Commit Graph

102 Commits

Author SHA1 Message Date
James Shubin
a4f368fc9f engine: resources: Add a sysctl resource
Very useful since these are easy to forget!
2024-09-15 23:07:03 -04:00
James Shubin
e7b57a32fd engine: resources: Add a tar resource
This makes tar archives from a list of files and/or directories. It
correctly includes empty directories as well. The code structure is
similar to the gzip resource. While this resource is arguably more
useful than gzip, it was invaluable to write the gzip resource first
since that made writing this one much easier.
2024-09-13 20:04:53 -04:00
James Shubin
29eebd0d07 lang: core: Move template to golang namespace
I don't think this template function should be in any way authoritative,
so let's namespace it.
2024-09-13 15:51:24 -04:00
James Shubin
5bbc06d8bc engine: resources: Add new gzip resource
This may have lots of uses, particularly for bootstrapping and handoff
if we want to compress payloads. It is also a good model resource for
how to implement such a resource to avoid re-computing the result on
every CheckApply call. Of course if the computation is cheaper than the
hashing of the data this isn't the optimal approach.
2024-09-13 03:32:10 -04:00
James Shubin
2e774215e4 lang: core: deploy: Add a function to get the binary path
Useful for bootstrapping new machines.
2024-09-10 23:22:53 -04:00
James Shubin
118f266211 lang: core: local: Add a new vardir function
This gives us a function to return a created vardir folder. It is not
locally namespaced, and a future function will have to namespace one to
each scope.
2024-09-09 18:04:18 -04:00
James Shubin
3d0660559e examples: lang: Add a join example
Good reminder that a lot of the golang stdlib functions are available.
2024-09-09 14:54:45 -04:00
James Shubin
d0ed004b24 examples: lang: Test that each of the mcl examples compiles
We let these rot, so fixup the issues and test them!
2024-07-31 17:29:42 -04:00
James Shubin
6de7d8b254 lang: funcs: Catch non-specific type build error
If you had ambiguous code, and specified an invalid type, this could
sneak through and become a runtime error, instead of a compile-time
error. We fix this and add a test.
2024-07-31 17:29:42 -04:00
Omar Al-Shuha
ccb0e55d5a examples: lang: Fix env0 example
Change function calls to the correct
one, remove extra argument in getenv
call, and fix typo.
2024-07-08 02:22:57 +02:00
James Shubin
aa03b5ce2f lang: core: iter: Add filter iterator function
This was fun to write and adds a new core iterator function.
2024-07-03 21:25:19 -04:00
James Shubin
e747e12002 examples: lang: Fixup a few examples
We might change unification to allow naked single strings with fancier
unification, but let's leave it as is for now and see how often it comes
up.
2024-07-02 23:50:24 -04:00
James Shubin
d1753c592a lang: core: iter: Misc formatting fixes
Also fix up the examples.
2024-07-02 23:49:56 -04:00
James Shubin
388d08e245 engine: resources: Add a dhcp range resource
This adds the ability to offer a dhcp lease to someone when we don't
know their mac address in advance.

This also uses the extended autogrouping API to keep the internal API
simpler.
2024-03-20 17:45:06 -04:00
James Shubin
849de648f0 engine: resources: Add a creates field for exec
This adds a standard gate that prevents execution if a file exists. Of
note, this also adds a watch on it, so we can have a proper watched exec
resource without a watch cmd.
2024-03-16 01:21:32 -04:00
James Shubin
c5e3e0ee70 engine: resources: Add firewalld resource
This is a simple firewalld resource to make the seamless opening of
firewall ports in standalone laptop (and other) environments easy.
2024-03-16 00:36:06 -04:00
James Shubin
589a5f9aeb cli, lib, lang: Port to new cli library
The new version of the urfave/cli library is moving to generics, and
it's completely unclear to me why this is an improvement. Their new API
is very complicated to understand, which for me, defeats the purpose of
golang.

In parallel, I needed to do some upcoming cli API refactoring, so this
was a good time to look into new libraries. After a review of the
landscape, I found the alexflint/go-arg library which has a delightfully
elegant API. It does have a few rough edges, but it's otherwise very
usable, and I think it would be straightforward to add features and fix
issues.

Thanks Alex!
2024-03-01 21:02:55 -05:00
James Shubin
a05b6a927e lang: funcs: core: os: Add an args function to read argv 2024-02-22 13:15:39 -05:00
James Shubin
26bce5ddc0 engine: resources: Add a new http:proxy resource
This adds a new caching http proxy resource that can be autogrouped into
the core http:server resource, and which caches and serves files that it
received from a different http server. It first pulls them down when
they are initially requested, which makes it possible for us to use this
for provisioning a Linux installation without having to pre-rsync the
entire package repository.
2024-02-22 13:15:39 -05:00
James Shubin
fc49888ba2 examples: lang: Add a simpler map-iterator example 2024-01-31 16:02:47 -05:00
James Shubin
7e1a4dea6c engine: resources: The http:file resource should allow directories
This expands the use of the http:file resource to allow it to be used as
a directory root.
2024-01-28 23:16:25 -05:00
James Shubin
67c72a0e86 engine: resources: Add an http:flag resource
This adds a new http:flag resource which can autogroup into an
http:server resource to receive actions from client HTTP requests, and
forward these values on to other resources.
2024-01-22 20:37:53 -05:00
James Shubin
377d62999f engine: resources: The kv resource can set un-mapped values
Previously the resource could only set values in a per-hostname
namespace, but for single, user-managed values, we'd like to be able to
control things entirely. Now this resource can do that.
2024-01-22 17:43:24 -05:00
James Shubin
aae0e16350 lang: ast, parser, interfaces: Implementation of nested class sugar
This implements a new type of syntactic sugar for the common pattern of
a base class which returns a child class, and so on. Instead of needing
to repeatedly indent the child classes, we can instead prefix them at
the definition site (where created with the class keyword) with the name
of the parent class, followed by a colon, to get the desired embedded
sugar.

For example, instead of writing:

class base() {
	class inner() {
		class deepest() {
		}
	}
}

You can instead write:

class base() {
}
class base:inner() {
}
class base:inner:deepest() {
}

Of course, you can only access any of the inner classes by first
including (with the include keyword) a parent class, and then
subsequently including the inner one.
2024-01-14 17:10:31 -05:00
James Shubin
44ee578a3a lang: parser, ast, interfaces: Implement include as
This adds support for `include as <identifier>` type statements which in
addition to pulling in any defined resources, it also makes the contents
of the scope of the class available to the scope of the include
statement, but prefixed by the identifier specified.

This makes passing data between scopes much more powerful, and it also
allows classes to return useful classes for subsequent use.

This also improves the SetScope procedure and adds to the Ordering
stage. It's unclear if the current Ordering stage can handle all code,
or if there exist corner-cases which are valid code, but which would
produce a wrong or imprecise topological sort.

Some extraneous scoping bugs still exist, which expose certain variables
that we should not depend on in future code.

Co-authored-by: Samuel Gélineau <gelisam@gmail.com>
2024-01-14 17:08:51 -05:00
James Shubin
a6b6aa570e test: Fix small test warnings
These were issues we should be catching.
2023-12-12 00:01:29 -05:00
James Shubin
18e1f08156 engine: graph: Allow send/recv to work with autogrouped resources
We've previously not received a value from within an autogrouped
resource. It turns out this would be quite useful, and so this patch
implements the additional plumbing and testing so that this works!

Testing that an autogrouped resource can still send values has not been
done at this time.
2023-12-08 18:18:17 -05:00
James Shubin
b7d8a769db lang: funcs: core: Add value functions
This adds a new series of "get*" functions which can read values from
the associated "value" resources. The key name of the function must
match the name value of the resource for things to work.

Type unification isn't yet perfect in these scenarios, so you should use
casually and with caution.
2023-12-03 18:24:20 -05:00
James Shubin
8251c8f259 lang: funcs: core: Add a second panic signature
This modifies the panic feature to accept a boolean or a string. If true
or not empty, then it will cause the panic. This makes some of the error
code a little less ugly.
2023-11-28 14:43:39 -05:00
James Shubin
9c0bde0b29 lang: funcs, parser: Add improved panic magic
This is a newer implementation of the panic magic. I kept the old commit
in for posterity and to show the difference. The two versions are
identical to the end-user with one exception: the newer version doesn't
include a useless panic resource in the graph when there is no panic. In
this version, the panic function returns false and the if statement it's
the condition of, doesn't produce the resource within. On error, we
still consume the function in the if expression, and doing so causes
everything to shutdown.

The other benefit is that the implementation is much cleaner and doesn't
need the interpolate hack.
2023-11-28 14:40:21 -05:00
James Shubin
2cbce963b7 engine: resources, lang: funcs, parser: Add panic magic
It's valuable to check your runtime values and to shut down the entire
engine in case something doesn't match. This patch adds some magic
plumbing to support a "panic" mechanism.

A new "panic" statement gets transparently converted into a panic
function and panic resource. The former errors if the input is not
empty. The latter must be present to consume the value, but doesn't
actually do anything.
2023-11-28 13:49:31 -05:00
James Shubin
d022d7f09e engine: resources: Rename the value field in value resource
Having a different name makes it more obvious, and also leaves us open
to add a string or bool field for more explicit typing.
2023-11-18 15:15:33 -05:00
James Shubin
88b414b9a3 engine: resources: Add new value resource
This is a strange resource which is probably most useful for passing
values between scopes. It supports a variant resource field, and should
only be used as a last resort and if you know exactly what you're doing.
2023-11-18 14:23:31 -05:00
James Shubin
6cc5adcd25 lang, test, examples: lang: Use new syntax for structlookup
Sugar makes the world go round.
2023-11-04 14:52:19 -04:00
James Shubin
96093984e4 test, examples: lang: Add new syntax for map lookup
Sugar at last!
2023-10-17 15:32:13 -04:00
James Shubin
d1c15bd0b7 lang: funcs: Rename the lookup functions
This will make things more consistent with future use.
2023-10-17 15:21:18 -04:00
James Shubin
b8f8ec8aa3 lang: parser, funcs: Take out the built-in history function
It's a normally named function for now, until we think of a common
package to move it into.

Hopefully this makes improving the lexer and parser easier for now. We
can consider bringing it back if needed.
2023-10-09 17:19:08 -04:00
James Shubin
05d570d250 lang: parser, funcs: Change the logical operators to OR, AND, NOT
This makes it easier to read for some, and easier to parse for us. This
also frees up more characters to use elsewhere.
2023-10-09 17:19:08 -04:00
James Shubin
2bc23c468e lang: funcs: core: iter: Finish map function
This was the goal all along. Proper iteration without for loops.

Yay!

Co-authored-by: Samuel Gélineau <gelisam@gmail.com>
2023-09-25 18:53:31 -04:00
James Shubin
2edae22a65 lang: funcs: core: test: Add a new test functions package
This adds a new test functions package and also a new "fastcount"
function which counts up from zero as fast as is possible. You probably
don't want to use this in production, but it is useful for performance
and deadlock testing the resource and function engines.
2023-08-30 21:58:38 -04:00
James Shubin
a41789a746 examples, lang: funcs: core: example: Update vumeter
This patch moves to use the sox package instead of arecord for getting
microphone data, and it also validates that both sox and rec and
installed. We also add a standalone example.
2023-02-08 12:14:11 -05:00
Samuel Gélineau
c23065aacd lang: funcs: Add SystemFunc
Runs a string as a shell command, then produces each line from stdout.
2022-09-11 21:58:39 -04:00
Jef Masereel
b26f842de1 engine: resources: Add hetzner:vm resource
Hetzner cloud resource using hcloud-go. Requires polling via Meta:poll param. This first commit provides a stable vm resource with support for the basic functions of creating, deleting and updating a live server instance. SSH key handling does still require some attention to make sure checkapply can detect and update live changes to the specified keylist. A dedicated hetzner:sshkeys resource might be in order to make sure the keyset is handled correctly if there are multiple hetzner:vm resources running under the same Hetzner project. All remarks for future improvements are indicated with a TODO prefix
2022-04-20 14:24:25 -04:00
James Shubin
e9791ff92c lang: funcs: core: fmt: Add variant verb for printf
There's no reason we can't support a %v variant verb. Of course it makes
type unification more difficult, and certain uses of this will produce
unsolvable situations, but it's useful for debugging, and fun to have.
2021-10-11 00:36:29 -04:00
James Shubin
400b58c0e9 lang: Improve string interpolation
The original string interpolation was based on hil which didn't allow
proper escaping, since they used a different escape pattern. Secondly,
the golang Unquote function didn't deal with the variable substitution,
which meant it had to be performed in a second step.

Most importantly, because we did this partial job in Unquote (the fact
that is strips the leading and trailing quotes tricked me into thinking
I was done with interpolation!) it was impossible to remedy the
remaining parts in a second pass with hil. Both operations needs to be
done in a single step. This is logical when you aren't tunnel visioned.

This patch replaces both of these so that string interpolation works
properly. This removes the ability to allow inline function calls in a
string, however this was an incidental feature, and it's not clear that
having it is a good idea. It also requires you wrap the var name with
curly braces. (They are not optional.)

This comes with a load of tests, but I think I got some of it wrong,
since I'm quite new at ragel. If you find something, please say so =D In
any case, this is much better than the original hil implementation, and
easy for a new contributor to patch to make the necessary fixes.
2021-02-17 03:35:12 -05:00
Joe Groocock
db95b6381f examples: lang: Reinstate mcl as unification bug is fixed
Struct field names now correctly map based on their `lang` tags in Go
structs, so this example now works as originally intended.

Signed-off-by: Joe Groocock <me@frebib.net>
2021-02-06 16:57:01 +00:00
Donald Bakong
5bc985663c docs: Add underscore issue to FAQ 2020-09-23 11:25:46 -04:00
James Shubin
8f60f42be3 engine: resources: Add http:server and http:file resources
This adds a new http server resource, as well as a http file resource
that is used to specify files to serve in that server. This allows you
to have an http server that is entirely server from memory without
needing files on disk.

It does this by using the autogrouping magic that is already available
in the engine.

The http resource is not meant to be a full-featured http server
replacement, and it might still be useful to use the venerable webserver
of your choice, however for integrated, pure-golang bootstrapping
environments, this might prove to be very useful.

It can be combined with the tftp and dhcp resources to build PXE setups
with mgmt!

This resource can be extended further to support an http:flag endpoint,
an http:ui endpoint, automatic edges, and more!
2020-04-11 03:23:04 -04:00
James Shubin
583344138a engine: resources: Add dhcp:server and dhcp:host resources
This adds a new dhcp server resource, as well as a dhcp host resource
used to specify the static mapping between mac address and ip address.
It also adds a simple, pure-golang example dhcp client which might make
testing easier.

The dhcp resource is not meant to be a full-featured dhcp server
replacement, and it might still be useful to use the venerable dhcpd,
however for integrated, pure-golang bootstrapping environments, this
might prove to be very useful.

It can be combined with the tftp resource to build PXE setups with mgmt.

This resource can be extended further to support a dhcp:range directive,
automatic edges, and more!
2020-04-11 02:45:18 -04:00
James Shubin
be39fbeff6 examples: lang: Update examples 2020-02-01 16:48:23 -05:00