422 Commits

Author SHA1 Message Date
James Shubin
f8b06f32ec engine: resources: Remove unused wait group 2024-11-06 21:09:50 -05:00
James Shubin
59a20f53eb lang: core: sys, engine: resource: Update hostname functionality
We didn't have a solid resource and sys.hostname() didn't have events!
2024-11-06 21:09:50 -05:00
James Shubin
83fd8b7e54 engine: util: Add more cmp utility functions 2024-11-06 20:02:10 -05:00
James Shubin
e4f062b006 engine: resources: Parse distro properly
I regressed here when patching this. Here's the fix. Would be beautiful
to have hardware to run end-to-end testing on. If you want to sponsor
this, please let me know!
2024-10-30 16:04:01 -04:00
James Shubin
20d4809e8e engine: resources: Print netmask nicely for our DHCP resource
Makes it easier to see what's going on.
2024-10-29 16:42:15 -04:00
James Shubin
583f90dc7b util: distro: Rename functions to avoid golang stutter warning 2024-10-25 02:56:12 -04:00
James Shubin
85e1d6c0e8 engine: resources: Make sure to set the netmask
Some clients would DECLINE if this was not set. This was reproduced my
using the stock coredhcp DHCPv4 server and disabling the netmask plugin.
One of the clients that would DECLINE is a Lenovo ThinkCentre m90n doing
a UEFI (PXE) netboot.

This was found in 1327752725 and is
hopefully now completely fixed!
2024-10-25 00:57:50 -04:00
James Shubin
067eef9007 util: distro, engine: resources: Update virt-builder res
A little easier to maintain if we support more distros eventually.
2024-10-15 20:36:51 -04:00
James Shubin
d403f18b2a util: distro: Put distro specific data in this util package
Try and see if we can put all our distro specific stuff in here...
2024-10-15 19:04:17 -04:00
James Shubin
1f12150d8f engine: resources, examples: lang: Expand on future deploy ideas
This is one idea for reference. But I doubt this is needed anytime soon
since we have a good working solution in the examples.
2024-10-13 20:47:47 -04:00
James Shubin
d3a7cefcc6 engine: resources: Add an archiving, deploy resource
This makes a bundle out of the code in the current deploy. Hopefully
this is useful for handoff!
2024-10-13 16:44:23 -04:00
James Shubin
a8c8f09aa3 gapi: Plumb through a URI mechanism
This is at least a stop-gap until we redo the whole filesystem API mess.
I think golang is partly to blame because they don't have proper API's
merged yet.
2024-10-13 16:40:50 -04:00
James Shubin
c937280664 engine: local, lang: core: local: Add a pool function
This adds a new local API for pool allocation, and with it a
corresponding function in the "local" import.
2024-10-04 22:37:39 -04:00
James Shubin
6419f931ee engine: resources: Add a virt-builder resource
This wraps the excellent virt-builder utility which makes setting up new
virtual machines a breeze.
2024-09-26 11:47:40 -04:00
James Shubin
8aac770bcb engine: resources: pkg: Add a small helper for simple installs
Add this utility function for doing single package installs.
2024-09-25 16:26:32 -04:00
James Shubin
80e8c9cadc util: arch: Use small arch util library
Put all of the arch stuff that we can into this library.
2024-09-25 16:25:55 -04:00
James Shubin
b9e093cd6b engine: resources: svc: Reduce unnecessary logging 2024-09-18 21:47:29 -04:00
James Shubin
06a023ca66 engine: resources: sysctl: Be more careful about dir path
More validation is not bad!
2024-09-18 21:39:33 -04:00
James Shubin
ccb4c6244d engine: resources: exec: Improve the docs for a common scenario
I knew this, and now you know it too!
2024-09-18 21:39:03 -04:00
James Shubin
4489e5ce6e engine: graph: autoedge: Quiet down the useless logs
These are so useful, let's silence them.
2024-09-18 21:38:29 -04:00
James Shubin
57b4a7efce lib, engine: graph: Let children directories be readable
We want to be able to put useful scripts in $vardir type places, but if
the perms at the higher levels block this, then that can't work. The
top-level should always be more permissive, and then it grows more
restricted as we descend.
2024-09-18 21:03:58 -04:00
James Shubin
fd508fbc0d engine: resources: Fix typo in svc
It's a typo, right?
2024-09-17 18:10:13 -04:00
James Shubin
a4f368fc9f engine: resources: Add a sysctl resource
Very useful since these are easy to forget!
2024-09-15 23:07:03 -04:00
James Shubin
e7b57a32fd engine: resources: Add a tar resource
This makes tar archives from a list of files and/or directories. It
correctly includes empty directories as well. The code structure is
similar to the gzip resource. While this resource is arguably more
useful than gzip, it was invaluable to write the gzip resource first
since that made writing this one much easier.
2024-09-13 20:04:53 -04:00
James Shubin
e34212a10b engine: resources: gzip: Check unhandled error
This is probably inconsequential, but let's do it since it's not in a
defer.
2024-09-13 19:48:00 -04:00
James Shubin
5f6e07b5e8 engine: resources: gzip: Fix typo 2024-09-13 16:24:01 -04:00
James Shubin
1465c5cdc9 engine: resource: gzip: Remove unneeded waitgroup
I think this was a copy+pasta mistake.
2024-09-13 16:13:03 -04:00
James Shubin
5bbc06d8bc engine: resources: Add new gzip resource
This may have lots of uses, particularly for bootstrapping and handoff
if we want to compress payloads. It is also a good model resource for
how to implement such a resource to avoid re-computing the result on
every CheckApply call. Of course if the computation is cheaper than the
hashing of the data this isn't the optimal approach.
2024-09-13 03:32:10 -04:00
James Shubin
1327752725 engine: resources: Special log message for unhandled decline
Not sure why a PXE client is sending these... Not sure if it's buggy
firmware or my inability to handle a DHCP corner case.
2024-09-10 23:22:47 -04:00
James Shubin
87a2dfc8f9 engine: local: Add a vardir API to our local API collection 2024-09-09 17:41:12 -04:00
James Shubin
fd3a2a1f0f engine: resources: Make consul optional
Licensing has made this non-free. Let's put that behind a build tag for
now, and remove it entirely if no suitable libre replacement is found.
2024-09-03 20:26:38 -04:00
James Shubin
c6e9175e3f engine: resources: Add missing build tag 2024-09-03 20:21:31 -04:00
James Shubin
bfa88e9b1c engine: resources: Workaround regression in wget2
Apparently wget2 has a serious regression that the HTTP 102 header
throws it off... So let's not send this for now... I'm pretty unhappy
about this, wget used to always be rock solid. Maybe curl deserves a
chance? (This works fine with curl btw.)
2024-08-30 20:33:41 -04:00
James Shubin
a0972c0752 lang, engine: Add a metaparam for catching accidental dollar signs
Let's make our life easier for users!
2024-08-22 20:41:48 -04:00
James Shubin
562eb643fc engine: resources: Display bytes copied when making a file 2024-08-06 15:24:45 -04:00
James Shubin
80178422db engine: graph, resources: Clean up log messages
The idea is to have a better user experience in the terminal.
2024-08-06 15:12:10 -04:00
James Shubin
e94f39bf2c engine: resources: Cleanups to the svc resource
Some new API's exist that take a context now too!
2024-08-06 14:31:39 -04:00
James Shubin
6c1a33066a engine: resources: The svc resource should reload on notification
Missing feature that is finally landing. I wish this wasn't needed, but
we need fancier plumbing to avoid it.
2024-08-06 14:22:15 -04:00
James Shubin
beca0c3ae6 engine: resources: Plumb through the context and constants
Basically a small cleanup.
2024-08-06 14:13:39 -04:00
James Shubin
7517c83953 engine: resources: Add the systemd service constants
Basically a cleanup to avoid duplicate strings everywhere. This makes it
easier to follow the code too.
2024-08-06 14:02:17 -04:00
James Shubin
0354082f89 engine: resources: Allow symbolic modes for missing files
In 83a747794e a bug was introduced with
the implementation of symbolic modes, that would prevent a file resource
from passing the Validate step if you were using a symbolic mode, and
the file didn't already exist. If you didn't use symbolic modes and
those files weren't absent, then you wouldn't have noticed.

It might be worth looking into the API for symbolic parsing as well.
2024-08-06 13:24:25 -04:00
James Shubin
c974820c56 engine: resources: Add log messages for chmod and chown 2024-08-01 20:32:56 -04:00
James Shubin
88670ae7a1 engine: resources: Improve output of log messages
I don't remember ever having this display a pointer address, but it is
now, so let's make this cleaner.
2024-08-01 18:44:44 -04:00
James Shubin
14577a0c46 lang: Add modern type unification implementation
This adds a modern type unification algorithm, which drastically
improves performance, particularly for bigger programs.

This required a change to the AST to add TypeCheck methods (for Stmt)
and Infer/Check methods (for Expr). This also changed how the functions
express their invariants, and as a result this was changed as well.

This greatly improves the way we express these invariants, and as a
result it makes adding new polymorphic functions significantly easier.

This also makes error output for the user a lot better in pretty much
all scenarios.

The one downside of this patch is that a good chunk of it is merged in
this giant single commit since it was hard to do it step-wise. That's
not the end of the world.

This couldn't be done without the guidance of Sam who helped me in
explaining, debugging, and writing all the sneaky algorithmic parts and
much more. Thanks again Sam!

Co-authored-by: Samuel Gélineau <gelisam@gmail.com>
2024-07-01 18:33:47 -04:00
Cian Yong Leow
09e53bfd3f engine: resources: file: Remove Validate owner/group Checks
The owner/group of a file should not be validated on the host until runtime. This removes the checks in Validate() that were happening before the execution of the resource graph (and therefore bound to fail if the system was being bootstrapped).
2024-06-17 13:52:04 +01:00
Joe Groocock
351a61c0cd engine: resources: docker: Update docker
Several types were renamed and moved

Signed-off-by: Joe Groocock <me@frebib.net>
2024-04-27 10:52:29 +00:00
James Shubin
388d08e245 engine: resources: Add a dhcp range resource
This adds the ability to offer a dhcp lease to someone when we don't
know their mac address in advance.

This also uses the extended autogrouping API to keep the internal API
simpler.
2024-03-20 17:45:06 -04:00
James Shubin
8a78907977 engine: autogroup, traits, graph: Extend autogroup API
This extends the autogrouping API so that a child can easily get a
reference to the parent that it is autogrouped in. This can simplify the
API for some resources when it makes sense to allow them access to the
parent handle. Use sparingly and intelligently!
2024-03-20 17:45:06 -04:00
James Shubin
849de648f0 engine: resources: Add a creates field for exec
This adds a standard gate that prevents execution if a file exists. Of
note, this also adds a watch on it, so we can have a proper watched exec
resource without a watch cmd.
2024-03-16 01:21:32 -04:00
James Shubin
4d18044851 util: Move recwatch package to util directory
This makes more sense and this is also a reminded that it needs a bit of
a cleanup.
2024-03-16 01:21:32 -04:00