Commit Graph

379 Commits

Author SHA1 Message Date
James Shubin
774d408e13 engine: Fix up some send/recv corner cases
Initially I wasn't 100% clear or decided on the send/recv semantics.
After some experimenting, I think this is much closer to what we want.
Nothing should break or regress here, this only enables more
possibilities.
2025-05-05 23:53:37 -04:00
James Shubin
ae1d9b94d4 engine: util: Add a debug utility
This is useful for some patches. Let's see if I can remember to use and
improve it!
2025-05-05 22:30:31 -04:00
James Shubin
267bcc144b engine: util: Clean up error messages 2025-05-05 22:29:10 -04:00
James Shubin
fd40c3b64f engine: util: Fix grammar typo 2025-05-05 20:21:29 -04:00
James Shubin
e2b6da01d8 engine: graph: Fix messy imports 2025-05-05 20:21:29 -04:00
James Shubin
807c4b3430 engine: resources: Add an http ui resource
Many years ago I built and demoed a prototype of a simple web ui with a
slider, and as you moved it left and right, it started up or shutdown
some number of virtual machines.

The webui was standalone code, but the rough idea of having events from
a high-level overview flow into mgmt, was what I wanted to test out. At
this stage, I didn't even have the language built yet. This prototype
helped convince me of the way a web ui would fit into everything.

Years later, I build an autogrouping prototype which looks quite similar
to what we have today. I recently picked it back up to polish it a bit
more. It's certainly not perfect, and might even be buggy, but it's
useful enough that it's worth sharing.

If I had more cycles, I'd probably consider removing the "store" mode,
and replace it with the normal "value" system, but we would need the
resource "mutate" API if we wanted this. This would allow us to directly
change the "value" field, without triggering a graph swap, which would
be a lot less clunky than the "store" situation.

Of course I'd love to see a GTK version of this concept, but I figured
it would be more practical to have a web ui over HTTP.

One notable missing feature, is that if the "web ui" changes (rather
than just a value changing) we need to offer to the user to reload it.
It currently doesn't get an event for that, and so don't confuse your
users. We also need to be better at validating "untrusted" input here.

There's also no major reason to use the "gin" framework, we should
probably redo this with the standard library alone, but it was easier
for me to push out something quick this way. We can optimize that later.

Lastly, this is all quite ugly since I'm not a very good web dev, so if
you want to make this polished, please do! The wasm code is also quite
terrible due to limitations in the compiler, and maybe one day when that
works better and doesn't constantly deadlock, we can improve it.
2025-05-02 02:14:14 -04:00
James Shubin
412e480b44 engine: local: Get the logic right
I think we were not benefitting from the cache and sending unnecessary
events. It would be great to have tests for this, but commit this fix
for now, and be embarrassed in the future if I got this code wrong.
2025-05-02 00:04:00 -04:00
James Shubin
cc2a235fbb engine: resources: Add a reminder about events
I might want to do this some day, it could be important. Look into it.
2025-05-02 00:04:00 -04:00
James Shubin
7c77efec1d engine: resources: Cleanup this old code
This is equivalent and cleaner.
2025-05-02 00:04:00 -04:00
James Shubin
de970ee557 engine: resources: Add symlink param to the file res
This adds initial symlink support to the file resource, and while it is
hopefully correct, there are always sneaky edge cases around symlinks
and security, so review and tests are highly encouraged!
2025-04-22 02:21:58 -04:00
James Shubin
748f05732a engine, etcd: Watch on star pattern for all hostnames
We forgot to watch on star hostname matches.
2025-04-05 15:45:44 -04:00
James Shubin
148bd50e9f engine, etcd: Prevent engine thrashing
These two small bugs would allow thrashing to occur since we'd
constantly delete and re-add exports, and constantly think that a noop
etcd operation made a change.
2025-04-05 15:28:54 -04:00
James Shubin
6c1c08ceda engine: resources: Test to make sure metaparams are preserved
We should ensure these get preserved across encoding/decoding. We rely
on this behaviour.
2025-04-05 12:45:23 -04:00
James Shubin
045b29291e engine, lang: Modern exported resources
I've been waiting to write this patch for a long time. I firmly believe
that the idea of "exported resources" was truly a brilliant one, but
which was never even properly understood by its original inventors! This
patch set aims to show how it should have been done.

The main differences are:

* Real-time modelling, since "once per run" makes no sense.
* Filter with code/functions not language syntax.
* Directed exporting to limit the intended recipients.

The next step is to add more "World" reading and filtering functions to
make it easy and expressive to make your selection of resources to
collect!
2025-04-05 12:45:23 -04:00
James Shubin
955112f64f engine: Let others use the ResUID struct
It's a useful key in maps.
2025-04-05 12:45:23 -04:00
James Shubin
7f341cee84 engine: resources: Improving logging even more
Messages should happen after the event on success. The error scenario
has its own pathway to report.
2025-04-05 12:45:23 -04:00
James Shubin
f71e623931 engine: resources: Print a message on empty file creation
We don't see this event happening which is confusing. There might be
other cases we didn't handle cleanly.
2025-04-05 12:45:23 -04:00
James Shubin
30aca74089 engine, yamlgraph: Disable the old exported resources stuff
These were really just stubs so that I could prove out the reactive
model very early, and I don't think they're really used anywhere.

I'm also not really using the yamlgraph frontend. If someone wants any
of that code, step up, or it will rot even more.
2025-04-05 12:45:23 -04:00
James Shubin
3dfca97f86 engine: Add a method to determine if a res kind is valid 2025-04-05 12:45:23 -04:00
James Shubin
5665259784 cli, engine, etcd, lib: Move the hostname value to the API
Every world implementation needs a unique UUID, might as well move this
to the API.
2025-03-19 05:41:04 -04:00
James Shubin
a7a5237b07 cli, engine, etcd, lib: Pass in init args
Improve the API and make it more general.
2025-03-18 04:54:13 -04:00
James Shubin
7ad54fe3e8 cli, engine, etcd, lib: Split out the deployer into world
This should hopefully make the refactor into a clean world API a bit
better. Still more to do though!
2025-03-18 04:54:13 -04:00
James Shubin
1a35ab61ca engine: Split out the world filesystem interface 2025-03-18 03:32:42 -04:00
James Shubin
59c33a354c engine, lang: core: world: Split out the scheduler interface 2025-03-18 03:32:42 -04:00
James Shubin
c853e24ded engine: Split out the str world interface
This is a core API and it should really be cleaned up if possible.
2025-03-18 03:32:42 -04:00
James Shubin
692db084e4 engine: Split off resource world interface 2025-03-18 03:32:42 -04:00
James Shubin
1edff3b3f5 engine: Move another interface method 2025-03-18 03:32:42 -04:00
James Shubin
b173d9f8ef engine: Split out the etcd cluster size options
This is clean up work so that it's easier to generalize the world
backends.
2025-03-18 03:32:42 -04:00
James Shubin
f313380480 engine: resources: Container stopped should be valid for no container 2025-03-13 01:03:11 -04:00
James Shubin
f8a4751290 engine: resources: Don't prematurely error docker watches
A subtlety about the engine is that while it guarantees CheckApply
happens in the listed edge-based dependency order, it doesn't stop
Watch from starting up in whatever order it wants to. As a result, we
can prematurely error since the docker service isn't running yet. It may
in fact be in the process of getting installed and started by mgmt
before we then try and use this resource! As a result, let it error once
for free and wait for CheckApply to get going before we start again.

Keep in mind, Watch has to use the .Running() method once to tell
CheckApply to do its initial event. So this concurrency is complex!

It's unclear if this is a bug in mgmt or not, but I'm leaning towards
not, particularly since there isn't an obvious way to fix it.
2025-03-12 06:14:38 -04:00
James Shubin
3ca1aa9cb1 engine: resources: Fix backwards docker ports
This wasn't setup properly, now it's fixed. Woops.
2025-03-12 05:45:27 -04:00
James Shubin
05306e3729 engine: resources: Modernize the docker resources
They made the assumption that there would be a based docker service
installed at Init which could not be guaranteed. Also use the internal
metaparameter timeout feature instead of private counters.
2025-03-12 05:45:26 -04:00
James Shubin
46035fee83 engine: resources: Add simple configuration steps to virt builder
This adds some simplistic configuration management / provisioning
functionality to this virt:builder resource which makes it easier to
kick off special functionality that we might want to build.
2025-03-11 04:18:02 -04:00
James Shubin
219d25b330 engine: resources, modules: virtualization: Add a seeds option
This makes it easier to configure the machine by giving it an automatic
initial setup of an mgmt client.
2025-03-11 04:18:02 -04:00
James Shubin
181aab9c81 engine: resources: Fix small cmp typo in virt builder res 2025-03-10 19:01:05 -04:00
Edward Toroshchyn
63d7b8e51e engine: resources: exec: Fix wrong err variable being checked in test 2025-02-27 14:50:38 -05:00
Edward Toroshchyn
c25a2a257b misc: Fix typos and spelling errors 2025-02-24 16:01:46 +01:00
Lourenço
b5384d1278 engine: resources: Adding logic for svc unit state
This is a small patch that adds logic for checking what's the state of
the unit file and making the CheckApply function more robust
2025-02-18 17:14:27 -05:00
James Shubin
d80ec4aaa7 engine: resources: Detect simple self-referential frags
It would be a likely mistake to create a self-referential frag, and mgmt
would spin forever updating the file... We probably don't want this, so
let's just catch this case in Validate.

Of course you could get around this with multiple files, and a fancier
search could statically check the graph, but the goal isn't to prevent
any bad code, since that's not likely to be possible.
2025-02-15 06:58:15 -05:00
James Shubin
82489c3fe0 engine: resources: Add shell field to user resource 2025-02-07 18:08:25 -05:00
James Shubin
f51a1200d1 util: Add a helper to get the users shell entry 2025-02-07 17:57:36 -05:00
James Shubin
096ef4cc66 engine: resources: Modernize the user resource
Do some small fixups like adding ctx and fixing obvious bugs.
2025-02-07 17:57:36 -05:00
hades
8cd7fa27e2 engine: resources: exec: Add a bit of documentation to exec res 2025-02-06 08:18:48 -05:00
Edward Toroshchyn
042ae02428 engine: resources: exec: Add tests to check env values 2025-02-06 07:14:12 -05:00
James Shubin
0fb546ad61 engine: resources: Make some svc cleanups
We would often actually drop the refresh because of bad checks.
2025-02-02 01:43:14 -05:00
James Shubin
5044ef4e8a engine: resources: Add a virt builder password selector
This is very helpful for debugging, particularly with broken Debian
installs.
2025-01-30 22:56:24 -05:00
James Shubin
d30ff6cfae legal: Remove year
Instead of constantly making these updates, let's just remove the year
since things are stored in git anyways, and this is not an actual modern
legal risk anymore.
2025-01-26 16:24:51 -05:00
James Shubin
12a0600d38 engine: resources: Improve hostname change message
Make it clearer where empty strings are, and that it already happened.
2025-01-17 23:58:26 -05:00
James Shubin
a713c08585 engine: resources: Make net and firewalld resources quieter
They are too noisy!
2025-01-17 23:32:13 -05:00
James Shubin
a08ba0b0e9 engine: resources: Tar now accepts dirs without a trailing slash
If these are found, then the dir path itself is copied in as well.
2025-01-17 18:15:45 -05:00