There was and still is a bunch of terrible mess in this code. This does
some initial cleanup, and also fixes an important bug!
If you're provisioning a vmhost from scratch, then the function engine
might do some work to get the libvirt related services running before
the virt resource is used to build a vm. Since we had connection code in
Init() it would fail if it wasn't up already, meaning we'd have to write
fancy mcl code to avoid this, or we could do this refactor and keep
things more logical.
This mega patch primarily introduces a new function engine. The main
reasons for this new engine are:
1) Massively improved performance with lock-contended graphs.
Certain large function graphs could have very high lock-contention which
turned out to be much slower than I would have liked. This new algorithm
happens to be basically lock-free, so that's another helpful
improvement.
2) Glitch-free function graphs.
The function graphs could "glitch" (an FRP term) which could be
undesirable in theory. In practice this was never really an issue, and
I've not explicitly guaranteed that the new graphs are provably
glitch-free, but in practice things are a lot more consistent.
3) Simpler graph shape.
The new graphs don't require the private channels. This makes
understanding the graphs a lot easier.
4) Branched graphs only run half.
Previously we would run two pure side of an if statement, and while this
was mostly meant as an early experiment, it stayed in for far too long
and now was the right time to remove this. This also means our graphs
are much smaller and more efficient too.
Note that this changed the function API slightly. Everything has been
ported. It's possible that we introduce a new API in the future, but it
is unexpected to cause removal of the two current APIs.
In addition, we finally split out the "schedule" aspect from
world.schedule(). The "pick me" aspects now happen in a separate
resource, rather than as a yucky side-effect in the function. This also
lets us more precisely choose when we're scheduled, and we can observe
without being chosen too.
As usual many thanks to Sam for helping through some of the algorithmic
graph shape issues!
If the user is logged in, and we try to change from /home/james to
/home/james/ we'll get the error:
usermod: user james is currently used by process ????
and furthermore, it makes no sense to try and make this change since the
usermod function won't do anything if you run:
usermod --gid james --groups wheel --home /home/james/ james
when /etc/passwd has /home/james as the string.
If we want to receive more than on flag (key) value, then these
obviously need to autogroup together, because it's the same http server
request that comes in and which should be shared by everyone with the
same path.
These can "break" silently and not autogroup if we change the resource
and it no longer fulfills the interface. Add this compile time check to
prevent that.
Resources that can be grouped into the http:server resource must have
that prefix. Grouping is basically hierarchical, and without that common
prefix, it means we'd have to special-case our grouping algorithm.
Initially I wasn't 100% clear or decided on the send/recv semantics.
After some experimenting, I think this is much closer to what we want.
Nothing should break or regress here, this only enables more
possibilities.
Many years ago I built and demoed a prototype of a simple web ui with a
slider, and as you moved it left and right, it started up or shutdown
some number of virtual machines.
The webui was standalone code, but the rough idea of having events from
a high-level overview flow into mgmt, was what I wanted to test out. At
this stage, I didn't even have the language built yet. This prototype
helped convince me of the way a web ui would fit into everything.
Years later, I build an autogrouping prototype which looks quite similar
to what we have today. I recently picked it back up to polish it a bit
more. It's certainly not perfect, and might even be buggy, but it's
useful enough that it's worth sharing.
If I had more cycles, I'd probably consider removing the "store" mode,
and replace it with the normal "value" system, but we would need the
resource "mutate" API if we wanted this. This would allow us to directly
change the "value" field, without triggering a graph swap, which would
be a lot less clunky than the "store" situation.
Of course I'd love to see a GTK version of this concept, but I figured
it would be more practical to have a web ui over HTTP.
One notable missing feature, is that if the "web ui" changes (rather
than just a value changing) we need to offer to the user to reload it.
It currently doesn't get an event for that, and so don't confuse your
users. We also need to be better at validating "untrusted" input here.
There's also no major reason to use the "gin" framework, we should
probably redo this with the standard library alone, but it was easier
for me to push out something quick this way. We can optimize that later.
Lastly, this is all quite ugly since I'm not a very good web dev, so if
you want to make this polished, please do! The wasm code is also quite
terrible due to limitations in the compiler, and maybe one day when that
works better and doesn't constantly deadlock, we can improve it.
This adds initial symlink support to the file resource, and while it is
hopefully correct, there are always sneaky edge cases around symlinks
and security, so review and tests are highly encouraged!
These were really just stubs so that I could prove out the reactive
model very early, and I don't think they're really used anywhere.
I'm also not really using the yamlgraph frontend. If someone wants any
of that code, step up, or it will rot even more.
A subtlety about the engine is that while it guarantees CheckApply
happens in the listed edge-based dependency order, it doesn't stop
Watch from starting up in whatever order it wants to. As a result, we
can prematurely error since the docker service isn't running yet. It may
in fact be in the process of getting installed and started by mgmt
before we then try and use this resource! As a result, let it error once
for free and wait for CheckApply to get going before we start again.
Keep in mind, Watch has to use the .Running() method once to tell
CheckApply to do its initial event. So this concurrency is complex!
It's unclear if this is a bug in mgmt or not, but I'm leaning towards
not, particularly since there isn't an obvious way to fix it.
They made the assumption that there would be a based docker service
installed at Init which could not be guaranteed. Also use the internal
metaparameter timeout feature instead of private counters.
This adds some simplistic configuration management / provisioning
functionality to this virt:builder resource which makes it easier to
kick off special functionality that we might want to build.
It would be a likely mistake to create a self-referential frag, and mgmt
would spin forever updating the file... We probably don't want this, so
let's just catch this case in Validate.
Of course you could get around this with multiple files, and a fancier
search could statically check the graph, but the goal isn't to prevent
any bad code, since that's not likely to be possible.
Instead of constantly making these updates, let's just remove the year
since things are stored in git anyways, and this is not an actual modern
legal risk anymore.
