These are expected from our engine. We do care about timeout's and so
on. This allows os to return ctx.Err() whenever a <-ctx.Done() happens,
which is more idiomatic for what we really want, but which we weren't
thorough with before.
This mega patch primarily introduces a new function engine. The main
reasons for this new engine are:
1) Massively improved performance with lock-contended graphs.
Certain large function graphs could have very high lock-contention which
turned out to be much slower than I would have liked. This new algorithm
happens to be basically lock-free, so that's another helpful
improvement.
2) Glitch-free function graphs.
The function graphs could "glitch" (an FRP term) which could be
undesirable in theory. In practice this was never really an issue, and
I've not explicitly guaranteed that the new graphs are provably
glitch-free, but in practice things are a lot more consistent.
3) Simpler graph shape.
The new graphs don't require the private channels. This makes
understanding the graphs a lot easier.
4) Branched graphs only run half.
Previously we would run two pure side of an if statement, and while this
was mostly meant as an early experiment, it stayed in for far too long
and now was the right time to remove this. This also means our graphs
are much smaller and more efficient too.
Note that this changed the function API slightly. Everything has been
ported. It's possible that we introduce a new API in the future, but it
is unexpected to cause removal of the two current APIs.
In addition, we finally split out the "schedule" aspect from
world.schedule(). The "pick me" aspects now happen in a separate
resource, rather than as a yucky side-effect in the function. This also
lets us more precisely choose when we're scheduled, and we can observe
without being chosen too.
As usual many thanks to Sam for helping through some of the algorithmic
graph shape issues!
This should ensure this sort order of longer thing first, when running
the algorithm. We want to autogroup two or more http:server:flag
resources together first, before the whole grouped resource gets pulled
into the http:server one.
Resources that can be grouped into the http:server resource must have
that prefix. Grouping is basically hierarchical, and without that common
prefix, it means we'd have to special-case our grouping algorithm.
Initially I wasn't 100% clear or decided on the send/recv semantics.
After some experimenting, I think this is much closer to what we want.
Nothing should break or regress here, this only enables more
possibilities.
Many years ago I built and demoed a prototype of a simple web ui with a
slider, and as you moved it left and right, it started up or shutdown
some number of virtual machines.
The webui was standalone code, but the rough idea of having events from
a high-level overview flow into mgmt, was what I wanted to test out. At
this stage, I didn't even have the language built yet. This prototype
helped convince me of the way a web ui would fit into everything.
Years later, I build an autogrouping prototype which looks quite similar
to what we have today. I recently picked it back up to polish it a bit
more. It's certainly not perfect, and might even be buggy, but it's
useful enough that it's worth sharing.
If I had more cycles, I'd probably consider removing the "store" mode,
and replace it with the normal "value" system, but we would need the
resource "mutate" API if we wanted this. This would allow us to directly
change the "value" field, without triggering a graph swap, which would
be a lot less clunky than the "store" situation.
Of course I'd love to see a GTK version of this concept, but I figured
it would be more practical to have a web ui over HTTP.
One notable missing feature, is that if the "web ui" changes (rather
than just a value changing) we need to offer to the user to reload it.
It currently doesn't get an event for that, and so don't confuse your
users. We also need to be better at validating "untrusted" input here.
There's also no major reason to use the "gin" framework, we should
probably redo this with the standard library alone, but it was easier
for me to push out something quick this way. We can optimize that later.
Lastly, this is all quite ugly since I'm not a very good web dev, so if
you want to make this polished, please do! The wasm code is also quite
terrible due to limitations in the compiler, and maybe one day when that
works better and doesn't constantly deadlock, we can improve it.
These two small bugs would allow thrashing to occur since we'd
constantly delete and re-add exports, and constantly think that a noop
etcd operation made a change.
I've been waiting to write this patch for a long time. I firmly believe
that the idea of "exported resources" was truly a brilliant one, but
which was never even properly understood by its original inventors! This
patch set aims to show how it should have been done.
The main differences are:
* Real-time modelling, since "once per run" makes no sense.
* Filter with code/functions not language syntax.
* Directed exporting to limit the intended recipients.
The next step is to add more "World" reading and filtering functions to
make it easy and expressive to make your selection of resources to
collect!
Instead of constantly making these updates, let's just remove the year
since things are stored in git anyways, and this is not an actual modern
legal risk anymore.
We want to be able to put useful scripts in $vardir type places, but if
the perms at the higher levels block this, then that can't work. The
top-level should always be more permissive, and then it grows more
restricted as we descend.
This extends the autogrouping API so that a child can easily get a
reference to the parent that it is autogrouped in. This can simplify the
API for some resources when it makes sense to allow them access to the
parent handle. Use sparingly and intelligently!
With the recent merging of embedded package imports and the entry CLI
package, it is now possible for users to build in mcl code into a single
binary. This additional permission makes it explicitly clear that this
is permitted to make it easier for those users. The condition is phrased
so that the terms can be "patched" by the original author if it's
necessary for the project. For example, if the name of the language
(mcl) changes, has a differently named new version, someone finds a
phrasing improvement or a legal loophole, or for some other
reasonable circumstance. Now go write some beautiful embedded tools!
A regression in 4b0cdf9123 caused the
basic send/recv functionality to break for simple scenarios. This was
due to inadequate testing, and a partial misunderstanding of the
situation.
New testing should hopefully catch more cases, but send/recv and
compile-time checks are still not as complete as is probably possible.
It can be used in more places if it's not tied to the engine struct.
This also changes the signature so that more information is returned.
This can be used for logging or other useful things. Of note, this
happens to be the same struct as already exists. It's used for
convenience since it happens to match up! Of course they're related.
This improves the autogrouping algorithm to support hierarchical
autogrouping. It's not guaranteed to work if we replace the reachability
grouper with something more efficient, but it's good enough for now.
This fixes two small races we had while simultaneously reading from and
writing to the vertex timestamp, and simultaneously writing to the
vertex isStateOK (dirty) flag.
They were actually "safe" races, in that it doesn't matter if the
read/write race got the old or new value, or that the double write
happened. The time sequencing was correct (I believe) in both cases, but
this triggers the race detector now that we have tests for it.
We've previously not received a value from within an autogrouped
resource. It turns out this would be quite useful, and so this patch
implements the additional plumbing and testing so that this works!
Testing that an autogrouped resource can still send values has not been
done at this time.
This is a new API that is similar in spirit and plumbing to the World
API, but it intended for all local machine operations and will likely
only ever have one implementation.
If we have rare, but special *interface{} values in resource structs, we
should be able to handle them normally. It's really not recommended that
you use these unless you know exactly why they are useful.
This lets us add a resource that has an implementation with a field
whose type is determined at compile time. This let's us write more
flexible resources.
What's missing is additional type checking so that we guarantee that a
specific resource doesn't change types during run-time.
A back poke is the deferral or delay of a Process/CheckApply. This is
because we notice that we're not truly ready to CheckApply due to some
timestamp issue. When Process errors, we should accept that, but not
treat it as a success.
