Commit Graph

402 Commits

Author SHA1 Message Date
James Shubin
67c72a0e86 engine: resources: Add an http:flag resource
This adds a new http:flag resource which can autogroup into an
http:server resource to receive actions from client HTTP requests, and
forward these values on to other resources.
2024-01-22 20:37:53 -05:00
James Shubin
e7a89a4a42 engine: Resource Cmp should be relaxed
When graph swapping (which is quite common) we only use the newly-made
resource if the Cmp function between the two shows a difference. If the
old resource has previously received a value via send/recv, then when it
is compared to the new value, it will almost always be different. As a
result, we need to run send/recv on the newly made graph to make sure it
has up-to-date values before we compare. This has to happen after
autogrouping since the resources can often be autogrouped and any child
grouped resource will cause a remake of all of the other children and
parents.

It turns out that the actual send/recv properties were being compared as
well, and for unknown reasons (tunnel vision perhaps) they are often not
identical. Skip comparing these for now until we find a fix or
understanding of how to make them identical.
2024-01-22 20:37:53 -05:00
James Shubin
1630eafbe3 engine: graph: Allow Recv overriding for Send/Recv
This allows us to pass in an alternate implementation for Recv, if we
want to temporarily use a different data source.
2024-01-22 20:37:53 -05:00
James Shubin
091bf3a64c engine: graph: Add a function for printing Send/Recv logs 2024-01-22 20:37:53 -05:00
James Shubin
e5a189b8c6 engine: graph: Split SendRecv off from the engine
It can be used in more places if it's not tied to the engine struct.
This also changes the signature so that more information is returned.
This can be used for logging or other useful things. Of note, this
happens to be the same struct as already exists. It's used for
convenience since it happens to match up! Of course they're related.
2024-01-22 20:37:53 -05:00
James Shubin
f68b34a485 engine: Add a mapper function for comparing two res graphs
This gives us a simple mapping between a new resource and an old one. We
compare by kind and name because those two values are our uniqueness
constraint in the resource graphs.
2024-01-22 20:37:53 -05:00
James Shubin
e946b39960 engine: graph: Set closed flag when resume signal closes
We had this backwards. Woops...
2024-01-22 20:37:53 -05:00
James Shubin
0711d05232 engine: resources: http: Improve groupability of resources
This improves groupability of http resources so that we can have fancier
kinds!
2024-01-22 20:37:53 -05:00
James Shubin
9dd5dfdde2 engine: graph: autogroup: Improve the autogrouping algorithm
This improves the autogrouping algorithm to support hierarchical
autogrouping. It's not guaranteed to work if we replace the reachability
grouper with something more efficient, but it's good enough for now.
2024-01-22 20:37:53 -05:00
James Shubin
32916f9a6f engine: resources: Receive keys should match mcl, not golang
The capitalization of these keys was wrong and they weren't getting
seen. Add a test as well.
2024-01-22 19:40:15 -05:00
James Shubin
b670bb8d2c engine: resources: http: Recurse on Init and Watch
Make this resource handle sub-resources more powerfully.
2024-01-22 19:16:28 -05:00
James Shubin
1abf6547ff engine: resources: http: Improved CheckApply for grouped resources 2024-01-22 19:16:28 -05:00
James Shubin
3739fa401e engine: resources: Refactor HTTP constants 2024-01-22 17:43:24 -05:00
James Shubin
377d62999f engine: resources: The kv resource can set un-mapped values
Previously the resource could only set values in a per-hostname
namespace, but for single, user-managed values, we'd like to be able to
control things entirely. Now this resource can do that.
2024-01-22 17:43:24 -05:00
James Shubin
b4bb4ca397 engine: resources: Update kv error messages 2024-01-22 17:43:24 -05:00
James Shubin
6b4fb434da engine: resources: Rename var
More accurate.
2024-01-22 17:43:24 -05:00
James Shubin
9e70f53afa engine: resources: Add missing apply guard 2024-01-22 17:43:24 -05:00
James Shubin
81885dec63 engine: resources: Add some spacing 2024-01-22 17:43:24 -05:00
James Shubin
a8f194259b legal: Happy 2024 everyone...
Done with:

ack '2023+' -l | xargs sed -i -e 's/2023+/2024+/g'

Checked manually with:

git add -p

Hello to future James from 2025, and Happy Hacking!
2024-01-22 15:52:49 -05:00
James Shubin
c4a9560d53 engine: local: Fix benign race
Use the mutex in a safer manner to eliminate the benign data race.
2024-01-05 15:50:43 -05:00
James Shubin
bc63b7608e engine: graph: Use an atomic bool instead of a mutex
The isStateOK variable can be accessed concurrently as these are
supposed to be "benign" races. As such, they need to be labelled as such
so that we don't hit some undefined compiler behaviour.

Here are five good references relating to "benign" data races in golang.

1) https://web.archive.org/web/20181022150257/https://software.intel.com/en-us/blogs/2013/01/06/benign-data-races-what-could-possibly-go-wrong

2) https://go.dev/ref/mem - "Informal Overview" section.

3) https://docs.oracle.com/cd/E19205-01/820-0619/gecqt/index.html

4) https://www.usenix.org/legacy/event/hotpar11/tech/final_files/Boehm.pdf

5) https://go.dev/doc/articles/race_detector

TL;DR: wrap your benign races with sync/atomic or eliminate them.
2024-01-05 15:50:42 -05:00
James Shubin
c2f508e261 engine: graph: Use an rwmutex around read/write of vertex state
This fixes two small races we had while simultaneously reading from and
writing to the vertex timestamp, and simultaneously writing to the
vertex isStateOK (dirty) flag.

They were actually "safe" races, in that it doesn't matter if the
read/write race got the old or new value, or that the double write
happened. The time sequencing was correct (I believe) in both cases, but
this triggers the race detector now that we have tests for it.
2024-01-02 18:18:34 -05:00
James Shubin
18e1f08156 engine: graph: Allow send/recv to work with autogrouped resources
We've previously not received a value from within an autogrouped
resource. It turns out this would be quite useful, and so this patch
implements the additional plumbing and testing so that this works!

Testing that an autogrouped resource can still send values has not been
done at this time.
2023-12-08 18:18:17 -05:00
James Shubin
271a94e0c7 engine: resources: Improve logging on resource errors
When a subcommand errors, it's helpful to see what it was.
2023-12-03 21:21:55 -05:00
James Shubin
b7d8a769db lang: funcs: core: Add value functions
This adds a new series of "get*" functions which can read values from
the associated "value" resources. The key name of the function must
match the name value of the resource for things to work.

Type unification isn't yet perfect in these scenarios, so you should use
casually and with caution.
2023-12-03 18:24:20 -05:00
James Shubin
c05af3b9b3 lib, lang, engine: local: Add Value mechanisms to local API
This expands the Local API with the first (and in theory, only ever) API
for reading and writing simple values. This is a coordination point for
resources and functions to share things directly.
2023-12-03 18:24:20 -05:00
James Shubin
9d47b6843f engine, gapi, lang, lib: Plumb through new local API
This is a new API that is similar in spirit and plumbing to the World
API, but it intended for all local machine operations and will likely
only ever have one implementation.
2023-12-03 18:24:20 -05:00
James Shubin
7991b4ab25 engine, lang, lib: Re-order for consistency
Some small cleanups to make room for future patches.
2023-12-03 18:02:00 -05:00
James Shubin
984fb702e5 engine: resources: Make test resource easier for debugging
Let some fields not get printed if we request it.
2023-12-03 18:02:00 -05:00
James Shubin
b8d391024a engine: resources: Add a DoneCmd feature for exec resource
This makes a common pattern of `$cmd && echo done > /tmp/donefile` much
clearer and easier.
2023-11-28 15:24:22 -05:00
James Shubin
598bec0eab engine: resources: Clean up var naming and misc in exec 2023-11-28 15:24:11 -05:00
James Shubin
709bf7b246 engine: resources: Add missing WatchCmd UID
Not sure if this is even used anywhere, but might as well add it for
now.
2023-11-28 15:24:11 -05:00
James Shubin
2cbce963b7 engine: resources, lang: funcs, parser: Add panic magic
It's valuable to check your runtime values and to shut down the entire
engine in case something doesn't match. This patch adds some magic
plumbing to support a "panic" mechanism.

A new "panic" statement gets transparently converted into a panic
function and panic resource. The former errors if the input is not
empty. The latter must be present to consume the value, but doesn't
actually do anything.
2023-11-28 13:49:31 -05:00
James Shubin
dad3458ece mod: Try and update things
No idea why things break. go.mod sucks.
2023-11-27 21:00:03 -05:00
James Shubin
0e2c73a36d engine: resources: Reduce logging for file resource 2023-11-27 19:57:54 -05:00
James Shubin
ee88161808 engine: graph, resources: Reduce and clean up logging
Make the output more usable.
2023-11-22 22:37:35 -05:00
James Shubin
04d54e8d82 engine: resources: Mask normal closing error
This could possibly be fixed upstream, but we'll workaround it for now.
2023-11-22 22:11:50 -05:00
James Shubin
f4d70068b1 engine: resources: Add a default NBP option to dhcp:server
This let's us do some overrides and is a good alternative since
send/recv+autogrouping doesn't work yet.
2023-11-22 20:22:50 -05:00
James Shubin
3fe5a8d0d6 mod: Update tftp libraries 2023-11-22 20:21:40 -05:00
James Shubin
6670407c9c engine: resources: Make net resource reversible
This allows the net resource to be reversible if we want it to be. It
only flips the network state from up to down or vice-versa.
2023-11-22 20:21:40 -05:00
James Shubin
641b6067cd engine: resources: Remove net config file on down
This cleans up the file we would have created when building the resource.
2023-11-22 20:21:40 -05:00
James Shubin
b1d61fa90b engine: resources: Ignore cleanup errors that don't matter
If the file is already gone, ignore it.
2023-11-22 20:21:40 -05:00
James Shubin
0c93cf2600 engine: resources: Add constants for state up and down
Safer and cleaner code.
2023-11-22 20:21:40 -05:00
James Shubin
53753c0932 engine: resources: Improve net error messages
This is a common case of parse failures for not using CIDR. Make it
cleaner for the user.
2023-11-22 20:21:40 -05:00
James Shubin
029cfaf1f8 engine, lib: Misc log and comment changes
Just misc polishing.
2023-11-22 20:21:40 -05:00
James Shubin
66edc39ccb engine: resources: Improve exec docs
So that I don't hit this too often.
2023-11-22 20:21:40 -05:00
James Shubin
3f42f785ac engine: resources: Handle recurse field more appropriately
This smoothes out some edge cases. We should add comprehensive tests in
the future.
2023-11-22 20:21:40 -05:00
James Shubin
fd4d32351b engine: resources: Refactor file mode code
Small cleanup in old code.
2023-11-22 20:21:40 -05:00
James Shubin
75d3895e84 engine: Improve logging for resources and exec
This makes things a bit clearer and fixes some silly formatting bugs.
2023-11-22 20:21:40 -05:00
James Shubin
d022d7f09e engine: resources: Rename the value field in value resource
Having a different name makes it more obvious, and also leaves us open
to add a string or bool field for more explicit typing.
2023-11-18 15:15:33 -05:00