Commit Graph

146 Commits

Author SHA1 Message Date
James Shubin
43ff92bbe7 engine: resources: Clean up test log message 2019-09-11 03:40:22 -04:00
James Shubin
28adc7e563 engine: resource: Refactor helper functions
Maybe we can use them in other tests too.
2019-09-11 03:40:22 -04:00
James Shubin
9788411995 engine: resources: Add another validation check
This simple check should prevent some silly mistakes and make the logic
easier for other parts of the code that won't have to worry about this
pattern.
2019-09-11 03:40:22 -04:00
James Shubin
0c9e8cc50e engine: resources: Change the default file state
The default file state should be undefined. This is important because if
a reverse scenario that doesn't specify the state gets given this
default, it will be as if it was specified explicitly, which wouldn't
necessarily be what we want. Instead, an undefined state should
implicitly cause a file to get created if there's a reason to do so,
such as if content or another attribute is specified.

Hopefully this change doesn't introduce any bugs in the CheckApply code,
if it does, then it was due to a lack of implicit file creation.
2019-09-11 03:16:57 -04:00
James Shubin
34d572c523 engine: Improve the way we make a unique res path token
This is needed in the state directory.
2019-09-11 03:16:57 -04:00
James Shubin
011b496b3f engine: resources: Ensure the Kind and Name methods work
Triple check these work after decoding, by adding a test.
2019-09-11 03:16:57 -04:00
James Shubin
12b906eac6 engine: Refactor state dir into a separate function
This lets us re-use it, and know the path is fixed.
2019-09-11 03:16:57 -04:00
James Shubin
20937d05c3 engine: resources: file: Add undefined file state and validate it
We should consider using *string instead of the empty string, but let's
keep the diff smaller for now.
2019-09-11 03:16:57 -04:00
James Shubin
4943d37ccf engine: resources: file: Use constants for state values
More robustness is yay!
2019-09-11 03:16:57 -04:00
James Shubin
3a8fd215de engine: resources: file: Add Copy method to file res
This lets us implement the CopyableRes interface.
2019-09-11 03:16:57 -04:00
James Shubin
87572e8922 test: Catch capitalized error messages in tests 2019-09-06 03:28:49 -04:00
James Shubin
1161872324 etcd: fs: Errors should start with lower case 2019-07-26 04:20:04 -04:00
Felix Frank
191a2495a5 engine: resources: mount: Fix the dbus call for reloading systemd
The Reload method cannot just be invoked on the administrative DBus
object. Just like the method for reloading specific units, it needs
to be invoked on the proper DBus service, addressing the proper object
and using the right interface.

Added an additional constant for the systemd DBus service. Even though
it shares the same value as the interface base name, this is
happenstance and it's technically incorrect to open a connection to an
interface name. The connection needs a service name.

Fixes #509
2019-06-04 23:44:57 +02:00
James Shubin
7a756cacb9 engine: graph: Add a mutex around waits map access
If you ran some extremely absurd code, it turns out you can cause a
race. This was found by roiedelapluie experimenting! In this case, it
would panic with: fatal error: concurrent map read and map write. This
patch adds the mutex to avoid this rare race.
2019-05-14 10:53:36 -04:00
Jan Martens
3c1da423fa engine: resources: nspawn: Trim possible systemd version suffix 2019-05-14 16:00:26 +02:00
Jonathan Gold
8f1f5d33fd engine: resources: mount: Restart remote-fs target 2019-04-23 16:24:49 -04:00
James Shubin
a5842a41b2 etcd: Rewrite embed etcd implementation
This is a giant cleanup of the etcd code. The earlier version was
written when I was less experienced with golang.

This is still not perfect, and does contain some races, but at least
it's a decent base to start from. The automatic elastic clustering
should be considered an experimental feature. If you need a more
battle-tested cluster, then you should manage etcd manually and point
mgmt at your existing cluster.
2019-04-11 21:43:48 -04:00
James Shubin
e4d874cc69 engine: resources: Remove named return params
Named return params aren't a favourite feature of mine, and they're
rarely used in the resource writing. They keep popping up because I once
used them and we've been copying and pasting code ever since. Get rid of
them all to help prevent the unnecessary spread.
2019-03-24 15:30:02 -04:00
James Shubin
07f542b4d7 legal: Happy 2019 everyone...
Done with:

ack '2018+' -l | xargs sed -i -e 's/2018+/2019+/g'

Checked manually with:

git add -p

Hello to future James from 2020, and Happy Hacking!
2019-03-24 15:08:50 -04:00
James Shubin
ae56261961 engine: util, resources: virt: Clean up virt resource
Do some cleanups which were long overdue.
2019-03-15 15:24:40 -04:00
James Shubin
753d1104ef util: Port all multierr code to new errwrap package
This cleans things up and simplifies a lot of the code. Also it's easier
to just import one error package when needed.
2019-03-12 16:51:37 -04:00
James Shubin
880652f5d4 util: Port all code to new errwrap package
This should keep things more uniform.
2019-03-12 16:49:01 -04:00
James Shubin
54c81d6bb2 engine, pgp: Fixup incorrect error usage
Small fixups found by the next commit.
2019-03-12 16:49:01 -04:00
James Shubin
5bbb474db6 engine: resources: Clean up KV resource 2019-03-09 18:48:26 -05:00
James Shubin
de43569fa2 engine, lang: Improve send/recv significantly
Part of this was rotten, and not fully functional. This fixes the rot,
adds some tests, and improves the type checking that occurs when sending
and receiving values. In addition, a significant portion of this happens
at compile time.

There is still more work to be done here, but this should get us a good
chunk of the way for now.
2019-03-09 17:37:58 -05:00
James Shubin
4c104d55cb engine: util: Add a new utility function for send/recv
This new utility function makes verifying send/recv struct comparisons
consistent. Unfortunately it doesn't yet support coercing from *string
to string or from string to *string.
2019-03-09 16:03:30 -05:00
James Shubin
8a8215fabe engine: util: Improve StructTagToFieldName and add tests
This improves this function to make it more generic.
2019-03-09 16:02:33 -05:00
James Shubin
4badeafb98 engine: resources: Add missing struct tags
These are needed for send/recv.
2019-03-09 16:01:25 -05:00
James Shubin
7cb79bec49 engine: resources: Replace the cached values with a live calculation
This replaces the static obj.path and obj.isDir with live variants. I
don't know why I ever cared about caching these before, and if we ever
care we can memoize properly in the future.

This caused a small bug to actually be masked in the gob code. It is now
fixed in the previous commit.
2019-03-06 10:08:14 -05:00
James Shubin
8da0da02d9 engine: traits: Make encoded fields public
The fields that we might want to store with encoding/gob or any other
encoding package, need to be public. We currently don't use any of these
at the moment, but we might in the future.
2019-03-06 10:08:14 -05:00
James Shubin
efef260764 engine: resources: Improve file Cmp function 2019-03-06 07:09:39 -05:00
James Shubin
a56991d081 engine: resources: Remove possible panic from within file res
Not sure how I let this in, but we should never do this. Hopefully the
Validate should catch this issue in advance, and if not, at least we'll
only error.
2019-03-06 07:09:39 -05:00
James Shubin
f0196540ab resources: file: Make some small cleanups to file res
This does some small cleaning for consistency, since I haven't reviewed
this code in a long while.
2019-03-06 07:09:39 -05:00
James Shubin
426b15313e engine: resources: Fix missing file when specified without content
If the file res was defined with state => "exists" but no content
specified, it was not created. This patch fixes this bug and adds a test
and an example.
2019-03-06 07:09:39 -05:00
James Shubin
4f4091a9bd engine: resources: Improve test case readability 2019-03-04 10:26:51 -05:00
James Shubin
6b803656b2 engine: resources: Improve exec resource
The exec resource was an early addition to the project, and it was due
for some fixes and integration into our automated tests. This patch
fixes a number of issues, and makes it ready for more general use.
2019-03-04 10:16:21 -05:00
Nicolas Charles
2d45f95501 engine: resources: print: Add RefreshOnly option
Add option RefreshOnly (default to false) on print ressource, to print only when
notified by other resource. When a print is RefreshOnly, it can't be grouped anymore.
2019-02-27 15:31:45 +01:00
James Shubin
ae7ebeedd1 engine: resources: Add CheckApply event detection to resource tests
This adds the ability to wait with a timeout for CheckApply happenings
in a resource. This helps avoid unnecessary long sleeping and timing
guesses. This also adds a cleanup function to run at the end.
2019-02-24 14:10:01 -05:00
James Shubin
652b657809 resources: exec: Avoid possible deadlock race
Some of the early code I wrote probably wouldn't pass my own reviews
today. Here's one example of a rare deadlock that could sometimes occur
when a Process/CheckApply caused a shutdown, but the bufio tried to send
on a channel that nobody was going to read any more. Now we properly
unblock that send with a context.
2019-02-24 12:28:59 -05:00
James Shubin
aa165b5e17 engine: Add the retry loop around Process
This adds back the retry loop around Process. This is done as a
separate commit so you can more easily see the logic of the retry magic
This commit is similar but different to the earlier commit adding retry
around Watch.
2019-02-24 12:28:59 -05:00
James Shubin
f06e87377c engine: Add limit delay before Process can run
This adds back the limit delay around Process.
2019-02-24 12:28:59 -05:00
James Shubin
4c3bf9fc7a engine: Add the retry loop around Watch
This adds back the retry loop around Watch. This is done as a separate
commit so you can more easily see the logic of the retry magic.
2019-02-24 12:28:59 -05:00
James Shubin
253ed78cc6 engine: Rewrite the core algorithm
The engine core had some unfortunate bugs that were the result of some
early design errors when I wasn't as familiar with channels. I've
finally rewritten most of the bad parts, and I think it's much more
logical and stable now.

This also simplifies the resource API, since more of the work is done
completely in the engine, and hidden from view.

Lastly, this adds a few new metaparameters and associated code.

There are still some open problems left to solve, but hopefully this
brings us one step closer.
2019-02-24 12:28:59 -05:00
James Shubin
4860d833c7 converger: Rewrite the converger module
I found a deadlock in the converger code, and I realized the code was
sufficiently bad that it needed a good clean up.
2019-02-24 12:28:59 -05:00
James Shubin
4c8086977a engine: resources: file: Update the format string
The %s in the format string is not technically correct here.
2019-02-08 12:38:10 -05:00
James Shubin
b1f088e5fa engine: resources: Add a test running for testing individual resources
This adds a simulated engine that can run and test single resources. It
can't test all aspects and features that the engine supports, but is
probably pretty decent for testing the actual CheckApply and Watch
semantics. Be warned that it actually applies changes on your machine,
so please don't write tests that make undesirable changes.
2019-02-08 12:36:37 -05:00
Johan Bloemberg
749038c76d misc: Make build on macOS work 2019-02-08 00:14:17 +01:00
James Shubin
4a3e2c3611 engine: nspawn: Add an nspawn example with an improved exec
This adds the cwd fields to exec, better error messages to svc (which is
nested in nspawn) and a fancier nspawn example!
2019-02-01 09:44:55 -05:00
James Shubin
35c26f9ee5 engine: resources: virt: Clean up virt resource for lang 2019-02-01 07:58:38 -05:00
James Shubin
ce8c8c8eea engine: resources: Fix a small typo in error message 2019-02-01 03:49:08 -05:00