Commit Graph

253 Commits

Author SHA1 Message Date
James Shubin
7cb79bec49 engine: resources: Replace the cached values with a live calculation
This replaces the static obj.path and obj.isDir with live variants. I
don't know why I ever cared about caching these before, and if we ever
care we can memoize properly in the future.

This caused a small bug to actually be masked in the gob code. It is now
fixed in the previous commit.
2019-03-06 10:08:14 -05:00
James Shubin
8da0da02d9 engine: traits: Make encoded fields public
The fields that we might want to store with encoding/gob or any other
encoding package, need to be public. We currently don't use any of these
at the moment, but we might in the future.
2019-03-06 10:08:14 -05:00
James Shubin
efef260764 engine: resources: Improve file Cmp function 2019-03-06 07:09:39 -05:00
James Shubin
a56991d081 engine: resources: Remove possible panic from within file res
Not sure how I let this in, but we should never do this. Hopefully the
Validate should catch this issue in advance, and if not, at least we'll
only error.
2019-03-06 07:09:39 -05:00
James Shubin
f0196540ab resources: file: Make some small cleanups to file res
This does some small cleaning for consistency, since I haven't reviewed
this code in a long while.
2019-03-06 07:09:39 -05:00
James Shubin
426b15313e engine: resources: Fix missing file when specified without content
If the file res was defined with state => "exists" but no content
specified, it was not created. This patch fixes this bug and adds a test
and an example.
2019-03-06 07:09:39 -05:00
James Shubin
4f4091a9bd engine: resources: Improve test case readability 2019-03-04 10:26:51 -05:00
James Shubin
6b803656b2 engine: resources: Improve exec resource
The exec resource was an early addition to the project, and it was due
for some fixes and integration into our automated tests. This patch
fixes a number of issues, and makes it ready for more general use.
2019-03-04 10:16:21 -05:00
Nicolas Charles
2d45f95501 engine: resources: print: Add RefreshOnly option
Add option RefreshOnly (default to false) on print ressource, to print only when
notified by other resource. When a print is RefreshOnly, it can't be grouped anymore.
2019-02-27 15:31:45 +01:00
James Shubin
ae7ebeedd1 engine: resources: Add CheckApply event detection to resource tests
This adds the ability to wait with a timeout for CheckApply happenings
in a resource. This helps avoid unnecessary long sleeping and timing
guesses. This also adds a cleanup function to run at the end.
2019-02-24 14:10:01 -05:00
James Shubin
652b657809 resources: exec: Avoid possible deadlock race
Some of the early code I wrote probably wouldn't pass my own reviews
today. Here's one example of a rare deadlock that could sometimes occur
when a Process/CheckApply caused a shutdown, but the bufio tried to send
on a channel that nobody was going to read any more. Now we properly
unblock that send with a context.
2019-02-24 12:28:59 -05:00
James Shubin
253ed78cc6 engine: Rewrite the core algorithm
The engine core had some unfortunate bugs that were the result of some
early design errors when I wasn't as familiar with channels. I've
finally rewritten most of the bad parts, and I think it's much more
logical and stable now.

This also simplifies the resource API, since more of the work is done
completely in the engine, and hidden from view.

Lastly, this adds a few new metaparameters and associated code.

There are still some open problems left to solve, but hopefully this
brings us one step closer.
2019-02-24 12:28:59 -05:00
James Shubin
4c8086977a engine: resources: file: Update the format string
The %s in the format string is not technically correct here.
2019-02-08 12:38:10 -05:00
James Shubin
b1f088e5fa engine: resources: Add a test running for testing individual resources
This adds a simulated engine that can run and test single resources. It
can't test all aspects and features that the engine supports, but is
probably pretty decent for testing the actual CheckApply and Watch
semantics. Be warned that it actually applies changes on your machine,
so please don't write tests that make undesirable changes.
2019-02-08 12:36:37 -05:00
Johan Bloemberg
749038c76d misc: Make build on macOS work 2019-02-08 00:14:17 +01:00
James Shubin
4a3e2c3611 engine: nspawn: Add an nspawn example with an improved exec
This adds the cwd fields to exec, better error messages to svc (which is
nested in nspawn) and a fancier nspawn example!
2019-02-01 09:44:55 -05:00
James Shubin
35c26f9ee5 engine: resources: virt: Clean up virt resource for lang 2019-02-01 07:58:38 -05:00
James Shubin
ce8c8c8eea engine: resources: Fix a small typo in error message 2019-02-01 03:49:08 -05:00
James Shubin
fc48fda7e5 engine: resources: Fix a possible panic on closed channel
I don't know how often it happens, but we should catch it.
2019-02-01 03:48:24 -05:00
Kevin Kuehler
5d0efce278 engine: lang: util: Kill race in socketset
After some investigation, it appears that SocketSet.Shutdown() and
SocketSet.Close() are not synchronous operations. The sendto system call
called in SocketSet.Shutdown() is not a blocking send. That means there
is a race in which SocketSet.Shutdown() sends a message to a file
descriptor to unblock select, while SocketSet.Close() will close the
file descriptor that the message is being sent to. If SocketSet.Close()
wins the race, select is listening on a dead file descriptor and will
hang indefinitely.

This is fixed in the current master by putting SocketSet.Close() inside
of the goroutine in which data from the socket is being received. It
relies on SocketSet.Shutdown() being called to terminate the goroutine.
While this works most of the time, there is a race here. All the
goroutines can also be terminated by a closeChan. If the goroutine
receives an event (thus unblocking select) and then closeChan is
triggered, both SocketSet.Shutdown() and SocketSet.Close() race, leading
to undefined behavior.

This patch ensures the ordering of the two function calls by pulling
them both out of the goroutine and separating them with a WaitGroup.

Co-authored-by: James Shubin <james@shubin.ca>
2019-01-22 20:59:17 -08:00
Kevin Kuehler
8697f8f91f util: Libify socketset
Add the ReceiveBytes, ReceiveNetlinkMessage, and ReceiveUEvent methods.
This is because not everything passed through a netlink socket cannot
reliably be parsed using the ParseNetLinkMessage function.

With the ReceiveUEvent method, we add support for "uevent" kernel
events, which updates us about the state of devices currently on the
system. To make using this method easier, we add a UEvent struct, that
has the action (what event), Devpath (where the device lives in /proc or
/sysfs), and Subsystem (what subsystem this event belows to).
2019-01-22 20:59:16 -08:00
Kevin Kuehler
06c67685f1 util: Move socketset from net resource to util
Prepare the socketset api to be used outside of the scope the net
resource.
2019-01-22 20:59:11 -08:00
James Shubin
dc2e7de9e5 engine: resources: pkg: Clarify that correct state is newest
I accidentally typed "latest" which got me confused why everything was
broken. Surprised it didn't error earlier anyways.
2019-01-21 04:28:34 -05:00
Vincent Membré
7d6080d13f engine: resources: exec: Use WatchShell in Exec resource when needed instead of Shell 2019-01-03 10:28:22 +01:00
James Shubin
e3eefeb3fe engine: resources: pkg: Implement the CompatibleRes interface
This signals to an interested consumer that two or more compatible
resources can be merged safely. This is so that we can avoid the
"duplicate resource" design problem that puppet had.

To test this, you can run:

./mgmt run --tmp-prefix lang --lang 'pkg "cowsay" { state => "installed", } pkg "cowsay" { state => "newest", }'

which should work.
2018-12-29 02:54:55 -05:00
James Shubin
9ff21b68e4 engine: resources: pkg: Simplify state check
Refactor this code.
2018-12-28 20:33:51 -05:00
James Shubin
a69a7009f8 engine: resources: pkg: Replace state strings with constants
This helps avoid typos, and gives us something we can export in the
future.
2018-12-28 20:32:23 -05:00
James Shubin
d413fac4cb engine: resources: pkg: Remove old Compare method
This was legacy code. Get rid of it.
2018-12-28 20:06:00 -05:00
James Shubin
246ecd8607 engine: resources: cron: Fix typo in error message 2018-12-28 20:00:14 -05:00
James Shubin
b1eb6711b7 engine: resources: Work around a subtle embedded res bug
This is a subtle issue that was found that caused a panic. This should
solve things for now, but it would be wise to build embedded or
composite resources sparingly until we we're certain this would work the
way we wanted for all scenarios.
2018-12-16 16:07:42 -05:00
Jonathan Gold
da0ffa5e56 engine: resources: cron: Add auto edges from SvcRes 2018-12-16 15:12:58 -05:00
James Shubin
e14b14b88c engine: resources: svc: Add symmetric closing
This improves some of the closing in the svc resource. This still needs
lots of improvements, and it's sort of terrible because it was some very
early code written.
2018-12-16 08:27:26 -05:00
James Shubin
d5bfb7257e engine: resources: file: Require paths to be absolute
This is a requirement of our file resource, so we should validate this
and clearly express it in the documentation.
2018-12-16 07:24:07 -05:00
Jonathan Gold
dbf0c84f0b engine: resources: cron: Add support for user session timers 2018-12-15 10:47:35 -05:00
Jonathan Gold
27df3ae876 engine: resources: cron: Add a systemd-timer resource 2018-12-15 10:47:35 -05:00
Jonathan Gold
28f343ac50 engine: resources: svc: Use dbus session bus for user session svc
This patch adds a util function, SessionBusUsable, that makes and returns
a new usable dbus session bus. If the svc bool session is true, the resource
will use a bus created with that function.
2018-12-14 00:16:21 -05:00
Jonathan Gold
4297a39d03 engine: resources: group: Make group edgeable
This adds the edgeable trait to the group resource and adds an
AutoEdges method which returns nil, nil. These changes are necessary
to allow UserRes to make autoedges to GroupRes.
2018-12-13 23:01:42 -05:00
Jonathan Gold
b902e2d30b engine: resources: file: Fix bug preventing use of source field
This patch fixes a previously undiscovered bug which prevented
the use of the source field in the file resource. CheckApply was
returning early if obj.Content was nil. It is also necessary to
check that obj.Source is empty before returning, otherwise
syncCheckApply never runs.
2018-12-10 22:08:59 -05:00
Jonathan Gold
d2bab32b0e engine: resources: packagekit: Fix dbus addmatch rule
I broke packagekit with commit 299080f5 due to a missing equals sign
in the DBus AddMatch rule. This commit adds the necessary equals sign.
2018-12-09 11:12:48 -05:00
Jonathan Gold
9b5c4c50e7 engine: resources: net: Allow addr without gateway
In some scenarios it is desirable to set the addrs and gateway
independently, i.e. if a default gateway is already set on
the machine. This patch removes the requirement to set them
together.
2018-12-04 17:12:57 -05:00
Jonathan Gold
7c817802a8 engine: resources: net: test: Add some go tests
This patch adds go tests for NetRes.unitFileContents(), socketSet.fdSet(),
and socketSet.nfd(), in the net resource.
2018-12-03 23:42:42 -05:00
James Shubin
cc02e96a13 engine: resources: Add nodocker build tag
Make it easy to disable building docker which is enormous.
2018-11-29 08:22:05 -05:00
Jonathan Gold
51ec91dd16 engine: resources: docker: Add a docker container resource 2018-11-29 08:14:07 -05:00
James Shubin
a049af6262 engine: resources: print: Add missing Recvable trait
We we're receiving values, but we forgot to list the trait. This caused
an intentional engine panic, but is easily fixed :)
2018-05-22 19:32:40 -04:00
James Shubin
22c0ff3cf5 test: Improve golang tests with root and disabling cache
This allows golang tests to be marked as root or !root using build tags.
The matching tests are then run as expected using our test runner.

This also disables test caching which is unfriendly to repeated test
running and is an absurd golang default to add.

Lastly this hooks up the testing verbose flag to tests that accept a
debug variable.

These tests aren't enabled on travis yet because of how it installs
golang.
2018-05-09 13:44:01 -04:00
James Shubin
3ced981d28 engine: test: Pass in the go test verbose flag
This hooks up our debug variable to the go test verbose flag.
2018-05-09 12:11:35 -04:00
Jonathan Gold
299080f590 engine: DBus cleanup 2018-05-07 15:57:17 -04:00
James Shubin
a407771eaf test: Catch naked returns and check for canonically named imports
This catches scenarios where we forgot to prefix the error with return.
One of our contributors occasionally made this typo, and since core go
vet didn't (surprisingly) catch it, we should add a test!

It also adds a simple check for import naming aliases. Expanding this
test to add other cases and check for differently named values might
make sense.
2018-05-06 15:18:46 -04:00
Jonathan Gold
d26a6de759 engine: resources: mount: Add a mount resource 2018-05-04 15:53:05 -04:00
Jonathan Gold
9baad56197 util: Move dbus AddMatch const to util package 2018-05-04 15:46:14 -04:00