Commit Graph

1157 Commits

Author SHA1 Message Date
Toshaan Bharvani
88fcda2c99 lang: funcs: Added an uptime function
Signed-off-by: Toshaan Bharvani <toshaan@vantosh.com>
2019-02-24 12:20:58 -05:00
James Shubin
00db953c9f lang: funcs: funcgen: Clean up some small details
Some small changes were needed, here they are. Unfortunately this only
supports the `string` type at the moment.
2019-02-21 13:06:29 -05:00
Julien Pivotto
a0df4829a8 lang: Add more string functions, autogenerated
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2019-02-21 17:50:06 +01:00
James Shubin
b0e1f12c22 test: Add expanders when running in travis
Hopefully this makes things more readable.
2019-02-20 09:35:31 -05:00
James Shubin
ee56155ec4 test: Split travis tests into three blocks
Our tests were taking near 50 minutes which kills them. This also makes
it easier to spot small issues faster.
2019-02-20 09:35:02 -05:00
Jeff Waugh
16d7c6a933 build: Fix macOS build
Add pkg-config to fix builds with augeas and libvirt on macOS.
2019-02-14 23:06:18 +11:00
Johan Bloemberg
f7a06c1da9 etcd: Connection options (socket file, ipv6)
- Allow unix domain socket to be used as client url
- Using ::1 as clienturl should not create default local ipv4 listener
- Add shell tests
2019-02-13 18:55:20 +01:00
James Shubin
4c8086977a engine: resources: file: Update the format string
The %s in the format string is not technically correct here.
2019-02-08 12:38:10 -05:00
James Shubin
b1f088e5fa engine: resources: Add a test running for testing individual resources
This adds a simulated engine that can run and test single resources. It
can't test all aspects and features that the engine supports, but is
probably pretty decent for testing the actual CheckApply and Watch
semantics. Be warned that it actually applies changes on your machine,
so please don't write tests that make undesirable changes.
2019-02-08 12:36:37 -05:00
James Shubin
1247c789aa lang: Remove unnecessary log package 2019-02-08 10:23:44 -05:00
Johan Bloemberg
749038c76d misc: Make build on macOS work 2019-02-08 00:14:17 +01:00
Johan Bloemberg
0a052494c4 misc: Add goimports dep 2019-02-08 00:14:17 +01:00
James Shubin
90fa83a5cf lang: funcs: core: Move world API functions
Some of the core functions interact with the remote "world" API. Move
them all into the same package.
2019-02-07 12:32:32 -05:00
James Shubin
4eaff892c1 lang: funcs: core: Rename core module files
More cleanup...
2019-02-07 12:19:59 -05:00
James Shubin
f368f75209 lang: funcs: core: Drop unnecessary core prefix from imports
This unbreaks the mcl bindata code. Of course we could change the parser
to allow this prefix, but this is cleaner. The packages still have a
core prefix, which it seems we could also remove, but this isn't
particularly important for anything.
2019-02-07 09:33:20 -05:00
Lander Van den Bulcke
04048b13ed lang: funcs: Add strings.split function
Signed-off-by: Lander Van den Bulcke <landervdb@inuits.eu>
2019-02-07 10:55:39 +01:00
Lander Van den Bulcke
5acc33c751 lang: funcs: Add tests for sqrt function
Signed-off-by: Lander Van den Bulcke <landervdb@inuits.eu>
2019-02-06 17:11:42 +01:00
James Shubin
b449be89a7 examples: Add uncommited nspawn example 2019-02-06 08:57:11 -05:00
Lander Van den Bulcke
dac019290d lang: funcs: Add sqrt function
Signed-off-by: Lander Van den Bulcke <landervdb@inuits.eu>
2019-02-06 14:32:13 +01:00
Julien Pivotto
bdc424e39d lang: Add to_lower and to_upper functions
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2019-02-06 14:24:15 +01:00
Lander Van den Bulcke
10193a2796 make: Use gem --no-document instead of deprecated flags
Signed-off-by: Lander Van den Bulcke <landervdb@inuits.eu>
2019-02-06 12:02:10 +01:00
Julien Pivotto
2c9a12e941 docker: Update FROM to go:1.11
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2019-02-06 10:48:24 +01:00
Felix Frank
8ba6c40f0c langpuppet: Fix Cli method invocations for wrapped GAPIs
Since the langpuppet GAPI creates fresh new CliContext objects,
it has to make sure to provide the original parent context, because
the child GAPIs expect to be able to access its data.
2019-02-05 16:34:55 +01:00
James Shubin
bbfeb49cdf examples: Add more examples and clean up some 2019-02-04 05:03:37 -05:00
James Shubin
f61e1cb36d examples: Add missing mcl files
I forgot to add these, sorry.
2019-02-03 09:58:04 -05:00
James Shubin
4a3e2c3611 engine: nspawn: Add an nspawn example with an improved exec
This adds the cwd fields to exec, better error messages to svc (which is
nested in nspawn) and a fancier nspawn example!
2019-02-01 09:44:55 -05:00
James Shubin
81faec508c integration: Avoid duplicate events from recwatch 2019-02-01 07:58:38 -05:00
James Shubin
9966ca2e85 examples: Improve dynamic cpus virt example 2019-02-01 07:58:38 -05:00
James Shubin
35c26f9ee5 engine: resources: virt: Clean up virt resource for lang 2019-02-01 07:58:38 -05:00
James Shubin
b5e29771ab lang: funcs: Add a trim space function to the new strings module 2019-02-01 07:00:05 -05:00
James Shubin
f5f09d3640 lang: funcs: Add str2int example function
We might want to move this into a real module eventually.
2019-02-01 06:59:07 -05:00
James Shubin
5a531b7948 lang: funcs: Add a new readfile function
This adds a new function that reads files from the local host.
2019-02-01 05:20:22 -05:00
James Shubin
f716a3a73b lang: funcs: Rename template functions to remove periods
Due to a limitation in the template library, we need to rename some
functions. It's probably worth looking into modifying this library or
finding an alternate version.
2019-02-01 03:58:02 -05:00
James Shubin
ce8c8c8eea engine: resources: Fix a small typo in error message 2019-02-01 03:49:08 -05:00
James Shubin
fc48fda7e5 engine: resources: Fix a possible panic on closed channel
I don't know how often it happens, but we should catch it.
2019-02-01 03:48:24 -05:00
James Shubin
78936c5ce8 examples: lang: Update examples to fix imports and port from yaml
Some small fixes that are useful for demos!
2019-02-01 03:47:18 -05:00
Kevin Kuehler
5d0efce278 engine: lang: util: Kill race in socketset
After some investigation, it appears that SocketSet.Shutdown() and
SocketSet.Close() are not synchronous operations. The sendto system call
called in SocketSet.Shutdown() is not a blocking send. That means there
is a race in which SocketSet.Shutdown() sends a message to a file
descriptor to unblock select, while SocketSet.Close() will close the
file descriptor that the message is being sent to. If SocketSet.Close()
wins the race, select is listening on a dead file descriptor and will
hang indefinitely.

This is fixed in the current master by putting SocketSet.Close() inside
of the goroutine in which data from the socket is being received. It
relies on SocketSet.Shutdown() being called to terminate the goroutine.
While this works most of the time, there is a race here. All the
goroutines can also be terminated by a closeChan. If the goroutine
receives an event (thus unblocking select) and then closeChan is
triggered, both SocketSet.Shutdown() and SocketSet.Close() race, leading
to undefined behavior.

This patch ensures the ordering of the two function calls by pulling
them both out of the goroutine and separating them with a WaitGroup.

Co-authored-by: James Shubin <james@shubin.ca>
2019-01-22 20:59:17 -08:00
Kevin Kuehler
0c17a0b4f2 util: Add TestShutdown to socketset
Test to ensure that SocketSet is nonblocking and will close when
SocketSet.Shutdown() is called. Create a SocketSet that will never
receive any data and leave it running in a goroutine with a WaitGroup
for a second. If Shutdown is working correctly, the goroutine will be
terminated after the timer expires.
2019-01-22 20:59:17 -08:00
Kevin Kuehler
3f396a7c52 lang: funcs: Add cpucount fact
Adds a CPU count fact, that can be used to determine how many CPUs are
presently on the machine and ready for use (online). We get this by
reading from a netlink socket to the kernel, and the kernel sends us
uevents when CPUs are added, removed, and brought online or offline.
Whenever one of these events are received, we look in sysfs to update
the fact's Stream with the number of online CPUs.
2019-01-22 20:59:16 -08:00
Kevin Kuehler
8697f8f91f util: Libify socketset
Add the ReceiveBytes, ReceiveNetlinkMessage, and ReceiveUEvent methods.
This is because not everything passed through a netlink socket cannot
reliably be parsed using the ParseNetLinkMessage function.

With the ReceiveUEvent method, we add support for "uevent" kernel
events, which updates us about the state of devices currently on the
system. To make using this method easier, we add a UEvent struct, that
has the action (what event), Devpath (where the device lives in /proc or
/sysfs), and Subsystem (what subsystem this event belows to).
2019-01-22 20:59:16 -08:00
Kevin Kuehler
06c67685f1 util: Move socketset from net resource to util
Prepare the socketset api to be used outside of the scope the net
resource.
2019-01-22 20:59:11 -08:00
James Shubin
dc2e7de9e5 engine: resources: pkg: Clarify that correct state is newest
I accidentally typed "latest" which got me confused why everything was
broken. Surprised it didn't error earlier anyways.
2019-01-21 04:28:34 -05:00
James Shubin
db1dbe7a27 lang: Edges should allow lists of strings
This continues the earlier patch that allowed resource names to be lists
of strings so that edges can now allow the same. This also includes a
new fancy test!
2019-01-20 17:27:40 -05:00
James Shubin
d6bbb94be5 lang: test: Add a new giant test infra for matching static output
This greatly expands our test infra to allow us to drop in mcl tests and
look at their resource graph output. The only downside is that this only
runs the function engine once, so if the function graph would be
constantly changing over time, then this is not a good fit here.
2019-01-20 17:27:40 -05:00
James Shubin
e3b4c0aee3 test: Fix a small copy pasta typo 2019-01-20 17:27:40 -05:00
James Shubin
a1fbe152bb lang: unification: Fix up small typos in example code 2019-01-20 04:22:05 -05:00
James Shubin
9d28ff9b23 lang: unification: Catch unification error on typed var expr
This was similar to the typed if expr error.
2019-01-20 04:19:39 -05:00
James Shubin
43f0ddd25d lang: unification: Catch unification error on typed if expr
I found a case where we had two missing unification rules. Now fixed in
the previous commits, and including this test to show I'm responsible.
I've added the same test in two locations for redundancy and as an
example.
2019-01-20 04:19:39 -05:00
James Shubin
7a28b00d75 lang: If expression was missing two invariants
I forgot to ensure that the type of the final expression matched the
type of each of the branches. It's rare, but possible for this to occur.
Luckily, this never would have caused a panic, because the func engine
would have caught the issue anyways, but it's still better we catch it
here first!
2019-01-20 04:02:54 -05:00
James Shubin
32e29862f2 lang: Check that set type matches actual expression
I forgot to include these two invariants which are occasionally
necessary, although in most cases they're necessary to prevent incorrect
code from getting past unification. In any case, they would have been
caught by the engine.
2019-01-20 04:02:54 -05:00