Commit Graph

1192 Commits

Author SHA1 Message Date
James Shubin
4badeafb98 engine: resources: Add missing struct tags
These are needed for send/recv.
2019-03-09 16:01:25 -05:00
James Shubin
7cb79bec49 engine: resources: Replace the cached values with a live calculation
This replaces the static obj.path and obj.isDir with live variants. I
don't know why I ever cared about caching these before, and if we ever
care we can memoize properly in the future.

This caused a small bug to actually be masked in the gob code. It is now
fixed in the previous commit.
2019-03-06 10:08:14 -05:00
James Shubin
8da0da02d9 engine: traits: Make encoded fields public
The fields that we might want to store with encoding/gob or any other
encoding package, need to be public. We currently don't use any of these
at the moment, but we might in the future.
2019-03-06 10:08:14 -05:00
James Shubin
efef260764 engine: resources: Improve file Cmp function 2019-03-06 07:09:39 -05:00
James Shubin
a56991d081 engine: resources: Remove possible panic from within file res
Not sure how I let this in, but we should never do this. Hopefully the
Validate should catch this issue in advance, and if not, at least we'll
only error.
2019-03-06 07:09:39 -05:00
James Shubin
f0196540ab resources: file: Make some small cleanups to file res
This does some small cleaning for consistency, since I haven't reviewed
this code in a long while.
2019-03-06 07:09:39 -05:00
James Shubin
426b15313e engine: resources: Fix missing file when specified without content
If the file res was defined with state => "exists" but no content
specified, it was not created. This patch fixes this bug and adds a test
and an example.
2019-03-06 07:09:39 -05:00
James Shubin
11fc55d679 lang: funcs: Add a new test for readfile and fix a small bug
This adds a new test for readfile. Interestingly, it actually caught a
small bug, which was also fixed with this commit. I think the bug was
actually always masked, because it only occurred on shutdown, and in
this case we often don't care about how the stream exited, but it's a
good example of how a test case focused on just one small aspect can be
important.

As an aside, this test case also would have caught the bug fixed in
94c40909cc and by reverting that patch it
indeed fails.
2019-03-06 07:09:39 -05:00
James Shubin
de1691665f lang: funcs: Add live function stream test infrastructure
This adds the ability to test that functions return the expected
streams, and to model this behaviour over time. This is done via a
"timeline" which runs an ordered list of actions that can both push new
values into the function stream, and wait and expect particular outputs.
Hopefully this will make our function implementations more robust!
2019-03-06 07:09:39 -05:00
James Shubin
b1f93b40ae lang: funcs: Add runner pure func execution
This adds a function runner that runs pure functions. It will hopefully
be useful for speculative execution of functions for compile time
determination of types.
2019-03-05 11:42:33 -05:00
James Shubin
5e58251026 test: Improve govet log newline check
We don't match for log.Fatalf but we shouldn't really be using that.
2019-03-05 11:42:33 -05:00
James Shubin
4f4091a9bd engine: resources: Improve test case readability 2019-03-04 10:26:51 -05:00
James Shubin
e9fb41fdc8 test: shell: Fix rare breakage in load test
For some reason the load is occasionally zero. This broke the regexp.
Let's see if this ever happens with the other digits.
2019-03-04 10:16:21 -05:00
James Shubin
6b803656b2 engine: resources: Improve exec resource
The exec resource was an early addition to the project, and it was due
for some fixes and integration into our automated tests. This patch
fixes a number of issues, and makes it ready for more general use.
2019-03-04 10:16:21 -05:00
James Shubin
829741e2ac lang: Print a clear message on module import containing unused stmt
If you run an import, you only include everything that's part of a
scope. This includes, variables, classes, and functions. Anything else
should cause a compile error. This cleans up the error by adding a
String() method to each Stmt in our AST.
2019-02-28 09:35:13 -05:00
James Shubin
94c40909cc lang: funcs: Avoid erroneous empty message in readfile
Readfile had a bug where it sent an empty string on startup. This has
ben fixed, and it now waits until the file contents are ready before
sending a string.
2019-02-28 08:56:10 -05:00
James Shubin
95dab16e6e lang: funcs: Allow the len function to determine str length 2019-02-28 08:54:11 -05:00
James Shubin
c049413b47 examples: lang: Add is_debian and is_redhat family example
This is just the beginning.
2019-02-28 08:53:25 -05:00
Nicolas Charles
2d45f95501 engine: resources: print: Add RefreshOnly option
Add option RefreshOnly (default to false) on print ressource, to print only when
notified by other resource. When a print is RefreshOnly, it can't be grouped anymore.
2019-02-27 15:31:45 +01:00
Nicolas Charles
3cfc76b635 lang: funcs: Added a function to detect Debian and RedHat like systems 2019-02-26 18:13:34 +01:00
James Shubin
d88874845c test: shell: Improve load test
This might have failed once in travis because of a short timeout.
Hopefully if this happens again, we'll now know why.
0.0.18
2019-02-24 14:10:01 -05:00
James Shubin
5e38c1c8fe examples: Remove old hcl examples
The hcl frontend was removed a while back. Might as well remove these
examples too.
2019-02-24 14:10:01 -05:00
James Shubin
ae7ebeedd1 engine: resources: Add CheckApply event detection to resource tests
This adds the ability to wait with a timeout for CheckApply happenings
in a resource. This helps avoid unnecessary long sleeping and timing
guesses. This also adds a cleanup function to run at the end.
2019-02-24 14:10:01 -05:00
James Shubin
652b657809 resources: exec: Avoid possible deadlock race
Some of the early code I wrote probably wouldn't pass my own reviews
today. Here's one example of a rare deadlock that could sometimes occur
when a Process/CheckApply caused a shutdown, but the bufio tried to send
on a channel that nobody was going to read any more. Now we properly
unblock that send with a context.
2019-02-24 12:28:59 -05:00
James Shubin
62a6e0da1d misc: Add two test helpers
Hopefully these make testing and debugging easier!
2019-02-24 12:28:59 -05:00
James Shubin
0d0d48d9f6 test: Shell tests should use unified timeout command 2019-02-24 12:28:59 -05:00
James Shubin
ab5957f1e9 make: Clean up the Makefiles so the output is more elegant
This avoids printing erroneous messages when nothing is actually
happening.
2019-02-24 12:28:59 -05:00
James Shubin
463ba23003 util: Improve the sync primitives. 2019-02-24 12:28:59 -05:00
James Shubin
ccad6e7e1a test: Enable and fix up some more tests
An unstable engine probably masked some of these issues.
2019-02-24 12:28:59 -05:00
James Shubin
aa165b5e17 engine: Add the retry loop around Process
This adds back the retry loop around Process. This is done as a
separate commit so you can more easily see the logic of the retry magic
This commit is similar but different to the earlier commit adding retry
around Watch.
2019-02-24 12:28:59 -05:00
James Shubin
f06e87377c engine: Add limit delay before Process can run
This adds back the limit delay around Process.
2019-02-24 12:28:59 -05:00
James Shubin
4c3bf9fc7a engine: Add the retry loop around Watch
This adds back the retry loop around Watch. This is done as a separate
commit so you can more easily see the logic of the retry magic.
2019-02-24 12:28:59 -05:00
James Shubin
253ed78cc6 engine: Rewrite the core algorithm
The engine core had some unfortunate bugs that were the result of some
early design errors when I wasn't as familiar with channels. I've
finally rewritten most of the bad parts, and I think it's much more
logical and stable now.

This also simplifies the resource API, since more of the work is done
completely in the engine, and hidden from view.

Lastly, this adds a few new metaparameters and associated code.

There are still some open problems left to solve, but hopefully this
brings us one step closer.
2019-02-24 12:28:59 -05:00
James Shubin
4860d833c7 converger: Rewrite the converger module
I found a deadlock in the converger code, and I realized the code was
sufficiently bad that it needed a good clean up.
2019-02-24 12:28:59 -05:00
James Shubin
450d5c1a59 util: Add an easy ACK sync primitive 2019-02-24 12:28:59 -05:00
Toshaan Bharvani
88fcda2c99 lang: funcs: Added an uptime function
Signed-off-by: Toshaan Bharvani <toshaan@vantosh.com>
2019-02-24 12:20:58 -05:00
James Shubin
00db953c9f lang: funcs: funcgen: Clean up some small details
Some small changes were needed, here they are. Unfortunately this only
supports the `string` type at the moment.
2019-02-21 13:06:29 -05:00
Julien Pivotto
a0df4829a8 lang: Add more string functions, autogenerated
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2019-02-21 17:50:06 +01:00
James Shubin
b0e1f12c22 test: Add expanders when running in travis
Hopefully this makes things more readable.
2019-02-20 09:35:31 -05:00
James Shubin
ee56155ec4 test: Split travis tests into three blocks
Our tests were taking near 50 minutes which kills them. This also makes
it easier to spot small issues faster.
2019-02-20 09:35:02 -05:00
Jeff Waugh
16d7c6a933 build: Fix macOS build
Add pkg-config to fix builds with augeas and libvirt on macOS.
2019-02-14 23:06:18 +11:00
Johan Bloemberg
f7a06c1da9 etcd: Connection options (socket file, ipv6)
- Allow unix domain socket to be used as client url
- Using ::1 as clienturl should not create default local ipv4 listener
- Add shell tests
2019-02-13 18:55:20 +01:00
James Shubin
4c8086977a engine: resources: file: Update the format string
The %s in the format string is not technically correct here.
2019-02-08 12:38:10 -05:00
James Shubin
b1f088e5fa engine: resources: Add a test running for testing individual resources
This adds a simulated engine that can run and test single resources. It
can't test all aspects and features that the engine supports, but is
probably pretty decent for testing the actual CheckApply and Watch
semantics. Be warned that it actually applies changes on your machine,
so please don't write tests that make undesirable changes.
2019-02-08 12:36:37 -05:00
James Shubin
1247c789aa lang: Remove unnecessary log package 2019-02-08 10:23:44 -05:00
Johan Bloemberg
749038c76d misc: Make build on macOS work 2019-02-08 00:14:17 +01:00
Johan Bloemberg
0a052494c4 misc: Add goimports dep 2019-02-08 00:14:17 +01:00
James Shubin
90fa83a5cf lang: funcs: core: Move world API functions
Some of the core functions interact with the remote "world" API. Move
them all into the same package.
2019-02-07 12:32:32 -05:00
James Shubin
4eaff892c1 lang: funcs: core: Rename core module files
More cleanup...
2019-02-07 12:19:59 -05:00
James Shubin
f368f75209 lang: funcs: core: Drop unnecessary core prefix from imports
This unbreaks the mcl bindata code. Of course we could change the parser
to allow this prefix, but this is cleaner. The packages still have a
core prefix, which it seems we could also remove, but this isn't
particularly important for anything.
2019-02-07 09:33:20 -05:00