Commit Graph

1157 Commits

Author SHA1 Message Date
James Shubin
880652f5d4 util: Port all code to new errwrap package
This should keep things more uniform.
2019-03-12 16:49:01 -04:00
James Shubin
54c81d6bb2 engine, pgp: Fixup incorrect error usage
Small fixups found by the next commit.
2019-03-12 16:49:01 -04:00
James Shubin
2bf43eae24 test: Improve test depth
Make sure we're catching everything, including new, deeper code.
2019-03-12 16:49:01 -04:00
James Shubin
58961d23bb test: Improve test depth
Make sure we're catching everything, including new, deeper code.
2019-03-12 15:50:30 -04:00
James Shubin
6044ade373 util: Add errwrap package
Simplify working with errors across our code base. Instead of constantly
importing the necessary error helpers, assemble them all into one
package and import and use that as needed.
2019-03-12 15:45:39 -04:00
James Shubin
da1c96c6fd examples: lang: Refresh examples
Removed two old examples which were no longer valid.
2019-03-09 18:54:50 -05:00
James Shubin
5bbb474db6 engine: resources: Clean up KV resource 2019-03-09 18:48:26 -05:00
James Shubin
a0c909914d lang: funcs: Don't allow interpolation in printf format string
We'd like to pre-compute the interpolation if we can, so that we can run
this code properly... For now, we can't so it's a compile time error...
Hopefully we can remove this restriction in the future. The problem is
the string must be a constant, or it would be possible to switch it from
"%d %s" to "%s %d %d" or anything that changes the type signature.
2019-03-09 18:06:18 -05:00
James Shubin
170e56b34a lang: Improve test case with more specific errors 2019-03-09 17:37:58 -05:00
James Shubin
de43569fa2 engine, lang: Improve send/recv significantly
Part of this was rotten, and not fully functional. This fixes the rot,
adds some tests, and improves the type checking that occurs when sending
and receiving values. In addition, a significant portion of this happens
at compile time.

There is still more work to be done here, but this should get us a good
chunk of the way for now.
2019-03-09 17:37:58 -05:00
James Shubin
aa6b701b77 lang: Improve the test case infra so it can detect different errors
Let the tests detect which specific error we want to fail on.
2019-03-09 17:14:45 -05:00
James Shubin
d69eb27557 lang: Small fixes about send/recv 2019-03-09 16:07:22 -05:00
James Shubin
0ca57d6a09 examples: lang: Add a basic file example 2019-03-09 16:05:12 -05:00
James Shubin
4c104d55cb engine: util: Add a new utility function for send/recv
This new utility function makes verifying send/recv struct comparisons
consistent. Unfortunately it doesn't yet support coercing from *string
to string or from string to *string.
2019-03-09 16:03:30 -05:00
James Shubin
8a8215fabe engine: util: Improve StructTagToFieldName and add tests
This improves this function to make it more generic.
2019-03-09 16:02:33 -05:00
James Shubin
4badeafb98 engine: resources: Add missing struct tags
These are needed for send/recv.
2019-03-09 16:01:25 -05:00
James Shubin
7cb79bec49 engine: resources: Replace the cached values with a live calculation
This replaces the static obj.path and obj.isDir with live variants. I
don't know why I ever cared about caching these before, and if we ever
care we can memoize properly in the future.

This caused a small bug to actually be masked in the gob code. It is now
fixed in the previous commit.
2019-03-06 10:08:14 -05:00
James Shubin
8da0da02d9 engine: traits: Make encoded fields public
The fields that we might want to store with encoding/gob or any other
encoding package, need to be public. We currently don't use any of these
at the moment, but we might in the future.
2019-03-06 10:08:14 -05:00
James Shubin
efef260764 engine: resources: Improve file Cmp function 2019-03-06 07:09:39 -05:00
James Shubin
a56991d081 engine: resources: Remove possible panic from within file res
Not sure how I let this in, but we should never do this. Hopefully the
Validate should catch this issue in advance, and if not, at least we'll
only error.
2019-03-06 07:09:39 -05:00
James Shubin
f0196540ab resources: file: Make some small cleanups to file res
This does some small cleaning for consistency, since I haven't reviewed
this code in a long while.
2019-03-06 07:09:39 -05:00
James Shubin
426b15313e engine: resources: Fix missing file when specified without content
If the file res was defined with state => "exists" but no content
specified, it was not created. This patch fixes this bug and adds a test
and an example.
2019-03-06 07:09:39 -05:00
James Shubin
11fc55d679 lang: funcs: Add a new test for readfile and fix a small bug
This adds a new test for readfile. Interestingly, it actually caught a
small bug, which was also fixed with this commit. I think the bug was
actually always masked, because it only occurred on shutdown, and in
this case we often don't care about how the stream exited, but it's a
good example of how a test case focused on just one small aspect can be
important.

As an aside, this test case also would have caught the bug fixed in
94c40909cc and by reverting that patch it
indeed fails.
2019-03-06 07:09:39 -05:00
James Shubin
de1691665f lang: funcs: Add live function stream test infrastructure
This adds the ability to test that functions return the expected
streams, and to model this behaviour over time. This is done via a
"timeline" which runs an ordered list of actions that can both push new
values into the function stream, and wait and expect particular outputs.
Hopefully this will make our function implementations more robust!
2019-03-06 07:09:39 -05:00
James Shubin
b1f93b40ae lang: funcs: Add runner pure func execution
This adds a function runner that runs pure functions. It will hopefully
be useful for speculative execution of functions for compile time
determination of types.
2019-03-05 11:42:33 -05:00
James Shubin
5e58251026 test: Improve govet log newline check
We don't match for log.Fatalf but we shouldn't really be using that.
2019-03-05 11:42:33 -05:00
James Shubin
4f4091a9bd engine: resources: Improve test case readability 2019-03-04 10:26:51 -05:00
James Shubin
e9fb41fdc8 test: shell: Fix rare breakage in load test
For some reason the load is occasionally zero. This broke the regexp.
Let's see if this ever happens with the other digits.
2019-03-04 10:16:21 -05:00
James Shubin
6b803656b2 engine: resources: Improve exec resource
The exec resource was an early addition to the project, and it was due
for some fixes and integration into our automated tests. This patch
fixes a number of issues, and makes it ready for more general use.
2019-03-04 10:16:21 -05:00
James Shubin
829741e2ac lang: Print a clear message on module import containing unused stmt
If you run an import, you only include everything that's part of a
scope. This includes, variables, classes, and functions. Anything else
should cause a compile error. This cleans up the error by adding a
String() method to each Stmt in our AST.
2019-02-28 09:35:13 -05:00
James Shubin
94c40909cc lang: funcs: Avoid erroneous empty message in readfile
Readfile had a bug where it sent an empty string on startup. This has
ben fixed, and it now waits until the file contents are ready before
sending a string.
2019-02-28 08:56:10 -05:00
James Shubin
95dab16e6e lang: funcs: Allow the len function to determine str length 2019-02-28 08:54:11 -05:00
James Shubin
c049413b47 examples: lang: Add is_debian and is_redhat family example
This is just the beginning.
2019-02-28 08:53:25 -05:00
Nicolas Charles
2d45f95501 engine: resources: print: Add RefreshOnly option
Add option RefreshOnly (default to false) on print ressource, to print only when
notified by other resource. When a print is RefreshOnly, it can't be grouped anymore.
2019-02-27 15:31:45 +01:00
Nicolas Charles
3cfc76b635 lang: funcs: Added a function to detect Debian and RedHat like systems 2019-02-26 18:13:34 +01:00
James Shubin
d88874845c test: shell: Improve load test
This might have failed once in travis because of a short timeout.
Hopefully if this happens again, we'll now know why.
0.0.18
2019-02-24 14:10:01 -05:00
James Shubin
5e38c1c8fe examples: Remove old hcl examples
The hcl frontend was removed a while back. Might as well remove these
examples too.
2019-02-24 14:10:01 -05:00
James Shubin
ae7ebeedd1 engine: resources: Add CheckApply event detection to resource tests
This adds the ability to wait with a timeout for CheckApply happenings
in a resource. This helps avoid unnecessary long sleeping and timing
guesses. This also adds a cleanup function to run at the end.
2019-02-24 14:10:01 -05:00
James Shubin
652b657809 resources: exec: Avoid possible deadlock race
Some of the early code I wrote probably wouldn't pass my own reviews
today. Here's one example of a rare deadlock that could sometimes occur
when a Process/CheckApply caused a shutdown, but the bufio tried to send
on a channel that nobody was going to read any more. Now we properly
unblock that send with a context.
2019-02-24 12:28:59 -05:00
James Shubin
62a6e0da1d misc: Add two test helpers
Hopefully these make testing and debugging easier!
2019-02-24 12:28:59 -05:00
James Shubin
0d0d48d9f6 test: Shell tests should use unified timeout command 2019-02-24 12:28:59 -05:00
James Shubin
ab5957f1e9 make: Clean up the Makefiles so the output is more elegant
This avoids printing erroneous messages when nothing is actually
happening.
2019-02-24 12:28:59 -05:00
James Shubin
463ba23003 util: Improve the sync primitives. 2019-02-24 12:28:59 -05:00
James Shubin
ccad6e7e1a test: Enable and fix up some more tests
An unstable engine probably masked some of these issues.
2019-02-24 12:28:59 -05:00
James Shubin
aa165b5e17 engine: Add the retry loop around Process
This adds back the retry loop around Process. This is done as a
separate commit so you can more easily see the logic of the retry magic
This commit is similar but different to the earlier commit adding retry
around Watch.
2019-02-24 12:28:59 -05:00
James Shubin
f06e87377c engine: Add limit delay before Process can run
This adds back the limit delay around Process.
2019-02-24 12:28:59 -05:00
James Shubin
4c3bf9fc7a engine: Add the retry loop around Watch
This adds back the retry loop around Watch. This is done as a separate
commit so you can more easily see the logic of the retry magic.
2019-02-24 12:28:59 -05:00
James Shubin
253ed78cc6 engine: Rewrite the core algorithm
The engine core had some unfortunate bugs that were the result of some
early design errors when I wasn't as familiar with channels. I've
finally rewritten most of the bad parts, and I think it's much more
logical and stable now.

This also simplifies the resource API, since more of the work is done
completely in the engine, and hidden from view.

Lastly, this adds a few new metaparameters and associated code.

There are still some open problems left to solve, but hopefully this
brings us one step closer.
2019-02-24 12:28:59 -05:00
James Shubin
4860d833c7 converger: Rewrite the converger module
I found a deadlock in the converger code, and I realized the code was
sufficiently bad that it needed a good clean up.
2019-02-24 12:28:59 -05:00
James Shubin
450d5c1a59 util: Add an easy ACK sync primitive 2019-02-24 12:28:59 -05:00