Commit Graph

1263 Commits

Author SHA1 Message Date
James Shubin
4109045fa4 github: Add new needinfo tag 2020-01-29 11:38:53 -05:00
James Shubin
90fd8023dd lang, engine: Add a facility for resources to export constants
Since we focus on safety, it would be nice to reduce the chance of any
runtime errors if we made a typo for a resource parameter. With this
patch, each resource can export constants into the global namespace so
that typos would cause a compile error.

Of course in the future if we had a more advanced type system, then we
could support precise types for each individual resource param, but in
an attempt to keep things simple, we'll leave that for another day. It
would add complexity too if we ever wanted to store a parameter
externally.

Lastly, we might consider adding "special case" parsing so that directly
specified fields would parse intelligently. For example, we could allow:

	file "/tmp/hello" {
		state => exists,	# magic sugar!
	}

This isn't supported for now, but if it works after all the other parser
changes have been made, it might be something to consider.
2020-01-29 11:16:04 -05:00
James Shubin
f67ad9c061 test: Add a check for too long or badly reflowed docstrings
This ensures that docstring comments are wrapped to 80 chars. ffrank
seemed to be making this mistake far too often, and it's a silly thing
to look for manually. As it turns out, I've made it too, as have many
others. Now we have a test that checks for most cases. There are still a
few stray cases that aren't checked automatically, but this can be
improved upon if someone is motivated to do so.

Before anyone complains about the 80 character limit: this only checks
docstring comments, not source code length or inline source code
comments. There's no excuse for having docstrings that are badly
reflowed or over 80 chars, particularly if you have an automated test.
2020-01-25 04:43:33 -05:00
James Shubin
525e2bafee misc: Let sigtee work on older golang versions 2020-01-25 04:43:33 -05:00
James Shubin
b65a9abf8e misc: Add two scripts to help debug things
This adds two new helper scripts that are good for debugging mgmt. Just
build and add `sigtee` to your ~/bin/ along with filter-golang-stack.py
and mgmt_debug.sh and use the later to call mgmt as you would normally.

For example, I might do:

$ mgmt_debug.sh ./mgmt run --tmp-prefix lang examples/lang/hello0.mcl

And if I kill it with ^\ then I'll get a filtered trace at the end in my
$PAGER (which is assumed to be `less`) and this should make my life
easier.

As a cool bonus, this means we use bash, python, and golang all
together!
2020-01-13 01:07:00 -05:00
James Shubin
fec94aa53a engine, lang: Fix simple test failures
Two bugs sneaked in while pushing old stuff.
2020-01-12 19:35:11 -05:00
James Shubin
3d4b345728 examples: lang: Improve reverse example
It's cool to show just the mode changes.
2020-01-12 17:42:20 -05:00
James Shubin
579975f08d engine: graph: Don't error when state file is missing
For some reason we get errors when we try to remove a non-existent state
file. There's a slight possibility that it could be a bug we're working
around, but it's not clear that this is the case, and I think it's
possible that a state file could have gotten nuked by the user somehow,
although this was occurring "naturally" when running reverse1.mcl so
let's keep that working for now.
2020-01-12 16:41:09 -05:00
James Shubin
3707b39fef engine: graph: Improve comments
Clarify that we're referring to cycles in the graph, since it needs to
be a DAG.
2020-01-12 16:39:32 -05:00
James Shubin
f07387225b engine: resources: Log more info about tftp errors
This helps for debugging this kind of issue:
https://github.com/pin/tftp/issues/41#issuecomment-570744056
2020-01-03 20:42:34 -05:00
James Shubin
2648fb1bb1 legal: Happy 2020 everyone...
Done with:

ack '2019+' -l | xargs sed -i -e 's/2019+/2020+/g'

Checked manually with:

git add -p

Hello to future James from 2021, and Happy Hacking!
2020-01-03 20:08:37 -05:00
James Shubin
d34715b4ba engine: resources: pippet: Cleanup and proper wrapping
Felix, please configure your editor to wrap at 80 chars and/or help us
write a test for this please =D
2020-01-03 01:20:00 -05:00
Felix Frank
63af50bf98 engine: resources: pippet: Initial implementation for new resource type
The pippet resource implements faster integration of Puppet resources
in mgmt at runtime, by piping synchronization commands to a Puppet
process that keeps running alongside mgmt. This avoids huge overhead
through launching a Puppet process for each operation on a resource
that is delegated to Puppet.
2020-01-03 01:19:37 -05:00
James Shubin
456550c1d4 engine: resources: docker: Make a few fixups
Here are a few fixups to the docker resources. All miscellaneous stuff,
nothing major.
2020-01-03 00:53:20 -05:00
Jonathan Gold
8174b88ec3 engine: resources: docker: Add sensible defaults 2020-01-03 00:30:01 -05:00
Jonathan Gold
3233973748 engine: resources: docker: Add AutoEdges between container and image 2020-01-03 00:30:01 -05:00
Jonathan Gold
bdfb1cf33e engine: resources: docker: Ensure image is specified for containers 2020-01-03 00:30:01 -05:00
Jonathan Gold
1c5fcd59e7 engine: resources: docker: Add a docker image resource 2020-01-03 00:30:01 -05:00
James Shubin
5cc960527e lang: funcs: Differentiate between empty and nil values
It would be good to differentiate between receiving an empty value or
not having received a value yet. This is similar to the previous commit.
2020-01-03 00:28:54 -05:00
James Shubin
762c53fb8d lang: funcs: Send empty values when appropriate
I seem to have forgotten to differentiate between the empty string and
no data because the zero value for the stored result was the empty
string. This turns it into a pointer so that we don't block the function
engine if a template or one of the other patched functions sends an
empty string as the first value.
2019-12-30 12:35:08 -05:00
James Shubin
ff20e67d07 lang: funcs: Improve template function
Template function should be able to be called with just one arg (no
input vars) and we should correctly use the known arg name and not the
string "a" or "b".
2019-12-30 11:21:43 -05:00
Francois Rompre-Lanctot
c0cea013d1 pgraph: Add test for SetValue function 2019-12-18 20:00:47 -05:00
James Shubin
5526bbba64 engine: resources: Add a tftp server and tftp file resource
This adds a tftp server and tftp file resource to help you run a small
pure golang tftp server embedded inside the mgmt resource model.
2019-12-17 03:41:45 -05:00
James Shubin
f0aa96ea8c etcd: Remove the capnslog stuff and switch to zap
Unfortunately, this doesn't give us a way to pass in our own logger
function, and afaict by reading the source, it's not possible because
the necessary methods are private. In any case, this is left as a future
exercise.
2019-12-17 03:40:44 -05:00
James Shubin
e73007c398 etcd: Bump to new 3.4.x version
This moves to the newest etcd release, and also updates the imports to
the new go.etcd.io path. I think this is a bit of a pain, but might as
well get it done.
2019-12-17 02:45:38 -05:00
Jonathan Gold
fdc459ec5b vagrant: Update Vagrantfile
This patch updates the Vagrantfile to Fedora 31, and updates the
install process to match the quick start guide.
2019-11-17 20:25:06 -05:00
Felix Frank
bdb523ece1 lang: funcs: funcgen: Suppress informational messages
Send non-error log messages to stdout rather than stderr. Any messages
outside the main function are expected to be purely informational. By
sending to stdout rather than stderr, they can be discarded during the
build.

Fixes #568
2019-11-17 20:21:52 -05:00
James Shubin
164a9479ad test: Add a new test to the commit message checking
Fix the missing "s" bug.
2019-11-17 20:21:52 -05:00
James Shubin
e18adc781f git: Add a simple gitignore helper
Run `touch 1 && chmod ugo-w 1` to prevent silly scripting bugs from
running without being caught.
2019-11-12 17:33:58 -05:00
Felix Frank
33d89c2739 examples: lib: Update code for urfave/cli v2 2019-11-12 21:07:38 +01:00
Felix Frank
7cc9ab9083 lib: Update for urfave/cli v2 2019-11-12 21:07:38 +01:00
Donald Bakong
4b4b7dc169 engine: resources: Adding tests to file mode 2019-11-08 10:02:30 -05:00
Yohan Belval
71ad5c5f05 examples: lang: Added os check for pkg example fix 2019-11-06 10:20:32 -05:00
Yohan Belval
39368bb5cb examples: lang: Fixed pkg example with cowsay 2019-11-06 10:03:20 -05:00
James Shubin
7a587ee8d1 misc: mkosi: Switch to copy-git-more
I added a new feature in mkosi which got merged in:

31801e89e77188e697ed937ca6b8668fde4c4a4d

This allows us to pull in all of the git repository so that we can see
the version number that comes from git.
2019-11-02 07:34:04 -04:00
James Shubin
77346527f3 docs: Update style guide with more review items
Hopefully this helps new contributors understand review changes and
avoid making them too!
2019-11-01 22:01:38 -04:00
James Shubin
1eba5833d5 engine: resources: Consistency changes and cleanup for file mode
This makes a few consistency changes and cleanups to the file mode
feature so that it's more in style with the rest of the code base.
2019-11-01 22:01:38 -04:00
Derek Buckley
83a747794e engine: resources: Adds symbolic mode to file resource
Adds a symbolic parsing function to the util package for parsing in the
file resource.
2019-11-01 21:57:10 -04:00
Julien Pivotto
3e16d1da46 engine: resources: Add new consul resource
This commit adds a new consul:kv resource which allows us to set and
watch keys inside a consul kv datastore.

This was started by roidelapluie, and was finished during pair
programming with purpleidea.

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
Signed-off-by: James Shubin <james@shubin.ca>
2019-11-01 21:38:08 -04:00
Julien Pivotto
ae1860e859 lang: funcs: Add datetime.format function
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2019-11-01 19:26:17 -04:00
Julien Pivotto
2ebc8fdf2a lang: funcs: Add datetime.hour function
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2019-11-01 20:45:45 +01:00
Julien Pivotto
be4023be66 docs: Update resource-guide.md 2019-11-01 10:03:36 -04:00
Julien Pivotto
7f4ad76298 lang: funcs: Fix autogenerated comments
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2019-11-01 11:18:48 +01:00
Julien Pivotto
0cbfaf98f3 lang: funcs: Support for []byte
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2019-10-30 22:14:08 +01:00
James Shubin
631124e658 lang: funcs: Add nitpicks from funcgen
Discussed nitpicks with roidelapluie to clean up slightly for
consistency.
2019-10-30 08:51:11 -04:00
Julien Pivotto
1685ee1ecb lang: funcs: Autogenerated a lot of new functions
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2019-10-30 08:42:53 -04:00
James Shubin
9b4d11f220 lang: funcs: Move convert into correct folder
This got merged in the wrong folder by accident.
2019-10-30 06:21:34 -04:00
James Shubin
46a71296a9 engine: resources: Add a Purge option to the file resource
This adds a "purge" parameter to the file resource. To do this, we have
to add the API hooks so the file resource can query other resources in
the graph to know if they are present, and as a result whether they
should be excluded from the purge or not.

This is useful for when we have a managed directory with some managed
contents. If a managed file is removed from the directory, then it will
be removed by the file (directory) resource if it has Purge set.
Alternatively, you can use the Reverse meta param, which is sometimes
preferable for this use case and sometimes not. This will be discussed
elsewhere.

This also adds a bunch of tests for this feature.

This also makes a few somewhat related cleanups in the file code.
2019-10-29 11:54:08 -04:00
James Shubin
1285588b62 engine: resources: Fix file absent helper
We should check this nil case too.
2019-10-29 07:15:43 -04:00
James Shubin
d96392f65e engine: resources: Improve error message in test
Nothing major, just improve testing here.
2019-10-29 07:15:43 -04:00