Commit Graph

1228 Commits

Author SHA1 Message Date
James Shubin
77346527f3 docs: Update style guide with more review items
Hopefully this helps new contributors understand review changes and
avoid making them too!
2019-11-01 22:01:38 -04:00
James Shubin
1eba5833d5 engine: resources: Consistency changes and cleanup for file mode
This makes a few consistency changes and cleanups to the file mode
feature so that it's more in style with the rest of the code base.
2019-11-01 22:01:38 -04:00
Derek Buckley
83a747794e engine: resources: Adds symbolic mode to file resource
Adds a symbolic parsing function to the util package for parsing in the
file resource.
2019-11-01 21:57:10 -04:00
Julien Pivotto
3e16d1da46 engine: resources: Add new consul resource
This commit adds a new consul:kv resource which allows us to set and
watch keys inside a consul kv datastore.

This was started by roidelapluie, and was finished during pair
programming with purpleidea.

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
Signed-off-by: James Shubin <james@shubin.ca>
2019-11-01 21:38:08 -04:00
Julien Pivotto
ae1860e859 lang: funcs: Add datetime.format function
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2019-11-01 19:26:17 -04:00
Julien Pivotto
2ebc8fdf2a lang: funcs: Add datetime.hour function
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2019-11-01 20:45:45 +01:00
Julien Pivotto
be4023be66 docs: Update resource-guide.md 2019-11-01 10:03:36 -04:00
Julien Pivotto
7f4ad76298 lang: funcs: Fix autogenerated comments
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2019-11-01 11:18:48 +01:00
Julien Pivotto
0cbfaf98f3 lang: funcs: Support for []byte
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2019-10-30 22:14:08 +01:00
James Shubin
631124e658 lang: funcs: Add nitpicks from funcgen
Discussed nitpicks with roidelapluie to clean up slightly for
consistency.
2019-10-30 08:51:11 -04:00
Julien Pivotto
1685ee1ecb lang: funcs: Autogenerated a lot of new functions
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2019-10-30 08:42:53 -04:00
James Shubin
9b4d11f220 lang: funcs: Move convert into correct folder
This got merged in the wrong folder by accident.
2019-10-30 06:21:34 -04:00
James Shubin
46a71296a9 engine: resources: Add a Purge option to the file resource
This adds a "purge" parameter to the file resource. To do this, we have
to add the API hooks so the file resource can query other resources in
the graph to know if they are present, and as a result whether they
should be excluded from the purge or not.

This is useful for when we have a managed directory with some managed
contents. If a managed file is removed from the directory, then it will
be removed by the file (directory) resource if it has Purge set.
Alternatively, you can use the Reverse meta param, which is sometimes
preferable for this use case and sometimes not. This will be discussed
elsewhere.

This also adds a bunch of tests for this feature.

This also makes a few somewhat related cleanups in the file code.
2019-10-29 11:54:08 -04:00
James Shubin
1285588b62 engine: resources: Fix file absent helper
We should check this nil case too.
2019-10-29 07:15:43 -04:00
James Shubin
d96392f65e engine: resources: Improve error message in test
Nothing major, just improve testing here.
2019-10-29 07:15:43 -04:00
James Shubin
d1c5a736ae engine: resources: Allow nil helper functions in tests
This reduces some of the boilerplate when writing resource tests.
2019-10-29 07:15:43 -04:00
James Shubin
6b1e038c5c engine: resources: Add file exists helper
This will allow us to test even more things!
2019-10-29 07:15:43 -04:00
James Shubin
eaab1aae28 engine: graph, resources: Add filtered graph function
This lets a resource query the resource graph in a controlled way.
2019-10-29 07:15:43 -04:00
James Shubin
31030343a2 engine: Add graph queryable trait
This is a trait that specifies that your resource can be queried by
others. You can either enable it plainly to add wholesale access by
everyone, or you can implement your own Allow method to limit what is
permitted.
2019-10-29 07:15:43 -04:00
James Shubin
325ca03a13 engine: graph: Pass through the graph struct
We want to use it in the resources.
2019-10-29 07:15:43 -04:00
James Shubin
dea8e63df2 util: Add more tests to HasPathPrefix
We need to ensure this behaviour in a future commit, so might as well
double check that this works as expected!
2019-10-29 07:15:43 -04:00
James Shubin
58421fd31a engine: resources: Add fragments support to file resource
This adds a "fragments" mode to the file resource. In addition to
"content" and "source", you can now alternatively build a file from the
file fragments of other files.
2019-10-29 07:15:43 -04:00
James Shubin
b961c96862 lang: Include automatic edges in our test case
When running this test, we didn't attempt to build any automatic edges.
Since we'd like to test this here as well, let's add it.
2019-10-24 04:30:49 -04:00
Donald Bakong
2d23c1b0f3 lang: funcs: Add to_int and to_float functions 2019-10-22 08:34:29 -04:00
James Shubin
06952c224b engine: resources: nspawn: Use populate variable
We referred to the wrong variable. Not a major bug, but would produce a
useless or confusing error message otherwise.
2019-10-21 07:38:42 -04:00
Donald Bakong
2ea492c965 docs: Fix error on language-guide.md 2019-10-19 18:44:56 -04:00
Donald Bakong
dbf84f6879 docs: Fix typo on language-guide.md 2019-10-19 18:44:11 -04:00
James Shubin
0fa3d6c462 github: Update funding information file
Just got added to the GH sponsors thing and there are more fields
available in this file now. Let's hope this works!
2019-10-09 19:56:19 -04:00
James Shubin
d57f7aa03f misc: Specific mkosi fixes for centos-7
Seems we need golang from epel, to mask out the old git version, and to
workaround mkosi bugs.
2019-10-05 01:49:42 -04:00
Jimmy Tang
d64f9f5401 misc: Add CentOS 7 rpm build 2019-10-04 23:47:12 -04:00
James Shubin
a3029afc41 art: Rename art file to clarify it's about Winnie the Pooh
Pretty cool that we have our first meme =D Have a look in the FAQ to see
the real reason we watch our resources. (TL;DR: It allows us to make
graph changes very fast.)
2019-10-04 08:01:23 -04:00
James Shubin
6a7d904fae misc: Improve tagging script
This way we can push the tag *after* all the builds succeed. If
something goes wrong, we can always delete our local tag and try again.
0.0.21
2019-10-04 06:49:04 -04:00
James Shubin
d4043d3f86 misc, make: Add full file path into fpm script
This is needed for our fancier, unique file names.
2019-10-04 06:43:17 -04:00
James Shubin
b4902a4f58 make: Add a unique token to the package file name
This unique token is necessary so that storing the files in the same dir
(basically a GitHub release) or in the SHA1SUMS file doesn't cause a
conflict.
2019-10-04 06:06:44 -04:00
James Shubin
ffe402f201 misc: Add fedora-30 mkosi+fpm build environment
Good example of how to add a new distro or version.
2019-10-04 06:02:08 -04:00
James Shubin
09cc7da282 misc: Add proper archlinux prefix in build script 2019-10-04 06:01:23 -04:00
James Shubin
2d2dad41f4 todo: Update the TODO file so that it has a sane purpose
We stored some stuff in GitHub, and some stuff here. We can keep using
this, but let's do it for the stuff that hasn't changed in a while.
2019-10-04 04:11:26 -04:00
bjanssens
5f7c0a86dd art: Add the requested art
Signed-off-by: bjanssens <bjanssens@inuits.eu>
2019-10-04 09:23:20 +02:00
Donald Bakong
fc1c631c98 engine: resources: Change Res API from Compare to Cmp
This will be done by refactoring the current method, to return an error
message instead of a boolean value. This will also update a typo on the
user res.
2019-09-27 18:10:58 -04:00
James Shubin
89bdafacb8 misc: Refactor Makefile slightly
We could make this even better in the future with lists.
2019-09-23 06:57:26 -04:00
James Shubin
73b6b3f129 misc: Remove old image building cruft 2019-09-23 06:50:26 -04:00
James Shubin
b2a495f593 misc: Add mkosi target for ubuntu bionic
The name of these is pretty weird.
2019-09-23 06:50:00 -04:00
James Shubin
65ee904377 misc: Work around old golang in ubuntu
Hopefully this helps.
2019-09-23 06:48:55 -04:00
James Shubin
13f59230b5 misc: Split Makefile PHONY target into multiple lines
AIUI this is valid make. Please correct me if I'm wrong.
2019-09-23 06:48:55 -04:00
James Shubin
36d2a0de1e misc: Make mkosi building suitable for different distro versions
We'd like to be able to build both Fedora N and N-1 at the same time if
possible. This makes it more generally applicable for this scenario, as
well as for other distros.
2019-09-23 06:48:55 -04:00
James Shubin
a4db9fc8e5 misc: Add mkosi based package building with fpm
Building distro packages is great, however if they aren't built in the
correct environment with the associated dependencies, then they won't
work properly on those distros.

This patch adds an `mkosi` based image building environment that builds
the packages in their respective distros, and then copies them out into
our releases directory.

You'll now want to `make tag && make mkosi && make release` to get a new
release out. We use a small hack to trick the `make release` portion to
not re-build the distro packages if they're already present in the
releases/ directory for that version.

This commit depends on a very recent version of mkosi (it was tested
with git master) and also depends on two currently unmerged patches:
https://github.com/systemd/mkosi/pull/363 and
https://github.com/systemd/mkosi/pull/365
2019-09-20 12:32:41 -04:00
James Shubin
9dae5ef83b engine: resources: Improve the file res and add strict state
This might be slightly controversial, in that you must specify the state
if a file would need to be created to perform the action. We no longer
implicitly assume that just specifying content is enough. As it turns
out, I believe this is safer and more correct. The code to implement
this turns out to be much more logical and simplified, and does this
removes an ambiguous corner case from the reversed resource code.

Some discussion in: https://github.com/purpleidea/mgmt/issues/540

This patch also does a bit of related cleanup.
2019-09-14 16:07:53 -04:00
James Shubin
e8842a740c lang: Remove duplicate log message
Looks like we had two copies of the same message by accident.
2019-09-11 04:26:15 -04:00
James Shubin
0d3807ad09 lang, test: Fix copy paste error with log message
This changes this to the correct error message.
2019-09-11 04:26:15 -04:00
James Shubin
5c27a249b7 engine: resources: Add reversible API and file resource
This adds the first reversible resource (file) and the necessary engine
API hooks to make it all work. This allows a special "reversed" resource
to be added to the subsequent graph in the stream when an earlier
version "disappears". This disappearance can happen if it was previously
in an if statement that then becomes false.

It might be wise to combine the use of this meta parameter with the use
of the `realize` meta parameter to ensure that your reversed resource
actually runs at least once, if there's a chance that it might be gone
for a while.

This patch also adds a new test harness for testing resources. It
doesn't test the "live" aspect of resources, as it doesn't run Watch,
but it was designed to ensure CheckApply works as intended, and it runs
very quickly with a simplified timeline of happenings.
2019-09-11 03:40:22 -04:00