Commit Graph

1185 Commits

Author SHA1 Message Date
James Shubin
13f59230b5 misc: Split Makefile PHONY target into multiple lines
AIUI this is valid make. Please correct me if I'm wrong.
2019-09-23 06:48:55 -04:00
James Shubin
36d2a0de1e misc: Make mkosi building suitable for different distro versions
We'd like to be able to build both Fedora N and N-1 at the same time if
possible. This makes it more generally applicable for this scenario, as
well as for other distros.
2019-09-23 06:48:55 -04:00
James Shubin
a4db9fc8e5 misc: Add mkosi based package building with fpm
Building distro packages is great, however if they aren't built in the
correct environment with the associated dependencies, then they won't
work properly on those distros.

This patch adds an `mkosi` based image building environment that builds
the packages in their respective distros, and then copies them out into
our releases directory.

You'll now want to `make tag && make mkosi && make release` to get a new
release out. We use a small hack to trick the `make release` portion to
not re-build the distro packages if they're already present in the
releases/ directory for that version.

This commit depends on a very recent version of mkosi (it was tested
with git master) and also depends on two currently unmerged patches:
https://github.com/systemd/mkosi/pull/363 and
https://github.com/systemd/mkosi/pull/365
2019-09-20 12:32:41 -04:00
James Shubin
9dae5ef83b engine: resources: Improve the file res and add strict state
This might be slightly controversial, in that you must specify the state
if a file would need to be created to perform the action. We no longer
implicitly assume that just specifying content is enough. As it turns
out, I believe this is safer and more correct. The code to implement
this turns out to be much more logical and simplified, and does this
removes an ambiguous corner case from the reversed resource code.

Some discussion in: https://github.com/purpleidea/mgmt/issues/540

This patch also does a bit of related cleanup.
2019-09-14 16:07:53 -04:00
James Shubin
e8842a740c lang: Remove duplicate log message
Looks like we had two copies of the same message by accident.
2019-09-11 04:26:15 -04:00
James Shubin
0d3807ad09 lang, test: Fix copy paste error with log message
This changes this to the correct error message.
2019-09-11 04:26:15 -04:00
James Shubin
5c27a249b7 engine: resources: Add reversible API and file resource
This adds the first reversible resource (file) and the necessary engine
API hooks to make it all work. This allows a special "reversed" resource
to be added to the subsequent graph in the stream when an earlier
version "disappears". This disappearance can happen if it was previously
in an if statement that then becomes false.

It might be wise to combine the use of this meta parameter with the use
of the `realize` meta parameter to ensure that your reversed resource
actually runs at least once, if there's a chance that it might be gone
for a while.

This patch also adds a new test harness for testing resources. It
doesn't test the "live" aspect of resources, as it doesn't run Watch,
but it was designed to ensure CheckApply works as intended, and it runs
very quickly with a simplified timeline of happenings.
2019-09-11 03:40:22 -04:00
James Shubin
7e41860b28 docs: Add missing docs on the rewatch and realize meta params
Sometimes it's hard to keep this in sync.
2019-09-11 03:40:22 -04:00
James Shubin
43ff92bbe7 engine: resources: Clean up test log message 2019-09-11 03:40:22 -04:00
James Shubin
28adc7e563 engine: resource: Refactor helper functions
Maybe we can use them in other tests too.
2019-09-11 03:40:22 -04:00
James Shubin
9788411995 engine: resources: Add another validation check
This simple check should prevent some silly mistakes and make the logic
easier for other parts of the code that won't have to worry about this
pattern.
2019-09-11 03:40:22 -04:00
James Shubin
0c9e8cc50e engine: resources: Change the default file state
The default file state should be undefined. This is important because if
a reverse scenario that doesn't specify the state gets given this
default, it will be as if it was specified explicitly, which wouldn't
necessarily be what we want. Instead, an undefined state should
implicitly cause a file to get created if there's a reason to do so,
such as if content or another attribute is specified.

Hopefully this change doesn't introduce any bugs in the CheckApply code,
if it does, then it was due to a lack of implicit file creation.
2019-09-11 03:16:57 -04:00
James Shubin
34d572c523 engine: Improve the way we make a unique res path token
This is needed in the state directory.
2019-09-11 03:16:57 -04:00
James Shubin
011b496b3f engine: resources: Ensure the Kind and Name methods work
Triple check these work after decoding, by adding a test.
2019-09-11 03:16:57 -04:00
James Shubin
12b906eac6 engine: Refactor state dir into a separate function
This lets us re-use it, and know the path is fixed.
2019-09-11 03:16:57 -04:00
James Shubin
20937d05c3 engine: resources: file: Add undefined file state and validate it
We should consider using *string instead of the empty string, but let's
keep the diff smaller for now.
2019-09-11 03:16:57 -04:00
James Shubin
4943d37ccf engine: resources: file: Use constants for state values
More robustness is yay!
2019-09-11 03:16:57 -04:00
James Shubin
3a8fd215de engine: resources: file: Add Copy method to file res
This lets us implement the CopyableRes interface.
2019-09-11 03:16:57 -04:00
James Shubin
87572e8922 test: Catch capitalized error messages in tests 2019-09-06 03:28:49 -04:00
James Shubin
f1eedc7a01 lang: Clarify error message about missing field
User probably just mistyped a field name. Make that clear.
2019-09-06 03:28:49 -04:00
Donald Bakong
b79e48dd77 docs: Fix typo on quick-start-guide.md 2019-08-25 22:53:43 -04:00
James Shubin
18872194af misc: Warn users with weird computers
A user seemed to experience a weird golang issue when they had deps from
both package managers installed. I won't block or fail their install,
but we can print a warning message so that someone sees it in their
logs.
2019-08-23 22:10:34 -04:00
James Shubin
bafd7ba282 misc: Use apt install of apt-get where possible
The future is now!
2019-08-23 22:10:12 -04:00
Donald Bakong
b186481181 pgraph: Add a test for FindEdge() function 2019-08-08 00:43:26 -04:00
James Shubin
09ca6d11ad lang: funcs: Module name should be public
For consistency with the rest of the core functions.
2019-07-29 11:17:43 -04:00
James Shubin
e68e4e786d docs: Add newly recorded talks and blog post 2019-07-26 06:52:01 -04:00
James Shubin
ee638254c3 lang: Remove the specialized info structs
Since this was an early form of the modern data struct, remove those and
pass in the correct data. This is also important in case we have
something more complex inside our string interpolation!
2019-07-26 04:20:04 -04:00
James Shubin
1e678905c4 util: Fix typo 2019-07-26 04:20:04 -04:00
James Shubin
10804c4b25 lang: Improve the gapi copying
We hit a weird bug where dirs would not get copied properly. I thought
the solution might be to add the missing dirs so they'd get a proper
mkdir, but in the end that didn't work well, so we just use `mkdirall`
and that seems to work. Let's leave it like this for now. Some of the
previous work for that is in the previous commit.
2019-07-26 04:20:04 -04:00
James Shubin
4bf9b4d41b util: Add some path helper functions
In the end, I'm not sure how useful these will be, but we'll keep them
in for now.
2019-07-26 04:20:04 -04:00
James Shubin
1161872324 etcd: fs: Errors should start with lower case 2019-07-26 04:20:04 -04:00
James Shubin
98cb570896 util: Add new mkdirall variants for the copy functions
This adds versions that recursively `mkdir` and all don't error as
easily. This works around some bugs we were having with file copying.
2019-07-26 04:20:04 -04:00
James Shubin
ed4ee3b58e lang: funcs: Add deploy package with readfile related functions
This adds a readfile function to actually access files from our deploy.
A fun side effect is that we can even access our own code! In general,
it's a good reminder that you should only run trusted code on your own
infrastructure. This also includes a fancy new test case.
2019-07-26 03:38:26 -04:00
James Shubin
066048f4de lang: Pass through the Fs and the FsURI
This should give us options as to how a function should interact with an
FS. I feel like it's cleaner to go through the World API, and passing in
the FsURI lets us do that, but I passed in the Fs at the same time in
case it's useful for some reason. I think using it is a boundary
violation, but it's just a hunch. Does anything break when we move from
one deploy to the next?
2019-07-26 03:07:08 -04:00
James Shubin
4b6b91c08b lang: Make sure to call Init for functions that arrive via import
We weren't calling Init on some functions which should have had this
done. I'm not sure whether this is the right place, or if it should be
elsewhere as part of the scope building process. Good enough for now.
2019-07-22 06:49:02 -04:00
James Shubin
2980523a5b lang: Add a new function interface to accept data
Sometimes certain internal functions might want to get some data from
the AST or from something relating to the state of the language. This
adds a method to pass in that data. For now it's a very simple method,
but we could generalize it in the future if it becomes more useful.
2019-07-22 06:46:04 -04:00
James Shubin
f2f9c043bf lang, gapi: Work around a copy bug in the deploy
It seems when we had a files/ dir that we added to our deploy, it would
get copied into /files/files/whatever instead of /files/whatever where
it should be. Hopefully this works around the issue forever.
2019-07-22 06:40:47 -04:00
James Shubin
5d59cfd2c9 util: Ensure the afero copy function is working as intended
The destination should be a dir sometimes.
2019-07-22 06:38:02 -04:00
James Shubin
f94474e24f lang: Add the world implementation to our test suite
This allows our tests to actual run the World API in them.
2019-07-22 06:36:37 -04:00
James Shubin
a63fc6d9ba util: Add a remove path suffix util function
This pairs with a similar one we already had.
2019-07-22 06:35:13 -04:00
James Shubin
076adeef80 lang: funcs: Fix a copypasta error with the not equals operator
Woops, sorry!
2019-07-22 06:08:37 -04:00
James Shubin
a0e756317c lang: Add tests for slow unification
These used to be cases where our algorithm was unusably slow.

Thanks to foxxx0 for the report!
2019-07-21 03:15:06 -04:00
James Shubin
252cb5f2f3 lang: Detect windows style CR and return a better error
If you get a sneaky \r in your code, the error just looks like
whitespace, so this way we can warn you explicitly.
2019-07-21 03:10:21 -04:00
James Shubin
64288b4914 lang, test: Inline some overly indented tests
Sometimes you're busy hacking and it's nice for future you to fix up
your code!
2019-07-21 01:19:15 -04:00
James Shubin
9ca6c6a315 test: Split up long tests into multiple sub tests again
I think we need this for non --race tests too.
2019-07-21 00:55:36 -04:00
James Shubin
3651ab5c0c lang: Add more tests for function 2019-07-20 22:27:21 -04:00
James Shubin
b3f15e1ddc lang: Add more tests for class and include 2019-07-20 01:33:42 -04:00
James Shubin
da2a5f72bd lib: Update dep for uuid
Apparently the package has moved.
0.0.20
2019-07-17 04:07:24 -04:00
James Shubin
591e6b68e0 test: Split up long tests into multiple sub tests
Hopefully this avoids the timeouts running the lang package.
2019-07-17 02:45:04 -04:00
James Shubin
0119abdcdd travis: Try to work around CI slowdowns
I think they throttle strangely on their garbage machines.
2019-07-17 01:21:29 -04:00