Commit Graph

33 Commits

Author SHA1 Message Date
James Shubin
7ccda7e99b engine: Add a ctx to the CheckApply API
This is just a rough port, there are lots of optimizations to be done
and lots of timeout values that should be replaced by a new timeout meta
param!
2023-09-02 01:34:42 -04:00
James Shubin
8299c04fc6 engine: graph, util: Clean up error printing
We should improve on this more, but at least as a quick fix, stop
splitting the error across two lines. This makes the logs really ugly.
2023-09-01 16:53:40 -04:00
James Shubin
c1850e0e20 engine: resources, lang: Set resource fields more accurately
There were some bugs about setting resource fields that were structs
with various fields. This makes things more strict and correct. Now we
check for duplicate field names earlier (duplicates due to identical
aliases) and we also don't try and set private fields, or incorrectly
set partial structs.

Most interestingly, this also cleans up all of the resources and ensures
that each one has nicer docs and a clear struct tag for fields that we
want to use in mcl. These are mandatory now, and if you're missing the
tag, then we will ignore the field.
2023-08-26 01:11:50 -04:00
James Shubin
514927c0b3 engine: resources, graph: Change the Close method to Cleanup
This also changes a few similarly named methods. Clearer what it's doing
in terms of cleanup vs. causing some action.
2023-08-08 01:11:29 -04:00
James Shubin
963393e3d9 engine: graph, resources: Change Watch to use ctx
This is a general port. There are many optimizations and cleanups we can
do now that we have a proper context passed in. That's for a future
patch.
2023-08-08 01:11:29 -04:00
Ofek Atar
eddd16283d engine: util: Add test for StructKindToFieldNameTypeMap function 2023-05-12 14:45:42 +03:00
James Shubin
a7624a2bf9 legal: Happy 2023 everyone...
Done with:

ack '2022+' -l | xargs sed -i -e 's/2022+/2023+/g'

Checked manually with:

git add -p

Hello to future James from 2024, and Happy Hacking!
2023-03-05 18:31:52 -05:00
James Shubin
aff6331211 misc, docs: Move to golang 1.18
Hopefully this fixes some yucky things.
2022-09-11 20:55:37 -04:00
James Shubin
3cea422365 legal: Happy 2022 everyone...
Done with:

ack '2021+' -l | xargs sed -i -e 's/2021+/2022+/g'

Checked manually with:

git add -p

Hello to future James from 2023, and Happy Hacking!
2022-08-05 23:06:27 -04:00
James Shubin
784d15b012 all: Misc housekeeping for new golang versions 2022-08-04 14:16:33 -04:00
Joe Groocock
bf7e45439b engine: util: Skip unexported fields in struct field mapping
Unexported fields should be ignored when mapping structs from Golang to
mcl, as this avoids issues where certain structs cannot be used in mcl
representation due to their structure that may be incompatible with
conversion. Disallowing private fields allows those to still be included
in the struct but ignored when mapping the types, allowing the mcl
representation of the type to stil be valid

Resolves: https://github.com/purpleidea/mgmt/issues/684
Signed-off-by: Joe Groocock <me@frebib.net>
2021-12-01 22:06:06 +00:00
James Shubin
5927a54208 docs: Improve autogenerate godoc
There were a bunch of packages that weren't well documented. With the
recent split up of the lang package, I figured it would be more helpful
for new contributors who want to learn the structure of the project.
2021-10-26 00:12:18 -04:00
James Shubin
88516546fa lib: Move to go modules and shed a few tears
The old system with vendor/ and git submodules worked great,
unfortunately FUD around git submodules seemed to scare people away and
golang moved to a go.mod system that adds a new lock file format instead
of using the built-in git version. It's now almost impossible to use
modern golang without this, so we've switched.

So much for the golang compatibility promise-- turns out it doesn't
apply to the useful parts that I actually care about like this.

Thanks to frebib for his incredibly valuable contributions to this
patch. This snide commit message is mine alone.

This patch also mixes in some changes due to legacy golang as we've also
bumped the minimum version to 1.16 in the docs and tests.

Lastly, we had to disable some tests and fix up a few other misc things
to get this passing. We've definitely hot bugs in the go.mod system, and
our Makefile tries to workaround those.
2021-10-05 08:34:51 -04:00
Joe Groocock
c950568f1b lang: Move StructTag const into lang/types
This constant value is strongly tied to the language, and little to do
with the engine. Move the definition into the lang/types package to
prevent circular imports between lang/types and engine/util.

Signed-off-by: Joe Groocock <me@frebib.net>
2021-02-06 16:56:57 +00:00
James Shubin
336a38081a legal: Happy 2021 everyone...
Done with:

ack '2020+' -l | xargs sed -i -e 's/2020+/2021+/g'

Checked manually with:

git add -p

Hello to future James from 2022, and Happy Hacking!
2021-01-31 16:52:46 -05:00
James Shubin
d75f763c99 misc, docs: Move to golang 1.12 2020-09-23 12:34:54 -04:00
Donald Bakong
e99dd749a0 engine: resources: Fixing return bug 2020-09-23 11:10:38 -04:00
James Shubin
f67ad9c061 test: Add a check for too long or badly reflowed docstrings
This ensures that docstring comments are wrapped to 80 chars. ffrank
seemed to be making this mistake far too often, and it's a silly thing
to look for manually. As it turns out, I've made it too, as have many
others. Now we have a test that checks for most cases. There are still a
few stray cases that aren't checked automatically, but this can be
improved upon if someone is motivated to do so.

Before anyone complains about the 80 character limit: this only checks
docstring comments, not source code length or inline source code
comments. There's no excuse for having docstrings that are badly
reflowed or over 80 chars, particularly if you have an automated test.
2020-01-25 04:43:33 -05:00
James Shubin
2648fb1bb1 legal: Happy 2020 everyone...
Done with:

ack '2019+' -l | xargs sed -i -e 's/2019+/2020+/g'

Checked manually with:

git add -p

Hello to future James from 2021, and Happy Hacking!
2020-01-03 20:08:37 -05:00
Donald Bakong
4b4b7dc169 engine: resources: Adding tests to file mode 2019-11-08 10:02:30 -05:00
James Shubin
1eba5833d5 engine: resources: Consistency changes and cleanup for file mode
This makes a few consistency changes and cleanups to the file mode
feature so that it's more in style with the rest of the code base.
2019-11-01 22:01:38 -04:00
Derek Buckley
83a747794e engine: resources: Adds symbolic mode to file resource
Adds a symbolic parsing function to the util package for parsing in the
file resource.
2019-11-01 21:57:10 -04:00
James Shubin
34d572c523 engine: Improve the way we make a unique res path token
This is needed in the state directory.
2019-09-11 03:16:57 -04:00
James Shubin
07f542b4d7 legal: Happy 2019 everyone...
Done with:

ack '2018+' -l | xargs sed -i -e 's/2018+/2019+/g'

Checked manually with:

git add -p

Hello to future James from 2020, and Happy Hacking!
2019-03-24 15:08:50 -04:00
James Shubin
ae56261961 engine: util, resources: virt: Clean up virt resource
Do some cleanups which were long overdue.
2019-03-15 15:24:40 -04:00
James Shubin
880652f5d4 util: Port all code to new errwrap package
This should keep things more uniform.
2019-03-12 16:49:01 -04:00
James Shubin
4c104d55cb engine: util: Add a new utility function for send/recv
This new utility function makes verifying send/recv struct comparisons
consistent. Unfortunately it doesn't yet support coercing from *string
to string or from string to *string.
2019-03-09 16:03:30 -05:00
James Shubin
8a8215fabe engine: util: Improve StructTagToFieldName and add tests
This improves this function to make it more generic.
2019-03-09 16:02:33 -05:00
Jonathan Gold
a5977b993a engine: util: Add EdgeCombiner() for combining auto edges 2018-12-15 10:47:35 -05:00
Jonathan Gold
27df3ae876 engine: resources: cron: Add a systemd-timer resource 2018-12-15 10:47:35 -05:00
James Shubin
22c0ff3cf5 test: Improve golang tests with root and disabling cache
This allows golang tests to be marked as root or !root using build tags.
The matching tests are then run as expected using our test runner.

This also disables test caching which is unfriendly to repeated test
running and is an absurd golang default to add.

Lastly this hooks up the testing verbose flag to tests that accept a
debug variable.

These tests aren't enabled on travis yet because of how it installs
golang.
2018-05-09 13:44:01 -04:00
Jonathan Gold
9baad56197 util: Move dbus AddMatch const to util package 2018-05-04 15:46:14 -04:00
James Shubin
9969286224 engine: Resources package rewrite
This giant patch makes some much needed improvements to the code base.

* The engine has been rewritten and lives within engine/graph/
* All of the common interfaces and code now live in engine/
* All of the resources are in one package called engine/resources/
* The Res API can use different "traits" from engine/traits/
* The Res API has been simplified to hide many of the old internals
* The Watch & Process loops were previously inverted, but is now fixed
* The likelihood of package cycles has been reduced drastically
* And much, much more...

Unfortunately, some code had to be temporarily removed. The remote code
had to be taken out, as did the prometheus code. We hope to have these
back in new forms as soon as possible.
2018-04-19 01:10:58 -04:00