302 Commits

Author SHA1 Message Date
James Shubin
f4d70068b1 engine: resources: Add a default NBP option to dhcp:server
This let's us do some overrides and is a good alternative since
send/recv+autogrouping doesn't work yet.
2023-11-22 20:22:50 -05:00
James Shubin
3fe5a8d0d6 mod: Update tftp libraries 2023-11-22 20:21:40 -05:00
James Shubin
6670407c9c engine: resources: Make net resource reversible
This allows the net resource to be reversible if we want it to be. It
only flips the network state from up to down or vice-versa.
2023-11-22 20:21:40 -05:00
James Shubin
641b6067cd engine: resources: Remove net config file on down
This cleans up the file we would have created when building the resource.
2023-11-22 20:21:40 -05:00
James Shubin
b1d61fa90b engine: resources: Ignore cleanup errors that don't matter
If the file is already gone, ignore it.
2023-11-22 20:21:40 -05:00
James Shubin
0c93cf2600 engine: resources: Add constants for state up and down
Safer and cleaner code.
2023-11-22 20:21:40 -05:00
James Shubin
53753c0932 engine: resources: Improve net error messages
This is a common case of parse failures for not using CIDR. Make it
cleaner for the user.
2023-11-22 20:21:40 -05:00
James Shubin
66edc39ccb engine: resources: Improve exec docs
So that I don't hit this too often.
2023-11-22 20:21:40 -05:00
James Shubin
3f42f785ac engine: resources: Handle recurse field more appropriately
This smoothes out some edge cases. We should add comprehensive tests in
the future.
2023-11-22 20:21:40 -05:00
James Shubin
fd4d32351b engine: resources: Refactor file mode code
Small cleanup in old code.
2023-11-22 20:21:40 -05:00
James Shubin
75d3895e84 engine: Improve logging for resources and exec
This makes things a bit clearer and fixes some silly formatting bugs.
2023-11-22 20:21:40 -05:00
James Shubin
d022d7f09e engine: resources: Rename the value field in value resource
Having a different name makes it more obvious, and also leaves us open
to add a string or bool field for more explicit typing.
2023-11-18 15:15:33 -05:00
James Shubin
88b414b9a3 engine: resources: Add new value resource
This is a strange resource which is probably most useful for passing
values between scopes. It supports a variant resource field, and should
only be used as a last resort and if you know exactly what you're doing.
2023-11-18 14:23:31 -05:00
James Shubin
8d7d2fb1f1 engine: resources: Add missing struct tags for password res
Just noticed these.
2023-11-04 16:59:09 -04:00
James Shubin
e37876afec engine: resources: Small formatting cleanups 2023-10-07 11:44:47 -04:00
James Shubin
c851322826 engine: resources: Fix up test resource
Woops...
2023-09-19 09:12:59 -04:00
James Shubin
6c0775ba59 engine: resources: Consider passing in funcs if possible
Let this sit in the test resource for now.
2023-09-04 16:33:05 -04:00
James Shubin
7ccda7e99b engine: Add a ctx to the CheckApply API
This is just a rough port, there are lots of optimizations to be done
and lots of timeout values that should be replaced by a new timeout meta
param!
2023-09-02 01:34:42 -04:00
James Shubin
41493993a2 engine: resources: Respect the apply variable
Woops!
2023-09-01 22:51:44 -04:00
James Shubin
9545e409d4 engine: Create a resource kind+name specific stateful store
This adds a meta state store that is preserved between graph switches if
the kind and name match. This is useful so that rapid graph changes
don't necessarily reset their retry count if they've only changed one
resource field.
2023-09-01 21:57:05 -04:00
James Shubin
c1850e0e20 engine: resources, lang: Set resource fields more accurately
There were some bugs about setting resource fields that were structs
with various fields. This makes things more strict and correct. Now we
check for duplicate field names earlier (duplicates due to identical
aliases) and we also don't try and set private fields, or incorrectly
set partial structs.

Most interestingly, this also cleans up all of the resources and ensures
that each one has nicer docs and a clear struct tag for fields that we
want to use in mcl. These are mandatory now, and if you're missing the
tag, then we will ignore the field.
2023-08-26 01:11:50 -04:00
James Shubin
514927c0b3 engine: resources, graph: Change the Close method to Cleanup
This also changes a few similarly named methods. Clearer what it's doing
in terms of cleanup vs. causing some action.
2023-08-08 01:11:29 -04:00
James Shubin
63f05e12ca engine: resources: net: Clean up weird error handling
Not sure how this happened.
2023-08-08 01:11:29 -04:00
James Shubin
b87d70c71c engine: resources: Move wait out of Close method
Watch should wait until everything is done before exiting.
2023-08-08 01:11:29 -04:00
James Shubin
00a37b6289 engine: resources: config_etcd: Fix missing return
Woops!
2023-08-08 01:11:29 -04:00
James Shubin
963393e3d9 engine: graph, resources: Change Watch to use ctx
This is a general port. There are many optimizations and cleanups we can
do now that we have a proper context passed in. That's for a future
patch.
2023-08-08 01:11:29 -04:00
James Shubin
53a878bf61 engine: resources, graph: Change the done channel into a ctx
This is part one of porting Watch to context.
2023-08-08 01:11:29 -04:00
James Shubin
5eac48094b engine: resources: Disable broken docker test
It's not my fault it broke, afaict!
2023-08-08 01:11:29 -04:00
James Shubin
c5efe7a17b lang, engine: Remove unneeded error wrapping
These situations basically never fail, and if they do, we certainly
don't need more context. This simplifies things a bit.
2023-04-20 18:02:40 -04:00
James Shubin
c598e4d289 engine, etcd: Update code for latest gofmt fixes
Latest version of golang broken gofmt again...
2023-03-14 16:43:08 -04:00
James Shubin
a7624a2bf9 legal: Happy 2023 everyone...
Done with:

ack '2022+' -l | xargs sed -i -e 's/2022+/2023+/g'

Checked manually with:

git add -p

Hello to future James from 2024, and Happy Hacking!
2023-03-05 18:31:52 -05:00
James Shubin
3cea422365 legal: Happy 2022 everyone...
Done with:

ack '2021+' -l | xargs sed -i -e 's/2021+/2022+/g'

Checked manually with:

git add -p

Hello to future James from 2023, and Happy Hacking!
2022-08-05 23:06:27 -04:00
James Shubin
784d15b012 all: Misc housekeeping for new golang versions 2022-08-04 14:16:33 -04:00
Jef Masereel
b26f842de1 engine: resources: Add hetzner:vm resource
Hetzner cloud resource using hcloud-go. Requires polling via Meta:poll param. This first commit provides a stable vm resource with support for the basic functions of creating, deleting and updating a live server instance. SSH key handling does still require some attention to make sure checkapply can detect and update live changes to the specified keylist. A dedicated hetzner:sshkeys resource might be in order to make sure the keyset is handled correctly if there are multiple hetzner:vm resources running under the same Hetzner project. All remarks for future improvements are indicated with a TODO prefix
2022-04-20 14:24:25 -04:00
James Shubin
5927a54208 docs: Improve autogenerate godoc
There were a bunch of packages that weren't well documented. With the
recent split up of the lang package, I figured it would be more helpful
for new contributors who want to learn the structure of the project.
2021-10-26 00:12:18 -04:00
James Shubin
88516546fa lib: Move to go modules and shed a few tears
The old system with vendor/ and git submodules worked great,
unfortunately FUD around git submodules seemed to scare people away and
golang moved to a go.mod system that adds a new lock file format instead
of using the built-in git version. It's now almost impossible to use
modern golang without this, so we've switched.

So much for the golang compatibility promise-- turns out it doesn't
apply to the useful parts that I actually care about like this.

Thanks to frebib for his incredibly valuable contributions to this
patch. This snide commit message is mine alone.

This patch also mixes in some changes due to legacy golang as we've also
bumped the minimum version to 1.16 in the docs and tests.

Lastly, we had to disable some tests and fix up a few other misc things
to get this passing. We've definitely hot bugs in the go.mod system, and
our Makefile tries to workaround those.
2021-10-05 08:34:51 -04:00
James Shubin
c555478b54 engine, lang: Misc fixes for golang lint 2021-08-09 16:55:31 -04:00
Joe Groocock
fe2b8c9fee engine: resources: exec: AutoEdge to User/Group/File
Fixes https://github.com/purpleidea/mgmt/issues/221

Signed-off-by: Joe Groocock <me@frebib.net>
2021-05-11 16:51:02 -04:00
Joe Groocock
7f1baea3b0 engine: resources: docker: Replace deprecated NewClient() with NewClientWithOpts()
docker/client.NewClient() is deprecated in favour of NewClientWithOpts()
which takes a series of client.Opt functions to configure the API
client. As mgmt only passes the API version through, this simplifies the
NewClient() calls.

Signed-off-by: Joe Groocock <me@frebib.net>
2021-05-02 10:39:04 +01:00
Joe Groocock
6a61774fb7 docker: Bump docker dependencies, add containerd
These dependencies are maintained because the upstream repos bundle
vendor directories into the repos, which cause namespacing issues during
build. Git submodules don't strip the vendor directory whereas most
vendoring tools would.

Signed-off-by: Joe Groocock <me@frebib.net>
2021-02-09 21:14:34 +00:00
James Shubin
336a38081a legal: Happy 2021 everyone...
Done with:

ack '2020+' -l | xargs sed -i -e 's/2020+/2021+/g'

Checked manually with:

git add -p

Hello to future James from 2022, and Happy Hacking!
2021-01-31 16:52:46 -05:00
Adam Sigal
a5152b82e9 engine: resources: exec: Add Env
Add functionality to specify environment variables in exec.
2020-04-19 20:30:43 -06:00
James Shubin
e9af8a2595 engine: resources: exec: Clean up error handling
Some quick fixes, this whole resource should be looked at for
discrepancies, since it was written very early.
2020-04-14 22:59:33 -04:00
Adam Sigal
84b5b60d49 engine: resources: exec: Fix typo
Typo in description of cwd field fixed.
2020-04-14 22:36:25 -04:00
James Shubin
8f60f42be3 engine: resources: Add http:server and http:file resources
This adds a new http server resource, as well as a http file resource
that is used to specify files to serve in that server. This allows you
to have an http server that is entirely server from memory without
needing files on disk.

It does this by using the autogrouping magic that is already available
in the engine.

The http resource is not meant to be a full-featured http server
replacement, and it might still be useful to use the venerable webserver
of your choice, however for integrated, pure-golang bootstrapping
environments, this might prove to be very useful.

It can be combined with the tftp and dhcp resources to build PXE setups
with mgmt!

This resource can be extended further to support an http:flag endpoint,
an http:ui endpoint, automatic edges, and more!
2020-04-11 03:23:04 -04:00
James Shubin
583344138a engine: resources: Add dhcp:server and dhcp:host resources
This adds a new dhcp server resource, as well as a dhcp host resource
used to specify the static mapping between mac address and ip address.
It also adds a simple, pure-golang example dhcp client which might make
testing easier.

The dhcp resource is not meant to be a full-featured dhcp server
replacement, and it might still be useful to use the venerable dhcpd,
however for integrated, pure-golang bootstrapping environments, this
might prove to be very useful.

It can be combined with the tftp resource to build PXE setups with mgmt.

This resource can be extended further to support a dhcp:range directive,
automatic edges, and more!
2020-04-11 02:45:18 -04:00
James Shubin
016d021d5a engine: resources: tftp: Improve validation and error messages
Just some small cleanups for our tftp resource. We also rename the
struct to make it consistent, since golint complains about similar
protocols when it is not all capitalized.
2020-04-11 02:45:18 -04:00
James Shubin
115dc4bfa4 engine: resources: net: Add an IP forward field
This adds an IP forward field (boolean) and improves the documentation.
2020-04-11 02:02:35 -04:00
Francois Rompre-Lanctot
f3b4a8d055 engine: resources: Add a test case for resource owner check
This adds FileOwnerExpect as a new Step which allows validating if the
owner was set properly on a resource.
2020-02-10 22:17:42 -05:00
Francois Rompre-Lanctot
104af7e86f engine: resources: Fix typo 2020-02-03 22:57:13 -05:00