There's always the fear that there is either a panic or a deadlock in
the highly concurrent engine resource code. I have not seen one recently
and I've been running some pretty concurrent tests. In the meantime, and
with my hopefully improved knowledge of concurrency, I decided to
rewrite some of the "uglier" parts of the engine. I think it is a lot
clearer now, and much less likely that there is a concurrency issue.
This has been tested by running the examples/lang/fastcount.mcl example.
There were some bugs about setting resource fields that were structs
with various fields. This makes things more strict and correct. Now we
check for duplicate field names earlier (duplicates due to identical
aliases) and we also don't try and set private fields, or incorrectly
set partial structs.
Most interestingly, this also cleans up all of the resources and ensures
that each one has nicer docs and a clear struct tag for fields that we
want to use in mcl. These are mandatory now, and if you're missing the
tag, then we will ignore the field.
Hetzner cloud resource using hcloud-go. Requires polling via Meta:poll param. This first commit provides a stable vm resource with support for the basic functions of creating, deleting and updating a live server instance. SSH key handling does still require some attention to make sure checkapply can detect and update live changes to the specified keylist. A dedicated hetzner:sshkeys resource might be in order to make sure the keyset is handled correctly if there are multiple hetzner:vm resources running under the same Hetzner project. All remarks for future improvements are indicated with a TODO prefix
Unexported fields should be ignored when mapping structs from Golang to
mcl, as this avoids issues where certain structs cannot be used in mcl
representation due to their structure that may be incompatible with
conversion. Disallowing private fields allows those to still be included
in the struct but ignored when mapping the types, allowing the mcl
representation of the type to stil be valid
Resolves: https://github.com/purpleidea/mgmt/issues/684
Signed-off-by: Joe Groocock <me@frebib.net>
There were a bunch of packages that weren't well documented. With the
recent split up of the lang package, I figured it would be more helpful
for new contributors who want to learn the structure of the project.
The old system with vendor/ and git submodules worked great,
unfortunately FUD around git submodules seemed to scare people away and
golang moved to a go.mod system that adds a new lock file format instead
of using the built-in git version. It's now almost impossible to use
modern golang without this, so we've switched.
So much for the golang compatibility promise-- turns out it doesn't
apply to the useful parts that I actually care about like this.
Thanks to frebib for his incredibly valuable contributions to this
patch. This snide commit message is mine alone.
This patch also mixes in some changes due to legacy golang as we've also
bumped the minimum version to 1.16 in the docs and tests.
Lastly, we had to disable some tests and fix up a few other misc things
to get this passing. We've definitely hot bugs in the go.mod system, and
our Makefile tries to workaround those.
Since we'll want to use them elsewhere, we should make these helper
functions. It also makes the code look a lot neater. Unfortunately, it
adds a bit more indirection, but this isn't a critical flaw here.
docker/client.NewClient() is deprecated in favour of NewClientWithOpts()
which takes a series of client.Opt functions to configure the API
client. As mgmt only passes the API version through, this simplifies the
NewClient() calls.
Signed-off-by: Joe Groocock <me@frebib.net>
These dependencies are maintained because the upstream repos bundle
vendor directories into the repos, which cause namespacing issues during
build. Git submodules don't strip the vendor directory whereas most
vendoring tools would.
Signed-off-by: Joe Groocock <me@frebib.net>
This constant value is strongly tied to the language, and little to do
with the engine. Move the definition into the lang/types package to
prevent circular imports between lang/types and engine/util.
Signed-off-by: Joe Groocock <me@frebib.net>
This adds a new http server resource, as well as a http file resource
that is used to specify files to serve in that server. This allows you
to have an http server that is entirely server from memory without
needing files on disk.
It does this by using the autogrouping magic that is already available
in the engine.
The http resource is not meant to be a full-featured http server
replacement, and it might still be useful to use the venerable webserver
of your choice, however for integrated, pure-golang bootstrapping
environments, this might prove to be very useful.
It can be combined with the tftp and dhcp resources to build PXE setups
with mgmt!
This resource can be extended further to support an http:flag endpoint,
an http:ui endpoint, automatic edges, and more!
This adds a new dhcp server resource, as well as a dhcp host resource
used to specify the static mapping between mac address and ip address.
It also adds a simple, pure-golang example dhcp client which might make
testing easier.
The dhcp resource is not meant to be a full-featured dhcp server
replacement, and it might still be useful to use the venerable dhcpd,
however for integrated, pure-golang bootstrapping environments, this
might prove to be very useful.
It can be combined with the tftp resource to build PXE setups with mgmt.
This resource can be extended further to support a dhcp:range directive,
automatic edges, and more!
Just some small cleanups for our tftp resource. We also rename the
struct to make it consistent, since golint complains about similar
protocols when it is not all capitalized.
Since we focus on safety, it would be nice to reduce the chance of any
runtime errors if we made a typo for a resource parameter. With this
patch, each resource can export constants into the global namespace so
that typos would cause a compile error.
Of course in the future if we had a more advanced type system, then we
could support precise types for each individual resource param, but in
an attempt to keep things simple, we'll leave that for another day. It
would add complexity too if we ever wanted to store a parameter
externally.
Lastly, we might consider adding "special case" parsing so that directly
specified fields would parse intelligently. For example, we could allow:
file "/tmp/hello" {
state => exists, # magic sugar!
}
This isn't supported for now, but if it works after all the other parser
changes have been made, it might be something to consider.
This ensures that docstring comments are wrapped to 80 chars. ffrank
seemed to be making this mistake far too often, and it's a silly thing
to look for manually. As it turns out, I've made it too, as have many
others. Now we have a test that checks for most cases. There are still a
few stray cases that aren't checked automatically, but this can be
improved upon if someone is motivated to do so.
Before anyone complains about the 80 character limit: this only checks
docstring comments, not source code length or inline source code
comments. There's no excuse for having docstrings that are badly
reflowed or over 80 chars, particularly if you have an automated test.
For some reason we get errors when we try to remove a non-existent state
file. There's a slight possibility that it could be a bug we're working
around, but it's not clear that this is the case, and I think it's
possible that a state file could have gotten nuked by the user somehow,
although this was occurring "naturally" when running reverse1.mcl so
let's keep that working for now.
