lourenco/mgmt
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

cli, puppet, langpuppet: Remove puppet GAPI's

Browse Source 
I'm currently refactoring the CLI code. Unfortunately this means a
pretty big churn in the various GAPI frontends. Since nobody is actively
using the puppet frontend code, I'm removing it for now. If someone is
actively using it, and wants to either port it to the new API, or
sponsor the porting of it to the new API, I'm happy to allow it back in.

Sorry Felix, it was a fun idea, and I loved seeing it work, but I can't
personally afford the maintenance cost of having this in right now.
This commit is contained in:
James Shubin
2024-03-01 13:54:36 -05:00
parent 62295e370c
commit e767655ede
14 changed files with 4 additions and 1405 deletions
Download Patch File Download Diff File Expand all files Collapse all files

27
docs/documentation.md
View File

@@ -131,33 +131,6 @@ execute via a `remote` resource.
You can read the introductory blog post about this topic here:
[https://purpleidea.com/blog/2016/10/07/remote-execution-in-mgmt/](https://purpleidea.com/blog/2016/10/07/remote-execution-in-mgmt/)
### Puppet support
You can supply a Puppet manifest instead of creating the (YAML) graph manually.
Puppet must be installed and in `mgmt`'s search path. You also need the
[ffrank-mgmtgraph Puppet module](https://forge.puppet.com/ffrank/mgmtgraph).
Invoke `mgmt` with the `--puppet` switch, which supports 3 variants:
1. Request the configuration from the Puppet Master (like `puppet agent` does)
`mgmt run puppet --puppet agent`
2. Compile a local manifest file (like `puppet apply`)
`mgmt run puppet --puppet /path/to/my/manifest.pp`
3. Compile an ad hoc manifest from the commandline (like `puppet apply -e`)
`mgmt run puppet --puppet 'file { "/etc/ntp.conf": ensure => file }'`
For more details and caveats see [puppet-guide.md](puppet-guide.md).
#### Blog post
An introductory post on the Puppet support is on
[Felix's blog](http://ffrank.github.io/features/2016/06/19/puppet-powered-mgmt/).
## Reference
Please note that there are a number of undocumented options. For more

1
docs/index.rst
View File

@@ -14,4 +14,3 @@ Welcome to mgmt's documentation!
quick-start-guide
resource-guide
prometheus
puppet-guide

316
docs/puppet-guide.md
View File

@@ -1,316 +0,0 @@
# Puppet guide
`mgmt` can use Puppet as its source for the configuration graph.
This document goes into detail on how this works, and lists
some pitfalls and limitations.
For basic instructions on how to use the Puppet support, see
the [main documentation](documentation.md#puppet-support).
## Prerequisites
You need Puppet installed in your system. It is not important how you
get it. On the most common Linux distributions, you can use packages
from the OS maintainer, or upstream Puppet repositories. An alternative
that will also work on OSX is the `puppet` Ruby gem. It also has the
advantage that you can install any desired version in your home directory
or any other location.
Any release of Puppet's 3.x and 4.x series should be suitable for use with
`mgmt`. Most importantly, make sure to install the `ffrank-mgmtgraph` Puppet
module (referred to below as "the translator module").
```
puppet module install ffrank-mgmtgraph
```
Please note that the module is not required on your Puppet master (if you
use a master/agent setup). It's needed on the machine that runs `mgmt`.
You can install the module on the master anyway, so that it gets distributed
to your agents through Puppet's `pluginsync` mechanism.
### Testing the Puppet side
The following command should run successfully and print a YAML hash on your
terminal:
```puppet
puppet mgmtgraph print --code 'file { "/tmp/mgmt-test": ensure => present }'
```
You can use this CLI to test any manifests before handing them straight
to `mgmt`.
## Writing a suitable manifest
### Unsupported attributes
`mgmt` inherited its resource module from Puppet, so by and large, it's quite
possible to express `mgmt` graphs in terms of Puppet manifests. However,
there isn't (and likely never will be) full feature parity between the
respective resource types. In consequence, a manifest can have semantics that
cannot be transferred to `mgmt`.
For example, at the time of writing this, the `file` type in `mgmt` had no
notion of permissions (the file `mode`) yet. This lead to the following
warning (among others that will be discussed below):
```
$ puppet mgmtgraph print --code 'file { "/tmp/foo": mode => "0600" }'
Warning: cannot translate: File[/tmp/foo] { mode => "600" } (attribute is ignored)
```
This is a heads-up for the user, because the resulting `mgmt` graph will
in fact not pass this information to the `/tmp/foo` file resource, and
`mgmt` will ignore this file's permissions. Including such attributes in
manifests that are written expressly for `mgmt` is not sensible and should
be avoided.
### Unsupported resources
Puppet has a fairly large number of
[built-in types](https://docs.puppet.com/puppet/latest/reference/type.html),
and countless more are available through
[modules](https://forge.puppet.com/). It's unlikely that all of them will
eventually receive native counterparts in `mgmt`.
When encountering an unknown resource, the translator module will replace
it with an `exec` resource in its output. This resource will run the equivalent
of a `puppet resource` command to make Puppet apply the original resource
itself. This has quite abysmal performance, because processing such a
resource requires the forking of at least one Puppet process (two if it
is found to be out of sync). This comes with considerable overhead.
On most systems, starting up any Puppet command takes several seconds.
Compared to the split second that the actual work usually takes,
this overhead can amount to several orders of magnitude.
Avoid Puppet types that `mgmt` does not implement (yet).
### Avoiding common warnings
Many resource parameters in Puppet take default values. For the most part,
the translator module just ignores them. However, there are cases in which
Puppet will default to convenient behavior that `mgmt` cannot quite replicate.
For example, translating a plain `file` resource will lead to a warning message:
```
$ puppet mgmtgraph print --code 'file { "/tmp/mgmt-test": }'
Warning: File[/tmp/mgmt-test] uses the 'puppet' file bucket, which mgmt cannot
do. There will be no backup copies!
```
The reason is that per default, Puppet assumes the following parameter value
(among others)
```puppet
file { "/tmp/mgmt-test":
backup => 'puppet',
}
```
To avoid this, specify the parameter explicitly:
```bash
puppet mgmtgraph print --code 'file { "/tmp/mgmt-test": backup => false }'
```
This is tedious in a more complex manifest. A good simplification is the
following [resource default](https://docs.puppet.com/puppet/latest/reference/lang_defaults.html)
anywhere on the top scope of your manifest:
```puppet
File { backup => false }
```
If you encounter similar warnings from other types and/or parameters,
use the same approach to silence them if possible.
## Configuring Puppet
Since `mgmt` uses an actual Puppet CLI behind the scenes, you might
need to tweak some of Puppet's runtime options in order to make it
do what you want. Reasons for this could be among the following:
* You use the `--puppet agent` variant and need to configure
`servername`, `certname` and other master/agent-related options.
* You don't want runtime information to end up in the `vardir`
that is used by your regular `puppet agent`.
* You install specific Puppet modules for `mgmt` in a non-standard
location.
`mgmt` exposes only one Puppet option in order to allow you to
control all of them, through its `--puppet-conf` option. It allows
you to specify which `puppet.conf` file should be used during
translation.
```
mgmt run puppet --puppet /opt/my-manifest.pp --puppet-conf /etc/mgmt/puppet.conf
```
Within this file, you can just specify any needed options in the
`[main]` section:
```
[main]
server=mgmt-master.example.net
vardir=/var/lib/mgmt/puppet
```
## Caveats
Please see the [README](https://github.com/ffrank/puppet-mgmtgraph/blob/master/README.md)
of the translator module for the current state of supported and unsupported
language features.
You should probably make sure to always use the latest release of
both `ffrank-mgmtgraph` and `ffrank-yamlresource` (the latter is
getting pulled in as a dependency of the former).
## Using Puppet in conjunction with the mcl lang
The graph that Puppet generates for `mgmt` can be united with a graph
that is created from native `mgmt` code in its mcl language. This is
useful when you are in the process of replacing Puppet with mgmt. You
can translate your custom modules into mgmt's language one by one,
and let mgmt run the current mix.
Instead of the usual `--puppet-conf` flag and argv for `puppet` and `mcl` input,
you need to use alternative flags to make this work:
* `--lp-lang` to specify the mcl input
* `--lp-puppet` to specify the puppet input
* `--lp-puppet-conf` to point to the optional puppet.conf file
`mgmt` will derive a graph that contains all edges and vertices from
both inputs. You essentially get two unrelated subgraphs that run in
parallel. To form edges between these subgraphs, you have to define
special vertices that will be merged. This works through a hard-coded
naming scheme.
### Mixed graph example 1 - No merges
```mcl
# lang
file "/tmp/mgmt_dir/" { state => "present" }
file "/tmp/mgmt_dir/a" { state => "present" }
```
```puppet
# puppet
file { "/tmp/puppet_dir": ensure => "directory" }
file { "/tmp/puppet_dir/a": ensure => "file" }
```
These very simple inputs (including implicit edges from directory to
respective file) result in two subgraphs that do not relate.
```
File[/tmp/mgmt_dir/] -> File[/tmp/mgmt_dir/a]
File[/tmp/puppet_dir] -> File[/tmp/puppet_dir/a]
```
### Mixed graph example 2 - Merged vertex
In order to have merged vertices in the resulting graph, you will
need to include special resources and classes in the respective
input code.
* On the lang side, add `noop` resources with names starting in `puppet_`.
* On the Puppet side, add **empty** classes with names starting in `mgmt_`.
```mcl
# lang
noop "puppet_handover_to_mgmt" {}
file "/tmp/mgmt_dir/" { state => "present" }
file "/tmp/mgmt_dir/a" { state => "present" }
Noop["puppet_handover_to_mgmt"] -> File["/tmp/mgmt_dir/"]
```
```puppet
# puppet
class mgmt_handover_to_mgmt {}
include mgmt_handover_to_mgmt
file { "/tmp/puppet_dir": ensure => "directory" }
file { "/tmp/puppet_dir/a": ensure => "file" }
File["/tmp/puppet_dir/a"] -> Class["mgmt_handover_to_mgmt"]
```
The new `noop` resource is merged with the new class, resulting in
the following graph:
```
File[/tmp/puppet_dir] -> File[/tmp/puppet_dir/a]
|
V
Noop[handover_to_mgmt]
|
V
File[/tmp/mgmt_dir/] -> File[/tmp/mgmt_dir/a]
```
You put all your ducks in a row, and the resources from the Puppet input
run before those from the mcl input.
**Note:** The names of the `noop` and the class must be identical after the
respective prefix. The common part (here, `handover_to_mgmt`) becomes the name
of the merged resource.
## Mixed graph example 3 - Multiple merges
In most scenarios, it will not be possible to define a single handover
point like in the previous example. For example, if some Puppet resources
need to run in between two stages of native resources, you need at least
two merged vertices:
```mcl
# lang
noop "puppet_handover" {}
noop "puppet_handback" {}
file "/tmp/mgmt_dir/" { state => "present" }
file "/tmp/mgmt_dir/a" { state => "present" }
file "/tmp/mgmt_dir/puppet_subtree/state-file" { state => "present" }
File["/tmp/mgmt_dir/"] -> Noop["puppet_handover"]
Noop["puppet_handback"] -> File["/tmp/mgmt_dir/puppet_subtree/state-file"]
```
```puppet
# puppet
class mgmt_handover {}
class mgmt_handback {}
include mgmt_handover, mgmt_handback
class important_stuff {
file { "/tmp/mgmt_dir/puppet_subtree":
ensure => "directory"
}
# ...
}
Class["mgmt_handover"] -> Class["important_stuff"] -> Class["mgmt_handback"]
```
The resulting graph looks roughly like this:
```
File[/tmp/mgmt_dir/] -> File[/tmp/mgmt_dir/a]
|
V
Noop[handover] -> ( class important_stuff resources )
|
V
Noop[handback]
|
V
File[/tmp/mgmt_dir/puppet_subtree/state-file]
```
You can add arbitrary numbers of merge pairs to your code bases,
with relationships as needed. From our limited experience, code
readability suffers quite a lot from these, however. We advise
to keep these structures simple.

7
docs/resource-guide.md
View File

@@ -60,10 +60,7 @@ it is not specified, but others cannot, and some might poorly infer if the
struct name is ambiguous.
If you'd like your resource to be accessible by the `YAML` graph API (GAPI),
then you'll need to include the appropriate YAML fields as shown below. This is
used by the `Puppet` compiler as well, so make sure you include these struct
tags if you want existing `Puppet` code to be able to run using the `mgmt`
engine.
then you'll need to include the appropriate YAML fields as shown below.
#### Example
@@ -623,7 +620,7 @@ func init() { // special golang method that runs once
To support YAML unmarshalling for your resource, you must implement an
additional method. It is recommended if you want to use your resource with the
`Puppet` compiler.
`yaml` compiler.
```golang
UnmarshalYAML(unmarshal func(interface{}) error) error // optional