From 4bd53d5ab025e131ea0587f2143eae6a80269833 Mon Sep 17 00:00:00 2001 From: Felix Frank Date: Mon, 12 Sep 2016 02:35:05 +0200 Subject: [PATCH] add puppet support documentation --- DOCUMENTATION.md | 28 ++++++++ Puppet.md | 163 +++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 191 insertions(+) create mode 100644 Puppet.md diff --git a/DOCUMENTATION.md b/DOCUMENTATION.md index 323a0763..64a39eb9 100644 --- a/DOCUMENTATION.md +++ b/DOCUMENTATION.md @@ -35,6 +35,7 @@ along with this program. If not, see . * [Autogrouping - Automatic resource grouping](#autogrouping) * [Automatic clustering - Automatic cluster management](#automatic-clustering) * [Remote mode - Remote "agent-less" execution](#remote-agent-less-mode) + * [Puppet support - write manifest code for mgmt](#puppet-support) 5. [Usage/FAQ - Notes on usage and frequently asked questions](#usage-and-frequently-asked-questions) 6. [Reference - Detailed reference](#reference) * [Graph definition file](#graph-definition-file) @@ -169,6 +170,33 @@ which need to exchange information that is only available at run time. An introductory blog post about this topic will follow soon. +###Puppet support + +You can supply a Puppet manifest instead of creating the (YAML) graph manually. +Puppet must be installed and in `mgmt`'s search path. You also need the +[ffrank-mgmtgraph Puppet module](https://forge.puppet.com/ffrank/mgmtgraph). + +Invoke `mgmt` with the `--puppet` switch, which supports 3 variants: + +1. Request the configuration from the Puppet Master (like `puppet agent` does) + + mgmt run --puppet agent + +2. Compile a local manifest file (like `puppet apply`) + + mgmt run --puppet /path/to/my/manifest.pp + +3. Compile an ad hoc manifest from the commandline (like `puppet apply -e`) + + mgmt run --puppet 'file { "/etc/ntp.conf": ensure => file }' + +For more details and caveats see [Puppet.md](Puppet.md). + +####Blog post + +An introductory post on the Puppet support is on +[Felix's blog](http://ffrank.github.io/features/2016/06/19/puppet-powered-mgmt/). + ##Usage and frequently asked questions (Send your questions as a patch to this FAQ! I'll review it, merge it, and respond by commit with the answer.) diff --git a/Puppet.md b/Puppet.md new file mode 100644 index 00000000..f8fb4286 --- /dev/null +++ b/Puppet.md @@ -0,0 +1,163 @@ +#mgmt Puppet support + +1. [Prerequisites](#prerequisites) + * [Testing the Puppet side](#testing-the-puppet-side) +2. [Writing a suitable manifest](#writing-a-suitable-manifest) + * [Unsupported attributes](#unsupported-attributes) + * [Unsupported resources](#unsupported-resources) + * [Avoiding common warnings](#avoiding-common-warnings) +3. [Configuring Puppet](#configuring-puppet) +4. [Caveats](#caveats) + +`mgmt` can use Puppet as its source for the configuration graph. +This document goes into detail on how this works, and lists +some pitfalls and limitations. + +For basic instructions on how to use the Puppet support, see +the [main documentation](DOCUMENTATION.md#puppet-support). + +##Prerequisites + +You need Puppet installed in your system. It is not important how you +get it. On the most common Linux distributions, you can use packages +from the OS maintainer, or upstream Puppet repositories. An alternative +that will also work on OSX is the `puppet` Ruby gem. It also has the +advantage that you can install any desired version in your home directory +or any other location. + +Any release of Puppet's 3.x and 4.x series should be suitable for use with +`mgmt`. Most importantly, make sure to install the `ffrank-mgmtgraph` Puppet +module (referred to below as "the translator module"). + + puppet module install ffrank-mgmtgraph + +Please note that the module is not required on your Puppet master (if you +use a master/agent setup). It's needed on the machine that runs `mgmt`. +You can install the module on the master anyway, so that it gets distributed +to your agents through Puppet's `pluginsync` mechanism. + +###Testing the Puppet side + +The following command should run successfully and print a YAML hash on your +terminal: + +```puppet +puppet mgmtgraph print --code 'file { "/tmp/mgmt-test": ensure => present }' +``` + +You can use this CLI to test any manifests before handing them straight +to `mgmt`. + +##Writing a suitable manifest + +###Unsupported attributes + +`mgmt` inherited its resource module from Puppet, so by and large, it's quite +possible to express `mgmt` graphs in terms of Puppet manifests. However, +there isn't (and likely never will be) full feature parity between the +respective resource types. In consequence, a manifest can have semantics that +cannot be transferred to `mgmt`. + +For example, at the time of writing this, the `file` type in `mgmt` had no +notion of permissions (the file `mode`) yet. This lead to the following +warning (among others that will be discussed below): + + $ puppet mgmtgraph print --code 'file { "/tmp/foo": mode => "0600" }' + Warning: cannot translate: File[/tmp/foo] { mode => "600" } (attribute is ignored) + +This is a heads-up for the user, because the resulting `mgmt` graph will +in fact not pass this information to the `/tmp/foo` file resource, and +`mgmt` will ignore this file's permissions. Including such attributes in +manifests that are written expressly for `mgmt` is not sensible and should +be avoided. + +###Unsupported resources + +Puppet has a fairly large number of +[built-in types](https://docs.puppet.com/puppet/latest/reference/type.html), +and countless more are available through +[modules](https://forge.puppet.com/). It's unlikely that all of them will +eventually receive native counterparts in `mgmt`. + +When encountering an unknown resource, the translator module will replace +it with an `exec` resource in its output. This resource will run the equivalent +of a `puppet resource` command to make Puppet apply the original resource +itself. This has quite abysmal performance, because processing such a +resource requires the forking of at least one Puppet process (two if it +is found to be out of sync). This comes with considerable overhead. +On most systems, starting up any Puppet command takes several seconds. +Compared to the split second that the actual work usually takes, +this overhead can amount to several orders of magnitude. + +Avoid Puppet types that `mgmt` does not implement (yet). + +###Avoiding common warnings + +Many resource parameters in Puppet take default values. For the most part, +the translator module just ignores them. However, there are cases in which +Puppet will default to convenient behavior that `mgmt` cannot quite replicate. +For example, translating a plain `file` resource will lead to a warning message: + + $ puppet mgmtgraph print --code 'file { "/tmp/mgmt-test": }' + Warning: File[/tmp/mgmt-test] uses the 'puppet' file bucket, which mgmt cannot do. There will be no backup copies! + +The reason is that per default, Puppet assumes the following parameter value +(among others) + +```puppet +file { "/tmp/mgmt-test": + backup => 'puppet', +} +``` + +To avoid this, specify the parameter explicitly: + + $ puppet mgmtgraph print --code 'file { "/tmp/mgmt-test": backup => false }' + +This is tedious in a more complex manifest. A good simplification is the +following [resource default](https://docs.puppet.com/puppet/latest/reference/lang_defaults.html) +anywhere on the top scope of your manifest: + +```puppet +File { backup => false } +``` + +If you encounter similar warnings from other types and/or parameters, +use the same approach to silence them if possible. + +##Configuring Puppet + +Since `mgmt` uses an actual Puppet CLI behind the scenes, you might +need to tweak some of Puppet's runtime options in order to make it +do what you want. Reasons for this could be among the following: + +* You use the `--puppet agent` variant and need to configure +`servername`, `certname` and other master/agent-related options. +* You don't want runtime information to end up in the `vardir` +that is used by your regular `puppet agent`. +* You install specific Puppet modules for `mgmt` in a non-standard +location. + +`mgmt` exposes only one Puppet option in order to allow you to +control all of them, through its `--puppet-conf` option. It allows +you to specify which `puppet.conf` file should be used during +translation. + + mgmt run --puppet /opt/my-manifest.pp --puppet-conf /etc/mgmt/puppet.conf + +Within this file, you can just specify any needed options in the +`[main]` section: + + [main] + server=mgmt-master.example.net + vardir=/var/lib/mgmt/puppet + +##Caveats + +Please see the [README](https://github.com/ffrank/puppet-mgmtgraph/blob/master/README.md) +of the translator module for the current state of supported and unsupported +language features. + +You should probably make sure to always use the latest release of +both `ffrank-mgmtgraph` and `ffrank-yamlresource` (the latter is +getting pulled in as a dependency of the former).