1b01f908e3
Sorry for the size of this patch, I was busy hacking and plumbing away and it got out of hand! I'm allowing this because there doesn't seem to be anyone hacking away on parts of the code that this would break, since the resource code is fairly stable in this change. In particular, it revisits and refreshes some areas of the code that didn't see anything new or innovative since the project first started. I've gotten rid of a lot of cruft, and in particular cleaned up some things that I didn't know how to do better before! Here's hoping I'll continue to learn and have more to improve upon in the future! (Well let's not hope _too_ hard though!) The logical goal of this patch was to make logical grouping of resources possible. For example, it might be more efficient to group three package installations into a single transaction, instead of having to run three separate transactions. This is because a package installation typically has an initial one-time per run cost which shouldn't need to be repeated. Another future goal would be to group file resources sharing a common base path under a common recursive fanotify watcher. Since this depends on fanotify capabilities first, this hasn't been implemented yet, but could be a useful method of reducing the number of separate watches needed, since there is a finite limit. It's worth mentioning that grouping resources typically _reduces_ the parallel execution capability of a particular graph, but depending on the cost/benefit tradeoff, this might be preferential. I'd submit it's almost universally beneficial for pkg resources. This monster patch includes: * the autogroup feature * the grouping interface * a placeholder algorithm * an extensive test case infrastructure to test grouping algorithms * a move of some base resource methods into pgraph refactoring * some config/compile clean ups to remove code duplication * b64 encoding/decoding improvements * a rename of the yaml "res" entries to "kind" (more logical) * some docs * small fixes * and more!
8.6 KiB
#mgmt
##mgmt by James ####Available from: ####https://github.com/purpleidea/mgmt/
####This documentation is available in: Markdown or PDF format.
####Table of Contents
- Overview
- Project description - What the project does
- Setup - Getting started with mgmt
- Features - All things mgmt can do
- Usage/FAQ - Notes on usage and frequently asked questions
- Reference - Detailed reference
- Examples - Example configurations
- Development - Background on module development and reporting bugs
- Authors - Authors and contact information
##Overview
The mgmt tool is a research prototype to demonstrate next generation config
management techniques. Hopefully it will evolve into a useful, robust tool.
##Project Description
The mgmt tool is a distributed, event driven, config management tool, that supports parallel execution, and librarification to be used as the management foundation in and for, new and existing software.
For more information, you may like to read some blog posts from the author:
There is also an introductory video available.
##Setup
During this prototype phase, the tool can be run out of the source directory.
You'll probably want to use ./run.sh run --file examples/graph1.yaml to
get started. Beware that this can cause data loss. Understand what you're
doing first, or perform these actions in a virtual environment such as the one
provided by Oh-My-Vagrant.
##Features
This section details the numerous features of mgmt and some caveats you might need to be aware of.
###Autoedges
Automatic edges, or AutoEdges, is the mechanism in mgmt by which it will automatically create dependencies for you between resources. For example, since mgmt can discover which files are installed by a package it will automatically ensure that any file resource you declare that matches a file installed by your package resource will only be processed after the package is installed.
####Controlling autodeges
Though autoedges is likely to be very helpful and avoid you having to declare all dependencies explicitly, there are cases where this behaviour is undesirable.
Some distributions allow package installations to automatically start the
service they ship. This can be problematic in the case of packages like MySQL
as there are configuration options that need to be set before MySQL is ever
started for the first time (or you'll need to wipe the data directory). In
order to handle this situation you can disable autoedges per resource and
explicitly declare that you want my.cnf to be written to disk before the
installation of the mysql-server package.
You can disable autoedges for a resource by setting the autoedge key on
the meta attributes of that resource to false.
###Autogrouping
Automatic grouping or AutoGroup is the mechanism in mgmt by which it will automatically group multiple resource vertices into a single one. This is particularly useful for grouping multiple package resources into a single resource, since the multiple installations can happen together in a single transaction, which saves a lot of time because package resources typically have a large fixed cost to running (downloading and verifying the package repo) and if they are grouped they share this fixed cost. This grouping feature can be used for other use cases too.
You can disable autogrouping for a resource by setting the autogroup key on
the meta attributes of that resource to false.
##Usage and frequently asked questions (Send your questions as a patch to this FAQ! I'll review it, merge it, and respond by commit with the answer.)
###Why did you start this project?
I wanted a next generation config management solution that didn't have all of the design flaws or limitations that the current generation of tools do, and no tool existed!
###Why did you use etcd? What about consul?
Etcd and consul are both written in golang, which made them the top two contenders for my prototype. Ultimately a choice had to be made, and etcd was chosen, but it was also somewhat arbitrary. If there is available interest, good reasoning, and patches, then we would consider either switching or supporting both, but this is not a high priority at this time.
###You didn't answer my question, or I have a question!
It's best to ask on IRC to see if someone can help you. Once we get a big enough community going, we'll add a mailing list. If you don't get any response from the above, you can contact me through my technical blog and I'll do my best to help. If you have a good question, please add it as a patch to this documentation. I'll merge your question, and add a patch with the answer!
##Reference Please note that there are a number of undocumented options. For more information on these options, please view the source at: https://github.com/purpleidea/mgmt/. If you feel that a well used option needs documenting here, please patch it!
###Overview of reference
- Graph definition file: Main graph definition file.
- Command line: Command line parameters.
###Graph definition file graph.yaml is the compiled graph definition file. The format is currently undocumented, but by looking through the examples/ you can probably figure out most of it, as it's fairly intuitive.
###Command line
The main interface to the mgmt tool is the command line. For the most recent
documentation, please run mgmt --help.
####--file <graph.yaml>
Point to a graph file to run.
####--converged-timeout <seconds>
Exit if the machine has converged for approximately this many seconds.
####--max-runtime <seconds>
Exit when the agent has run for approximately this many seconds. This is not
generally recommended, but may be useful for users who know what they're doing.
##Examples For example configurations, please consult the examples/ directory in the git source repository. It is available from:
https://github.com/purpleidea/mgmt/tree/master/examples
##Development
This is a project that I started in my free time in 2013. Development is driven by all of our collective patches! Dive right in, and start hacking! Please contact me if you'd like to invite me to speak about this at your event.
You can follow along on my technical blog.
To report any bugs, please file a ticket at: https://github.com/purpleidea/mgmt/issues.
##Authors
Copyright (C) 2013-2016+ James Shubin and the project contributors
Please see the AUTHORS file for more information.