432 Commits

Author SHA1 Message Date
James Shubin
9546949945 git: Improve tagging script 2017-03-11 08:29:53 -05:00
James Shubin
8ff048d055 test: Disable prometheus-3.sh test temporarily
It seems to be failing, and I'm not sure where the regression is, or if
there is a race. Sorry roidelapluie.
2017-03-09 11:46:21 -05:00
James Shubin
95a1c6e7fb pgraph, resources: Discard BackPokes during pause and resume
This prevents some nasty races where a BackPoke could arrive on a paused
vertex either during a resume or pause operation. Previously we might
also have poked an excessive number of resources on resume.

The solution was to discard BackPokes during pause or resume. On pause,
they can be discarded because we've asked the graph to quiesce, and any
further work can be done on resume, and on resume we ignore them because
this should only happen during the unrolling (reverse topological resume
of the graph) and at the end of this the indegree == 0 vertices will
initiate a series of pokes which should deal with any BackPoke that was
possibly discarded.

One other aspect of this which is important: if an indegree == 0 vertex
is poked (Process runs) but it's already in the correct state, it should
still transmit the Poke through itself so that subsequent vertices know
to run. Currently this is done correctly in Process().

I'm a bit ashamed that this wasn't done properly in the engine earlier,
but I suppose that's what comes out of running fancier graphs and really
thinking in detail about what's truly correct. Hopefully I got it right
this time!
2017-03-09 06:35:15 -05:00
James Shubin
0b1a4a0f30 pgraph, resources: Quiesce when pausing or exiting the resource
This prevents a nasty race that can happen in a graph with more than one
resource. If a resource has someone that it can BackPoke, and then
suppose an event comes in. It runs the obj.Event() method (from inside
its Watch loop) and then *before* the resulting Process method can run
it receives a pause event and pauses. Then the parent resource pauses as
well. Finally (it's a race) the Process gets around to running, and
decides it needs to BackPoke. At this point since the parent resource is
paused, it receives the BackPoke at a time when it can't handle
receiving one, and it panics!

As a result, we now track the number of running Process possibilities
via a WaitGroup which gets incremented from the obj.Event() and we don't
finish our pause or exit operations until it has quiesced and our
WaitGroup lets us know via Wait(). Lastly in order to prevent repeated
replays, we detect when we're quiescing and suspend replaying until post
pause. We don't need to save the replay (playback variable) explicitly
because its state remains during pause, and on exit it would get
re-checked anyways.
2017-03-09 02:50:55 -05:00
James Shubin
22b48e296a resources, yamlgraph: Drop the kind capitalization
This stopped making sense now that we have a resource with two primary
capitals. It was just a silly formatting hack anyways. Welcome kv!
2017-03-09 02:50:55 -05:00
James Shubin
c696ebf53c resources: svc: Add failed state
Services can be in a failed state too.
2017-03-08 19:23:33 -05:00
James Shubin
a0686b7d2b pgraph: graphviz: Update Graphviz lib to quote names properly
This also moves the library to after the graph starts so that the kind
fields will be visible.
2017-03-08 19:23:33 -05:00
James Shubin
8d94be8924 resources: kv: Add new KV resource which sets key value pairs
This is a new resource for setting key value pairs in our global world
database. Currently only etcd is supported. Some of the implications and
possibilities of this resource will become more obvious with future
commits!

You can bother/test this resource with these commands:

ETCDCTL_API=3 etcdctl get "/_mgmt/strings/" --prefix=true
ETCDCTL_API=3 etcdctl put "/_mgmt/strings/KEY/HOSTNAME" 42

Replace the KEY and HOSTNAME variables with the actual values you'd like
to use. The 42 is the value that is set.
2017-03-08 19:23:33 -05:00
James Shubin
e97ac5033f resources: Split util functions into separate file
This also adds errwrap to their implementation.
2017-03-08 19:23:33 -05:00
James Shubin
44771a0049 gapi: Move the World interface into resources
This was necessary to fix some "import cycle" errors I was having when
adding the World api to the resource Data struct.

I think this is a good hint that I need to start splitting up existing
packages into sub files, and cleaning up and inter-package problems too.
2017-03-08 19:23:33 -05:00
James Shubin
32aae8f57a lib, pgraph, resources: Refactor data association API
This should make things cleaner and help avoid as much churn every time
we change a property.
2017-03-07 22:51:11 -05:00
James Shubin
8207e23cd9 lib: Refactor instantiation of world API 2017-03-07 22:51:11 -05:00
James Shubin
a469029698 etcd: Switch to buffered channel to remove duplicates
Since we don't return the actual values and instead only tell about
events (which leaves the `Get` of the value as a second operation) then
we don't have to use a channel with backpressure since all the events
are identical.
2017-03-07 22:51:11 -05:00
James Shubin
203d866643 gapi, etcd: Define and implement a string sharing API for the World
This adds a new set of methods to the World API (for sharing data
throughout the cluster) and adds an etcd backed implementation.
2017-03-07 22:51:11 -05:00
Michael Borden
1488e5ec4d travis: Add go_import_path
This should fix an issue with turning on travis for any mgmt fork.
2017-03-07 14:17:53 -08:00
Michael Borden
af66138a17 misc: Add pacman support 2017-03-03 21:04:29 -08:00
James Shubin
5f060d60a7 test: Avoid matching three X's
This helps my "WIP" detector script avoid false positives. It is a
simple script which helps me find release critical problems.
2017-03-01 22:37:08 -05:00
James Shubin
73ccbb69ea travis: Ensure recent golang versions in test
This ensures travis picks the latest versions. Apparently writing 1.x is
also a valid choice.
2017-03-01 21:45:02 -05:00
James Shubin
be60440b20 readme: Add new blog post about metaparameters
This also documents the recent semaphore work.
2017-03-01 16:18:14 -05:00
James Shubin
837efb78e6 spelling: Fix typos as found by goreportcard 2017-02-28 23:48:34 -05:00
James Shubin
4a62a290d8 pgraph: Clean up tests
This splits the tests into multiple files.
2017-02-28 16:47:16 -05:00
James Shubin
018399cb1f semaphore, pgraph: Add semaphore grouping and tests
If two resources are grouped, then the result should contain the
semaphores of both resources. This is because the user is expecting
(independently) resource A and resource B to have a limiting choke
point. If when combined those choke points aren't preserved, then we
have broken an important promise to the user.
2017-02-28 16:40:53 -05:00
James Shubin
646a576358 remote: Replace builtin semaphore type with common util lib
Refactor code to use the new fancy lib!
2017-02-27 02:57:06 -05:00
James Shubin
d8e19cd79a semaphore: Create a semaphore metaparam
This adds a P/V style semaphore mechanism to the resource graph. This
enables the user to specify a number of "id:count" tags associated with
each resource which will reduce the parallelism of the CheckApply
operation to that maximum count.

This is particularly interesting because (assuming I'm not mistaken) the
implementation is dead-lock free assuming that no individual resource
permanently ever blocks during execution! I don't have a formal proof of
this, but I was able to convince myself on paper that it was the case.

An actual proof that N P/V counting semaphores in a DAG won't ever
dead-lock would be particularly welcome! Hint: the trick is to acquire
them in alphabetical order while respecting the DAG flow. Disclaimer,
this assumes that the lock count is always > 0 of course.
2017-02-27 02:57:06 -05:00
James Shubin
757cb0cf23 test: Small fixups to t4 and a rename 2017-02-26 20:54:22 -05:00
Julien Pivotto
7d92ab335a prometheus: Add mgmt_pgraph_start_time_seconds metric
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2017-02-26 15:28:43 +01:00
James Shubin
46c6d6f656 compiling: Add -i flag to go build to speed up builds
So builds take about 30s on my shitty machine which is pretty long.
Turns out golang caches things in $GOPATH/pkg/ but is not clever enough
to erase things from there that are out of date. As a result, it was
rebuilding everything (including the unchanged dependencies) every
build!

By completely wiping out $GOPATH/pkg/ and then running `go build -i`,
this now takes builds down to about 8 seconds. (After one full build is
finished.)

This is basically the same as running `go install`, but without copying
junk to $GOPATH/bin.

Hopefully the tooling will be smart enough to know when to throw out
stuff in $GOPATH/pkg automatically and avoid this problem entirely.

Is it wrong to send Google a bill for all the extra cpu cycles I've
used? ;)
2017-02-26 04:06:36 -05:00
Julien Pivotto
46260749c1 prometheus: Move the prometheus nil check inside the prometheus function
That pattern will be reused in future metrics.

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2017-02-26 09:33:34 +01:00
Julien Pivotto
50664fe115 compilation: Make build the default target
It is really strange that whe I run make, it does not build mgmt. This
commit makes build the default target, without moving the target,
therefore we keep as much as we can the order of the file.

This also removes the confusion for designers that would run "make"
instead of "make art", whose work would be disrupted when we add a --
let's say -- make alpharelese command.

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2017-02-26 09:04:20 +01:00
James Shubin
c480bd94db resources: virt: Remove unnecessary early exit from CheckApply
I don't think this early exit is necessary any more, since the main
CheckApply function really just spawns out to the different sub workers
which all individually check the apply variable.

If I'm wrong, we can revert this. It was @roidelapluie that noticed the
check here to begin with.
2017-02-25 21:29:08 -05:00
James Shubin
79923a939b resources: virt: Catch bad calls to CheckApply
If the engine cheats, we'll know!
2017-02-25 21:28:35 -05:00
James Shubin
327b22113a resources: virt: Don't block exit in callbacks
This prevents us blocking an exit if we close when a callback was about
to run. This is because the callbacks are called from the
EventRunDefaultImpl method, which waits for their return to exit and
release the WaitGroup.

I think we should probably get rid of the obj.wg since the engine is
supposed to guarantee that Close doesn't happen before Watch finishes.
2017-02-25 21:04:36 -05:00
James Shubin
12160ab539 pgraph: Wait for Process routine to exit
Wait for the innerWorker's Process routine to exit, before we exit too.
2017-02-25 21:01:02 -05:00
James Shubin
2462ea0892 pgraph, resources: Wait for innerWorker to exit cleanly
Don't run the Close() method until the innerWorker has exited cleanly.
This is a guarantee which we make to the resources.
2017-02-25 21:00:38 -05:00
James Shubin
8be09eadd4 test: Fix up probable timeout failures due to slow ci
We had *occasional* failures most likely due to slow ci and compounded
by low entropy. We also weren't pointing at the right test!
2017-02-22 23:08:09 -05:00
James Shubin
98bc96c911 golint: Fixup issues found in the report
This also increases the max allowed to 5% -- I'm happy to make this
lower if someone asks.
2017-02-22 22:18:55 -05:00
James Shubin
b0fce6a80d travis: Start building golang 1.8 as well
Require success from 1.7 since we'll probably move to it as the default
within six months.
2017-02-22 22:18:55 -05:00
James Shubin
53b8a21d1e resources: virt: Cleanup cleanly on Close
Don't block accidentally on error!
2017-02-22 22:18:55 -05:00
James Shubin
1346492d72 yamlgraph: Close the recwatcher properly after use
Don't leave it running unnecessarily! This might have contributed to a
block, but it was hard to isolate if this was the cause or if this was
one of many causes.
2017-02-22 18:37:43 -05:00
James Shubin
e5bb8d7992 recwatch: Close the ConfigWatch properly
This improves the shutdown process so that there is no change of
blocking if the sender runs close without having emptied the channel.
2017-02-22 17:45:16 -05:00
James Shubin
49594b8435 pgraph, resources: Clean up the event system around the resources
This cleans up some of the resource events and also reorganizes the
struct for simplicity. This should hopefully kill off at least one race
which would cause unnecessary blocking!

Yes this patch is a bit yucky, but so was the bug I was fighting with!
2017-02-22 17:45:16 -05:00
James Shubin
3bd37a7906 test: Don't block on graph transitions
Improvements in the engine have uncovered some annoying race conditions
which would cause the engine to block between transitions. This is a
test which catches the most obvious file based ones.

This requires inotify to work in the test environment.
2017-02-22 17:45:16 -05:00
James Shubin
e070a85ae0 lib: Misc cleanups and new log message 2017-02-22 17:45:16 -05:00
James Shubin
c189278e24 recwatch: Unblock from sending on exit
If we receive an exit signal while we are waiting to send a message, we
should still be able to shut down.
2017-02-22 16:41:47 -05:00
James Shubin
2a8606bd98 recwatch: Close cleanly after Watcher finishes
This cleans up the recwatch code to avoid some possible races. While you
were previously able to call Close more than once, this was really
something that would mask bugs, rather than a useful feature.
2017-02-22 16:41:26 -05:00
James Shubin
18ea05c837 pgraph, resources: Add proper start/stop signals
We need to perform some operations in lock step between graph
transitions. This should help with that!
2017-02-21 18:48:27 -05:00
James Shubin
86c3072515 resources: The user should not call Init
Even in tests...
2017-02-21 18:42:07 -05:00
James Shubin
fccf508dde resources, pgraph: Refactor Worker and simplify API
I'm still working on reducing the size of the monster patches that I
land, but I'm exercising the priviledge as the initial author. In any
case, this refactors worker into two, and cleans up the passing around
of the processChan. This puts common code into Init and Close.
2017-02-21 18:42:07 -05:00
James Shubin
2da21f90f4 pgraph, resources: Improve Init/Close and Worker status
This should do some rough cleanups around the Init/Close of resources,
and tracking of Worker function status.
2017-02-21 18:42:07 -05:00
James Shubin
bec7f1726f resources: virt: Allow hotplugging
This allows hot (un)plugging of CPU's! It also includes some general
cleanups which were necessary to support this as well as some other
features to the virt resource. Hotunplug requires Fedora 25.

It also comes with a mini shell script to help demo this capability.

Many thanks to pkrempa for his help with the libvirt API!
2017-02-21 18:42:07 -05:00
James Shubin
74dfb9d88d test: Make test status more clear 2017-02-21 18:40:31 -05:00
James Shubin
02dddfc227 test: Fix yamlfmt test
Last chance before we kill this entirely.
2017-02-21 16:16:41 -05:00
Julien Pivotto
545016b38f project: Add $me to AUTHORS
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2017-02-21 07:07:41 +01:00
Felix Frank
0ccceaf226 project: Reluctantly add myself as an author
For the record, I currently don't feel that my level of contributions
warrant a spot here, but I'm entering myself upon express invitation
by James =) ...also in anticipation of much more substantial work.
2017-02-20 22:53:30 -05:00
James Shubin
a601115650 test: Fix false negative on go vet
This was my fault, now it is fixed :) It passed locally due to a bug.
2017-02-20 18:12:30 -05:00
Kaushal M
ae6267c906 build: Use GOTAGS for static builds as well 2017-02-20 17:16:03 -05:00
James Shubin
ac142694f5 test: Improve go vet so that it is less noisy 2017-02-20 17:08:48 -05:00
James Shubin
69b0913315 test: Fix tests by hooking up go test properly
The internal golang tests broke when we turned everything into packages.
This resurrects them with the hopes that we'll add more!
2017-02-20 16:40:40 -05:00
Julien Pivotto
421bacd7dc travis: Run apt update before installing dependencies
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2017-02-20 22:23:06 +01:00
James Shubin
573a76eedb build: Require golang 1.6 or greater 2017-02-20 15:17:49 -05:00
James Shubin
b7948c7f40 resources: Specify defaults for the MetaParams
When creating new resources, we didn't specify the defaults, which for
the limit metaparam caused invalid resources by default. It would be
nice to change the limit param to have the 1/X (reciprocal) as the
default, although the problem with that is that (1) it is illogical, and
(2) it's not clear if the precision for the common cases is enough.

If someone wants to investigate this further, please do! Zero value
structs are definitely more useful! In any case, we can now specify the
default. It's not entirely obvious to me if this is the best way to do
it, or if there is a superior method.
2017-02-16 21:08:46 -05:00
James Shubin
2647d09b8f resources: file: Don't modify resource in Init
This didn't break anything previously, but technically wasn't correct.
Pure functions are superior in this case!
2017-02-16 21:06:58 -05:00
James Shubin
57e919d7e5 resources: Remove "NewRes" constructors
Remove the New constructors since calling Init should be done by the
engine, and not by the user even when using mgmt as a lib. This is also
the case in tests! It used to be the case that a user might want to call
Init manually, but that is no longer the case!
2017-02-16 21:06:12 -05:00
James Shubin
f456aa1407 resources: file: Small fixups and force additions 2017-02-16 20:46:51 -05:00
Mildred Ki'Lya
d0d62892c8 resources: file: Allow creation of empty directories 2017-02-16 20:41:59 -05:00
James Shubin
a981cfa053 legal: Oh yeah, it is 2017 2017-02-16 01:34:32 -05:00
James Shubin
55290dd1e3 docs: Add missing newline and remove extra one 2017-02-16 01:13:42 -05:00
Julien Pivotto
9c4e255994 documentation: Implement sphinx documentation
Licence removed due to to a read the docs (or sphinx/recommonmark) bug:
everything after the comment is not rendered.

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2017-02-16 07:08:44 +01:00
Julien Pivotto
f9c7d5f7bc resources: augeas: comments: Improvements
- Remove obvious statements
- Fix typo

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2017-02-15 22:42:42 +01:00
Julien Pivotto
49baea5f6a documentation: Embed command-line syntax in a code block
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2017-02-15 15:30:32 -05:00
James Shubin
6209cf3933 resources: file: Use the computed path in our resource
We compute the actual path in Init() and forget to use it everywhere.
2017-02-15 15:25:03 -05:00
Mildred Ki'Lya
d170a523c3 misc: Split OPTS variable of systemd unit
Systemd split variables when specified like $VAR and not when specified
like ${VAR}. Since OPTS must contain multiple options, we must ensure
systemd will split it.

See also systemd.service(5)
2017-02-15 19:31:44 +01:00
Julien Pivotto
be5040e7a8 prometheus: Remove mgmt_process_start_time_seconds metric
That metric is useless as by default the prometheus golang client
provides the `process_start_time_seconds` metric.

This reverts commit 25e2af7c89.
2017-02-14 22:56:12 +01:00
James Shubin
ecbaa5bfc1 github: Show a friendly message in issues template 2017-02-14 16:45:35 -05:00
Julien Pivotto
25e2af7c89 prometheus: Add mgmt_process_start_time_seconds metric 2017-02-14 22:14:59 +01:00
Julien Pivotto
605688426d resources: file: Do not error on os.Stat in noop mode
Fix #142.

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2017-02-14 20:54:32 +01:00
James Shubin
0e069f1e75 docs: Update the quick start guide with git clone and vagrant
There was a bit of a catch-22, since go get wouldn't succeed without
some system build libs installed first. We also added a Vagrant
environment.
2017-02-14 12:13:43 -05:00
James Shubin
e9adbf18d3 test: prometheus: Fix up test 2017-02-14 12:10:54 -05:00
Sean Jones
610202097a test: Enable macOS shell testing
* Check and install libvirt with Homebrew

  macOS does not have apt, dnf or yum. Add checking for homebrew for
  installing libvirt.

* Use platform timeout for tests
    * Add timeout detection to test/util.sh
    * Use $timeout for shell test requiring timeout
2017-02-14 11:59:44 -05:00
Mildred Ki'Lya
8c2c552164 resources: file: Implement file attributes
Add owner which must be username or uid of the file owner, group which is
the group name or gid of the file, and mode which is the octal unix file
permissions.

Add separate implementation for Go 1.6 and lower.
2017-02-14 11:55:00 -05:00
Julien Pivotto
b9976cf693 documentation: prometheus: Improvements
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2017-02-13 21:59:05 +01:00
Julien Pivotto
3261c405bd resources: augeas: Make augeas support optional
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2017-02-13 07:14:14 +01:00
James Shubin
35d3328e3e etcd: Remove stuttering in package
This is a good first step to cleaning up the package.
2017-02-12 22:51:46 -05:00
James Shubin
e96041d76f resources: augeas: Turn augeas namespace into a constant 2017-02-12 19:46:16 -05:00
Felix Frank
c2034bc0c0 vagrant: Add Vagrantfile and some basic configuration 2017-02-12 19:18:33 -05:00
Julien Pivotto
e8855f7621 prometheus: Implement mgmt_checkapply_total metric
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2017-02-12 23:45:47 +01:00
Julien Pivotto
bdb8368e89 resources: augeas: New resource
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2017-02-12 23:02:12 +01:00
Julien Pivotto
f160db2032 compilation: virt: Make libvirt support optional
refs #114

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2017-02-12 22:09:46 +01:00
Julien Pivotto
de9a32a273 recwatch: Remove watcher on file move
Fix #120

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2017-02-11 13:45:32 +01:00
Steve Milner
6ba7422c3b spec: Fix rpmlint warnings
- Switched spaces to tabs
- Added quiet switch to setup
2017-02-10 11:55:58 -05:00
Julien Pivotto
5cbb0ceb80 test: commit: Improve commit message testing
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2017-02-10 11:12:26 +01:00
James Shubin
5b29358b37 test: Small nitpicks with messages 2017-02-09 11:16:18 -05:00
Julien Pivotto
90147f3dfb travis: more strict commit messages tests
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2017-02-09 16:33:06 +01:00
Julien Pivotto
72873abe05 test: file: test the behaviour of inotify on parent dir moves
This is a test for #124. It is disabled until #124 is fixed, so it can
already me merged.

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2017-02-09 16:01:09 +01:00
Julien Pivotto
de1810ba68 travis: add a test regarding commit messages
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2017-02-09 16:01:09 +01:00
Julien Pivotto
7b7c765d78 prometheus: Add a new test, with --prometheus-listen
Also: rename t9 to prometheus-1

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2017-02-09 00:05:19 +01:00
Felix Frank
806d4660cf tests: simplify shell code, skip YAML test if Ruby is too old 2017-02-08 09:29:00 -05:00
Julien Pivotto
5ae5d364bb Prometheus: fix URL to the "Default port allocations"
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2017-02-08 13:41:09 +01:00
Julien Pivotto
1af67e72d4 prometheus: Implement basic Prometheus support
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2017-02-08 12:13:33 +01:00
James Shubin
ed268ad683 pgraph, yamlgraph: Allow specifying notify value from YAML 2017-02-06 16:29:02 -05:00
James Shubin
5bdd2ca02f examples: Update the examples 2017-02-06 16:28:16 -05:00
Daniele Sluijters
eb59861d4d TODO: Point to current file improvement issue 2017-02-06 13:31:26 +01:00
Tom Ritserveldt
427e46a2aa point to the correct puppet-guide path in docs. 2017-02-06 10:35:39 +01:00
James Shubin
68a8649292 resources: Parse YAML infinity specifications correctly
This makes it easier to specify an infinite rate.
2017-02-05 21:01:52 -05:00
James Shubin
2aff8709a5 gapi: Unblock from a waiting send on GAPI close
There was a race condition that would sometimes occur in that if we
stopped reading from the gapiChan (on shutdown) but then a new message
was available before we managed to close the GAPI, then we would wait
forever to finish the close because the channel never sent, and the
WaitGroup wouldn't let us exit.

This fixes this horrible, horrible race.
2017-02-05 21:01:52 -05:00
James Shubin
62c3add888 lib: Make initial one-shot start pattern more elegant
This should do the same thing, in a more elegant and efficient way.
2017-02-05 19:00:14 -05:00
James Shubin
3ac878db62 recwatch: Do not send to channel if closed.
If we close the recwatcher right after it's opened, we might have
previously tried to send on the now closed channel.
2017-02-05 18:57:08 -05:00
James Shubin
c247cd8fea resources: Don't double close on Running restart
If we use the "retry" metaparam on a Watch, we want to avoid a double
close due to the second Running() signal. This avoids this with a simple
flag.
2017-02-05 18:47:40 -05:00
James Shubin
b6772b7280 examples: etcd: Simplify the etcd examples 2017-02-03 19:52:29 -05:00
James Shubin
807a3df9d1 travis: Limit notifications now that travis cron is enabled
This should hopefully cause less noise!
2017-02-01 08:45:57 -05:00
James Shubin
491d60e267 examples: Have libmgmt work around the lack of rate defaults
The rate limit lib needs a proper default. Until that's the case,
specify this so it doesn't block the resources.
2017-01-27 18:41:09 -05:00
James Shubin
4811eafd67 etcd: Update example to ignore pgp
This makes demos go faster since pgp is wip anyways.
2017-01-27 18:24:47 -05:00
James Shubin
8dedbb9620 examples: Update example to be Validate safe
This is also safer too!
2017-01-27 18:24:05 -05:00
James Shubin
dd8454161f pgraph: Set the false starter value too
We might have left re-used nodes as true, even if they no longer were
anymore due to graph changes, which would have caused additional pokes.
2017-01-27 17:43:24 -05:00
James Shubin
9421f2cddd resources: Rename GetUIDs to UIDs
This is more in line with the style guide for golang.
2017-01-25 14:51:23 -05:00
James Shubin
d8c4f78ec1 virt: Allow the use of ~ to expand to home directory
This makes examples slightly nicer to commit, since you don't have to
have a hardcoded ~/james/ in their source value. It's also probably a
useful feature for the resource.
2017-01-25 13:06:28 -05:00
James Shubin
54296da647 converger: Remove converger boilerplate from the resources
This simplifies the resource code by now removing all the converger
related material. Happy resource writing!
2017-01-25 11:30:47 -05:00
James Shubin
357102fdb5 converger: Block converging in the engine
This more appropriately blocks converging in the engine, since we are
now 1-1 decoupled from the Watch resource. This simplifies resource
writing, and should be more accurate around small converged timeouts.

We don't block in the Worker routine when we are polling, because we
expect to get constant poll events, and we can instead be more careful
about these by looking at CheckApply results.

If we can do this for all resources in the future, it would be
excellent!
2017-01-25 11:06:02 -05:00
James Shubin
7e15a9e181 pgraph: Add debug messages
These two messages in particular make graph analysis easier.
2017-01-25 09:52:34 -05:00
James Shubin
12e0b2d6f7 pgraph: Parallelize the BackPoke facility
I can't guarantee this has a significant effect, but it's likely to add
some efficiency when sending multiple BackPoke's at the same time, so
that erroneous ones can be cancelled out easier.
2017-01-25 09:13:59 -05:00
James Shubin
11b40bf32f resources: Update state checks
The mgmt graph depends on state tracking to eliminate redundant pokes.
With the Watch loop now able to produce events quickly, it should no
longer play a part in determining the vertex state. This simplifies the
resource API as well!
2017-01-25 09:13:59 -05:00
James Shubin
8d2b53373f pgraph: Remove unnecessary indentation in Process code path
We can indent the simple BackPoke code path instead!
2017-01-25 09:13:59 -05:00
James Shubin
9ecc49e592 resources: Force a sane default for zero value limiting
The default UnmarshalYAML on *BaseRes doesn't work properly at the
moment, so hack in a default so that we don't need to specify one if the
MetaParams struct isn't specified. The problem is that if there isn't a
meta value added, its UnmarshalYAML doesn't get a chance to run.
2017-01-25 09:13:59 -05:00
James Shubin
4f34f7083b resources: rate limiting: Implement resource rate limiting
This adds rate limiting with the limit and burst meta parameters. The
limits apply to how often the Process check is called. As a result, it
might get called more often than there are Watch events due to possible
Poke/BackPoke events.

This system might need to get rethought in the future depending on its
usefulness.
2017-01-25 09:13:59 -05:00
James Shubin
2a6df875ec resources: Improve composition of Validate API in resources
This now appropriately calls the Base method.
2017-01-22 06:00:37 -05:00
James Shubin
51c83116a2 resources: Overhaul legacy code around the resource API
This patch makes a number of changes in the engine surrounding the
resource API. In particular:

* Cleanup of send/read event.
* Cleanup of DoSend (now Event) in the Watch method.
* Events are now more consistently pointers.
* Exiting within Watch is now done in a single place.
* Multiple incoming events will be combined into a single action.
* Events in flight during an action are played back after CheckApply.
* Addition of Close method to API

This gets things ready for rate limiting and semaphore metaparams!
2017-01-22 05:59:15 -05:00
James Shubin
74435aac76 docs: Small fixes to the README 2017-01-22 05:47:42 -05:00
goberghen
5dfdb5b5f9 docs: Fix headers formating 2017-01-22 16:05:27 +06:00
James Shubin
ac892a3f3d docs: Add missing newline 2017-01-16 11:15:59 -05:00
James Shubin
1a2e99f559 docs: Add new blog post link 2017-01-16 11:14:19 -05:00
James Shubin
e97bba0524 docs: Add an introductory paragraph to the quick start guide 2017-01-16 11:12:07 -05:00
James Shubin
0538f0c524 docs: Improve README 2017-01-16 11:01:27 -05:00
James Shubin
fc3e35868d docs: Rename puppet guide and small cleanups 2017-01-16 10:27:17 -05:00
James Shubin
f1e0cfea1c docs: Split out a separate quick start guide 2017-01-16 10:27:17 -05:00
James Shubin
56efef69ba resources: Officially add Validate method
This officially adds the Validate method to the resource API, and also
cleans up the ordering in existing resources.
2017-01-09 05:10:26 -05:00
James Shubin
668ec8a248 docs: Add missing headings to table of contents 2017-01-09 04:35:48 -05:00
James Shubin
60912bd01c resources: Add a Default method to the resource API
This provides sensible defaults for when they're not the zero value.
2017-01-09 04:35:48 -05:00
Daniel P. Berrange
0b416e44f8 resources: virt: convert to use github.com/libvirt/libvirt-go
Convert the code to use the new libvirt Go bindings.

Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
2017-01-05 04:55:51 -05:00
James Shubin
ecc4aa09d3 resources: timer: Polish the timer resource
We should probably use this more correct data type which now matches
what is used by the Poll metaparam.
2016-12-24 00:51:39 -05:00
James Shubin
b921aabbed resources: Add poll metaparameter
This allows a resource to use polling instead of the event based
mechanism. This isn't recommended, but it could be useful, and it was
certainly fun to code!
2016-12-24 00:51:39 -05:00
James Shubin
6ad8ac0b6b converger: Wait till the timer exits
Wait till the timer loop exits, otherwise we might race and panic if
ConvergedTimer gets called after we've asked for an exit, but before it
does.
2016-12-24 00:51:39 -05:00
James Shubin
44e7e0e970 resources: virt: Add missing converger call
This was previously left out accidentally I suppose.
2016-12-23 23:33:13 -05:00
James Shubin
45820b4ce3 resources: Rename Converger to ConvergerUID
This is more correct since we need the Converger method to return the
actual converger!
2016-12-23 23:32:05 -05:00
Lars Kulseng
3a098377cb readme: Changed wording of quick start section 2016-12-23 23:29:27 +01:00
James Shubin
35875485ee todo: Make TODO file more current 2016-12-21 03:42:44 -05:00
James Shubin
19760be0bc golint: Fix some golint issues 2016-12-21 03:10:25 -05:00
James Shubin
b3ea33f88d test: Allow devel versions to run gofmt
Let tip builds pass in travis too!
2016-12-21 02:48:50 -05:00
James Shubin
5b3425a689 pgraph: Remember to unpause the vertices!
Forgot this part earlier, sorry! Should work correctly now :)
2016-12-21 02:39:54 -05:00
James Shubin
a3d157bde6 pgraph: Mutex must be a pointer
This should be done the same way as the WaitGroup so that we don't
panic!
2016-12-20 05:49:17 -05:00
James Shubin
2c8c9264a4 pgraph: Simplify graph exit waiting
I think the vertex resource exiting can be done in a single stage
instead of the previous two stage exit.
2016-12-20 05:49:17 -05:00
James Shubin
0009d9b20e pgraph, resources: Integrate properly with the startup logic
This signals which resources have to run their initial pokes, and
removes the racy retry timer. We actually get a proper signal when
things are running too!
2016-12-20 05:49:17 -05:00
James Shubin
dd8d17232f pgraph: Build the sync group into the graph structure
This hides the sync/wait logic inside the graph itself.
2016-12-20 05:49:17 -05:00
James Shubin
6312b9225f gapi: Rename SwitchStream to Next
This is more concise and I think more logical. Complains welcome!
2016-12-20 05:49:17 -05:00
James Shubin
68cc09fef2 resources: file: Fix small typo in the compare function 2016-12-20 05:49:16 -05:00
James Shubin
0651c9de65 docs: Add resource guide
Sorry I never published this earlier. Thanks to everyone who has managed
to write a native resource without this.
2016-12-20 05:49:16 -05:00
James Shubin
38261ec809 resources: msg: Remove legacy comment
This doesn't apply anymore. Remove it!
2016-12-20 05:47:40 -05:00
James Shubin
067932aebf resources: Remove SetWatching/IsWatching code from Watch
This removes some boilerplate from the Watch methods which can be baked
into the engine instead.

This code should be checked for races and locks to make sure we only
start resources when it makes sense to.
2016-12-20 05:47:40 -05:00
James Shubin
af47511d58 resources: Don't dirty resource when poked with activity
When we receive a poke with the activity flag set it probably means we
are receiving a refresh notification. This doesn't necessarily mean that
the resource state should be dirty as a result, in particular if the
resource doesn't support refreshing. As a result, don't automatically
mark it as dirty. (The engine knows not to skip the CheckApply when the
refresh flag is set!)
2016-12-20 05:47:40 -05:00
James Shubin
36b916f27f resources: Simplify resource Converger and Startup code
This takes the Converged initialization and Startup patterns that are
common in all resources, and bakes it into the core engine. This way
resource writing is much more concise and there is less boilerplate!
2016-12-20 05:47:40 -05:00
James Shubin
e519811893 docs: Create a dedicated documentation folder 2016-12-09 17:32:50 -05:00
James Shubin
4803be1987 misc: Rename mgmtmain to lib and remove global package
This refactor should make it cleaner to use mgmt.
2016-12-08 23:31:45 -05:00
James Shubin
1f415db44f readme: Add new blog post about send/recv 2016-12-07 14:22:01 -05:00
James Shubin
0e316b1d55 gapi: Add world interface and refactor existing code to use it
This is the initial base of what will hopefully become a powerful API
that machines will use to communicate. It will be the basis of the
stateful data store that can be used for exported resources, fact
exchange, state machine flags, locks, and much more.
2016-12-07 02:39:14 -05:00
James Shubin
eb545e75fb resources: Re-order send/recv display messages.
The updated order is more logical, and follows the time sequence.
2016-12-06 14:42:06 -05:00
James Shubin
6edb5c30d5 resources: Actually verify which send/recv elements changed
When updating the code, I forgot to actually verify if there were
changes or not. This caused erroneous changed messages when none were
actually sent.
2016-12-06 14:22:34 -05:00
James Shubin
597ed6eaa0 resources: Polish the password PoC and build out send/recv
This polishes the password resource so that it can actually avoid
writing the password to disk, and so that the work actually happens in
CheckApply where it can properly interact with the graph. This resource
now re-generates the password when it receives a notification.

The send/recv plumbing has been extended so that receivers can detect
when they're receiving new values. This is particularly important if
they might otherwise not expect those values to change and cache them
for efficiency purposes.
2016-12-06 02:29:47 -05:00
Nicolas Nadeau
2b47d7494e pgp: Base pgp code 2016-12-05 02:10:15 -05:00
James Shubin
213a88f62f misc: Improve gofmt test case
Add new golang versions, and fail if one is not found.
2016-12-04 21:11:36 -05:00
James Shubin
07fd2e88a2 resources: Fix poke/refresh race
Clearly the use of errgroup is flawed.
1) You can't pass in variables, so this is likely to race.
2) You can't get a set of errors, so this is a bad API.

For the second problem, it would be much more sane to return a multierr
or a list of errors. If there's no fix for the first, I think it should
be removed from the lib.
2016-12-04 21:06:08 -05:00
James Shubin
639afe881c resources: Reduce logging on Send/Recv
This was too noisy, let's tone it down a bit.
2016-12-03 01:44:36 -05:00
James Shubin
2e718c0e9d resources: Improve notification system and notify refreshes
Resources can send "refresh" notifications along edges. These messages
are sent whenever the upstream (initiating vertex) changes state. When
the changed state propagates downstream, it will be paired with a
refresh flag which can be queried in the CheckApply method of that
resource.

Future work will include a stateful refresh tracking mechanism so that
if a refresh event is generated and not consumed, it will be saved
across an interrupt (shutdown) or a crash so that it can be re-applied
on the subsequent run. This is important because the unapplied refresh
is a form of hysteresis which needs to be tracked and remembered or we
won't be able to determine that the state is wrong!

Still to do:
* Update the autogrouping code to handle the edge notify properties!
* Actually finish the stateful bool code
2016-12-03 01:35:31 -05:00
James Shubin
b0a8fc165c resources: Improve the state/cache system
Refactor the state cache into the engine. This makes resource writing
less error prone, and paves the way for better notifications.
2016-12-03 00:07:29 -05:00
James Shubin
ba6044e9e8 resources, pgraph: split logical chunks into separate files 2016-12-03 00:07:29 -05:00
James Shubin
7f1c13a576 resources: Implement Send -> Recv
This is a new design idea which I had. Whether it stays around or not is
up for debate. For now it's a rough POC.

The idea is that any resource can _produce_ data, and any resource can
_consume_ data. This is what we call send and recv. By linking the two
together, data can be passed directly between resources, which will
maximize code re-use, and allow for some interesting logical graphs.

For example, you might have an HTTP resource which puts its output in a
particular file. This avoids having to overload the HTTP resource with
all of the special behaviours of the File resource.

For our POC, I implemented a `password` resource which generates a
random string which can then be passed to a receiver such as a file. At
this point the password resource isn't recommended for sensitive
applications because it caches the password as plain text.

Still to do:
* Statically check all of the type matching before we run the graph
* Verify that our autogrouping works correctly around this feature
* Verify that appropriate edges exist between send->recv pairs
* Label the password as generated instead of storing the plain text
* Consider moving password logic from Init() to CheckApply()
* Consider combining multiple send values (list?) into a single receiver
* Consider intermediary transformation nodes for value combining
2016-12-03 00:07:29 -05:00
James Shubin
63c5e35e2b misc: Cleanup unnecessary use of %v and small nitpicks 2016-12-03 00:07:29 -05:00
James Shubin
62e6a7d7fa resources: Add VarDir support
This gives resources a private directory where they can store state.
2016-12-03 00:07:29 -05:00
James Shubin
e5a3dae332 misc: Exclude vendor/ directory from ack matching
This is almost always what we want.
2016-12-03 00:07:29 -05:00
James Shubin
b45a7663b3 readme: Add video from NLUUG
This video shows the virt+cockpit PoC & the live remote execution demo.
2016-12-03 00:06:03 -05:00
Vinzenz Feenstra
6ef904f62b misc: Prefer dnf over yum when present
Signed-off-by: Vinzenz Feenstra <vfeenstr@redhat.com>
2016-11-22 12:26:17 +01:00
James Shubin
6d21cf3084 readme: Add video link from High Load Strategy 2016-11-18 16:16:51 -05:00
James Shubin
32bd96b6e2 resources: nspawn: Update grammer for Joe
Joe says this is the correct grammar. I incorrectly changed it wrongly.
2016-11-18 16:15:21 -05:00
James Shubin
fb5da76247 resources: Add stable go-systemd branch for now
We'll update this to point to upstream or JoeJulian once it stops
randomly changing and breaking git master!
2016-11-14 20:03:19 -05:00
James Shubin
e588f51824 resources: nspawn: Use new API
We broke the API (*cough* Joe), this updates it with the new version.
2016-11-14 19:36:13 -05:00
Joe Julian
3e419c4955 nspawn: bump go-systemd commit
joejulian/go-systemd was updated to provide more machine1 features and
was rebased to the latest master
2016-11-14 11:45:07 -08:00
James Shubin
606d2bafac resources: nspawn: Tweaks and updates
Here are some small fixes to enhance the original nspawn patch.
2016-11-12 00:43:55 -05:00
Joe Julian
8ac3c49286 nspawn: Add systemd-machined support for nspawn containers
This adds a rudimentary resource for systemd-machined's nspawn
containers, ensuring they're either started or stopped.
2016-11-11 14:55:14 -08:00
James Shubin
534aa84ed0 etcd: Watch for obvious failures on first startup
We should probably wait for this signal elsewhere too.
2016-11-11 07:37:36 -05:00
Vinzenz Feenstra
04d17cb580 examples: rename hostname.yml to hostname.yaml
Signed-off-by: Vinzenz Feenstra <vfeenstr@redhat.com>
2016-11-11 12:51:55 +01:00
Vinzenz Feenstra
d039006eb4 resources: Add new hostname resource
This resource allows to set and watch the hostname on a system.

Signed-off-by: Vinzenz Feenstra <vfeenstr@redhat.com>
2016-11-11 12:42:04 +01:00
James Shubin
fb04f62115 resources: file: Allow undefined file contents
An undefined file contents means we aren't managing that state!
2016-11-10 06:06:41 -05:00
James Shubin
3bffccc48e resources: Clean up errors and string printing 2016-11-08 03:49:27 -05:00
Vinzenz Feenstra
eef9abf0bf virt: Authentication support
Signed-off-by: Vinzenz Feenstra <vfeenstr@redhat.com>
2016-11-07 13:28:30 +01:00
Vinzenz Feenstra
de5ada30b7 virt: Avoid parsing URI all the time
Signed-off-by: Vinzenz Feenstra <vfeenstr@redhat.com>
2016-11-07 13:22:40 +01:00
Joe Julian
12f7d0a516 virt: Add logic to parse the libvirt uri
Guess the domain and os types from the libvirt uri and add the ability
to set the init for lxc containers.
2016-11-07 13:22:40 +01:00
Joe Julian
0aa9c7c592 Add IntelliJ Idea to .gitignore 2016-11-07 13:22:40 +01:00
Joe Julian
2216c8dc1c virt: Do not restrict VIR_ERR_NO_DOMAIN to qemu.
Consider it sufficent that a libvirt.VIR_ERR_NO_DOMAIN was reported
without concerning ourselves over who reported it.
2016-11-07 13:22:40 +01:00
James Shubin
984270ebe1 pgraph: Fix ineffassign warning
This ineffassign didn't seem to cause problems, but perhaps we didn't
exhaustively test all the areas.

Watch out to make sure this doesn't break any of the sync requirements,
eg: "A WaitGroup must not be copied after first use."
https://golang.org/pkg/sync/#WaitGroup
2016-11-04 02:47:13 -04:00
James Shubin
2e2658ab6f examples: make the libmgmt example more fun
You can try it out yourself by running `go build` and then calling it.
Use a bare integer argument to create that number of noop resources.
There are clearly some performance optimizations that we could do for
extremely large graphs.
2016-11-03 04:18:26 -04:00
James Shubin
1370f2a76b gapi: Split out graph generation into a proper graph API
This is a monster patch that splits out the yaml and puppet based graph
generation and pushes them behind a common API. In addition alternate
pluggable GAPI's can be easily added! The important side benefit is that
you can now write a custom GAPI for embedding mgmt!

This also includes some slight clean ups that I didn't find it worth
splitting into separate patches.
2016-11-03 03:56:16 -04:00
Joe Julian
75dedf391a virt: don't set emulator path
Remove the implicit emulator path from the domain definition. Libvirt is
already configured to use the correct emulator for kvm or qemu and
specifying it creates distro dependence.

Fixes #85
2016-10-27 15:20:06 -04:00
Juergen Hoetzel
7b5c640d05 readme: Fix go get command
"go get" requires a package name
2016-10-27 18:30:00 +02:00
James Shubin
aa9a21b4d0 cli: Pass through program and version strings
We forgot to pass these through. If they're undefined, it errors.
2016-10-24 17:41:03 -04:00
James Shubin
71de8014d5 main: Libify mgmt with a golang API
This is an initial implementation of a possible golang API. In this
particular version, the *gconfig.GraphConfig data structures are
emitted, instead of possibly building a pgraph. As long as we can
represent any local graph as the data structure, then this is fine!

Is there a way to merge the gconfig Vertex and the pgraph Vertex?
2016-10-24 17:33:31 -04:00
Marc Antoine Dumont
80476d19f9 Add the link to a new dependencies in README.md
Add the link to the dependencise github.com/rgbkrk/libvirt-go
2016-10-24 16:16:05 -04:00
James Shubin
15103d18ef readme: Update README file to make it clearer for new hackers 2016-10-23 20:47:12 -04:00
James Shubin
0dbd2004ad main: Split apart logic in main
This splits most of the main logic from the cli logic so that they can
be used independently, in particular for if we ever libify mgmt.
2016-10-23 20:23:04 -04:00
James Shubin
8c92566889 resources: virt: Update CPUs variable to new uint16 signature
Now things are consistent after my new patch upstream!
2016-10-23 02:41:32 -04:00
James Shubin
fb9449038b resources: Update constructor signature to return error as well
Update the helper functions so they're easier to properly use!
2016-10-23 01:36:34 -04:00
James Shubin
e06c4a873d resources: Set the defaults for metaparameters
This now lets us have defaults for metaparameters that aren't the zero
value for that type.
2016-10-23 01:14:02 -04:00
James Shubin
c4c28c6c82 spec: Improve the rpm package
This still needs a lot of work by a packaging specialist.
2016-10-19 20:10:11 -04:00
James Shubin
42ff9b803a resources: Use Events() method instead of raw channel
This makes things easier if we ever split resources out into separate
packages.
2016-10-19 20:08:53 -04:00
James Shubin
3831e9739c resources: virt: Update to new function signature
This changed in git master, and is now more idiomatic.
2016-10-19 13:59:20 -04:00
James Shubin
f196e5cca2 test: Fix travis so it pulls in our deps 2016-10-19 13:51:38 -04:00
Joe Julian
d3af9105ee Use the download-only flag when fetching dependencies 2016-10-19 09:54:15 -07:00
James Shubin
6d685ae4d6 misc: Add libvirt header file dependency 2016-10-19 04:23:15 -04:00
James Shubin
8381d8246a resources: virt: Add a virt resource based on libvirt
This adds an initial implementation of a virt resource based on libvirt.
It is not complete and requires more testing. The initial skeleton was
written by nseps but was not merged. It was later cleaned up and merged
in its current form by purpleidea. Many thanks to nseps for getting this
going, and hopefully we'll get you contributing more in the future!
2016-10-19 04:11:17 -04:00
James Shubin
b26322fc20 all: Rename UUID to UID.
Felix pointed out that these ID's are unique, but not universally unique
across the cluster, which might be confusing to new programmers. As a
result, rename them all.
2016-10-18 23:03:55 -04:00
James Shubin
1c1e8127d8 resources: Check that resource kind is set.
This could be a Fatal instead, but might as well fail gracefully.
2016-10-18 14:07:27 -04:00
James Shubin
1b3b4406ff resources: Interfaces parameters can be named to help documentation 2016-10-18 14:04:47 -04:00
James Shubin
cf0b77518a resources: List resources in alphabetical order 2016-10-18 14:04:21 -04:00
James Shubin
afdbf44e23 make: Sed needs g option to replace multiple PROGRAM names 2016-10-13 10:23:18 -04:00
Alexandre-Xavier Labonté-Lamoureux
ec87781956 test: Tokens should always have a colon 2016-10-11 13:46:59 -04:00
James Shubin
a6ae958be7 etcd: Fix type issue
I was lazy and pushed the previous fix too quickly. Sorry, fixed now!
2016-10-07 15:59:53 -04:00
James Shubin
312103ef1b test: update lint checker to support packages 2016-10-07 15:51:58 -04:00
James Shubin
c2911bb2b7 etcd: Verify struct is not nil before accessing retries value
This didn't happen often because there's a nominateCallback race, but is
a bug which happened occasionally.
2016-10-07 15:36:09 -04:00
James Shubin
8ca5e38121 readme: Update repository with information about remote execution 2016-10-07 15:35:29 -04:00
James Shubin
4b8ad3a8a7 godoc: Document packagekit package 2016-10-03 15:28:25 -04:00
James Shubin
f219c2649d godoc: Document resources package 2016-10-03 15:26:41 -04:00
James Shubin
cfde54261b golint: Outdent else statement 2016-10-03 15:21:08 -04:00
James Shubin
71a82b0a34 resources: Use named fields to avoid bothering go vet
This removes an invalid warning which tries to prevent ambiguous changes
I suppose.
2016-10-03 15:02:41 -04:00
James Shubin
b7bd2d2664 recwatch: Rearrange condition to unconfuse golinter
This wasn't a bug, but the go linter is unhappy because it is dumb.
2016-10-03 14:58:08 -04:00
James Shubin
cd26a0770d test: Catch go vet issues in subpackages
Improve our tests now that we have multiple packages to test.
2016-10-03 14:50:28 -04:00
James Shubin
46893e84c3 recwatch: Split recursive watching into sub-package
This splits the recursive watching bit of the file file resource into
it's own package. This also de-duplicates the configwatch code and puts
it into the same package. With these bits refactored, it was also easier
to clean up the error code in main.
2016-10-03 14:48:57 -04:00
James Shubin
567dcaf79d resources: fix deadlock on recursion
This was a sneaky deadlock which wasn't 100% reproducible. It was pretty
tricky to find because the same deadlocked behaviour was seen due to the
regression in the fsnotify module.

The event system needs a bit of a cleaning, but this can happen later.
2016-10-01 08:19:46 -04:00
James Shubin
9368c7e05f resources: msg: Turn on journal logging in the example
Make this more useful by default.
2016-09-28 05:59:55 -04:00
James Shubin
654b3e9dbe file: Fix regression in fsnotify code
This was a major deadlock that hit the file resource. I didn't notice it
earlier because I was using an older version of fsnotify and I hadn't
done a go get -u to refresh it. I finally tracked this down, and will
vendor the repository until a fix upstream or a workaround downstream is
added.

The upstream issue is: https://github.com/fsnotify/fsnotify/issues/123
2016-09-28 03:26:33 -04:00
James Shubin
f09db490f0 golint: Fix nested else statement 2016-09-27 13:33:58 -04:00
Felix Frank
30d93cfde7 resources: msg: Introduce new resource type to log arbitrary messages
Untested things:
* systemd journal

Unimplemented things:
* syslog
2016-09-27 13:31:16 -04:00
James Shubin
41b3db7d6b golint: Fix issues caught by the linter 2016-09-27 12:14:12 -04:00
James Shubin
2a60debceb resources: svc: Catch reload states too 2016-09-27 11:47:55 -04:00
James Shubin
eb30642b6f readme: Add link to godoc 2016-09-27 08:54:02 -04:00
James Shubin
ea85e2af6b golang: Update to version 1.6 as the minimum
Etcd now requires golang version 1.6 or greater.
2016-09-27 08:08:01 -04:00
James Shubin
ef979a0839 vendor: Update etcd to working version and fix grpc stuff 2016-09-27 07:57:04 -04:00
Felix Frank
e0107b1dda have 'make clean' get rid of generated code files again 2016-09-27 05:20:56 -04:00
James Shubin
ccc00f913d Resources: Update interface to support errors
This was an early interface mistake I made and is now corrected.
We'll plumb in checking the error state of Init() and running Validate()
later on.
2016-09-27 05:16:37 -04:00
James Shubin
ad3c6bdc88 TODO: add TODO item about DHT idea 2016-09-27 05:07:34 -04:00
James Shubin
8fe3891ea9 Makefile: Limit depth for yamlfmt 2016-09-27 05:06:43 -04:00
James Shubin
63f21952f4 golang: Split things into packages
This makes this logically more separate! :) As an aside...

I really hate the way golang does dependencies and packages. Yes, some
people insist on nesting their code deep into a $GOPATH, which is fine
if you're a google dev and are forced to work this way, but annoying for
the rest of the world. Your code shouldn't need a git commit to switch
to a a different vcs host! Gah I hate this so much.
2016-09-26 12:30:28 -04:00
James Shubin
361d643ce7 resources: Compare Delay and Retry metaparams on graph change
We forgot to add a number of important properties to the Compare. This
means that we'll rebuild elements for these changes too now.
2016-09-20 05:35:04 -04:00
Antoine Racine
abe1ffaab6 Add pcap.h dependency on Debian/Ubuntu to fix make deps error 2016-09-19 22:51:10 -04:00
James Shubin
fc24c91dde Resources: Add retry and retry delay meta parameters
All resources can now set a retry limit (-1 for infinite) and a delay
between retries. This applies to both the CheckApply methods, and the
Watch methods as well. They each have their own separate counts, but use
the same input meta param, since I decided it wouldn't be useful to have
a separate watchRetry and watchDelay set of meta parameters.

In the process, we got rid of about 15 error cases which would normally
panic.

This patch required a slight overhaul of the Event system.

The previous commit is an earlier version of this patch which I decided
to leave in to "show my work" as I used to have to do in math class.
It's slightly more correct with the current event system, and this
version is less correct and has a few bugs, but that is because the
event system needs a massive overhaul, and once that's done this should
all work properly for the corner cases.
2016-09-19 06:32:21 -04:00
James Shubin
53cabd5ee4 Resources: Prototype retry and retry delay meta parameters
This was the initial cut of the retry and delay meta parameters.
Instead, I decided to move the delay action into the common space
outside of the Watch resource. This is more complicated in the short
term, but will be more beneficial in the long run as each resource won't
have to implement this part itself (even if it uses boiler plate).

This is the first version of this patch without this fix. I decided to
include it because I think it has more correct event processing.
2016-09-19 06:32:21 -04:00
James Shubin
2b1e8cdbee travis: bump to newer golang version (fedora 23) 2016-09-19 03:39:09 -04:00
Michaël Faille
9715146495 travis : use go1.7 to be future proof.
And, to reflect my own config `;-)`
2016-09-19 03:37:55 -04:00
Michaël Faille
22b0b89949 Let env find bash in shebang.
On Nixos and GNUIX-SD, bash is chroot in package store path.
In my case : /nix/store/qvccmr6fsis4kqlvlk8pb1c8c0r0cwai-system-path/bin/bash

In any case, using `/usr/bin/env bash` is the recommended way to get bash
portable across UNIX-like systems.
2016-09-19 03:36:38 -04:00
Michaël Faille
2ebc23a777 cli : Prefer github.com/urfave/cli as cli
Over  github.com/codegangsta/cli
Note : I just adapt the README.md and dependency manager
2016-09-19 02:26:28 -04:00
James Shubin
0199285319 readme: Add information for new users 2016-09-18 05:59:46 -04:00
James Shubin
277ab2fe44 readme: We now have an arch aur build.
Thanks Joe Julian!
2016-09-18 05:52:48 -04:00
James Shubin
8a96dfdc8a art: Add a logo
Sarah Jane Cox made us a logo. Thanks SJ!
2016-09-18 05:47:33 -04:00
James Shubin
66fbbb940a test: temporarily disable test 2016-09-18 03:46:42 -04:00
James Shubin
716ea1bb3c etcd: Update examples to be more useful for single machine tests 2016-09-14 23:00:59 -04:00
James Shubin
3d701d3daa todo: Update TODO file 2016-09-12 02:01:44 -04:00
James Shubin
598c74657c file: Overhaul file resource and add recursion
The file resource contained some of the early golang code that I wrote
for this project. Needless to say, some of it was quite yucky, and it
was also lacking a number of important features. This patch builds upon
it so that it starts being usable for directories of files too.

Many thanks to Sam Gélineau for helping with the recursive watching. My
brain officially didn't want to look at that code anymore.
2016-09-12 01:55:31 -04:00
Felix Frank
4bd53d5ab0 add puppet support documentation 2016-09-12 02:40:09 +02:00
James Shubin
70f8d54a31 thanks: More people really needed to be on the list 2016-09-07 19:41:39 -04:00
James Shubin
4ef25a33fc docs: Add an FAQ entry about difference between two similar methods
Felix wins the points for first asking the question I knew would
eventually come but didn't document earlier.
2016-09-07 19:31:13 -04:00
Felix Frank
f5dd90a8dd add unit tests for UUID comparison and resource event passing 2016-09-08 01:12:56 +02:00
James Shubin
a84defd689 readme: Add new blog posts by Felix
The last post here was one of my favourites! Super impressive stuff!
2016-09-07 01:08:21 -04:00
James Shubin
1cf88d9540 test: Increase timeouts of t8
Increase the timeouts in the rare chance that this is slow performing
travis, and not just an etcd regression.
2016-09-02 02:27:39 -04:00
James Shubin
644a0ee8c8 puppet: golint fixes 2016-09-02 02:25:41 -04:00
James Shubin
e9d5dc8fee packagekit: golint fixes 2016-09-02 02:23:13 -04:00
James Shubin
8003202beb resources: golint fixes 2016-09-02 02:03:26 -04:00
James Shubin
b46432b5b6 misc: Golint fixes 2016-09-02 01:46:45 -04:00
James Shubin
5e3f03df06 etcd: Catch possible raft grpc error 2016-09-02 00:35:05 -04:00
James Shubin
8ab8e6679a test: provider usage text for shell test runner 2016-09-01 22:52:32 -04:00
James Shubin
786b896018 remote: small cleanups to update misc notes 2016-08-31 22:56:00 -04:00
James Shubin
40723f8705 docs: Add additional documentation about remote execution 2016-08-31 22:42:09 -04:00
James Shubin
2a0721bddf remote: allow converge during corner cases
This allows the system to converge during corner cases where there is an
error, or when there are no remotes being used, but we are using the
--no-watch variable.

I deliberately left this in as a separate commit instead of rebasing
into the remote execution development branch because the placement of
the Unregister() and semaphore.V(1) were quite subtle and easy to forget
about.
2016-08-31 22:42:09 -04:00
James Shubin
ff01e4a5e7 remote: Add distributed converged timeout
This patch extends the --converged-timeout argument so that when used
with --remote it waits for the entire set of remote mgmt agents to
converge simultaneously before exiting.

purpleidea says: This particular part of the patch probably took as much
work as all of the work required for the initial remote patches alone!
2016-08-31 21:55:19 -04:00
James Shubin
6794aff77c miscellaneous cleanups and fixes 2016-08-31 21:55:19 -04:00
James Shubin
636f2a36b1 etcd: Use new converged timers and allow skipping them
This implements the new extensions to the converged UUID API so that we
can keep a consistent timer running and reset it when needed. This is
useful because we now allow certain Watcher callbacks and other
operations to explicitly _not_ cause a convergerUUID to un-converge.
This is a necessary dependency for the distributed remote
converged-timeout work.
2016-08-31 21:55:19 -04:00
James Shubin
eee652cefe converger: Add new timer system for determining convergence
This adds a new method of marking whether a particular UUID has
converged or not. You can now Start, Stop, or Reset a convergence timer
on the individual UUID's. This wraps the existing SetConverged calls
with a hidden go routine. It is not recommended to use the SetConverged
calls and the Timer calls on the same UUID.
2016-08-31 21:55:19 -04:00
James Shubin
6d45cd45d1 converger: Add new methods to the API
This adds new helper methods to the API such as the ability to query the
set timeout, and to set the state change function after initialization.
The first makes it easier to pass in the timeout value to functions and
structs, because you don't need to pass it separately from the converger
object. The second makes it easy to specify the state change functon
when you don't know what it is at creation time. In addition, it is more
powerful now, and tells you when we converge or un-converge in case we
want to take different actions on each.
2016-08-31 21:55:19 -04:00
James Shubin
f5fb135793 converger: Update the API for errors and naming
We updated the API and behaviour to make two changes:

1) We remove the log.* stuff, and replace the Fatal calls with straight
calls to panic(). These are meant to be like an assert, and shouldn't
happen unless there is a user error or a bug.

2) We made the !uuid.IsValid() checks return an error instead of causing
a panic. These returns errors instead, and makes the process safer for
users who are okay calling a function that won't have an effect.
2016-08-31 21:55:19 -04:00
James Shubin
6bf32c978a etcd: Rename loop to be more consistent in messages
Small nitpick fixups
2016-08-31 21:55:19 -04:00
James Shubin
8d3011fb9c etcd: Add a timeout for etcd server to start correctly
This also updates etcd to a newer version with a fix that allows this
detection and timeout operation to be possible.
2016-08-31 21:55:19 -04:00
James Shubin
9260066fa3 tests: Workaround regression in two host etcd clusters
If you don't give your two host cluster enough time to "feel healthy",
it will generate an error if you do operations within five seconds. This
is a regression and the five seconds is also quite arbitrary. This is
detailed at: https://github.com/coreos/etcd/issues/6305

This seems to be a bit of a race condition, even with a 10s timer, so
this also disables the StrictReconfigCheck. Re-enable this as soon as
possible.
2016-08-31 21:55:19 -04:00
James Shubin
5e45c5805b Improve internal etcd error handling 2016-08-31 21:55:19 -04:00
James Shubin
db4de12767 Add more flexibility around the prefixes available
This allows you to specify a custom prefix, or a tmp prefix which is
chosen automatically.
2016-08-31 21:55:19 -04:00
James Shubin
d429795737 Improve the configWatcher array to allow N files
This simplifies the code in main and hides it in the watcher!
2016-08-31 21:55:19 -04:00
James Shubin
276219a691 Run tests with a tmp prefix
This will avoid failures if /var/lib/mgmt/ isn't writable.
2016-08-31 21:55:19 -04:00
James Shubin
03c1df98f4 Improve prefix creation and feedback
This makes the default /var/lib/mgmt/ directory, but also allows you to
use a prefix in /tmp/ automatically if you can't write anywhere else.
2016-08-31 21:55:19 -04:00
James Shubin
79ba750dd5 Automatically update remote files on change
This extends the automatic watching of graph definition files across the
remote SSH boundary.
2016-08-31 21:55:19 -04:00
James Shubin
1d0e187838 Etcd: switch to using a directory prefix 2016-08-31 21:55:19 -04:00
James Shubin
ad1e48aa2d Add caching for remote execution
This speeds up copying of the binary for slow connections. It also
finally adds a universal directory prefix for mgmt!
2016-08-31 21:55:19 -04:00
James Shubin
7032eea045 Remote "agent-less" mode
This is a new mode to be used for bootstrapping mgmt clusters or in
situations with tight operational restrictions.

This includes the basics, additional functionality will follow!
2016-08-31 21:55:19 -04:00
James Shubin
bdb970203c Start converger even if graph is empty 2016-08-30 17:47:25 -04:00
James Shubin
fa4f5abc78 Fix up tests, and one small bug 2016-08-30 17:47:25 -04:00
James Shubin
0c7b05b233 Check for exit status of tests 2016-08-30 17:47:25 -04:00
Joe Julian
4ca98b5f17 Allow make overrides of version if git fails
If the $(shell ...) command fails, the ':=' operator fails as well
preventing variable overrides from functioning.

Wrap the assignment in an $(or ...) to prevent the shell script from
running if the variable is set from the command line or the environment.

Fixes issue #58
2016-08-30 14:44:03 -07:00
Joe Julian
4e00c78410 Change "uname -i" to "uname -m" to be portable
According to the documentation for uname:

       -m, --machine
              print the machine hardware name

       -i, --hardware-platform
              print the hardware platform (non-portable)

So use the portable -m version.
2016-08-30 11:55:45 -07:00
James Shubin
17adb19c0d Fix typo 2016-08-04 00:44:50 -04:00
James Shubin
1db936e253 Update docs because they were out of date 2016-08-03 05:28:18 -04:00
James Shubin
7194ba7e0e Update docs to add automatic clustering 2016-08-03 05:16:47 -04:00
James Shubin
59b9b6f091 Docs: Add FAQ entry about similarly named band 2016-08-02 04:29:56 -04:00
James Shubin
c1ec8d15f3 Improve README for first time users
We need --recursive because we have a dependency vendored this way.
2016-08-02 04:25:35 -04:00
James Shubin
24ba6abc6b Don't block on member exits
The MemberList() function which apparently looks at all of the endpoints
was blocking right after a member exited, because we still had a stale
list of endpoints, and while a member that exited safely would update
the endpoints lists, the other member would (unavoidably) race to get
that message, and so it might call the MemberList() with the now stale
endpoint list. This way we invalidate an endpoint we know to be gone
immediately.

This also adds a simple test case to catch this scenario.
2016-07-26 04:23:46 -04:00
James Shubin
f6c1bba3b6 Avoid a rare panic if DestroyServer is called early
I never actually hit this bug, but I noticed it was possible when
examining the WaitGroup code that gets .Done() by DestroyServer().
2016-07-26 01:58:13 -04:00
James Shubin
a606961a22 Be safe when closing in destroy in case client is nil 2016-07-25 21:46:08 -04:00
James Shubin
cafe0e4ec2 Round of golint fixes to improve documentation.
This is really boring :(
2016-07-25 21:36:09 -04:00
James Shubin
e28c1266cf Do some gofmt simplifications 2016-07-25 20:56:33 -04:00
James Shubin
c1605a4f22 Add test case for urfave regression
Credit to jerith for helping me hack this together :)
2016-07-25 20:25:08 -04:00
James Shubin
7aeb55de70 Port embedded etcd to embed API
This also updates the vendored version of etcd to current git master,
which is the only place this is supported at the moment.
2016-07-25 19:10:09 -04:00
James Shubin
8ca65f9fda Revert "Copy in out of tree patches"
This reverts commit d26b503dca.

Use new etcd "embed" API.
2016-07-24 00:08:58 -04:00
James Shubin
94524d1156 Revert "Revert "Allow 1.6 to fail for now""
This reverts commit 78d769797f.
2016-07-20 03:09:17 -04:00
Sharad Ganapathy
a1ed03478b Adding timer resource and usage examples 2016-07-17 14:01:36 -04:00
James Shubin
402a6379b9 Add exec3 example
This is meant to be easier to understand than just sleep's.
2016-07-14 17:37:36 -04:00
Jeremy Thurgood
5d45bcd552 Check for old golang versions while installing dependencies. 2016-07-07 10:14:56 +02:00
Jack Henschel
f1fa64c170 Add recording and slides from DebConf16 presentation by James Shubin 2016-07-06 08:40:30 +02:00
Jack Henschel
50fc78564c Remove noop functionality from TODO
noop functionality (Bug #21) has been implemented:
6bbce039aa
9f56e4a582
2016-07-05 21:33:59 +02:00
James Shubin
3e5863dc8a Get travis results faster!
Thanks to jerith for the tip
2016-07-05 07:07:44 -04:00
James Shubin
94b447a9c5 Remove manual etcd usage. No longer needed. 2016-07-05 06:58:04 -04:00
James Shubin
78d769797f Revert "Allow 1.6 to fail for now"
This reverts commit f63b1cd56d.

This now works with stable version of etcd. Let's hope it stays that
way! :)
2016-07-05 04:32:38 -04:00
James Shubin
672baae126 Bump to etcd v3.0.0 2016-07-05 04:21:43 -04:00
James Shubin
e942d71ed2 Include video 2016-06-20 12:57:51 -04:00
James Shubin
f5d24cf86c New blog post! 2016-06-20 12:30:37 -04:00
James Shubin
f63b1cd56d Allow 1.6 to fail for now
I'm going to let 1.6 fail for now until F24 comes out and I can start
testing on 1.6 -- I'd like to be able to test all versions, but I don't
have all the resources to do so right now. If you want to help, please
let me know!
2016-06-20 00:43:54 -04:00
James Shubin
66719b3cda Update links 2016-06-20 00:36:46 -04:00
James Shubin
a5e9f6a6fc Fix stupid gofmt issue 2016-06-19 02:47:36 -04:00
James Shubin
f821afdf3e Update cli library path 2016-06-19 02:35:45 -04:00
James Shubin
2c61de83c6 Add go report card! 2016-06-19 02:35:24 -04:00
James Shubin
6da6f75b88 Add vendored etcd
This is a git submodule. Once etcd v3 becomes stable, this might not be
necessary anymore. We'll have to wait and see!
2016-06-18 04:43:19 -04:00
James Shubin
a55807a708 Split formatting into two targets 2016-06-18 04:43:19 -04:00
James Shubin
fce86b0d08 docs: add faq entry about using external etcd cluster 2016-06-18 04:43:19 -04:00
James Shubin
d26b503dca Copy in out of tree patches
These patches are proposed upstream changes and code for and from etcd.
Ideally we would revert this patch when/if things are merged upstream!
The majority of the work is in: https://github.com/coreos/etcd/pull/5584
2016-06-18 04:43:19 -04:00
James Shubin
5363839ac8 Embedded etcd
This monster patch embeds the etcd server. It took a good deal of
iterative work to tweak small details, and survived a rewrite from the
initial etcd v2 API implementation to the beta version of v3.

It has a notable race, and is missing some features, but it is ready for
git master and external developer consumption.
2016-06-18 04:43:19 -04:00
James Shubin
715a4bf393 Update links 2016-06-18 00:35:16 -04:00
Felix Frank
8f83ecee65 add puppet integration code
Puppet can be used on the basis of the ffrank-mgmtgraph module.
There are three modes available:
* fetching catalogs from the master (--puppet agent)
* compiling a manifest from a local file (--puppet /path/to/file.pp)
* compiling a manifest from the cli (--puppet "<manifest>")

Catalogs from the master are currently never refreshed. We should
add some more code to re-run the parsing function at an interval
equal to Puppet's local 'runinterval' setting.

There is also still a distinct lack of tests.

Still, this fixes #8
2016-06-01 00:34:40 +02:00
Felix Frank
2eed4bda42 README: add the capnslog package to the build dependencies 2016-06-01 00:32:36 +02:00
Raphaël Pinson
f4e1e24ca7 Typo in DOCUMENTATION.md 2016-05-26 10:50:51 +02:00
Paul Morgan
05c540e6cc move doc files to /usr/share/doc/mgmt and tag as docs
After this commit:

* doc files are in `/usr/share/doc/mgmt-*/`
* `rpm -qd mgmt` lists the files
* docs are listed on single lines in spec file
  to minimize future diff churn
* docs are in alpha order
2016-05-24 02:23:43 +00:00
James Shubin
9656390c87 Add quote from notable automation specialist and author 2016-05-22 02:16:19 -04:00
James Shubin
4b6470d1e1 Remove pesky spaces 2016-05-21 12:57:28 -04:00
Michał Czeraszkiewicz
56471c2fe4 Add Docker support 2016-05-21 11:08:14 +02:00
James Shubin
9f56e4a582 Add global --noop support
This is part two of the earlier patch in
6bbce039aa

We also rename GetMeta to just Meta to clean up the API.
2016-05-18 14:28:34 -04:00
James Shubin
12ea860eba Expand the event system slightly
This also adds a cleaner exit for the inner main loop. I'm not sure if
it's absolutely needed, but this will give me more confidence that we
won't end in the middle of some action.
2016-05-18 11:57:36 -04:00
James Shubin
b876c29862 Add logging workaround when embedding etcd
This was discussed in: https://github.com/coreos/etcd/issues/4115
2016-05-18 11:56:51 -04:00
Martin Alfke
6bbce039aa noop as resource meta param
first part of #21
tested with example/noop1.yaml on CentOS 6
2016-05-17 11:43:30 -04:00
Martin Alfke
1584f20220 make go vet installation optional 2016-05-17 16:47:52 +02:00
Martin Alfke
dcad5abc1c remove vet
go 1.6 has vet included
https://github.com/hashicorp/vault/issues/1310#issuecomment-207922338
2016-05-17 16:47:33 +02:00
James Shubin
ab73261fd4 Fix cli API change
I guess this is why builds were breaking. Remember to go get -u ...
2016-05-14 16:26:09 -04:00
James Shubin
05b75c0a44 Pkg: Immediately unconverge on events
It's more correct to run this as soon as possible. This wasn't because
of any known bug, but it's more correct stylistically.
2016-05-14 15:27:01 -04:00
James Shubin
ba7ef0788e Pkg: cache state when it is correct
We forgot to cache the state when we are converged. This avoids
repetitive checking when we hit repeated backpoke()'s for example.
2016-05-14 12:28:42 -04:00
Felix Frank
3aaa80974e rename the 'stateok' return value to 'checkok'
The naming was confusing because the boolean return value expresses
whether the resource needed changing (the check failed) as opposed to
the state not being not OK.

purpleidea note: The "stateok" (now properly renamed to "checkok") is
actually the historical bool return value of the Check() -> bool
function which is now part of the CheckApply() amalgamation. This is an
easy way to think about it if you're trying to understand why at the end
of a successful apply we return false, nil.
2016-05-14 18:15:06 +02:00
Felix Frank
995ca32eee packagekit: support Debian's 'all' architecture 2016-05-14 18:15:06 +02:00
Martin Alfke
bf5f48b85b find on OS X needs the dot 2016-05-13 14:30:23 +02:00
James Shubin
d6e386a555 Add a base method for GroupCmp
You'll want to override this when implementing a resource that does
grouping.
2016-04-28 15:37:19 -04:00
James Shubin
a0a71f683c Fix typo
Exec doesn't group, so this never impacted anything, but wasn't correct.
2016-04-28 15:35:32 -04:00
James Shubin
7adf88b55b Update TODO file 2016-04-26 01:15:59 -04:00
Paul Morgan
8a9d47fc4b move systemd tip from README to DOCUMENTATION
This consolidates how-to at the right place for the docs.
2016-04-26 03:07:59 +00:00
James Shubin
2a0a69c917 Update Makefile to be more useful for hackers 2016-04-25 22:52:08 -04:00
James Shubin
aeab8f55bd Fix go generate issue with path
Gah, I hate you $GOPATH...
2016-04-25 22:52:08 -04:00
James Shubin
9407050598 Add flatten helper to take apart messy nested lists
This isn't 100% necessary, but it's a friendly feature to add, and it
was a fun function to write.
2016-04-25 22:52:08 -04:00
Paul Morgan
b99da63306 add myself as a contributor 2016-04-26 02:19:24 +00:00
Paul Morgan
f0d6cfaae4 add systemd unit file
Update the spec file with the rpm macro to put the unit file
in the system-wide unit file directory based on:

    [root@1713bbf19a0b /]# rpmbuild --eval '%{_unitdir}'
    /usr/lib/systemd/system

Allow user to create a drop directory to specify options
via environment variables.

Resolves https://github.com/purpleidea/mgmt/issues/12.
2016-04-26 02:19:24 +00:00
Paul Morgan
3120628d8a allow to specify CLI options via environment vars
Use `git show -w` to inspect this commit diff since
it changes whitespace due to `gofmt`.

This commit allows to use environment variables in place of
CLI parameters to change program behavior. This supports the
notion of [12-factor apps](http://12factor.net/)
and makes it easier to dockerize the app as well as create
a systemd unit file.

It establishes a pattern of `MGMT_*` naming convention
for environment variables.
2016-04-26 02:19:24 +00:00
Paul Morgan
2654384461 add make target to build static binary
This is useful to generate a binary that can be dropped
onto any arbitrary distro, such as CoreOS, without having
to worry about glibc or other dependencies.

Specifically: CoreOS uses glibc, but it does not have a
package manager. It also has a read-only OS (`/usr/`).
Thus I'd like to compile a binary that can be dropped
into CoreOS and have zero dependencies.

* `make build` builds the same as it did before this commit.
* `make all` builds both dynamic and static bins, as expected.

I struggled with a way to DRY this up _and_ avoid diff churn.
In the end, I went with simplicity even though it's not DRY.
2016-04-26 02:17:20 +00:00
Paul Morgan
eac3b25dc9 improve portability of the Makefile
I like to build on Alpine Linux when possible, but
coreutils on Alpine does not include the
[`arch`](http://git.savannah.gnu.org/gitweb/?p=coreutils.git;a=blob_plain;f=src/coreutils-arch.c;hb=HEAD)
wrapper to print the kernel architecture.

`uname` and `arch` are both provided by coreutils rpm
on RedHat derivatives, so this change does not impact
the build on those distros.
2016-04-26 00:43:49 +00:00
Paul Morgan
7788f91dd5 make life easier for people who have EditorConfig plugin
If a person has the EditorConfig plugin,
this overrides default editor configs to play nicely with go files
and the Makefile.
2016-04-26 00:40:35 +00:00
James Shubin
d0c9b7170c Small nitpick fixups 2016-04-07 21:08:26 -04:00
Felix Frank
d84caa5528 don't abort test suite on first failure
The test.sh script aborts as soon as a test fails. This can save time
on the local machine, but is inconvenient in CI where an early minor
failure can mask more severe errors that will be found later.
2016-04-07 00:50:57 +02:00
James Shubin
2ab72bdf94 Allow stubborn users to avoid having to move their project
Some users like to have their project in ~/code/mgmt/ for example, but
this is not compatible with a $GOPATH which is elsewhere. This isn't a
problem unless you need vendored directories in ~/code/mgmt/vendor/
which doesn't work because ~/code/mgmt/ isn't in $GOPATH. With this
symlink and the provided ~/bin/go wrapper, vendored directories work
exactly as expected, and we also get a local $GOPATH pointing to the
same thing. Since $GOPATH must have a src/ dir, and vendor/ must NOT,
then we symlink the two together accordingly.

An important part of this is that those who like to put everything in
$GOPATH won't be affected by any of this!
2016-04-04 01:25:23 -04:00
James Shubin
f6833fde29 Update docs and README to mention new blog post
woo... it's done! thanks to my reviewers!
2016-03-30 06:08:31 -04:00
James Shubin
fa8a50b525 Update TODO file 2016-03-30 03:49:49 -04:00
James Shubin
d80c6bbf1d Add additional autogrouping example 2016-03-30 03:40:17 -04:00
James Shubin
6f3ac4bf2a Rework the converged detection and provide a clean interface
The old converged detection was hacked in code, instead of something
with a nice interface. This cleans it up, splits it into a separate
file, and removes a race condition that happened with the old code.

We also take the time to get rid of the ugly Set* methods and replace
them all with a single AssociateData method. This might be unnecessary
if we can pass in the Converger method at Resource construction.

Lastly, and most interesting, we suspend the individual timeout callers
when they've already converged, thus reducing unnecessary traffic, and
avoiding fast (eg: < 5 second) timers triggering more than once if they
stay converged!

A quick note on theory for any future readers... What happens if we have
--converged-timeout=0 ? Well, for this and any other positive value,
it's important to realize that deciding if something is converged is
actually a race between if the converged timer will fire and if some
random new event will get triggered. This is because there is nothing
that can actually predict if or when a new event will happen (eg the
user modifying a file). As a result, a race is always inherent, and
actually not a negative or "incorrect" algorithm.

A future improvement could be to add a global lock to each resource, and
to lock all resources when computing if we are converged or not. In
practice, this hasn't been necessary. The worst case scenario would be
(in theory, because this hasn't been tested) if an event happens
*during* the converged calculation, and starts running, the exit command
then runs, and the event finishes, but it doesn't get a chance to notify
some service to restart. A lock could probably fix this theoretical
case.
2016-03-29 20:27:38 -04:00
James Shubin
a6dc81a38e Allow failures in 1.4.x because of etcd problem
The error seen is:

req.Cancel undefined
(type *http.Request has no field or method Cancel)

And is possibly the result of something etcd folks changed. Sadly, the
fast building 1.4.x won't work at the moment :(
2016-03-28 21:28:08 -04:00
James Shubin
81c5ce40d4 Add grouping algorithm
This might not be fully correct, but it seems to be accurate so far. Of
particular note, the vertex order needs to be deterministic for this
algorithm, which isn't provided by a map, since golang intentionally
randomizes it. As a result, this also adds a sorted version of
GetVertices called GetVerticesSorted.
2016-03-28 21:16:03 -04:00
James Shubin
c59f45a37b Force process events to be synchronous
This avoids messing up the converged-timeout state!
2016-03-28 21:16:03 -04:00
James Shubin
1b01f908e3 Add resource auto grouping
Sorry for the size of this patch, I was busy hacking and plumbing away
and it got out of hand! I'm allowing this because there doesn't seem to
be anyone hacking away on parts of the code that this would break, since
the resource code is fairly stable in this change. In particular, it
revisits and refreshes some areas of the code that didn't see anything
new or innovative since the project first started. I've gotten rid of a
lot of cruft, and in particular cleaned up some things that I didn't
know how to do better before! Here's hoping I'll continue to learn and
have more to improve upon in the future! (Well let's not hope _too_ hard
though!)

The logical goal of this patch was to make logical grouping of resources
possible. For example, it might be more efficient to group three package
installations into a single transaction, instead of having to run three
separate transactions. This is because a package installation typically
has an initial one-time per run cost which shouldn't need to be
repeated.

Another future goal would be to group file resources sharing a common
base path under a common recursive fanotify watcher. Since this depends
on fanotify capabilities first, this hasn't been implemented yet, but
could be a useful method of reducing the number of separate watches
needed, since there is a finite limit.

It's worth mentioning that grouping resources typically _reduces_ the
parallel execution capability of a particular graph, but depending on
the cost/benefit tradeoff, this might be preferential. I'd submit it's
almost universally beneficial for pkg resources.

This monster patch includes:
* the autogroup feature
* the grouping interface
* a placeholder algorithm
* an extensive test case infrastructure to test grouping algorithms
* a move of some base resource methods into pgraph refactoring
* some config/compile clean ups to remove code duplication
* b64 encoding/decoding improvements
* a rename of the yaml "res" entries to "kind" (more logical)
* some docs
* small fixes
* and more!
2016-03-28 20:54:41 -04:00
James Shubin
9720812a78 Graph cleanups to make way for the autogroup feature! 2016-03-28 20:54:41 -04:00
James Shubin
05b4066ba6 Add initial plumbing for autogroups
This adds some of the API changes and improvements to the pkg resource
so that it can make use of this feature.
2016-03-28 20:54:41 -04:00
James Shubin
50c458b6cc Support "noarch" style packages in arch check
Forgot about these :)
2016-03-28 20:54:41 -04:00
James Shubin
2ab6d61a61 Avoid errors on golint test with travis
No idea why this happens on travis, and I'm done wasting time trying to
figure it out.
2016-03-28 20:54:41 -04:00
James Shubin
b77a39bdff Add golint test improvements so we detect branches more cleverly
Also fix up a count reversal, hopefully I was just tunnel visioned
before and this is the correct fix!
2016-03-28 05:27:28 -04:00
James Shubin
d3f7432861 Update context since etcd upstream moved to vendor/ dir
Follow suit with what etcd upstream is doing.
2016-03-28 05:27:28 -04:00
Daniele Sluijters
7f3ef5bf85 docs: Add Features and autoedges section 2016-03-14 09:20:53 +01:00
James Shubin
659fb3eb82 Split the Res interface into a Base sub piece
I didn't know this was possible until I was browsing through some golang
docs recently. This should hopefully make it clearer which the common
methods to all resources are (which don't need to be reimplemented each
time) and which ones are unique and need to be created for each
resource.
2016-03-14 01:48:58 -04:00
James Shubin
d1315bb092 Refactor out noop resource into a separate file
I still think it's a useful resource for demonstrating concepts and
perhaps for other future purposes.
2016-03-14 01:28:44 -04:00
James Shubin
b4ac0e2e7c Update README and docs with information about new blog post 2016-03-14 01:03:35 -04:00
James Shubin
bfe619272e Update make deps script to make it better for debian folks
Hopefully this should make it easier for debian users, or for users who
run the script in the wrong directory.
2016-03-11 18:45:28 -05:00
Xavi S.B
963f025011 Fix broken link to graph.yaml section 2016-03-10 20:42:47 +01:00
James Shubin
b8cdcaeb75 Switch bc for awk
This is useful in dumb environments *cough* travis, that apparently
don't have bc as a default :(
2016-03-10 04:23:29 -05:00
James Shubin
6b6dc75152 Add a threshold based golint test case
This lets some golint errors in, but fails if you're over a certain
threshold. The current threshold of 15% (of LOC) is arbitrary and
subject to change. The algorithm should be extended to check a range of
commits, although it's unclear how to detect what range of commits make
up a patch set.
2016-03-10 04:06:57 -05:00
James Shubin
23647445d7 Golint fixes 2016-03-10 03:29:51 -05:00
James Shubin
e60dda5027 Add some of the pkg and svc autoedge logic
This adds another chunk of it, and makes some other small fixes.
2016-03-10 03:29:51 -05:00
James Shubin
f39551952f Add pkg auto edge basics with packagekit improvements
This is a monster patch that finally gets the iterative pkg auto edges
working the way they should. For each file, as soon as one matches, we
don't want to keep add dependencies on other file objects under that
tree structure. This reduces the number of necessary edges considerably,
and allows the graph to run more concurrently.
2016-03-10 03:29:51 -05:00
James Shubin
a9538052bf Split out some of the pkg CheckApply logic
This avoids code duplication since we need to reuse this logic
elsewhere, and as well this adds support for resolving the mapping for
multiple numbers of packages at the same time.

Note that on occasion, InstallPackages has hung waiting for Finished or
Destroy signals which never came! Increasing the PkBufferSize seemed to
help, but it's not clear if this is a permanent fix, or if more advanced
debugging and/or work arounds are needed. Is it possible that not
*waiting* to receive the destroy signals is causing this?
2016-03-10 03:29:51 -05:00
James Shubin
267d5179f5 Exec variable should use pre-existing error variable
Small fixup
2016-03-10 03:29:51 -05:00
James Shubin
10b8c93da4 Make resource "kind" determination more obvious
By adding the "kind" to the base resource, it is still identifiable even
when the resource specific elements are gone in calls that are only
defined in the base resource. This is also more logical for building
resources!

This also switches resources to use an Init() method. This will be
useful for when resources have more complex initialization to do.
2016-03-10 03:29:50 -05:00
James Shubin
c999f0c2cd Add initial "autoedge" plumbing
This allows for resources to automatically add necessary edges to the
graph so that the event system doesn't have to work overtime due to
sub-optimal execution order.
2016-03-10 03:29:50 -05:00
James Shubin
54615dc03b Actually update to 1.6 in travis 2016-03-10 03:22:36 -05:00
James Shubin
9aea95ce85 Bump go versions in travis
Failures might be caused by inotify not working correctly in travis.
This is probably because travis has things mounted over NFS.
https://github.com/travis-ci/travis-ci/issues/2342
2016-02-28 18:44:19 -05:00
James Shubin
80f48291f3 Update Makefile plumbing and associated misc things 2016-02-28 16:57:18 -05:00
James Shubin
1a164cee3e Fix file resource regression
I added a regression to the file resource. This was caused by two
different bugs that I added when I switched the API to use checkapply. I
would have caught these issues, except my test cases *also* had a bug! I
think I've fixed all three issues now.

Lastly, when running on travis, the tests behave very differently! Some
of the tests actually fail, and it's not clear why. As a result, we had
to disable them! I guess you get what you pay for.
2016-02-28 02:18:46 -05:00
James Shubin
da494cdc7c Clean up the examples/ directory
Naming things numerically isn't very obvious. This is better for now.
2016-02-26 02:32:13 -05:00
James Shubin
06635dfa75 Rename GetRes() which is not descriptive, to Kind()
This used to be GetType(), but since now things are "resources", we want
to know what "kind" they are, since asking what "type" they are is
confusing, and makes less logical sense than "Kind".
2016-02-23 00:24:43 -05:00
James Shubin
a56fb3c8cd Update items for Jenkins tests 2016-02-22 20:00:15 -05:00
James Shubin
2dc3c62bbd The svc resource should use the private system bus of course
I didn't realize there was a difference until I was debugging an issue
with the pkg resource, which also uses dbus!
2016-02-22 19:44:10 -05:00
James Shubin
0339d0caa8 Improve go vet testing 2016-02-22 19:43:55 -05:00
James Shubin
3b5678dd91 Add package (pkg) resource
This is based on PackageKit, which means events, *and* we automatically
get support for any of the backends that PackageKit supports. This means
dpkg, and rpm are both first class citizens! Many other backends will
surely work, although thorough testing is left as an exercise to the
reader, or to someone who would like to write more test cases!

Unfortunately at the moment, there are a few upstream PackageKit bugs
which cause us issues, but those have been apparently resolved upstream.
If you experience issues with an old version of PackageKit, test if it
is working correctly before blaming mgmt :)

In parallel, mgmt might increase the testing surface for PackageKit, so
hopefully this makes it more robust for everyone involved!

Lastly, I'd like to point out that many great things that are typically
used for servers do start in the GNOME desktop world. Help support your
GNOME GNU/Linux desktop today!
2016-02-22 19:05:24 -05:00
James Shubin
82ff34234d Fix Makefile warnings on Travis 2016-02-22 01:36:57 -05:00
James Shubin
f3d1369764 Add some fixes for building in tip or with golang 1.6 2016-02-22 01:30:12 -05:00
James Shubin
ed61444d82 Fix golang 1.6 vet issue 2016-02-22 01:05:27 -05:00
James Shubin
ce0d68a8ba Make sure to error on test
This works around set -e not working in this scenario.
2016-02-22 00:59:36 -05:00
James Shubin
74aadbadb8 Update README with small fixes 2016-02-22 00:37:13 -05:00
James Shubin
58f41eddd9 Change API from StateOK/Apply to CheckApply()
This simplifies the API, and reduces code duplication for most
resources. It became obvious when writing the pkg resource, that the two
operations should really be one. Hopefully this will last! Comments
welcome.
2016-02-21 22:07:49 -05:00
James Shubin
4726445ec4 Add great article by felix frank
This is very important work that is doubly hard because the API isn't
stable yet. If you see felix, buy him a beverage.

PS: Sorry felix that I just broke the api. I'll send you the patch to
fix it!
2016-02-21 19:42:21 -05:00
James Shubin
3a85384377 Rename type to resource (res) and service to svc
Naming the resources "type" was a stupid mistake, and is a huge source
of confusion when also talking about real types. Fix this before it gets
out of hand.
2016-02-21 15:51:52 -05:00
James Shubin
d20b529508 Update README with new video
Also did some small formatting fixes
2016-02-20 14:35:52 -05:00
Rob Wilson
7199f558e8 build requires (at least) stringer - add package which contains missing dependency 2016-02-20 10:42:26 +00:00
James Shubin
674cb24a1a Update link to video 2016-02-18 16:00:57 -05:00
James Shubin
02c7336315 Add link to slides from Walter Heck 2016-02-17 01:57:17 -05:00
James Shubin
cde052d819 Add DevConf.cz video recording 2016-02-16 14:26:11 -05:00
James Shubin
989cc8d236 Add more mgmtlove tags 2016-02-16 13:06:22 -05:00
James Shubin
1186d63653 Add a link to my article about debugging golang 2016-02-15 17:37:06 -05:00
James Shubin
6e68d6dda0 Update web articles 2016-02-15 14:18:59 -05:00
James Shubin
7d876701b3 Add test case to match headers 2016-02-14 14:24:26 -05:00
James Shubin
dbbb483853 Switch to CentOS-7.1 for OMV tests.
There is an occasional hang when booting up F23 on Vagrant which causes
jenkins to wait indefinitely. This might be fixed by getting F23 back to
using eth0/eth1 in the base image.
2016-02-12 17:36:51 -05:00
James Shubin
85e9473d56 Make debugging easier when running on Jenkins 2016-02-12 15:55:31 -05:00
James Shubin
427d424707 Update docs to refer to #mgmtlove patches 2016-02-12 15:01:15 -05:00
James Shubin
f90c5fafa4 This now works. w00t 2016-02-12 14:48:00 -05:00
246 changed files with 26034 additions and 4396 deletions

1
.ackrc
View File

@@ -1,2 +1,3 @@
--ignore-dir=old/ --ignore-dir=old/
--ignore-dir=tmp/ --ignore-dir=tmp/
--ignore-dir=vendor/

1
.dockerignore Normal file
View File

@@ -0,0 +1 @@
docker

19
.editorconfig Normal file
View File

@@ -0,0 +1,19 @@
; This file is for unifying the coding style for different editors and IDEs.
; Plugins are available for notepad++, emacs, vim, gedit,
; textmate, visual studio, and more.
;
; See http://editorconfig.org for details.
# Top-most EditorConfig file.
root = true
[*]
end_of_line = lf
insert_final_newline = true
trim_trailing_whitespace = true
[*.go]
indent_style = tab
[Makefile]
indent_style = tab

9
.github/ISSUE_TEMPLATE.md vendored Normal file
View File

@@ -0,0 +1,9 @@
## Versions:
* mgmt version (eg: `mgmt --version`):
* operating system/distribution (eg: `uname -a`):
* golang version (eg: `go version`):
## Description:

4
.gitignore vendored
View File

@@ -1,9 +1,11 @@
.idea/
.omv/ .omv/
.ssh/ .ssh/
.vagrant/ .vagrant/
mgmt-documentation.pdf
old/ old/
tmp/ tmp/
*_stringer.go *_stringer.go
mgmt mgmt
mgmt.static
mgmt.iml
rpmbuild/ rpmbuild/

18
.gitmodules vendored Normal file
View File

@@ -0,0 +1,18 @@
[submodule "vendor/github.com/coreos/etcd"]
path = vendor/github.com/coreos/etcd
url = https://github.com/coreos/etcd/
[submodule "vendor/google.golang.org/grpc"]
path = vendor/google.golang.org/grpc
url = https://github.com/grpc/grpc-go
[submodule "vendor/github.com/grpc-ecosystem/grpc-gateway"]
path = vendor/github.com/grpc-ecosystem/grpc-gateway
url = https://github.com/grpc-ecosystem/grpc-gateway
[submodule "vendor/gopkg.in/fsnotify.v1"]
path = vendor/gopkg.in/fsnotify.v1
url = https://gopkg.in/fsnotify.v1
[submodule "vendor/github.com/purpleidea/go-systemd"]
path = vendor/github.com/purpleidea/go-systemd
url = https://github.com/purpleidea/go-systemd
[submodule "vendor/honnef.co/go/augeas"]
path = vendor/honnef.co/go/augeas
url = https://github.com/dominikh/go-augeas/

View File

@@ -1,15 +1,22 @@
language: go language: go
go: go:
- 1.4.3 - 1.6.x
- 1.5.2 - 1.7.x
- 1.8.x
- tip - tip
go_import_path: github.com/purpleidea/mgmt
sudo: true
dist: trusty dist: trusty
sudo: required before_install:
- sudo apt update
- git fetch --unshallow
install: 'make deps' install: 'make deps'
script: 'make test' script: 'make test'
matrix: matrix:
fast_finish: true
allow_failures: allow_failures:
- go: tip - go: tip
- go: 1.8.x
notifications: notifications:
irc: irc:
channels: channels:
@@ -22,4 +29,7 @@ notifications:
use_notice: false use_notice: false
skip_join: false skip_join: false
email: email:
- travis-ci@shubin.ca recipients:
- travis-ci@shubin.ca
on_failure: change
on_success: change

View File

@@ -4,4 +4,7 @@ If you appreciate the work of one of the contributors, thank them a beverage!
For a more exhaustive list please run: git log --format='%aN' | sort -u For a more exhaustive list please run: git log --format='%aN' | sort -u
This list is sorted alphabetically by first name. This list is sorted alphabetically by first name.
Felix Frank
James Shubin James Shubin
Julien Pivotto
Paul Morgan

View File

@@ -1,5 +1,5 @@
Mgmt Mgmt
Copyright (C) 2013-2016+ James Shubin and the project contributors Copyright (C) 2013-2017+ James Shubin and the project contributors
Written by James Shubin <james@shubin.ca> and the project contributors Written by James Shubin <james@shubin.ca> and the project contributors
This program is free software: you can redistribute it and/or modify This program is free software: you can redistribute it and/or modify

View File

@@ -1,143 +0,0 @@
#mgmt
<!--
Mgmt
Copyright (C) 2013-2016+ James Shubin and the project contributors
Written by James Shubin <james@shubin.ca> and the project contributors
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU Affero General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU Affero General Public License for more details.
You should have received a copy of the GNU Affero General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
-->
##mgmt by [James](https://ttboj.wordpress.com/)
####Available from:
####[https://github.com/purpleidea/mgmt/](https://github.com/purpleidea/mgmt/)
####This documentation is available in: [Markdown](https://github.com/purpleidea/mgmt/blob/master/DOCUMENTATION.md) or [PDF](https://pdfdoc-purpleidea.rhcloud.com/pdf/https://github.com/purpleidea/mgmt/blob/master/DOCUMENTATION.md) format.
####Table of Contents
1. [Overview](#overview)
2. [Project description - What the project does](#project-description)
3. [Setup - Getting started with mgmt](#setup)
4. [Usage/FAQ - Notes on usage and frequently asked questions](#usage-and-frequently-asked-questions)
5. [Reference - Detailed reference](#reference)
* [graph.yaml](#graph.yaml)
* [Command line](#command-line)
6. [Examples - Example configurations](#examples)
7. [Development - Background on module development and reporting bugs](#development)
8. [Authors - Authors and contact information](#authors)
##Overview
The `mgmt` tool is a research prototype to demonstrate next generation config
management techniques. Hopefully it will evolve into a useful, robust tool.
##Project Description
The mgmt tool is a distributed, event driven, config management tool, that
supports parallel execution, and librarification to be used as the management
foundation in and for, new and existing software.
##Setup
During this prototype phase, the tool can be run out of the source directory.
You'll probably want to use ```./run.sh run --file examples/graph1.yaml``` to
get started. Beware that this _can_ cause data loss. Understand what you're
doing first, or perform these actions in a virtual environment such as the one
provided by [Oh-My-Vagrant](https://github.com/purpleidea/oh-my-vagrant).
##Usage and frequently asked questions
(Send your questions as a patch to this FAQ! I'll review it, merge it, and
respond by commit with the answer.)
###Why did you start this project?
I wanted a next generation config management solution that didn't have all of
the design flaws or limitations that the current generation of tools do, and no
tool existed!
###Why did you use etcd? What about consul?
Etcd and consul are both written in golang, which made them the top two
contenders for my prototype. Ultimately a choice had to be made, and etcd was
chosen, but it was also somewhat arbitrary. If there is available interest,
good reasoning, *and* patches, then we would consider either switching or
supporting both, but this is not a high priority at this time.
###You didn't answer my question, or I have a question!
It's best to ask on [IRC](https://webchat.freenode.net/?channels=#mgmtconfig)
to see if someone can help you. Once we get a big enough community going, we'll
add a mailing list. If you don't get any response from the above, you can
contact me through my [technical blog](https://ttboj.wordpress.com/contact/)
and I'll do my best to help. If you have a good question, please add it as a
patch to this documentation. I'll merge your question, and add a patch with the
answer!
##Reference
Please note that there are a number of undocumented options. For more
information on these options, please view the source at:
[https://github.com/purpleidea/mgmt/](https://github.com/purpleidea/mgmt/).
If you feel that a well used option needs documenting here, please patch it!
###Overview of reference
* [graph.yaml](#graph.yaml): Main graph definition file.
* [Command line](#command-line): Command line parameters.
###graph.yaml
This is the compiled graph definition file. The format is currently
undocumented, but by looking through the [examples/](https://github.com/purpleidea/mgmt/tree/master/examples)
you can probably figure out most of it, as it's fairly intuitive.
###Command line
The main interface to the `mgmt` tool is the command line. For the most recent
documentation, please run `mgmt --help`.
####`--file <graph.yaml>`
Point to a graph file to run.
####`--converged-timeout <seconds>`
Exit if the machine has converged for approximately this many seconds.
####`--max-runtime <seconds>`
Exit when the agent has run for approximately this many seconds. This is not
generally recommended, but may be useful for users who know what they're doing.
##Examples
For example configurations, please consult the [examples/](https://github.com/purpleidea/mgmt/tree/master/examples) directory in the git
source repository. It is available from:
[https://github.com/purpleidea/mgmt/tree/master/examples](https://github.com/purpleidea/mgmt/tree/master/examples)
##Development
This is a project that I started in my free time in 2013. Development is driven
by all of our collective patches! Dive right in, and start hacking!
Please contact me if you'd like to invite me to speak about this at your event.
You can follow along [on my technical blog](https://ttboj.wordpress.com/).
To report any bugs, please file a ticket at: [https://github.com/purpleidea/mgmt/issues](https://github.com/purpleidea/mgmt/issues).
##Authors
Copyright (C) 2013-2016+ James Shubin and the project contributors
Please see the
[AUTHORS](https://github.com/purpleidea/mgmt/tree/master/AUTHORS) file
for more information.
* [github](https://github.com/purpleidea/)
* [&#64;purpleidea](https://twitter.com/#!/purpleidea)
* [https://ttboj.wordpress.com/](https://ttboj.wordpress.com/)

141
Makefile
View File

@@ -1,5 +1,5 @@
# Mgmt # Mgmt
# Copyright (C) 2013-2016+ James Shubin and the project contributors # Copyright (C) 2013-2017+ James Shubin and the project contributors
# Written by James Shubin <james@shubin.ca> and the project contributors # Written by James Shubin <james@shubin.ca> and the project contributors
# #
# This program is free software: you can redistribute it and/or modify # This program is free software: you can redistribute it and/or modify
@@ -15,29 +15,71 @@
# You should have received a copy of the GNU Affero General Public License # You should have received a copy of the GNU Affero General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>. # along with this program. If not, see <http://www.gnu.org/licenses/>.
SHELL = /bin/bash SHELL = /usr/bin/env bash
.PHONY: all version program path deps run race build clean test format docs rpmbuild rpm srpm spec tar upload upload-sources upload-srpms upload-rpms copr .PHONY: all art cleanart version program path deps run race generate build clean test gofmt yamlfmt format docs rpmbuild mkdirs rpm srpm spec tar upload upload-sources upload-srpms upload-rpms copr
.SILENT: clean .SILENT: clean
SVERSION := $(shell git describe --match '[0-9]*\.[0-9]*\.[0-9]*' --tags --dirty --always) SVERSION := $(or $(SVERSION),$(shell git describe --match '[0-9]*\.[0-9]*\.[0-9]*' --tags --dirty --always))
VERSION := $(shell git describe --match '[0-9]*\.[0-9]*\.[0-9]*' --tags --abbrev=0) VERSION := $(or $(VERSION),$(shell git describe --match '[0-9]*\.[0-9]*\.[0-9]*' --tags --abbrev=0))
PROGRAM := $(shell basename --suffix=-$(VERSION) $(notdir $(CURDIR))) PROGRAM := $(shell echo $(notdir $(CURDIR)) | cut -f1 -d"-")
OLDGOLANG := $(shell go version | grep -E 'go1.3|go1.4')
ifeq ($(VERSION),$(SVERSION)) ifeq ($(VERSION),$(SVERSION))
RELEASE = 1 RELEASE = 1
else else
RELEASE = untagged RELEASE = untagged
endif endif
ARCH = $(shell arch) ARCH = $(uname -m)
SPEC = rpmbuild/SPECS/mgmt.spec SPEC = rpmbuild/SPECS/$(PROGRAM).spec
SOURCE = rpmbuild/SOURCES/mgmt-$(VERSION).tar.bz2 SOURCE = rpmbuild/SOURCES/$(PROGRAM)-$(VERSION).tar.bz2
SRPM = rpmbuild/SRPMS/mgmt-$(VERSION)-$(RELEASE).src.rpm SRPM = rpmbuild/SRPMS/$(PROGRAM)-$(VERSION)-$(RELEASE).src.rpm
SRPM_BASE = mgmt-$(VERSION)-$(RELEASE).src.rpm SRPM_BASE = $(PROGRAM)-$(VERSION)-$(RELEASE).src.rpm
RPM = rpmbuild/RPMS/mgmt-$(VERSION)-$(RELEASE).$(ARCH).rpm RPM = rpmbuild/RPMS/$(PROGRAM)-$(VERSION)-$(RELEASE).$(ARCH).rpm
USERNAME := $(shell cat ~/.config/copr 2>/dev/null | grep username | awk -F '=' '{print $$2}' | tr -d ' ') USERNAME := $(shell cat ~/.config/copr 2>/dev/null | grep username | awk -F '=' '{print $$2}' | tr -d ' ')
SERVER = 'dl.fedoraproject.org' SERVER = 'dl.fedoraproject.org'
REMOTE_PATH = 'pub/alt/$(USERNAME)/mgmt' REMOTE_PATH = 'pub/alt/$(USERNAME)/$(PROGRAM)'
ifneq ($(GOTAGS),)
BUILD_FLAGS = -tags '$(GOTAGS)'
endif
all: docs default: build
#
# art
#
art: art/mgmt_logo_default_symbol.png art/mgmt_logo_default_tall.png art/mgmt_logo_default_wide.png art/mgmt_logo_reversed_symbol.png art/mgmt_logo_reversed_tall.png art/mgmt_logo_reversed_wide.png art/mgmt_logo_white_symbol.png art/mgmt_logo_white_tall.png art/mgmt_logo_white_wide.png
cleanart:
rm -f art/mgmt_logo_default_symbol.png art/mgmt_logo_default_tall.png art/mgmt_logo_default_wide.png art/mgmt_logo_reversed_symbol.png art/mgmt_logo_reversed_tall.png art/mgmt_logo_reversed_wide.png art/mgmt_logo_white_symbol.png art/mgmt_logo_white_tall.png art/mgmt_logo_white_wide.png
# NOTE: the widths are arbitrary
art/mgmt_logo_default_symbol.png: art/mgmt_logo_default_symbol.svg
inkscape --export-background='#ffffff' --without-gui --export-png "$@" --export-width 300 $(@:png=svg)
art/mgmt_logo_default_tall.png: art/mgmt_logo_default_tall.svg
inkscape --export-background='#ffffff' --without-gui --export-png "$@" --export-width 400 $(@:png=svg)
art/mgmt_logo_default_wide.png: art/mgmt_logo_default_wide.svg
inkscape --export-background='#ffffff' --without-gui --export-png "$@" --export-width 800 $(@:png=svg)
art/mgmt_logo_reversed_symbol.png: art/mgmt_logo_reversed_symbol.svg
inkscape --export-background='#231f20' --without-gui --export-png "$@" --export-width 300 $(@:png=svg)
art/mgmt_logo_reversed_tall.png: art/mgmt_logo_reversed_tall.svg
inkscape --export-background='#231f20' --without-gui --export-png "$@" --export-width 400 $(@:png=svg)
art/mgmt_logo_reversed_wide.png: art/mgmt_logo_reversed_wide.svg
inkscape --export-background='#231f20' --without-gui --export-png "$@" --export-width 800 $(@:png=svg)
art/mgmt_logo_white_symbol.png: art/mgmt_logo_white_symbol.svg
inkscape --export-background='#231f20' --without-gui --export-png "$@" --export-width 300 $(@:png=svg)
art/mgmt_logo_white_tall.png: art/mgmt_logo_white_tall.svg
inkscape --export-background='#231f20' --without-gui --export-png "$@" --export-width 400 $(@:png=svg)
art/mgmt_logo_white_wide.png: art/mgmt_logo_white_wide.svg
inkscape --export-background='#231f20' --without-gui --export-png "$@" --export-width 800 $(@:png=svg)
all: docs $(PROGRAM).static
# show the current version # show the current version
version: version:
@@ -53,39 +95,56 @@ deps:
./misc/make-deps.sh ./misc/make-deps.sh
run: run:
find -maxdepth 1 -type f -name '*.go' -not -name '*_test.go' | xargs go run -ldflags "-X main.program=$(PROGRAM) -X main.version=$(SVERSION)" find . -maxdepth 1 -type f -name '*.go' -not -name '*_test.go' | xargs go run -ldflags "-X main.program=$(PROGRAM) -X main.version=$(SVERSION)"
# include race flag # include race flag
race: race:
find -maxdepth 1 -type f -name '*.go' -not -name '*_test.go' | xargs go run -race -ldflags "-X main.program=$(PROGRAM) -X main.version=$(SVERSION)" find . -maxdepth 1 -type f -name '*.go' -not -name '*_test.go' | xargs go run -race -ldflags "-X main.program=$(PROGRAM) -X main.version=$(SVERSION)"
build: mgmt generate:
mgmt: main.go
@echo "Building: $(PROGRAM), version: $(SVERSION)."
go generate go generate
# avoid equals sign in old golang versions eg in: -X foo=bar
if go version | grep -qE 'go1.3|go1.4'; then \ build: $(PROGRAM)
go build -ldflags "-X main.program $(PROGRAM) -X main.version $(SVERSION)" -o mgmt; \
else \ $(PROGRAM): main.go
go build -ldflags "-X main.program=$(PROGRAM) -X main.version=$(SVERSION)" -o mgmt; \ @echo "Building: $(PROGRAM), version: $(SVERSION)..."
fi ifneq ($(OLDGOLANG),)
@# avoid equals sign in old golang versions eg in: -X foo=bar
time go build -ldflags "-X main.program $(PROGRAM) -X main.version $(SVERSION)" -o $(PROGRAM) $(BUILD_FLAGS);
else
time go build -i -ldflags "-X main.program=$(PROGRAM) -X main.version=$(SVERSION)" -o $(PROGRAM) $(BUILD_FLAGS);
endif
$(PROGRAM).static: main.go
@echo "Building: $(PROGRAM).static, version: $(SVERSION)..."
go generate
ifneq ($(OLDGOLANG),)
@# avoid equals sign in old golang versions eg in: -X foo=bar
go build -a -installsuffix cgo -tags netgo -ldflags '-extldflags "-static" -X main.program $(PROGRAM) -X main.version $(SVERSION)' -o $(PROGRAM).static $(BUILD_FLAGS);
else
go build -a -installsuffix cgo -tags netgo -ldflags '-extldflags "-static" -X main.program=$(PROGRAM) -X main.version=$(SVERSION)' -o $(PROGRAM).static $(BUILD_FLAGS);
endif
clean: clean:
[ ! -e mgmt ] || rm mgmt [ ! -e $(PROGRAM) ] || rm $(PROGRAM)
rm -f *_stringer.go # generated by `go generate` rm -f *_stringer.go # generated by `go generate`
rm -f *_mock.go # generated by `go generate`
test: test:
./test.sh ./test.sh
format: gofmt:
find -type f -name '*.go' -not -path './old/*' -not -path './tmp/*' -exec gofmt -w {} \; find . -maxdepth 3 -type f -name '*.go' -not -path './old/*' -not -path './tmp/*' -exec gofmt -w {} \;
find -type f -name '*.yaml' -not -path './old/*' -not -path './tmp/*' -not -path './omv.yaml' -exec ruby -e "require 'yaml'; x=YAML.load_file('{}').to_yaml.each_line.map(&:rstrip).join(10.chr)+10.chr; File.open('{}', 'w').write x" \;
docs: mgmt-documentation.pdf yamlfmt:
find . -maxdepth 3 -type f -name '*.yaml' -not -path './old/*' -not -path './tmp/*' -not -path './omv.yaml' -exec ruby -e "require 'yaml'; x=YAML.load_file('{}').to_yaml.each_line.map(&:rstrip).join(10.chr)+10.chr; File.open('{}', 'w').write x" \;
mgmt-documentation.pdf: DOCUMENTATION.md format: gofmt yamlfmt
pandoc DOCUMENTATION.md -o 'mgmt-documentation.pdf'
docs: $(PROGRAM)-documentation.pdf
$(PROGRAM)-documentation.pdf: docs/documentation.md
pandoc docs/documentation.md -o docs/'$(PROGRAM)-documentation.pdf'
# #
# build aliases # build aliases
@@ -116,21 +175,21 @@ upload: upload-sources upload-srpms upload-rpms
$(RPM): $(SPEC) $(SOURCE) $(RPM): $(SPEC) $(SOURCE)
@echo Running rpmbuild -bb... @echo Running rpmbuild -bb...
rpmbuild --define '_topdir $(shell pwd)/rpmbuild' -bb $(SPEC) && \ rpmbuild --define '_topdir $(shell pwd)/rpmbuild' -bb $(SPEC) && \
mv rpmbuild/RPMS/$(ARCH)/mgmt-$(VERSION)-$(RELEASE).*.rpm $(RPM) mv rpmbuild/RPMS/$(ARCH)/$(PROGRAM)-$(VERSION)-$(RELEASE).*.rpm $(RPM)
$(SRPM): $(SPEC) $(SOURCE) $(SRPM): $(SPEC) $(SOURCE)
@echo Running rpmbuild -bs... @echo Running rpmbuild -bs...
rpmbuild --define '_topdir $(shell pwd)/rpmbuild' -bs $(SPEC) rpmbuild --define '_topdir $(shell pwd)/rpmbuild' -bs $(SPEC)
# renaming is not needed because we aren't using the dist variable # renaming is not needed because we aren't using the dist variable
#mv rpmbuild/SRPMS/mgmt-$(VERSION)-$(RELEASE).*.src.rpm $(SRPM) #mv rpmbuild/SRPMS/$(PROGRAM)-$(VERSION)-$(RELEASE).*.src.rpm $(SRPM)
# #
# spec # spec
# #
$(SPEC): rpmbuild/ mgmt.spec.in $(SPEC): rpmbuild/ spec.in
@echo Running templater... @echo Running templater...
#cat mgmt.spec.in > $(SPEC) cat spec.in > $(SPEC)
sed -e s/__VERSION__/$(VERSION)/ -e s/__RELEASE__/$(RELEASE)/ < mgmt.spec.in > $(SPEC) sed -e s/__PROGRAM__/$(PROGRAM)/g -e s/__VERSION__/$(VERSION)/g -e s/__RELEASE__/$(RELEASE)/g < spec.in > $(SPEC)
# append a changelog to the .spec file # append a changelog to the .spec file
git log --format="* %cd %aN <%aE>%n- (%h) %s%d%n" --date=local | sed -r 's/[0-9]+:[0-9]+:[0-9]+ //' >> $(SPEC) git log --format="* %cd %aN <%aE>%n- (%h) %s%d%n" --date=local | sed -r 's/[0-9]+:[0-9]+:[0-9]+ //' >> $(SPEC)
@@ -140,7 +199,7 @@ $(SPEC): rpmbuild/ mgmt.spec.in
$(SOURCE): rpmbuild/ $(SOURCE): rpmbuild/
@echo Running git archive... @echo Running git archive...
# use HEAD if tag doesn't exist yet, so that development is easier... # use HEAD if tag doesn't exist yet, so that development is easier...
git archive --prefix=mgmt-$(VERSION)/ -o $(SOURCE) $(VERSION) 2> /dev/null || (echo 'Warning: $(VERSION) does not exist. Using HEAD instead.' && git archive --prefix=mgmt-$(VERSION)/ -o $(SOURCE) HEAD) git archive --prefix=$(PROGRAM)-$(VERSION)/ -o $(SOURCE) $(VERSION) 2> /dev/null || (echo 'Warning: $(VERSION) does not exist. Using HEAD instead.' && git archive --prefix=$(PROGRAM)-$(VERSION)/ -o $(SOURCE) HEAD)
# TODO: if git archive had a --submodules flag this would easier! # TODO: if git archive had a --submodules flag this would easier!
@echo Running git archive submodules... @echo Running git archive submodules...
# i thought i would need --ignore-zeros, but it doesn't seem necessary! # i thought i would need --ignore-zeros, but it doesn't seem necessary!
@@ -149,14 +208,14 @@ $(SOURCE): rpmbuild/
temp="$${temp#\'}"; \ temp="$${temp#\'}"; \
path=$$temp; \ path=$$temp; \
[ "$$path" = "" ] && continue; \ [ "$$path" = "" ] && continue; \
(cd $$path && git archive --prefix=mgmt-$(VERSION)/$$path/ HEAD > $$p/rpmbuild/tmp.tar && tar --concatenate --file=$$p/$(SOURCE) $$p/rpmbuild/tmp.tar && rm $$p/rpmbuild/tmp.tar); \ (cd $$path && git archive --prefix=$(PROGRAM)-$(VERSION)/$$path/ HEAD > $$p/rpmbuild/tmp.tar && tar --concatenate --file=$$p/$(SOURCE) $$p/rpmbuild/tmp.tar && rm $$p/rpmbuild/tmp.tar); \
done done
# TODO: ensure that each sub directory exists # TODO: ensure that each sub directory exists
rpmbuild/: rpmbuild/:
mkdir -p rpmbuild/{BUILD,BUILDROOT,RPMS,SOURCES,SPECS,SRPMS} mkdir -p rpmbuild/{BUILD,BUILDROOT,RPMS,SOURCES,SPECS,SRPMS}
rpmbuild: mkdirs:
mkdir -p rpmbuild/{BUILD,BUILDROOT,RPMS,SOURCES,SPECS,SRPMS} mkdir -p rpmbuild/{BUILD,BUILDROOT,RPMS,SOURCES,SPECS,SRPMS}
# #

102
README.md
View File

@@ -1,65 +1,83 @@
# *mgmt*: This is: mgmt! # *mgmt*: next generation config management!
[![Build Status](https://secure.travis-ci.org/purpleidea/mgmt.png?branch=master)](http://travis-ci.org/purpleidea/mgmt) [![mgmt!](art/mgmt.png)](art/)
[![Documentation](https://img.shields.io/docs/markdown.png)](DOCUMENTATION.md)
[![IRC](https://img.shields.io/irc/%23mgmtconfig.png)](https://webchat.freenode.net/?channels=#mgmtconfig) [![Go Report Card](https://goreportcard.com/badge/github.com/purpleidea/mgmt?style=flat-square)](https://goreportcard.com/report/github.com/purpleidea/mgmt)
[![Jenkins](https://img.shields.io/jenkins/status.png)](https://ci.centos.org/job/purpleidea-mgmt/) [![Build Status](https://img.shields.io/travis/purpleidea/mgmt/master.svg?style=flat-square)](http://travis-ci.org/purpleidea/mgmt)
[![COPR](https://img.shields.io/copr/builds.png)](https://copr.fedoraproject.org/coprs/purpleidea/mgmt/) [![GoDoc](https://img.shields.io/badge/godoc-reference-5272B4.svg?style=flat-square)](https://godoc.org/github.com/purpleidea/mgmt)
[![IRC](https://img.shields.io/badge/irc-%23mgmtconfig-brightgreen.svg?style=flat-square)](https://webchat.freenode.net/?channels=#mgmtconfig)
[![Jenkins](https://img.shields.io/badge/jenkins-status-brightgreen.svg?style=flat-square)](https://ci.centos.org/job/purpleidea-mgmt/)
## Community: ## Community:
Come join us on IRC in [#mgmtconfig](https://webchat.freenode.net/?channels=#mgmtconfig) on Freenode! Come join us in the `mgmt` community!
You may like the [#mgmtconfig](https://twitter.com/hashtag/mgmtconfig) hashtag if you're on [Twitter](https://twitter.com/#!/purpleidea).
## Questions: | Medium | Link |
Please join the [#mgmtconfig](https://webchat.freenode.net/?channels=#mgmtconfig) IRC community! |---|---|---|
If you have a well phrased question that might benefit others, consider asking it by sending a patch to the documentation [FAQ](https://github.com/purpleidea/mgmt/blob/master/DOCUMENTATION.md#usage-and-frequently-asked-questions) section. I'll merge your question, and a patch with the answer! | IRC | [#mgmtconfig](https://webchat.freenode.net/?channels=#mgmtconfig) on Freenode |
| Twitter | [@mgmtconfig](https://twitter.com/mgmtconfig) & [#mgmtconfig](https://twitter.com/hashtag/mgmtconfig) |
| Mailing list | [mgmtconfig-list@redhat.com](https://www.redhat.com/mailman/listinfo/mgmtconfig-list) |
## Quick start: ## Status:
* Either get the golang dependencies on your own, or run `make deps` if you're comfortable with how we install them. Mgmt is a fairly new project.
* Run `make build` to get a fresh built `mgmt` binary. We're working towards being minimally useful for production environments.
* Run `cd $(mktemp --tmpdir -d tmp.XXX) && etcd` to get etcd running. The `mgmt` software will do this automatically for you in the future. We aren't feature complete for what we'd consider a 1.x release yet.
* Run `time ./mgmt run --file examples/graph0.yaml --converged-timeout=1` to try out a very simple example! With your help you'll be able to influence our design and get us there sooner!
* To run continuously in the default mode of operation, omit the `--converged-timeout` option.
* Have fun hacking on our future technology!
## Examples:
Please look in the [examples/](examples/) folder for more examples!
## Documentation: ## Documentation:
Please see: [DOCUMENTATION.md](DOCUMENTATION.md) or [PDF](https://pdfdoc-purpleidea.rhcloud.com/pdf/https://github.com/purpleidea/mgmt/blob/master/DOCUMENTATION.md). Please read, enjoy and help improve our documentation!
| Documentation | Additional Notes |
|---|---|
| [general documentation](docs/documentation.md) | for everyone |
| [quick start guide](docs/quick-start-guide.md) | for mgmt developers |
| [resource guide](docs/resource-guide.md) | for mgmt developers |
| [godoc API reference](https://godoc.org/github.com/purpleidea/mgmt) | for mgmt developers |
| [prometheus guide](docs/prometheus.md) | for everyone |
| [puppet guide](docs/puppet-guide.md) | for puppet sysadmins |
## Questions:
Please ask in the [community](#community)!
If you have a well phrased question that might benefit others, consider asking it by sending a patch to the documentation [FAQ](https://github.com/purpleidea/mgmt/blob/master/docs/documentation.md#usage-and-frequently-asked-questions) section. I'll merge your question, and a patch with the answer!
## Roadmap: ## Roadmap:
Please see: [TODO.md](TODO.md) for a list of upcoming work and TODO items. Please see: [TODO.md](TODO.md) for a list of upcoming work and TODO items.
Please get involved by working on one of these items or by suggesting something else! Please get involved by working on one of these items or by suggesting something else!
Feel free to grab one of the straightforward [#mgmtlove](https://github.com/purpleidea/mgmt/labels/mgmtlove) issues if you're a first time contributor to the project or if you're unsure about what to hack on!
## Bugs: ## Bugs:
Please set the `DEBUG` constant in [main.go](https://github.com/purpleidea/mgmt/blob/master/main.go) to `true`, and post the logs when you report the [issue](https://github.com/purpleidea/mgmt/issues). Please set the `DEBUG` constant in [main.go](https://github.com/purpleidea/mgmt/blob/master/main.go) to `true`, and post the logs when you report the [issue](https://github.com/purpleidea/mgmt/issues).
Bonus points if you provide a [shell](https://github.com/purpleidea/mgmt/tree/master/test/shell) or [OMV](https://github.com/purpleidea/mgmt/tree/master/test/omv) reproducible test case. Bonus points if you provide a [shell](https://github.com/purpleidea/mgmt/tree/master/test/shell) or [OMV](https://github.com/purpleidea/mgmt/tree/master/test/omv) reproducible test case.
Feel free to read my article on [debugging golang programs](https://ttboj.wordpress.com/2016/02/15/debugging-golang-programs/).
## Dependencies:
* golang 1.4 or higher (required, available in most distros)
* golang libraries (required, available with `go get`)
go get github.com/coreos/etcd/client
go get gopkg.in/yaml.v2
go get gopkg.in/fsnotify.v1
go get github.com/codegangsta/cli
go get github.com/coreos/go-systemd/dbus
go get github.com/coreos/go-systemd/util
* stringer (required for building), available as a package on some platforms, otherwise via `go get`
go get golang.org/x/tools/cmd/stringer
* pandoc (optional, for building a pdf of the documentation)
* graphviz (optional, for building a visual representation of the graph)
## Patches: ## Patches:
We'd love to have your patches! Please send them by email, or as a pull request. We'd love to have your patches! Please send them by email, or as a pull request.
## On the web: ## On the web:
* Introductory blog post: [https://ttboj.wordpress.com/2016/01/18/next-generation-configuration-mgmt/](https://ttboj.wordpress.com/2016/01/18/next-generation-configuration-mgmt/) | Author | Format | Subject |
* Julian Dunn at Cfgmgmtcamp 2016 [https://www.youtube.com/watch?v=kfF9IATUask&t=1949&html5=1](https://www.youtube.com/watch?v=kfF9IATUask&t=1949&html5=1) |---|---|---|
| James Shubin | blog | [Next generation configuration mgmt](https://ttboj.wordpress.com/2016/01/18/next-generation-configuration-mgmt/) |
| James Shubin | video | [Introductory recording from DevConf.cz 2016](https://www.youtube.com/watch?v=GVhpPF0j-iE&html5=1) |
| James Shubin | video | [Introductory recording from CfgMgmtCamp.eu 2016](https://www.youtube.com/watch?v=fNeooSiIRnA&html5=1) |
| Julian Dunn | video | [On mgmt at CfgMgmtCamp.eu 2016](https://www.youtube.com/watch?v=kfF9IATUask&t=1949&html5=1) |
| Walter Heck | slides | [On mgmt at CfgMgmtCamp.eu 2016](http://www.slideshare.net/olindata/configuration-management-time-for-a-4th-generation/3) |
| Marco Marongiu | blog | [On mgmt](http://syslog.me/2016/02/15/leap-or-die/) |
| Felix Frank | blog | [From Catalog To Mgmt (on puppet to mgmt "transpiling")](https://ffrank.github.io/features/2016/02/18/from-catalog-to-mgmt/) |
| James Shubin | blog | [Automatic edges in mgmt (...and the pkg resource)](https://ttboj.wordpress.com/2016/03/14/automatic-edges-in-mgmt/) |
| James Shubin | blog | [Automatic grouping in mgmt](https://ttboj.wordpress.com/2016/03/30/automatic-grouping-in-mgmt/) |
| John Arundel | tweet | [“Puppets days are numbered.”](https://twitter.com/bitfield/status/732157519142002688) |
| Felix Frank | blog | [Puppet, Meet Mgmt (on puppet to mgmt internals)](https://ffrank.github.io/features/2016/06/12/puppet,-meet-mgmt/) |
| Felix Frank | blog | [Puppet Powered Mgmt (puppet to mgmt tl;dr)](https://ffrank.github.io/features/2016/06/19/puppet-powered-mgmt/) |
| James Shubin | blog | [Automatic clustering in mgmt](https://ttboj.wordpress.com/2016/06/20/automatic-clustering-in-mgmt/) |
| James Shubin | video | [Recording from CoreOSFest 2016](https://www.youtube.com/watch?v=KVmDCUA42wc&html5=1) |
| James Shubin | video | [Recording from DebConf16](http://meetings-archive.debian.net/pub/debian-meetings/2016/debconf16/Next_Generation_Config_Mgmt.webm) ([Slides](https://annex.debconf.org//debconf-share/debconf16/slides/15-next-generation-config-mgmt.pdf)) |
| Felix Frank | blog | [Edging It All In (puppet and mgmt edges)](https://ffrank.github.io/features/2016/07/12/edging-it-all-in/) |
| Felix Frank | blog | [Translating All The Things (puppet to mgmt translation warnings)](https://ffrank.github.io/features/2016/08/19/translating-all-the-things/) |
| James Shubin | video | [Recording from systemd.conf 2016](https://www.youtube.com/watch?v=jB992Zb3nH0&html5=1) |
| James Shubin | blog | [Remote execution in mgmt](https://ttboj.wordpress.com/2016/10/07/remote-execution-in-mgmt/) |
| James Shubin | video | [Recording from High Load Strategy 2016](https://vimeo.com/191493409) |
| James Shubin | video | [Recording from NLUUG 2016](https://www.youtube.com/watch?v=MmpwOQAb_SE&html5=1) |
| James Shubin | blog | [Send/Recv in mgmt](https://ttboj.wordpress.com/2016/12/07/sendrecv-in-mgmt/) |
| James Shubin | blog | [Metaparameters in mgmt](https://ttboj.wordpress.com/2017/03/01/metaparameters-in-mgmt/) |
## ##

6
THANKS
View File

@@ -9,10 +9,16 @@ Chris Wright - For encouraging me to continue work on my prototype.
Daniel Riek - For supporting and sheltering this project from bureaucracy. Daniel Riek - For supporting and sheltering this project from bureaucracy.
Diego Ongaro - For good chats, particularly around distributed systems.
Felix Frank - For taking a difficult problem and building an inspiring solution.
Ira Cooper - For having an algorithmic design discussion with me. Ira Cooper - For having an algorithmic design discussion with me.
Jeff Darcy - For some algorithm recommendations, and NACKing my TopoSort idea! Jeff Darcy - For some algorithm recommendations, and NACKing my TopoSort idea!
Red Hat, inc. - For paying my salary, thus financially supporting my hacking. Red Hat, inc. - For paying my salary, thus financially supporting my hacking.
Samuel Gélineau - For help with programming language theory and design.
And many others... And many others...

48
TODO.md
View File

@@ -3,27 +3,54 @@ If you're looking for something to do, look here!
Let us know if you're working on one of the items. Let us know if you're working on one of the items.
## Package resource ## Package resource
- [ ] base type [bug](https://github.com/purpleidea/mgmt/issues/11) - [ ] getfiles support on debian [bug](https://github.com/hughsie/PackageKit/issues/118)
- [ ] directory info on fedora [bug](https://github.com/hughsie/PackageKit/issues/117)
- [ ] dnf blocker [bug](https://github.com/hughsie/PackageKit/issues/110) - [ ] dnf blocker [bug](https://github.com/hughsie/PackageKit/issues/110)
- [ ] install signal blocker [bug](https://github.com/hughsie/PackageKit/issues/109) - [ ] install signal blocker [bug](https://github.com/hughsie/PackageKit/issues/109)
## File resource ## File resource [bug](https://github.com/purpleidea/mgmt/issues/64) [:heart:](https://github.com/purpleidea/mgmt/labels/mgmtlove)
- [ ] ability to make/delete folders - [ ] chown/chmod support [:heart:](https://github.com/purpleidea/mgmt/labels/mgmtlove)
- [ ] recursive argument (can recursively watch/modify contents) - [ ] user/group support [:heart:](https://github.com/purpleidea/mgmt/labels/mgmtlove)
- [ ] force argument (can cause switch from file <-> folder) - [ ] recurse limit support [:heart:](https://github.com/purpleidea/mgmt/labels/mgmtlove)
- [ ] fanotify support [bug](https://github.com/go-fsnotify/fsnotify/issues/114) - [ ] fanotify support [bug](https://github.com/go-fsnotify/fsnotify/issues/114)
## Svc resource
- [ ] base resource improvements
## Exec resource ## Exec resource
- [ ] base resource improvements - [ ] base resource improvements
## Timer resource ## Timer resource
- [ ] increment algorithm (linear, exponential, etc...) [:heart:](https://github.com/purpleidea/mgmt/labels/mgmtlove)
## User/Group resource
- [ ] base resource [:heart:](https://github.com/purpleidea/mgmt/labels/mgmtlove)
- [ ] automatic edges to file resource [:heart:](https://github.com/purpleidea/mgmt/labels/mgmtlove)
## Virt (libvirt) resource
- [ ] base resource improvements [:heart:](https://github.com/purpleidea/mgmt/labels/mgmtlove)
- [ ] port to upstream https://github.com/libvirt/libvirt-go [:heart:](https://github.com/purpleidea/mgmt/labels/mgmtlove)
## Net (systemd-networkd) resource
- [ ] base resource - [ ] base resource
- [ ] reset on recompile
- [ ] increment algorithm (linear, exponential, etc...) ## Nspawn (systemd-nspawn) resource
- [ ] base resource [:heart:](https://github.com/purpleidea/mgmt/labels/mgmtlove)
## Mount (systemd-mount) resource
- [ ] base resource [:heart:](https://github.com/purpleidea/mgmt/labels/mgmtlove)
## Cron (systemd-timer) resource
- [ ] base resource [:heart:](https://github.com/purpleidea/mgmt/labels/mgmtlove)
## Http resource
- [ ] base resource [:heart:](https://github.com/purpleidea/mgmt/labels/mgmtlove)
## Etcd improvements ## Etcd improvements
- [ ] embedded etcd master - [ ] fix embedded etcd master race
- [ ] capnslog fixes [bug](https://github.com/coreos/etcd/issues/4115)
## Torrent/dht file transfer
- [ ] base plumbing
## Language improvements ## Language improvements
- [ ] language design - [ ] language design
@@ -35,9 +62,6 @@ Let us know if you're working on one of the items.
## Other ## Other
- [ ] better error/retry handling - [ ] better error/retry handling
- [ ] resource grouping
- [ ] automatic dependency adding (eg: packagekit file dependencies)
- [ ] rpm package target in Makefile
- [ ] deb package target in Makefile - [ ] deb package target in Makefile
- [ ] reproducible builds - [ ] reproducible builds
- [ ] add your suggestions! - [ ] add your suggestions!

38
Vagrantfile vendored Normal file
View File

@@ -0,0 +1,38 @@
Vagrant.configure(2) do |config|
config.ssh.forward_agent = true
config.ssh.username = 'vagrant'
config.vm.network "private_network", ip: "192.168.219.2"
config.vm.synced_folder ".", "/vagrant", disabled: true
config.vm.define "mgmt-dev" do |instance|
instance.vm.box = "fedora/24-cloud-base"
end
config.vm.provider "virtualbox" do |v|
v.memory = 1536
v.cpus = 2
end
config.vm.provision "file", source: "vagrant/motd", destination: ".motd"
config.vm.provision "shell", inline: "cp ~vagrant/.motd /etc/motd"
config.vm.provision "file", source: "vagrant/mgmt.bashrc", destination: ".mgmt.bashrc"
config.vm.provision "file", source: "~/.gitconfig", destination: ".gitconfig"
# copied from make-deps.sh (with added git)
config.vm.provision "shell", inline: "dnf install -y libvirt-devel golang golang-googlecode-tools-stringer hg git"
# set up vagrant home
script = <<-SCRIPT
grep -q 'mgmt\.bashrc' ~/.bashrc || echo '. ~/.mgmt.bashrc' >>~/.bashrc
. ~/.mgmt.bashrc
go get -u github.com/purpleidea/mgmt
cd ~/gopath/src/github.com/purpleidea/mgmt
make deps
SCRIPT
config.vm.provision "shell" do |shell|
shell.privileged = false
shell.inline = script
end
end

2
art/.gitignore vendored Normal file
View File

@@ -0,0 +1,2 @@
*.png
misc/

BIN
art/mgmt.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 24 KiB

View File

@@ -0,0 +1,94 @@
<?xml version="1.0" encoding="utf-8"?>
<svg version="1.1" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" x="0px" y="0px"
viewBox="0 0 120 107.1" style="enable-background:new 0 0 120 107.1;" xml:space="preserve">
<style type="text/css">
.st0{fill:#FFFFFF;}
.st1{fill:#E22434;}
.st2{display:none;}
.st3{fill:#1B3663;}
.st4{fill:#00B1D1;}
.st5{fill:#BFE6EF;}
.st6{fill:#69CBE0;}
.st7{fill:#0080BD;}
.st8{fill:#005DAB;}
.st9{fill:#183660;}
.st10{fill:none;stroke:#183660;stroke-width:1.9441;stroke-miterlimit:10;}
.st11{fill:none;stroke:#183660;stroke-width:1.6201;stroke-miterlimit:10;}
.st12{fill:none;stroke:#183660;stroke-width:1.2961;stroke-miterlimit:10;}
.st13{fill:none;stroke:#C0E6EF;stroke-width:1.9441;stroke-miterlimit:10;}
.st14{fill:#C0E6EF;}
.st15{fill:none;stroke:#C0E6EF;stroke-width:1.2961;stroke-miterlimit:10;}
.st16{fill:none;stroke:#C0E6EF;stroke-width:1.6201;stroke-miterlimit:10;}
.st17{fill:none;stroke:#FFFFFF;stroke-width:1.9441;stroke-miterlimit:10;}
.st18{fill:none;stroke:#FFFFFF;stroke-width:1.2961;stroke-miterlimit:10;}
.st19{fill:none;stroke:#FFFFFF;stroke-width:1.6201;stroke-miterlimit:10;}
</style>
<g id="Layer_2">
</g>
<g id="Layer_1">
<g>
<g>
<g>
<line class="st10" x1="29.2" y1="24.1" x2="52.1" y2="42.8"/>
<g>
<polygon class="st9" points="27.7,27.8 24.9,20.5 32.6,21.8 "/>
</g>
</g>
</g>
<circle class="st5" cx="16.1" cy="12.2" r="12.1"/>
<g>
<g>
<line class="st10" x1="52.1" y1="42.1" x2="74.1" y2="80"/>
<g>
<polygon class="st9" points="70.1,80.9 76.9,84.8 76.8,77.1 "/>
</g>
</g>
</g>
<g>
<g>
<line class="st10" x1="69.4" y1="46.7" x2="95.8" y2="52.4"/>
<g>
<polygon class="st9" points="69.7,50.7 63.9,45.6 71.3,43.2 "/>
</g>
</g>
</g>
<g>
<g>
<line class="st10" x1="52.1" y1="42.8" x2="71.9" y2="30.1"/>
<g>
<polygon class="st9" points="73.1,34 76.6,27.1 68.9,27.5 "/>
</g>
</g>
</g>
<g>
<g>
<line class="st10" x1="16.8" y1="49.6" x2="34.8" y2="46.5"/>
<g>
<polygon class="st9" points="34.3,50.5 40.3,45.6 33,42.9 "/>
</g>
</g>
</g>
<g>
<g>
<line class="st10" x1="92.3" y1="79.5" x2="107.8" y2="54.2"/>
<g>
<polygon class="st9" points="96.1,80.6 89.3,84.3 89.5,76.5 "/>
</g>
</g>
</g>
<g>
<g>
<line class="st10" x1="97.3" y1="36.5" x2="107.8" y2="54.2"/>
<g>
<polygon class="st9" points="94.6,39.4 94.5,31.7 101.2,35.5 "/>
</g>
</g>
</g>
<circle class="st3" cx="52.1" cy="42.8" r="12.1"/>
<circle class="st4" cx="12.2" cy="50.8" r="12.1"/>
<circle class="st7" cx="87.5" cy="21.7" r="12.1"/>
<circle class="st8" cx="83.5" cy="95" r="12.1"/>
<circle class="st6" cx="107.8" cy="54.2" r="12.1"/>
</g>
</g>
</svg>

After

Width:  |  Height:  |  Size: 2.8 KiB

View File

@@ -0,0 +1,132 @@
<?xml version="1.0" encoding="utf-8"?>
<svg version="1.1" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" x="0px" y="0px"
viewBox="0 0 168.3 131.6" style="enable-background:new 0 0 168.3 131.6;" xml:space="preserve">
<style type="text/css">
.st0{fill:#FFFFFF;}
.st1{fill:#E22434;}
.st2{display:none;}
.st3{fill:#1B3663;}
.st4{fill:#00B1D1;}
.st5{fill:#BFE6EF;}
.st6{fill:#69CBE0;}
.st7{fill:#0080BD;}
.st8{fill:#005DAB;}
.st9{fill:#183660;}
.st10{fill:none;stroke:#183660;stroke-width:1.9441;stroke-miterlimit:10;}
.st11{fill:none;stroke:#183660;stroke-width:1.6201;stroke-miterlimit:10;}
.st12{fill:none;stroke:#183660;stroke-width:1.2961;stroke-miterlimit:10;}
.st13{fill:none;stroke:#C0E6EF;stroke-width:1.9441;stroke-miterlimit:10;}
.st14{fill:#C0E6EF;}
.st15{fill:none;stroke:#C0E6EF;stroke-width:1.2961;stroke-miterlimit:10;}
.st16{fill:none;stroke:#C0E6EF;stroke-width:1.6201;stroke-miterlimit:10;}
.st17{fill:none;stroke:#FFFFFF;stroke-width:1.9441;stroke-miterlimit:10;}
.st18{fill:none;stroke:#FFFFFF;stroke-width:1.2961;stroke-miterlimit:10;}
.st19{fill:none;stroke:#FFFFFF;stroke-width:1.6201;stroke-miterlimit:10;}
</style>
<g id="Layer_2">
</g>
<g id="Layer_1">
<g>
<g>
<g>
<path class="st3" d="M4.7,105l0.1,1.8c1.1-1.4,2.6-2.1,4.4-2.1c1.9,0,3.2,0.9,4,2.6c1.1-1.7,2.6-2.6,4.7-2.6
c3.3,0,5,2.3,5.1,6.9V124h-5v-12.1c0-1.1-0.2-1.9-0.5-2.4c-0.3-0.5-0.8-0.7-1.5-0.7c-0.9,0-1.6,0.6-2.1,1.7l0,0.6V124H9v-12.1
c0-1.1-0.1-1.9-0.4-2.4c-0.3-0.5-0.8-0.7-1.6-0.7c-0.9,0-1.5,0.5-2,1.4V124H0v-19H4.7z"/>
<path class="st3" d="M26.4,113.9c0-3.1,0.6-5.4,1.7-7s2.7-2.3,4.7-2.3c1.7,0,3.1,0.7,4,2L37,105h4.5v19c0,2.4-0.7,4.3-2,5.6
c-1.4,1.3-3.3,1.9-5.9,1.9c-1,0-2.1-0.2-3.3-0.6s-2-0.9-2.6-1.6l1.7-3.4c0.5,0.5,1.1,0.9,1.8,1.2c0.7,0.3,1.5,0.5,2.1,0.5
c1.1,0,1.9-0.3,2.4-0.8s0.7-1.4,0.7-2.6v-1.6c-0.9,1.2-2.2,1.9-3.7,1.9c-2,0-3.6-0.8-4.7-2.4s-1.7-3.8-1.7-6.7V113.9z
M31.4,115.2c0,1.8,0.2,3,0.7,3.8s1.2,1.2,2.2,1.2c1,0,1.8-0.4,2.3-1.1V110c-0.5-0.8-1.3-1.2-2.2-1.2c-1,0-1.7,0.4-2.2,1.2
s-0.7,2.1-0.7,3.9V115.2z"/>
<path class="st3" d="M50.1,105l0.1,1.8c1.1-1.4,2.6-2.1,4.4-2.1c1.9,0,3.2,0.9,4,2.6c1.1-1.7,2.6-2.6,4.7-2.6
c3.3,0,5,2.3,5.1,6.9V124h-5v-12.1c0-1.1-0.2-1.9-0.5-2.4s-0.8-0.7-1.5-0.7c-0.9,0-1.6,0.6-2.1,1.7l0,0.6V124h-5v-12.1
c0-1.1-0.1-1.9-0.4-2.4s-0.8-0.7-1.6-0.7c-0.9,0-1.5,0.5-2,1.4V124h-5v-19H50.1z"/>
<path class="st3" d="M78.2,100.3v4.7h2.5v3.7h-2.5v9.5c0,0.8,0.1,1.3,0.3,1.5c0.2,0.3,0.6,0.4,1.2,0.4c0.5,0,0.9,0,1.2-0.1
l0,3.9c-0.8,0.3-1.8,0.5-2.7,0.5c-3.2,0-4.8-1.8-4.9-5.5v-10.1h-2.2V105h2.2v-4.7H78.2z"/>
<path class="st4" d="M90.6,122.6c1.4,0,2.4-0.4,3.1-1.1c0.7-0.8,1.1-1.9,1.2-3.3h1.9c-0.1,1.9-0.7,3.5-1.9,4.6
c-1.1,1.1-2.6,1.7-4.3,1.7c-2.3,0-4-0.7-5.1-2.2s-1.7-3.6-1.8-6.4v-2.3c0-2.9,0.6-5.1,1.7-6.6s2.9-2.2,5.1-2.2
c1.9,0,3.4,0.6,4.5,1.8s1.7,2.9,1.7,5H95c-0.1-1.6-0.5-2.8-1.2-3.6s-1.8-1.3-3.1-1.3c-1.7,0-2.9,0.6-3.7,1.7s-1.2,2.9-1.2,5.2
v2.2c0,2.4,0.4,4.2,1.2,5.3S89,122.6,90.6,122.6z"/>
<path class="st4" d="M100.5,113.6c0-2.7,0.6-4.9,1.9-6.5s2.9-2.4,5.1-2.4c2.2,0,3.9,0.8,5.1,2.4s1.9,3.7,1.9,6.5v2
c0,2.8-0.6,5-1.9,6.5s-2.9,2.3-5.1,2.3c-2.1,0-3.8-0.8-5-2.3s-1.9-3.6-1.9-6.3V113.6z M102.5,115.5c0,2.2,0.4,3.9,1.3,5.2
c0.9,1.3,2.1,1.9,3.7,1.9c1.6,0,2.8-0.6,3.7-1.8s1.3-2.9,1.3-5.2v-2c0-2.2-0.4-3.9-1.3-5.2c-0.9-1.3-2.1-1.9-3.7-1.9
c-1.5,0-2.7,0.6-3.6,1.8s-1.3,2.9-1.4,5.1V115.5z"/>
<path class="st4" d="M121.1,105l0.1,3c0.6-1,1.3-1.9,2.2-2.5s1.9-0.9,3-0.9c3.3,0,5,2.2,5.1,6.6V124h-1.9v-12.5
c0-1.7-0.3-3-0.9-3.8c-0.6-0.8-1.5-1.2-2.8-1.2c-1,0-1.9,0.4-2.8,1.2s-1.4,1.8-1.9,3.2V124h-2v-19H121.1z"/>
<path class="st4" d="M138.2,124v-17.3h-2.6V105h2.6v-2.5c0-1.9,0.5-3.3,1.3-4.3s2-1.5,3.5-1.5c0.7,0,1.3,0.1,1.9,0.3l-0.1,1.8
c-0.5-0.1-1-0.2-1.6-0.2c-0.9,0-1.7,0.4-2.2,1.1s-0.8,1.7-0.8,3v2.3h3.7v1.8h-3.7V124H138.2z"/>
<path class="st4" d="M148,99.5c0-0.4,0.1-0.7,0.3-1s0.5-0.4,0.9-0.4s0.7,0.1,1,0.4s0.3,0.6,0.3,1s-0.1,0.7-0.3,1s-0.5,0.4-1,0.4
s-0.7-0.1-0.9-0.4S148,99.9,148,99.5z M150.2,124h-2v-19h2V124z"/>
<path class="st4" d="M155.3,113.6c0-3,0.5-5.2,1.5-6.7s2.6-2.2,4.6-2.2c2.2,0,3.8,1,4.9,3l0.1-2.7h1.8v19.6c0,2.3-0.6,4-1.6,5.2
c-1.1,1.2-2.7,1.8-4.7,1.8c-1,0-2.1-0.3-3.2-0.8s-1.9-1.1-2.4-1.9l0.9-1.4c1.3,1.5,2.8,2.2,4.5,2.2c1.6,0,2.7-0.4,3.5-1.3
c0.7-0.8,1.1-2.1,1.1-3.8v-3c-1.1,1.8-2.7,2.7-4.9,2.7c-2,0-3.5-0.7-4.5-2.2s-1.6-3.6-1.6-6.5V113.6z M157.2,115.4
c0,2.4,0.4,4.2,1.1,5.4s1.9,1.7,3.5,1.7c2.1,0,3.6-1,4.5-3v-9.7c-0.9-2.2-2.4-3.3-4.4-3.3c-1.6,0-2.8,0.6-3.5,1.7
s-1.1,2.9-1.1,5.3V115.4z"/>
</g>
</g>
<g>
<g>
<g>
<line class="st11" x1="58.9" y1="20.1" x2="77.9" y2="35.6"/>
<g>
<polygon class="st9" points="57.6,23.2 55.3,17.1 61.7,18.2 "/>
</g>
</g>
</g>
<circle class="st5" cx="48" cy="10.1" r="10.1"/>
<g>
<g>
<line class="st11" x1="77.9" y1="35.1" x2="96.2" y2="66.7"/>
<g>
<polygon class="st9" points="93,67.5 98.6,70.7 98.6,64.2 "/>
</g>
</g>
</g>
<g>
<g>
<line class="st11" x1="92.3" y1="38.9" x2="114.4" y2="43.6"/>
<g>
<polygon class="st9" points="92.6,42.3 87.8,38 93.9,36 "/>
</g>
</g>
</g>
<g>
<g>
<line class="st11" x1="77.9" y1="35.6" x2="94.5" y2="25.1"/>
<g>
<polygon class="st9" points="95.4,28.3 98.4,22.6 91.9,22.9 "/>
</g>
</g>
</g>
<g>
<g>
<line class="st11" x1="48.5" y1="41.3" x2="63.5" y2="38.7"/>
<g>
<polygon class="st9" points="63.1,42.1 68.1,38 62,35.7 "/>
</g>
</g>
</g>
<g>
<g>
<line class="st11" x1="111.4" y1="66.3" x2="124.4" y2="45.2"/>
<g>
<polygon class="st9" points="114.6,67.2 109,70.2 109.1,63.8 "/>
</g>
</g>
</g>
<g>
<g>
<line class="st11" x1="115.6" y1="30.4" x2="124.4" y2="45.2"/>
<g>
<polygon class="st9" points="113.3,32.8 113.3,26.4 118.9,29.5 "/>
</g>
</g>
</g>
<circle class="st3" cx="77.9" cy="35.6" r="10.1"/>
<circle class="st4" cx="44.6" cy="42.4" r="10.1"/>
<circle class="st7" cx="107.4" cy="18.1" r="10.1"/>
<circle class="st8" cx="104.1" cy="79.1" r="10.1"/>
<circle class="st6" cx="124.4" cy="45.2" r="10.1"/>
</g>
</g>
</g>
</svg>

After

Width:  |  Height:  |  Size: 6.2 KiB

View File

@@ -0,0 +1,132 @@
<?xml version="1.0" encoding="utf-8"?>
<svg version="1.1" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" x="0px" y="0px"
viewBox="0 0 260.4 71.4" style="enable-background:new 0 0 260.4 71.4;" xml:space="preserve">
<style type="text/css">
.st0{fill:#FFFFFF;}
.st1{fill:#E22434;}
.st2{display:none;}
.st3{fill:#1B3663;}
.st4{fill:#00B1D1;}
.st5{fill:#BFE6EF;}
.st6{fill:#69CBE0;}
.st7{fill:#0080BD;}
.st8{fill:#005DAB;}
.st9{fill:#183660;}
.st10{fill:none;stroke:#183660;stroke-width:1.9441;stroke-miterlimit:10;}
.st11{fill:none;stroke:#183660;stroke-width:1.6201;stroke-miterlimit:10;}
.st12{fill:none;stroke:#183660;stroke-width:1.2961;stroke-miterlimit:10;}
.st13{fill:none;stroke:#C0E6EF;stroke-width:1.9441;stroke-miterlimit:10;}
.st14{fill:#C0E6EF;}
.st15{fill:none;stroke:#C0E6EF;stroke-width:1.2961;stroke-miterlimit:10;}
.st16{fill:none;stroke:#C0E6EF;stroke-width:1.6201;stroke-miterlimit:10;}
.st17{fill:none;stroke:#FFFFFF;stroke-width:1.9441;stroke-miterlimit:10;}
.st18{fill:none;stroke:#FFFFFF;stroke-width:1.2961;stroke-miterlimit:10;}
.st19{fill:none;stroke:#FFFFFF;stroke-width:1.6201;stroke-miterlimit:10;}
</style>
<g id="Layer_2">
</g>
<g id="Layer_1">
<g>
<g>
<g>
<path class="st3" d="M96.7,25.7l0.1,1.8c1.1-1.4,2.6-2.1,4.4-2.1c1.9,0,3.2,0.9,4,2.6c1.1-1.7,2.6-2.6,4.7-2.6
c3.3,0,5,2.3,5.1,6.9v12.5h-5V32.6c0-1.1-0.2-1.9-0.5-2.4s-0.8-0.7-1.5-0.7c-0.9,0-1.6,0.6-2.1,1.7l0,0.6v12.9h-5V32.6
c0-1.1-0.1-1.9-0.4-2.4s-0.8-0.7-1.6-0.7c-0.9,0-1.5,0.5-2,1.4v13.8h-5v-19H96.7z"/>
<path class="st3" d="M118.5,34.6c0-3.1,0.6-5.4,1.7-7s2.7-2.3,4.7-2.3c1.7,0,3.1,0.7,4,2l0.2-1.7h4.5v19c0,2.4-0.7,4.3-2,5.6
c-1.4,1.3-3.3,1.9-5.9,1.9c-1,0-2.1-0.2-3.3-0.6s-2-0.9-2.6-1.6l1.7-3.4c0.5,0.5,1.1,0.9,1.8,1.2c0.7,0.3,1.5,0.5,2.1,0.5
c1.1,0,1.9-0.3,2.4-0.8s0.7-1.4,0.7-2.6v-1.6c-0.9,1.2-2.2,1.9-3.7,1.9c-2,0-3.6-0.8-4.7-2.4s-1.7-3.8-1.7-6.7V34.6z
M123.5,35.9c0,1.8,0.2,3,0.7,3.8s1.2,1.2,2.2,1.2c1,0,1.8-0.4,2.3-1.1v-9.1c-0.5-0.8-1.3-1.2-2.2-1.2c-1,0-1.7,0.4-2.2,1.2
s-0.7,2.1-0.7,3.9V35.9z"/>
<path class="st3" d="M142.2,25.7l0.1,1.8c1.1-1.4,2.6-2.1,4.4-2.1c1.9,0,3.2,0.9,4,2.6c1.1-1.7,2.6-2.6,4.7-2.6
c3.3,0,5,2.3,5.1,6.9v12.5h-5V32.6c0-1.1-0.2-1.9-0.5-2.4c-0.3-0.5-0.8-0.7-1.5-0.7c-0.9,0-1.6,0.6-2.1,1.7l0,0.6v12.9h-5V32.6
c0-1.1-0.1-1.9-0.4-2.4c-0.3-0.5-0.8-0.7-1.6-0.7c-0.9,0-1.5,0.5-2,1.4v13.8h-5v-19H142.2z"/>
<path class="st3" d="M170.3,21v4.7h2.5v3.7h-2.5v9.5c0,0.8,0.1,1.3,0.3,1.5c0.2,0.3,0.6,0.4,1.2,0.4c0.5,0,0.9,0,1.2-0.1l0,3.9
c-0.8,0.3-1.8,0.5-2.7,0.5c-3.2,0-4.8-1.8-4.9-5.5V29.4h-2.2v-3.7h2.2V21H170.3z"/>
<path class="st4" d="M182.7,43.2c1.4,0,2.4-0.4,3.1-1.1s1.1-1.8,1.2-3.3h1.9c-0.1,1.9-0.7,3.5-1.9,4.6s-2.6,1.7-4.3,1.7
c-2.3,0-4-0.7-5.1-2.2s-1.7-3.6-1.8-6.4v-2.3c0-2.9,0.6-5.1,1.7-6.6s2.9-2.2,5.1-2.2c1.9,0,3.4,0.6,4.5,1.8s1.7,2.9,1.7,5H187
c-0.1-1.6-0.5-2.8-1.2-3.6s-1.8-1.3-3.1-1.3c-1.7,0-2.9,0.6-3.7,1.7c-0.8,1.1-1.2,2.9-1.2,5.2v2.2c0,2.4,0.4,4.2,1.2,5.3
S181,43.2,182.7,43.2z"/>
<path class="st4" d="M192.6,34.2c0-2.7,0.6-4.9,1.9-6.5s2.9-2.4,5.1-2.4c2.2,0,3.9,0.8,5.1,2.4c1.2,1.6,1.9,3.7,1.9,6.5v2
c0,2.8-0.6,5-1.9,6.5s-2.9,2.3-5.1,2.3s-3.8-0.8-5-2.3s-1.9-3.6-1.9-6.3V34.2z M194.6,36.2c0,2.2,0.4,3.9,1.3,5.2
c0.9,1.3,2.1,1.9,3.7,1.9c1.6,0,2.8-0.6,3.7-1.8c0.8-1.2,1.3-2.9,1.3-5.2v-2c0-2.2-0.4-3.9-1.3-5.2c-0.9-1.3-2.1-1.9-3.7-1.9
c-1.5,0-2.7,0.6-3.6,1.8c-0.9,1.2-1.3,2.9-1.4,5.1V36.2z"/>
<path class="st4" d="M213.2,25.7l0.1,3c0.6-1,1.3-1.9,2.2-2.5s1.9-0.9,3-0.9c3.3,0,5,2.2,5.1,6.6v12.7h-1.9V32.2
c0-1.7-0.3-3-0.9-3.8c-0.6-0.8-1.5-1.2-2.8-1.2c-1,0-1.9,0.4-2.8,1.2s-1.4,1.8-1.9,3.2v13.2h-2v-19H213.2z"/>
<path class="st4" d="M230.3,44.7V27.4h-2.6v-1.8h2.6v-2.5c0-1.9,0.5-3.3,1.3-4.3s2-1.5,3.5-1.5c0.7,0,1.3,0.1,1.9,0.3l-0.1,1.8
c-0.5-0.1-1-0.2-1.6-0.2c-0.9,0-1.7,0.4-2.2,1.1s-0.8,1.7-0.8,3v2.3h3.7v1.8h-3.7v17.3H230.3z"/>
<path class="st4" d="M240.1,20.2c0-0.4,0.1-0.7,0.3-1s0.5-0.4,0.9-0.4s0.7,0.1,1,0.4s0.3,0.6,0.3,1s-0.1,0.7-0.3,1
s-0.5,0.4-1,0.4s-0.7-0.1-0.9-0.4S240.1,20.6,240.1,20.2z M242.3,44.7h-2v-19h2V44.7z"/>
<path class="st4" d="M247.4,34.3c0-3,0.5-5.2,1.5-6.7s2.6-2.2,4.6-2.2c2.2,0,3.8,1,4.9,3l0.1-2.7h1.8v19.6c0,2.3-0.6,4-1.6,5.2
s-2.7,1.8-4.7,1.8c-1,0-2.1-0.3-3.2-0.8s-1.9-1.1-2.4-1.9l0.9-1.4c1.3,1.5,2.8,2.2,4.5,2.2c1.6,0,2.7-0.4,3.5-1.3
c0.7-0.8,1.1-2.1,1.1-3.8v-3c-1.1,1.8-2.7,2.7-4.9,2.7c-2,0-3.5-0.7-4.5-2.2s-1.6-3.6-1.6-6.5V34.3z M249.3,36.1
c0,2.4,0.4,4.2,1.1,5.4s1.9,1.7,3.5,1.7c2.1,0,3.6-1,4.5-3v-9.7c-0.9-2.2-2.4-3.3-4.4-3.3c-1.6,0-2.8,0.6-3.5,1.7
s-1.1,2.9-1.1,5.3V36.1z"/>
</g>
</g>
<g>
<g>
<g>
<line class="st12" x1="19.5" y1="16" x2="34.7" y2="28.5"/>
<g>
<polygon class="st9" points="18.4,18.5 16.6,13.7 21.7,14.5 "/>
</g>
</g>
</g>
<circle class="st5" cx="10.8" cy="8.1" r="8.1"/>
<g>
<g>
<line class="st12" x1="34.7" y1="28" x2="49.4" y2="53.3"/>
<g>
<polygon class="st9" points="46.8,54 51.2,56.5 51.2,51.4 "/>
</g>
</g>
</g>
<g>
<g>
<line class="st12" x1="46.2" y1="31.1" x2="63.9" y2="34.9"/>
<g>
<polygon class="st9" points="46.4,33.8 42.6,30.4 47.5,28.8 "/>
</g>
</g>
</g>
<g>
<g>
<line class="st12" x1="34.7" y1="28.5" x2="47.9" y2="20.1"/>
<g>
<polygon class="st9" points="48.7,22.7 51.1,18.1 45.9,18.3 "/>
</g>
</g>
</g>
<g>
<g>
<line class="st12" x1="11.2" y1="33.1" x2="23.2" y2="31"/>
<g>
<polygon class="st9" points="22.9,33.7 26.8,30.4 22,28.6 "/>
</g>
</g>
</g>
<g>
<g>
<line class="st12" x1="61.5" y1="53" x2="71.9" y2="36.1"/>
<g>
<polygon class="st9" points="64.1,53.7 59.5,56.2 59.7,51 "/>
</g>
</g>
</g>
<g>
<g>
<line class="st12" x1="64.2" y1="24.2" x2="70.7" y2="33.9"/>
<g>
<polygon class="st9" points="62.5,26.3 62.2,21.1 66.8,23.4 "/>
</g>
</g>
</g>
<circle class="st3" cx="34.7" cy="28.5" r="8.1"/>
<circle class="st4" cx="8.1" cy="33.9" r="8.1"/>
<circle class="st7" cx="58.3" cy="14.5" r="8.1"/>
<circle class="st8" cx="55.7" cy="63.3" r="8.1"/>
<circle class="st6" cx="71.9" cy="36.1" r="8.1"/>
</g>
</g>
</g>
</svg>

After

Width:  |  Height:  |  Size: 6.2 KiB

View File

@@ -0,0 +1,94 @@
<?xml version="1.0" encoding="utf-8"?>
<svg version="1.1" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" x="0px" y="0px"
viewBox="0 0 120 107.1" style="enable-background:new 0 0 120 107.1;" xml:space="preserve">
<style type="text/css">
.st0{fill:#FFFFFF;}
.st1{fill:#E22434;}
.st2{display:none;}
.st3{fill:#1B3663;}
.st4{fill:#00B1D1;}
.st5{fill:#BFE6EF;}
.st6{fill:#69CBE0;}
.st7{fill:#0080BD;}
.st8{fill:#005DAB;}
.st9{fill:#183660;}
.st10{fill:none;stroke:#183660;stroke-width:1.9441;stroke-miterlimit:10;}
.st11{fill:none;stroke:#183660;stroke-width:1.6201;stroke-miterlimit:10;}
.st12{fill:none;stroke:#183660;stroke-width:1.2961;stroke-miterlimit:10;}
.st13{fill:none;stroke:#C0E6EF;stroke-width:1.9441;stroke-miterlimit:10;}
.st14{fill:#C0E6EF;}
.st15{fill:none;stroke:#C0E6EF;stroke-width:1.2961;stroke-miterlimit:10;}
.st16{fill:none;stroke:#C0E6EF;stroke-width:1.6201;stroke-miterlimit:10;}
.st17{fill:none;stroke:#FFFFFF;stroke-width:1.9441;stroke-miterlimit:10;}
.st18{fill:none;stroke:#FFFFFF;stroke-width:1.2961;stroke-miterlimit:10;}
.st19{fill:none;stroke:#FFFFFF;stroke-width:1.6201;stroke-miterlimit:10;}
</style>
<g id="Layer_2">
</g>
<g id="Layer_1">
<g>
<g>
<g>
<line class="st13" x1="29.2" y1="24.1" x2="52.1" y2="42.8"/>
<g>
<polygon class="st14" points="27.7,27.8 24.9,20.5 32.6,21.8 "/>
</g>
</g>
</g>
<circle class="st3" cx="16.1" cy="12.2" r="12.1"/>
<g>
<g>
<line class="st13" x1="52.1" y1="42.1" x2="74.1" y2="80"/>
<g>
<polygon class="st14" points="70.1,80.9 76.9,84.8 76.8,77.1 "/>
</g>
</g>
</g>
<g>
<g>
<line class="st13" x1="69.4" y1="46.7" x2="95.8" y2="52.4"/>
<g>
<polygon class="st14" points="69.7,50.7 63.9,45.6 71.3,43.2 "/>
</g>
</g>
</g>
<g>
<g>
<line class="st13" x1="52.1" y1="42.8" x2="71.9" y2="30.1"/>
<g>
<polygon class="st14" points="73.1,34 76.6,27.1 68.9,27.5 "/>
</g>
</g>
</g>
<g>
<g>
<line class="st13" x1="16.8" y1="49.6" x2="34.8" y2="46.5"/>
<g>
<polygon class="st14" points="34.3,50.5 40.3,45.6 33,42.9 "/>
</g>
</g>
</g>
<g>
<g>
<line class="st13" x1="92.3" y1="79.5" x2="107.8" y2="54.2"/>
<g>
<polygon class="st14" points="96.1,80.6 89.3,84.3 89.5,76.5 "/>
</g>
</g>
</g>
<g>
<g>
<line class="st13" x1="97.2" y1="36.3" x2="107.8" y2="54.2"/>
<g>
<polygon class="st14" points="94.4,39.3 94.3,31.5 101.1,35.3 "/>
</g>
</g>
</g>
<circle class="st5" cx="52.1" cy="42.8" r="12.1"/>
<circle class="st4" cx="12.2" cy="50.8" r="12.1"/>
<circle class="st7" cx="87.5" cy="21.7" r="12.1"/>
<circle class="st8" cx="83.5" cy="95" r="12.1"/>
<circle class="st6" cx="107.8" cy="54.2" r="12.1"/>
</g>
</g>
</svg>

After

Width:  |  Height:  |  Size: 2.8 KiB

View File

@@ -0,0 +1,132 @@
<?xml version="1.0" encoding="utf-8"?>
<svg version="1.1" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" x="0px" y="0px"
viewBox="0 0 168.3 133" style="enable-background:new 0 0 168.3 133;" xml:space="preserve">
<style type="text/css">
.st0{fill:#FFFFFF;}
.st1{fill:#E22434;}
.st2{display:none;}
.st3{fill:#1B3663;}
.st4{fill:#00B1D1;}
.st5{fill:#BFE6EF;}
.st6{fill:#69CBE0;}
.st7{fill:#0080BD;}
.st8{fill:#005DAB;}
.st9{fill:#183660;}
.st10{fill:none;stroke:#183660;stroke-width:1.9441;stroke-miterlimit:10;}
.st11{fill:none;stroke:#183660;stroke-width:1.6201;stroke-miterlimit:10;}
.st12{fill:none;stroke:#183660;stroke-width:1.2961;stroke-miterlimit:10;}
.st13{fill:none;stroke:#C0E6EF;stroke-width:1.9441;stroke-miterlimit:10;}
.st14{fill:#C0E6EF;}
.st15{fill:none;stroke:#C0E6EF;stroke-width:1.2961;stroke-miterlimit:10;}
.st16{fill:none;stroke:#C0E6EF;stroke-width:1.6201;stroke-miterlimit:10;}
.st17{fill:none;stroke:#FFFFFF;stroke-width:1.9441;stroke-miterlimit:10;}
.st18{fill:none;stroke:#FFFFFF;stroke-width:1.2961;stroke-miterlimit:10;}
.st19{fill:none;stroke:#FFFFFF;stroke-width:1.6201;stroke-miterlimit:10;}
</style>
<g id="Layer_2">
</g>
<g id="Layer_1">
<g>
<g>
<g>
<path class="st0" d="M4.7,106.4l0.1,1.8c1.1-1.4,2.6-2.1,4.4-2.1c1.9,0,3.2,0.9,4,2.6c1.1-1.7,2.6-2.6,4.7-2.6
c3.3,0,5,2.3,5.1,6.9v12.5h-5v-12.1c0-1.1-0.2-1.9-0.5-2.4c-0.3-0.5-0.8-0.7-1.5-0.7c-0.9,0-1.6,0.6-2.1,1.7l0,0.6v12.9H9v-12.1
c0-1.1-0.1-1.9-0.4-2.4c-0.3-0.5-0.8-0.7-1.6-0.7c-0.9,0-1.5,0.5-2,1.4v13.8H0v-19H4.7z"/>
<path class="st0" d="M26.4,115.3c0-3.1,0.6-5.4,1.7-7s2.7-2.3,4.7-2.3c1.7,0,3.1,0.7,4,2l0.2-1.7h4.5v19c0,2.4-0.7,4.3-2,5.6
c-1.4,1.3-3.3,1.9-5.9,1.9c-1,0-2.1-0.2-3.3-0.6s-2-0.9-2.6-1.6l1.7-3.4c0.5,0.5,1.1,0.9,1.8,1.2c0.7,0.3,1.5,0.5,2.1,0.5
c1.1,0,1.9-0.3,2.4-0.8c0.5-0.5,0.7-1.4,0.7-2.6v-1.6c-0.9,1.2-2.2,1.9-3.7,1.9c-2,0-3.6-0.8-4.7-2.4s-1.7-3.8-1.7-6.7V115.3z
M31.4,116.6c0,1.8,0.2,3,0.7,3.8s1.2,1.2,2.2,1.2c1,0,1.8-0.4,2.3-1.1v-9.1c-0.5-0.8-1.3-1.2-2.2-1.2c-1,0-1.7,0.4-2.2,1.2
s-0.7,2.1-0.7,3.9V116.6z"/>
<path class="st0" d="M50.1,106.4l0.1,1.8c1.1-1.4,2.6-2.1,4.4-2.1c1.9,0,3.2,0.9,4,2.6c1.1-1.7,2.6-2.6,4.7-2.6
c3.3,0,5,2.3,5.1,6.9v12.5h-5v-12.1c0-1.1-0.2-1.9-0.5-2.4s-0.8-0.7-1.5-0.7c-0.9,0-1.6,0.6-2.1,1.7l0,0.6v12.9h-5v-12.1
c0-1.1-0.1-1.9-0.4-2.4s-0.8-0.7-1.6-0.7c-0.9,0-1.5,0.5-2,1.4v13.8h-5v-19H50.1z"/>
<path class="st0" d="M78.2,101.7v4.7h2.5v3.7h-2.5v9.5c0,0.8,0.1,1.3,0.3,1.5c0.2,0.3,0.6,0.4,1.2,0.4c0.5,0,0.9,0,1.2-0.1
l0,3.9c-0.8,0.3-1.8,0.5-2.7,0.5c-3.2,0-4.8-1.8-4.9-5.5v-10.1h-2.2v-3.7h2.2v-4.7H78.2z"/>
<path class="st4" d="M90.6,124c1.4,0,2.4-0.4,3.1-1.1c0.7-0.8,1.1-1.9,1.2-3.3h1.9c-0.1,1.9-0.7,3.5-1.9,4.6
c-1.1,1.1-2.6,1.7-4.3,1.7c-2.3,0-4-0.7-5.1-2.2s-1.7-3.6-1.8-6.4v-2.3c0-2.9,0.6-5.1,1.7-6.6s2.9-2.2,5.1-2.2
c1.9,0,3.4,0.6,4.5,1.8s1.7,2.9,1.7,5H95c-0.1-1.6-0.5-2.8-1.2-3.6s-1.8-1.3-3.1-1.3c-1.7,0-2.9,0.6-3.7,1.7s-1.2,2.9-1.2,5.2
v2.2c0,2.4,0.4,4.2,1.2,5.3S89,124,90.6,124z"/>
<path class="st4" d="M100.5,115c0-2.7,0.6-4.9,1.9-6.5s2.9-2.4,5.1-2.4c2.2,0,3.9,0.8,5.1,2.4s1.9,3.7,1.9,6.5v2
c0,2.8-0.6,5-1.9,6.5s-2.9,2.3-5.1,2.3c-2.1,0-3.8-0.8-5-2.3s-1.9-3.6-1.9-6.3V115z M102.5,116.9c0,2.2,0.4,3.9,1.3,5.2
c0.9,1.3,2.1,1.9,3.7,1.9c1.6,0,2.8-0.6,3.7-1.8s1.3-2.9,1.3-5.2v-2c0-2.2-0.4-3.9-1.3-5.2c-0.9-1.3-2.1-1.9-3.7-1.9
c-1.5,0-2.7,0.6-3.6,1.8s-1.3,2.9-1.4,5.1V116.9z"/>
<path class="st4" d="M121.1,106.4l0.1,3c0.6-1,1.3-1.9,2.2-2.5s1.9-0.9,3-0.9c3.3,0,5,2.2,5.1,6.6v12.7h-1.9v-12.5
c0-1.7-0.3-3-0.9-3.8c-0.6-0.8-1.5-1.2-2.8-1.2c-1,0-1.9,0.4-2.8,1.2s-1.4,1.8-1.9,3.2v13.2h-2v-19H121.1z"/>
<path class="st4" d="M138.2,125.4v-17.3h-2.6v-1.8h2.6v-2.5c0-1.9,0.5-3.3,1.3-4.3s2-1.5,3.5-1.5c0.7,0,1.3,0.1,1.9,0.3
l-0.1,1.8c-0.5-0.1-1-0.2-1.6-0.2c-0.9,0-1.7,0.4-2.2,1.1s-0.8,1.7-0.8,3v2.3h3.7v1.8h-3.7v17.3H138.2z"/>
<path class="st4" d="M148,100.9c0-0.4,0.1-0.7,0.3-1s0.5-0.4,0.9-0.4s0.7,0.1,1,0.4s0.3,0.6,0.3,1s-0.1,0.7-0.3,1
s-0.5,0.4-1,0.4s-0.7-0.1-0.9-0.4S148,101.3,148,100.9z M150.2,125.4h-2v-19h2V125.4z"/>
<path class="st4" d="M155.3,115c0-3,0.5-5.2,1.5-6.7s2.6-2.2,4.6-2.2c2.2,0,3.8,1,4.9,3l0.1-2.7h1.8V126c0,2.3-0.6,4-1.6,5.2
c-1.1,1.2-2.7,1.8-4.7,1.8c-1,0-2.1-0.3-3.2-0.8s-1.9-1.1-2.4-1.9l0.9-1.4c1.3,1.5,2.8,2.2,4.5,2.2c1.6,0,2.7-0.4,3.5-1.3
c0.7-0.8,1.1-2.1,1.1-3.8v-3c-1.1,1.8-2.7,2.7-4.9,2.7c-2,0-3.5-0.7-4.5-2.2s-1.6-3.6-1.6-6.5V115z M157.2,116.8
c0,2.4,0.4,4.2,1.1,5.4s1.9,1.7,3.5,1.7c2.1,0,3.6-1,4.5-3v-9.7c-0.9-2.2-2.4-3.3-4.4-3.3c-1.6,0-2.8,0.6-3.5,1.7
s-1.1,2.9-1.1,5.3V116.8z"/>
</g>
</g>
<g>
<g>
<g>
<line class="st16" x1="58.9" y1="20.1" x2="77.9" y2="35.6"/>
<g>
<polygon class="st14" points="57.6,23.2 55.3,17.1 61.7,18.2 "/>
</g>
</g>
</g>
<circle class="st3" cx="48" cy="10.1" r="10.1"/>
<g>
<g>
<line class="st16" x1="77.9" y1="35.1" x2="96.2" y2="66.7"/>
<g>
<polygon class="st14" points="93,67.5 98.6,70.7 98.6,64.2 "/>
</g>
</g>
</g>
<g>
<g>
<line class="st16" x1="92.3" y1="38.9" x2="114.4" y2="43.6"/>
<g>
<polygon class="st14" points="92.6,42.3 87.8,38 93.9,36 "/>
</g>
</g>
</g>
<g>
<g>
<line class="st16" x1="77.9" y1="35.6" x2="94.5" y2="25.1"/>
<g>
<polygon class="st14" points="95.4,28.3 98.4,22.6 91.9,22.9 "/>
</g>
</g>
</g>
<g>
<g>
<line class="st16" x1="48.5" y1="41.3" x2="63.5" y2="38.7"/>
<g>
<polygon class="st14" points="63.1,42.1 68.1,38 62,35.7 "/>
</g>
</g>
</g>
<g>
<g>
<line class="st16" x1="111.4" y1="66.3" x2="124.4" y2="45.2"/>
<g>
<polygon class="st14" points="114.6,67.2 109,70.2 109.1,63.8 "/>
</g>
</g>
</g>
<g>
<g>
<line class="st16" x1="115.5" y1="30.5" x2="124.4" y2="45.2"/>
<g>
<polygon class="st14" points="113.2,32.9 113.1,26.5 118.8,29.6 "/>
</g>
</g>
</g>
<circle class="st5" cx="77.9" cy="35.6" r="10.1"/>
<circle class="st4" cx="44.6" cy="42.4" r="10.1"/>
<circle class="st7" cx="107.4" cy="18.1" r="10.1"/>
<circle class="st8" cx="104.1" cy="79.1" r="10.1"/>
<circle class="st6" cx="124.4" cy="45.2" r="10.1"/>
</g>
</g>
</g>
</svg>

After

Width:  |  Height:  |  Size: 6.2 KiB

View File

@@ -0,0 +1,132 @@
<?xml version="1.0" encoding="utf-8"?>
<svg version="1.1" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" x="0px" y="0px"
viewBox="0 0 260.4 71.4" style="enable-background:new 0 0 260.4 71.4;" xml:space="preserve">
<style type="text/css">
.st0{fill:#FFFFFF;}
.st1{fill:#E22434;}
.st2{display:none;}
.st3{fill:#1B3663;}
.st4{fill:#00B1D1;}
.st5{fill:#BFE6EF;}
.st6{fill:#69CBE0;}
.st7{fill:#0080BD;}
.st8{fill:#005DAB;}
.st9{fill:#183660;}
.st10{fill:none;stroke:#183660;stroke-width:1.9441;stroke-miterlimit:10;}
.st11{fill:none;stroke:#183660;stroke-width:1.6201;stroke-miterlimit:10;}
.st12{fill:none;stroke:#183660;stroke-width:1.2961;stroke-miterlimit:10;}
.st13{fill:none;stroke:#C0E6EF;stroke-width:1.9441;stroke-miterlimit:10;}
.st14{fill:#C0E6EF;}
.st15{fill:none;stroke:#C0E6EF;stroke-width:1.2961;stroke-miterlimit:10;}
.st16{fill:none;stroke:#C0E6EF;stroke-width:1.6201;stroke-miterlimit:10;}
.st17{fill:none;stroke:#FFFFFF;stroke-width:1.9441;stroke-miterlimit:10;}
.st18{fill:none;stroke:#FFFFFF;stroke-width:1.2961;stroke-miterlimit:10;}
.st19{fill:none;stroke:#FFFFFF;stroke-width:1.6201;stroke-miterlimit:10;}
</style>
<g id="Layer_2">
</g>
<g id="Layer_1">
<g>
<g>
<g>
<path class="st0" d="M96.7,27.6l0.1,1.8c1.1-1.4,2.6-2.1,4.4-2.1c1.9,0,3.2,0.9,4,2.6c1.1-1.7,2.6-2.6,4.7-2.6
c3.3,0,5,2.3,5.1,6.9v12.5h-5V34.4c0-1.1-0.2-1.9-0.5-2.4c-0.3-0.5-0.8-0.7-1.5-0.7c-0.9,0-1.6,0.6-2.1,1.7l0,0.6v12.9h-5V34.5
c0-1.1-0.1-1.9-0.4-2.4c-0.3-0.5-0.8-0.7-1.6-0.7c-0.9,0-1.5,0.5-2,1.4v13.8h-5v-19H96.7z"/>
<path class="st0" d="M118.5,36.5c0-3.1,0.6-5.4,1.7-7s2.7-2.3,4.7-2.3c1.7,0,3.1,0.7,4,2l0.2-1.7h4.5v19c0,2.4-0.7,4.3-2,5.6
c-1.4,1.3-3.3,1.9-5.9,1.9c-1,0-2.1-0.2-3.3-0.6s-2-0.9-2.6-1.6l1.7-3.4c0.5,0.5,1.1,0.9,1.8,1.2c0.7,0.3,1.5,0.5,2.1,0.5
c1.1,0,1.9-0.3,2.4-0.8s0.7-1.4,0.7-2.6v-1.6c-0.9,1.2-2.2,1.9-3.7,1.9c-2,0-3.6-0.8-4.7-2.4s-1.7-3.8-1.7-6.7V36.5z
M123.5,37.7c0,1.8,0.2,3,0.7,3.8s1.2,1.2,2.2,1.2c1,0,1.8-0.4,2.3-1.1v-9.1c-0.5-0.8-1.3-1.2-2.2-1.2c-1,0-1.7,0.4-2.2,1.2
c-0.5,0.8-0.7,2.1-0.7,3.9V37.7z"/>
<path class="st0" d="M142.2,27.6l0.1,1.8c1.1-1.4,2.6-2.1,4.4-2.1c1.9,0,3.2,0.9,4,2.6c1.1-1.7,2.6-2.6,4.7-2.6
c3.3,0,5,2.3,5.1,6.9v12.5h-5V34.4c0-1.1-0.2-1.9-0.5-2.4c-0.3-0.5-0.8-0.7-1.5-0.7c-0.9,0-1.6,0.6-2.1,1.7l0,0.6v12.9h-5V34.5
c0-1.1-0.1-1.9-0.4-2.4c-0.3-0.5-0.8-0.7-1.6-0.7c-0.9,0-1.5,0.5-2,1.4v13.8h-5v-19H142.2z"/>
<path class="st0" d="M170.3,22.9v4.7h2.5v3.7h-2.5v9.5c0,0.8,0.1,1.3,0.3,1.5c0.2,0.3,0.6,0.4,1.2,0.4c0.5,0,0.9,0,1.2-0.1
l0,3.9c-0.8,0.3-1.8,0.5-2.7,0.5c-3.2,0-4.8-1.8-4.9-5.5V31.3h-2.2v-3.7h2.2v-4.7H170.3z"/>
<path class="st4" d="M182.7,45.1c1.4,0,2.4-0.4,3.1-1.1c0.7-0.8,1.1-1.9,1.2-3.3h1.9c-0.1,1.9-0.7,3.5-1.9,4.6s-2.6,1.7-4.3,1.7
c-2.3,0-4-0.7-5.1-2.2s-1.7-3.6-1.8-6.4V36c0-2.9,0.6-5.1,1.7-6.6s2.9-2.2,5.1-2.2c1.9,0,3.4,0.6,4.5,1.8s1.7,2.9,1.7,5H187
c-0.1-1.6-0.5-2.8-1.2-3.6s-1.8-1.3-3.1-1.3c-1.7,0-2.9,0.6-3.7,1.7s-1.2,2.9-1.2,5.2v2.2c0,2.4,0.4,4.2,1.2,5.3
S181,45.1,182.7,45.1z"/>
<path class="st4" d="M192.6,36.1c0-2.7,0.6-4.9,1.9-6.5s2.9-2.4,5.1-2.4c2.2,0,3.9,0.8,5.1,2.4c1.2,1.6,1.9,3.7,1.9,6.5v2
c0,2.8-0.6,5-1.9,6.5s-2.9,2.3-5.1,2.3s-3.8-0.8-5-2.3s-1.9-3.6-1.9-6.3V36.1z M194.6,38.1c0,2.2,0.4,3.9,1.3,5.2
c0.9,1.3,2.1,1.9,3.7,1.9c1.6,0,2.8-0.6,3.7-1.8c0.8-1.2,1.3-2.9,1.3-5.2v-2c0-2.2-0.4-3.9-1.3-5.2c-0.9-1.3-2.1-1.9-3.7-1.9
c-1.5,0-2.7,0.6-3.6,1.8c-0.9,1.2-1.3,2.9-1.4,5.1V38.1z"/>
<path class="st4" d="M213.2,27.6l0.1,3c0.6-1,1.3-1.9,2.2-2.5s1.9-0.9,3-0.9c3.3,0,5,2.2,5.1,6.6v12.7h-1.9V34.1
c0-1.7-0.3-3-0.9-3.8c-0.6-0.8-1.5-1.2-2.8-1.2c-1,0-1.9,0.4-2.8,1.2s-1.4,1.8-1.9,3.2v13.2h-2v-19H213.2z"/>
<path class="st4" d="M230.3,46.6V29.3h-2.6v-1.8h2.6v-2.5c0-1.9,0.5-3.3,1.3-4.3s2-1.5,3.5-1.5c0.7,0,1.3,0.1,1.9,0.3l-0.1,1.8
c-0.5-0.1-1-0.2-1.6-0.2c-0.9,0-1.7,0.4-2.2,1.1s-0.8,1.7-0.8,3v2.3h3.7v1.8h-3.7v17.3H230.3z"/>
<path class="st4" d="M240.1,22.1c0-0.4,0.1-0.7,0.3-1s0.5-0.4,0.9-0.4s0.7,0.1,1,0.4s0.3,0.6,0.3,1s-0.1,0.7-0.3,1
s-0.5,0.4-1,0.4s-0.7-0.1-0.9-0.4S240.1,22.5,240.1,22.1z M242.3,46.6h-2v-19h2V46.6z"/>
<path class="st4" d="M247.4,36.2c0-3,0.5-5.2,1.5-6.7s2.6-2.2,4.6-2.2c2.2,0,3.8,1,4.9,3l0.1-2.7h1.8v19.6c0,2.3-0.6,4-1.6,5.2
s-2.7,1.8-4.7,1.8c-1,0-2.1-0.3-3.2-0.8s-1.9-1.1-2.4-1.9l0.9-1.4c1.3,1.5,2.8,2.2,4.5,2.2c1.6,0,2.7-0.4,3.5-1.3
c0.7-0.8,1.1-2.1,1.1-3.8v-3c-1.1,1.8-2.7,2.7-4.9,2.7c-2,0-3.5-0.7-4.5-2.2s-1.6-3.6-1.6-6.5V36.2z M249.3,38
c0,2.4,0.4,4.2,1.1,5.4s1.9,1.7,3.5,1.7c2.1,0,3.6-1,4.5-3v-9.7c-0.9-2.2-2.4-3.3-4.4-3.3c-1.6,0-2.8,0.6-3.5,1.7
s-1.1,2.9-1.1,5.3V38z"/>
</g>
</g>
<g>
<g>
<g>
<line class="st15" x1="19.5" y1="16" x2="34.7" y2="28.5"/>
<g>
<polygon class="st14" points="18.4,18.5 16.6,13.7 21.7,14.5 "/>
</g>
</g>
</g>
<circle class="st3" cx="10.8" cy="8.1" r="8.1"/>
<g>
<g>
<line class="st15" x1="34.7" y1="28" x2="49.4" y2="53.3"/>
<g>
<polygon class="st14" points="46.8,54 51.2,56.5 51.2,51.4 "/>
</g>
</g>
</g>
<g>
<g>
<line class="st15" x1="46.2" y1="31.1" x2="63.9" y2="34.9"/>
<g>
<polygon class="st14" points="46.4,33.8 42.6,30.4 47.5,28.8 "/>
</g>
</g>
</g>
<g>
<g>
<line class="st15" x1="34.7" y1="28.5" x2="47.9" y2="20.1"/>
<g>
<polygon class="st14" points="48.7,22.7 51.1,18.1 45.9,18.3 "/>
</g>
</g>
</g>
<g>
<g>
<line class="st15" x1="11.2" y1="33.1" x2="23.2" y2="31"/>
<g>
<polygon class="st14" points="22.9,33.7 26.8,30.4 22,28.6 "/>
</g>
</g>
</g>
<g>
<g>
<line class="st15" x1="61.5" y1="53" x2="71.9" y2="36.1"/>
<g>
<polygon class="st14" points="64.1,53.7 59.5,56.2 59.7,51 "/>
</g>
</g>
</g>
<g>
<g>
<line class="st15" x1="64.8" y1="24.4" x2="71.9" y2="36.1"/>
<g>
<polygon class="st14" points="63,26.4 62.9,21.2 67.4,23.7 "/>
</g>
</g>
</g>
<circle class="st5" cx="34.7" cy="28.5" r="8.1"/>
<circle class="st4" cx="8.1" cy="33.9" r="8.1"/>
<circle class="st7" cx="58.3" cy="14.5" r="8.1"/>
<circle class="st8" cx="55.7" cy="63.3" r="8.1"/>
<circle class="st6" cx="71.9" cy="36.1" r="8.1"/>
</g>
</g>
</g>
</svg>

After

Width:  |  Height:  |  Size: 6.2 KiB

View File

@@ -0,0 +1,94 @@
<?xml version="1.0" encoding="utf-8"?>
<svg version="1.1" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" x="0px" y="0px"
viewBox="0 0 120 107.1" style="enable-background:new 0 0 120 107.1;" xml:space="preserve">
<style type="text/css">
.st0{fill:#FFFFFF;}
.st1{fill:#E22434;}
.st2{display:none;}
.st3{fill:#1B3663;}
.st4{fill:#00B1D1;}
.st5{fill:#BFE6EF;}
.st6{fill:#69CBE0;}
.st7{fill:#0080BD;}
.st8{fill:#005DAB;}
.st9{fill:#183660;}
.st10{fill:none;stroke:#183660;stroke-width:1.9441;stroke-miterlimit:10;}
.st11{fill:none;stroke:#183660;stroke-width:1.6201;stroke-miterlimit:10;}
.st12{fill:none;stroke:#183660;stroke-width:1.2961;stroke-miterlimit:10;}
.st13{fill:none;stroke:#C0E6EF;stroke-width:1.9441;stroke-miterlimit:10;}
.st14{fill:#C0E6EF;}
.st15{fill:none;stroke:#C0E6EF;stroke-width:1.2961;stroke-miterlimit:10;}
.st16{fill:none;stroke:#C0E6EF;stroke-width:1.6201;stroke-miterlimit:10;}
.st17{fill:none;stroke:#FFFFFF;stroke-width:1.9441;stroke-miterlimit:10;}
.st18{fill:none;stroke:#FFFFFF;stroke-width:1.2961;stroke-miterlimit:10;}
.st19{fill:none;stroke:#FFFFFF;stroke-width:1.6201;stroke-miterlimit:10;}
</style>
<g id="Layer_2">
</g>
<g id="Layer_1">
<g>
<g>
<g>
<line class="st17" x1="29.2" y1="24.1" x2="52.1" y2="42.8"/>
<g>
<polygon class="st0" points="27.7,27.8 24.9,20.5 32.6,21.8 "/>
</g>
</g>
</g>
<circle class="st0" cx="16.1" cy="12.2" r="12.1"/>
<g>
<g>
<line class="st17" x1="52.1" y1="42.1" x2="74.1" y2="80"/>
<g>
<polygon class="st0" points="70.1,80.9 76.9,84.8 76.8,77.1 "/>
</g>
</g>
</g>
<g>
<g>
<line class="st17" x1="69.4" y1="46.7" x2="95.8" y2="52.4"/>
<g>
<polygon class="st0" points="69.7,50.7 63.9,45.6 71.3,43.2 "/>
</g>
</g>
</g>
<g>
<g>
<line class="st17" x1="52.1" y1="42.8" x2="71.9" y2="30.1"/>
<g>
<polygon class="st0" points="73.1,34 76.6,27.1 68.9,27.5 "/>
</g>
</g>
</g>
<g>
<g>
<line class="st17" x1="16.8" y1="49.6" x2="34.8" y2="46.5"/>
<g>
<polygon class="st0" points="34.3,50.5 40.3,45.6 33,42.9 "/>
</g>
</g>
</g>
<g>
<g>
<line class="st17" x1="92.3" y1="79.5" x2="107.8" y2="54.2"/>
<g>
<polygon class="st0" points="96.1,80.6 89.3,84.3 89.5,76.5 "/>
</g>
</g>
</g>
<g>
<g>
<line class="st17" x1="97.2" y1="36.6" x2="107.8" y2="54.2"/>
<g>
<polygon class="st0" points="94.4,39.5 94.3,31.8 101.1,35.5 "/>
</g>
</g>
</g>
<circle class="st0" cx="52.1" cy="42.8" r="12.1"/>
<circle class="st0" cx="12.2" cy="50.8" r="12.1"/>
<circle class="st0" cx="87.5" cy="21.7" r="12.1"/>
<circle class="st0" cx="83.5" cy="95" r="12.1"/>
<circle class="st0" cx="107.8" cy="54.2" r="12.1"/>
</g>
</g>
</svg>

After

Width:  |  Height:  |  Size: 2.8 KiB

View File

@@ -0,0 +1,132 @@
<?xml version="1.0" encoding="utf-8"?>
<svg version="1.1" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" x="0px" y="0px"
viewBox="0 0 168.3 131.4" style="enable-background:new 0 0 168.3 131.4;" xml:space="preserve">
<style type="text/css">
.st0{fill:#FFFFFF;}
.st1{fill:#E22434;}
.st2{display:none;}
.st3{fill:#1B3663;}
.st4{fill:#00B1D1;}
.st5{fill:#BFE6EF;}
.st6{fill:#69CBE0;}
.st7{fill:#0080BD;}
.st8{fill:#005DAB;}
.st9{fill:#183660;}
.st10{fill:none;stroke:#183660;stroke-width:1.9441;stroke-miterlimit:10;}
.st11{fill:none;stroke:#183660;stroke-width:1.6201;stroke-miterlimit:10;}
.st12{fill:none;stroke:#183660;stroke-width:1.2961;stroke-miterlimit:10;}
.st13{fill:none;stroke:#C0E6EF;stroke-width:1.9441;stroke-miterlimit:10;}
.st14{fill:#C0E6EF;}
.st15{fill:none;stroke:#C0E6EF;stroke-width:1.2961;stroke-miterlimit:10;}
.st16{fill:none;stroke:#C0E6EF;stroke-width:1.6201;stroke-miterlimit:10;}
.st17{fill:none;stroke:#FFFFFF;stroke-width:1.9441;stroke-miterlimit:10;}
.st18{fill:none;stroke:#FFFFFF;stroke-width:1.2961;stroke-miterlimit:10;}
.st19{fill:none;stroke:#FFFFFF;stroke-width:1.6201;stroke-miterlimit:10;}
</style>
<g id="Layer_2">
</g>
<g id="Layer_1">
<g>
<g>
<g>
<path class="st0" d="M4.7,104.8l0.1,1.8c1.1-1.4,2.6-2.1,4.4-2.1c1.9,0,3.2,0.9,4,2.6c1.1-1.7,2.6-2.6,4.7-2.6
c3.3,0,5,2.3,5.1,6.9v12.5h-5v-12.1c0-1.1-0.2-1.9-0.5-2.4c-0.3-0.5-0.8-0.7-1.5-0.7c-0.9,0-1.6,0.6-2.1,1.7l0,0.6v12.9H9v-12.1
c0-1.1-0.1-1.9-0.4-2.4c-0.3-0.5-0.8-0.7-1.6-0.7c-0.9,0-1.5,0.5-2,1.4v13.8H0v-19H4.7z"/>
<path class="st0" d="M26.4,113.8c0-3.1,0.6-5.4,1.7-7s2.7-2.3,4.7-2.3c1.7,0,3.1,0.7,4,2l0.2-1.7h4.5v19c0,2.4-0.7,4.3-2,5.6
c-1.4,1.3-3.3,1.9-5.9,1.9c-1,0-2.1-0.2-3.3-0.6s-2-0.9-2.6-1.6l1.7-3.4c0.5,0.5,1.1,0.9,1.8,1.2c0.7,0.3,1.5,0.5,2.1,0.5
c1.1,0,1.9-0.3,2.4-0.8s0.7-1.4,0.7-2.6v-1.6c-0.9,1.2-2.2,1.9-3.7,1.9c-2,0-3.6-0.8-4.7-2.4s-1.7-3.8-1.7-6.7V113.8z M31.4,115
c0,1.8,0.2,3,0.7,3.8s1.2,1.2,2.2,1.2c1,0,1.8-0.4,2.3-1.1v-9.1c-0.5-0.8-1.3-1.2-2.2-1.2c-1,0-1.7,0.4-2.2,1.2
s-0.7,2.1-0.7,3.9V115z"/>
<path class="st0" d="M50.1,104.8l0.1,1.8c1.1-1.4,2.6-2.1,4.4-2.1c1.9,0,3.2,0.9,4,2.6c1.1-1.7,2.6-2.6,4.7-2.6
c3.3,0,5,2.3,5.1,6.9v12.5h-5v-12.1c0-1.1-0.2-1.9-0.5-2.4s-0.8-0.7-1.5-0.7c-0.9,0-1.6,0.6-2.1,1.7l0,0.6v12.9h-5v-12.1
c0-1.1-0.1-1.9-0.4-2.4s-0.8-0.7-1.6-0.7c-0.9,0-1.5,0.5-2,1.4v13.8h-5v-19H50.1z"/>
<path class="st0" d="M78.2,100.2v4.7h2.5v3.7h-2.5v9.5c0,0.8,0.1,1.3,0.3,1.5c0.2,0.3,0.6,0.4,1.2,0.4c0.5,0,0.9,0,1.2-0.1
l0,3.9c-0.8,0.3-1.8,0.5-2.7,0.5c-3.2,0-4.8-1.8-4.9-5.5v-10.1h-2.2v-3.7h2.2v-4.7H78.2z"/>
<path class="st0" d="M90.6,122.4c1.4,0,2.4-0.4,3.1-1.1c0.7-0.8,1.1-1.8,1.2-3.3h1.9c-0.1,1.9-0.7,3.5-1.9,4.6
c-1.1,1.1-2.6,1.7-4.3,1.7c-2.3,0-4-0.7-5.1-2.2s-1.7-3.6-1.8-6.4v-2.3c0-2.9,0.6-5.1,1.7-6.6s2.9-2.2,5.1-2.2
c1.9,0,3.4,0.6,4.5,1.8s1.7,2.9,1.7,5H95c-0.1-1.6-0.5-2.8-1.2-3.6s-1.8-1.3-3.1-1.3c-1.7,0-2.9,0.6-3.7,1.7
c-0.8,1.1-1.2,2.9-1.2,5.2v2.2c0,2.4,0.4,4.2,1.2,5.3S89,122.4,90.6,122.4z"/>
<path class="st0" d="M100.5,113.4c0-2.7,0.6-4.9,1.9-6.5s2.9-2.4,5.1-2.4c2.2,0,3.9,0.8,5.1,2.4s1.9,3.7,1.9,6.5v2
c0,2.8-0.6,5-1.9,6.5s-2.9,2.3-5.1,2.3c-2.1,0-3.8-0.8-5-2.3s-1.9-3.6-1.9-6.3V113.4z M102.5,115.3c0,2.2,0.4,3.9,1.3,5.2
c0.9,1.3,2.1,1.9,3.7,1.9c1.6,0,2.8-0.6,3.7-1.8c0.8-1.2,1.3-2.9,1.3-5.2v-2c0-2.2-0.4-3.9-1.3-5.2c-0.9-1.3-2.1-1.9-3.7-1.9
c-1.5,0-2.7,0.6-3.6,1.8c-0.9,1.2-1.3,2.9-1.4,5.1V115.3z"/>
<path class="st0" d="M121.1,104.8l0.1,3c0.6-1,1.3-1.9,2.2-2.5s1.9-0.9,3-0.9c3.3,0,5,2.2,5.1,6.6v12.7h-1.9v-12.5
c0-1.7-0.3-3-0.9-3.8c-0.6-0.8-1.5-1.2-2.8-1.2c-1,0-1.9,0.4-2.8,1.2s-1.4,1.8-1.9,3.2v13.2h-2v-19H121.1z"/>
<path class="st0" d="M138.2,123.8v-17.3h-2.6v-1.8h2.6v-2.5c0-1.9,0.5-3.3,1.3-4.3s2-1.5,3.5-1.5c0.7,0,1.3,0.1,1.9,0.3
l-0.1,1.8c-0.5-0.1-1-0.2-1.6-0.2c-0.9,0-1.7,0.4-2.2,1.1s-0.8,1.7-0.8,3v2.3h3.7v1.8h-3.7v17.3H138.2z"/>
<path class="st0" d="M148,99.3c0-0.4,0.1-0.7,0.3-1s0.5-0.4,0.9-0.4s0.7,0.1,1,0.4s0.3,0.6,0.3,1s-0.1,0.7-0.3,1s-0.5,0.4-1,0.4
s-0.7-0.1-0.9-0.4S148,99.7,148,99.3z M150.2,123.8h-2v-19h2V123.8z"/>
<path class="st0" d="M155.3,113.5c0-3,0.5-5.2,1.5-6.7s2.6-2.2,4.6-2.2c2.2,0,3.8,1,4.9,3l0.1-2.7h1.8v19.6c0,2.3-0.6,4-1.6,5.2
c-1.1,1.2-2.7,1.8-4.7,1.8c-1,0-2.1-0.3-3.2-0.8s-1.9-1.1-2.4-1.9l0.9-1.4c1.3,1.5,2.8,2.2,4.5,2.2c1.6,0,2.7-0.4,3.5-1.3
c0.7-0.8,1.1-2.1,1.1-3.8v-3c-1.1,1.8-2.7,2.7-4.9,2.7c-2,0-3.5-0.7-4.5-2.2s-1.6-3.6-1.6-6.5V113.5z M157.2,115.2
c0,2.4,0.4,4.2,1.1,5.4s1.9,1.7,3.5,1.7c2.1,0,3.6-1,4.5-3v-9.7c-0.9-2.2-2.4-3.3-4.4-3.3c-1.6,0-2.8,0.6-3.5,1.7
s-1.1,2.9-1.1,5.3V115.2z"/>
</g>
</g>
<g>
<g>
<g>
<line class="st19" x1="58.9" y1="20.1" x2="77.9" y2="35.6"/>
<g>
<polygon class="st0" points="57.6,23.2 55.3,17.1 61.7,18.2 "/>
</g>
</g>
</g>
<circle class="st0" cx="48" cy="10.1" r="10.1"/>
<g>
<g>
<line class="st19" x1="77.9" y1="35.1" x2="96.2" y2="66.7"/>
<g>
<polygon class="st0" points="93,67.5 98.6,70.7 98.6,64.2 "/>
</g>
</g>
</g>
<g>
<g>
<line class="st19" x1="92.3" y1="38.9" x2="114.4" y2="43.6"/>
<g>
<polygon class="st0" points="92.6,42.3 87.8,38 93.9,36 "/>
</g>
</g>
</g>
<g>
<g>
<line class="st19" x1="77.9" y1="35.6" x2="94.5" y2="25.1"/>
<g>
<polygon class="st0" points="95.4,28.3 98.4,22.6 91.9,22.9 "/>
</g>
</g>
</g>
<g>
<g>
<line class="st19" x1="48.5" y1="41.3" x2="63.5" y2="38.7"/>
<g>
<polygon class="st0" points="63.1,42.1 68.1,38 62,35.7 "/>
</g>
</g>
</g>
<g>
<g>
<line class="st19" x1="111.4" y1="66.3" x2="124.4" y2="45.2"/>
<g>
<polygon class="st0" points="114.6,67.2 109,70.2 109.1,63.8 "/>
</g>
</g>
</g>
<g>
<g>
<line class="st19" x1="115.4" y1="30.4" x2="124.4" y2="45.2"/>
<g>
<polygon class="st0" points="113.2,32.8 113,26.4 118.7,29.5 "/>
</g>
</g>
</g>
<circle class="st0" cx="77.9" cy="35.6" r="10.1"/>
<circle class="st0" cx="44.6" cy="42.4" r="10.1"/>
<circle class="st0" cx="107.4" cy="18.1" r="10.1"/>
<circle class="st0" cx="104.1" cy="79.1" r="10.1"/>
<circle class="st0" cx="124.4" cy="45.2" r="10.1"/>
</g>
</g>
</g>
</svg>

After

Width:  |  Height:  |  Size: 6.2 KiB

View File

@@ -0,0 +1,132 @@
<?xml version="1.0" encoding="utf-8"?>
<svg version="1.1" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" x="0px" y="0px"
viewBox="0 0 260.4 71.4" style="enable-background:new 0 0 260.4 71.4;" xml:space="preserve">
<style type="text/css">
.st0{fill:#FFFFFF;}
.st1{fill:#E22434;}
.st2{display:none;}
.st3{fill:#1B3663;}
.st4{fill:#00B1D1;}
.st5{fill:#BFE6EF;}
.st6{fill:#69CBE0;}
.st7{fill:#0080BD;}
.st8{fill:#005DAB;}
.st9{fill:#183660;}
.st10{fill:none;stroke:#183660;stroke-width:1.9441;stroke-miterlimit:10;}
.st11{fill:none;stroke:#183660;stroke-width:1.6201;stroke-miterlimit:10;}
.st12{fill:none;stroke:#183660;stroke-width:1.2961;stroke-miterlimit:10;}
.st13{fill:none;stroke:#C0E6EF;stroke-width:1.9441;stroke-miterlimit:10;}
.st14{fill:#C0E6EF;}
.st15{fill:none;stroke:#C0E6EF;stroke-width:1.2961;stroke-miterlimit:10;}
.st16{fill:none;stroke:#C0E6EF;stroke-width:1.6201;stroke-miterlimit:10;}
.st17{fill:none;stroke:#FFFFFF;stroke-width:1.9441;stroke-miterlimit:10;}
.st18{fill:none;stroke:#FFFFFF;stroke-width:1.2961;stroke-miterlimit:10;}
.st19{fill:none;stroke:#FFFFFF;stroke-width:1.6201;stroke-miterlimit:10;}
</style>
<g id="Layer_2">
</g>
<g id="Layer_1">
<g>
<g>
<g>
<path class="st0" d="M96.7,26l0.1,1.8c1.1-1.4,2.6-2.1,4.4-2.1c1.9,0,3.2,0.9,4,2.6c1.1-1.7,2.6-2.6,4.7-2.6
c3.3,0,5,2.3,5.1,6.9V45h-5V32.9c0-1.1-0.2-1.9-0.5-2.4s-0.8-0.7-1.5-0.7c-0.9,0-1.6,0.6-2.1,1.7l0,0.6V45h-5V32.9
c0-1.1-0.1-1.9-0.4-2.4s-0.8-0.7-1.6-0.7c-0.9,0-1.5,0.5-2,1.4V45h-5V26H96.7z"/>
<path class="st0" d="M118.5,34.9c0-3.1,0.6-5.4,1.7-7s2.7-2.3,4.7-2.3c1.7,0,3.1,0.7,4,2l0.2-1.7h4.5v19c0,2.4-0.7,4.3-2,5.6
c-1.4,1.3-3.3,1.9-5.9,1.9c-1,0-2.1-0.2-3.3-0.6s-2-0.9-2.6-1.6l1.7-3.4c0.5,0.5,1.1,0.9,1.8,1.2c0.7,0.3,1.5,0.5,2.1,0.5
c1.1,0,1.9-0.3,2.4-0.8s0.7-1.4,0.7-2.6v-1.6c-0.9,1.2-2.2,1.9-3.7,1.9c-2,0-3.6-0.8-4.7-2.4s-1.7-3.8-1.7-6.7V34.9z
M123.5,36.2c0,1.8,0.2,3,0.7,3.8s1.2,1.2,2.2,1.2c1,0,1.8-0.4,2.3-1.1V31c-0.5-0.8-1.3-1.2-2.2-1.2c-1,0-1.7,0.4-2.2,1.2
s-0.7,2.1-0.7,3.9V36.2z"/>
<path class="st0" d="M142.2,26l0.1,1.8c1.1-1.4,2.6-2.1,4.4-2.1c1.9,0,3.2,0.9,4,2.6c1.1-1.7,2.6-2.6,4.7-2.6
c3.3,0,5,2.3,5.1,6.9V45h-5V32.9c0-1.1-0.2-1.9-0.5-2.4c-0.3-0.5-0.8-0.7-1.5-0.7c-0.9,0-1.6,0.6-2.1,1.7l0,0.6V45h-5V32.9
c0-1.1-0.1-1.9-0.4-2.4c-0.3-0.5-0.8-0.7-1.6-0.7c-0.9,0-1.5,0.5-2,1.4V45h-5V26H142.2z"/>
<path class="st0" d="M170.3,21.3V26h2.5v3.7h-2.5v9.5c0,0.8,0.1,1.3,0.3,1.5c0.2,0.3,0.6,0.4,1.2,0.4c0.5,0,0.9,0,1.2-0.1l0,3.9
c-0.8,0.3-1.8,0.5-2.7,0.5c-3.2,0-4.8-1.8-4.9-5.5V29.7h-2.2V26h2.2v-4.7H170.3z"/>
<path class="st0" d="M182.7,43.5c1.4,0,2.4-0.4,3.1-1.1c0.7-0.8,1.1-1.9,1.2-3.3h1.9c-0.1,1.9-0.7,3.5-1.9,4.6s-2.6,1.7-4.3,1.7
c-2.3,0-4-0.7-5.1-2.2s-1.7-3.6-1.8-6.4v-2.3c0-2.9,0.6-5.1,1.7-6.6s2.9-2.2,5.1-2.2c1.9,0,3.4,0.6,4.5,1.8s1.7,2.8,1.7,5H187
c-0.1-1.6-0.5-2.8-1.2-3.6s-1.8-1.3-3.1-1.3c-1.7,0-2.9,0.6-3.7,1.7c-0.8,1.1-1.2,2.9-1.2,5.2v2.2c0,2.4,0.4,4.2,1.2,5.3
C179.8,43,181,43.5,182.7,43.5z"/>
<path class="st0" d="M192.6,34.5c0-2.7,0.6-4.9,1.9-6.5s2.9-2.4,5.1-2.4c2.2,0,3.9,0.8,5.1,2.4c1.2,1.6,1.9,3.7,1.9,6.5v2
c0,2.8-0.6,5-1.9,6.5s-2.9,2.3-5.1,2.3s-3.8-0.8-5-2.3s-1.9-3.6-1.9-6.3V34.5z M194.6,36.5c0,2.2,0.4,3.9,1.3,5.2
c0.9,1.3,2.1,1.9,3.7,1.9c1.6,0,2.8-0.6,3.7-1.8c0.8-1.2,1.3-2.9,1.3-5.2v-2c0-2.2-0.4-3.9-1.3-5.2c-0.9-1.3-2.1-1.9-3.7-1.9
c-1.5,0-2.7,0.6-3.6,1.8c-0.9,1.2-1.3,2.9-1.4,5.1V36.5z"/>
<path class="st0" d="M213.2,26l0.1,3c0.6-1,1.3-1.9,2.2-2.5s1.9-0.9,3-0.9c3.3,0,5,2.2,5.1,6.6V45h-1.9V32.5
c0-1.7-0.3-3-0.9-3.8c-0.6-0.8-1.5-1.2-2.8-1.2c-1,0-1.9,0.4-2.8,1.2s-1.4,1.8-1.9,3.2V45h-2V26H213.2z"/>
<path class="st0" d="M230.3,45V27.7h-2.6V26h2.6v-2.5c0-1.9,0.5-3.3,1.3-4.3s2-1.5,3.5-1.5c0.7,0,1.3,0.1,1.9,0.3l-0.1,1.8
c-0.5-0.1-1-0.2-1.6-0.2c-0.9,0-1.7,0.4-2.2,1.1s-0.8,1.7-0.8,3V26h3.7v1.8h-3.7V45H230.3z"/>
<path class="st0" d="M240.1,20.5c0-0.4,0.1-0.7,0.3-1s0.5-0.4,0.9-0.4s0.7,0.1,1,0.4s0.3,0.6,0.3,1s-0.1,0.7-0.3,1
s-0.5,0.4-1,0.4s-0.7-0.1-0.9-0.4S240.1,20.9,240.1,20.5z M242.3,45h-2V26h2V45z"/>
<path class="st0" d="M247.4,34.6c0-3,0.5-5.2,1.5-6.7s2.6-2.2,4.6-2.2c2.2,0,3.8,1,4.9,3l0.1-2.7h1.8v19.6c0,2.3-0.6,4-1.6,5.2
s-2.7,1.8-4.7,1.8c-1,0-2.1-0.3-3.2-0.8s-1.9-1.1-2.4-1.9l0.9-1.4c1.3,1.5,2.8,2.2,4.5,2.2c1.6,0,2.7-0.4,3.5-1.3
c0.7-0.8,1.1-2.1,1.1-3.8v-3c-1.1,1.8-2.7,2.7-4.9,2.7c-2,0-3.5-0.7-4.5-2.2s-1.6-3.6-1.6-6.5V34.6z M249.3,36.4
c0,2.4,0.4,4.2,1.1,5.4s1.9,1.7,3.5,1.7c2.1,0,3.6-1,4.5-3v-9.7c-0.9-2.2-2.4-3.3-4.4-3.3c-1.6,0-2.8,0.6-3.5,1.7
s-1.1,2.9-1.1,5.3V36.4z"/>
</g>
</g>
<g>
<g>
<g>
<line class="st18" x1="19.5" y1="16" x2="34.7" y2="28.5"/>
<g>
<polygon class="st0" points="18.4,18.5 16.6,13.7 21.7,14.5 "/>
</g>
</g>
</g>
<circle class="st0" cx="10.8" cy="8.1" r="8.1"/>
<g>
<g>
<line class="st18" x1="34.7" y1="28" x2="49.4" y2="53.3"/>
<g>
<polygon class="st0" points="46.8,54 51.2,56.5 51.2,51.4 "/>
</g>
</g>
</g>
<g>
<g>
<line class="st18" x1="46.2" y1="31.1" x2="63.9" y2="34.9"/>
<g>
<polygon class="st0" points="46.4,33.8 42.6,30.4 47.5,28.8 "/>
</g>
</g>
</g>
<g>
<g>
<line class="st18" x1="34.7" y1="28.5" x2="47.9" y2="20.1"/>
<g>
<polygon class="st0" points="48.7,22.7 51.1,18.1 45.9,18.3 "/>
</g>
</g>
</g>
<g>
<g>
<line class="st18" x1="11.2" y1="33.1" x2="23.2" y2="31"/>
<g>
<polygon class="st0" points="22.9,33.7 26.8,30.4 22,28.6 "/>
</g>
</g>
</g>
<g>
<g>
<line class="st18" x1="61.5" y1="53" x2="71.9" y2="36.1"/>
<g>
<polygon class="st0" points="64.1,53.7 59.5,56.2 59.7,51 "/>
</g>
</g>
</g>
<g>
<g>
<line class="st18" x1="64.7" y1="24.4" x2="71.9" y2="36.6"/>
<g>
<polygon class="st0" points="62.9,26.4 62.9,21.2 67.3,23.7 "/>
</g>
</g>
</g>
<circle class="st0" cx="34.7" cy="28.5" r="8.1"/>
<circle class="st0" cx="8.1" cy="33.9" r="8.1"/>
<circle class="st0" cx="58.3" cy="14.5" r="8.1"/>
<circle class="st0" cx="55.7" cy="63.3" r="8.1"/>
<circle class="st0" cx="71.9" cy="36.1" r="8.1"/>
</g>
</g>
</g>
</svg>

After

Width:  |  Height:  |  Size: 6.2 KiB

229
config.go
View File

@@ -1,229 +0,0 @@
// Mgmt
// Copyright (C) 2013-2016+ James Shubin and the project contributors
// Written by James Shubin <james@shubin.ca> and the project contributors
//
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU Affero General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// This program is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU Affero General Public License for more details.
//
// You should have received a copy of the GNU Affero General Public License
// along with this program. If not, see <http://www.gnu.org/licenses/>.
package main
import (
"errors"
"gopkg.in/yaml.v2"
"io/ioutil"
"log"
"strings"
)
type collectorTypeConfig struct {
Type string `yaml:"type"`
Pattern string `yaml:"pattern"` // XXX: Not Implemented
}
type vertexConfig struct {
Type string `yaml:"type"`
Name string `yaml:"name"`
}
type edgeConfig struct {
Name string `yaml:"name"`
From vertexConfig `yaml:"from"`
To vertexConfig `yaml:"to"`
}
type GraphConfig struct {
Graph string `yaml:"graph"`
Types struct {
Noop []NoopType `yaml:"noop"`
File []FileType `yaml:"file"`
Service []ServiceType `yaml:"service"`
Exec []ExecType `yaml:"exec"`
} `yaml:"types"`
Collector []collectorTypeConfig `yaml:"collect"`
Edges []edgeConfig `yaml:"edges"`
Comment string `yaml:"comment"`
}
func (c *GraphConfig) Parse(data []byte) error {
if err := yaml.Unmarshal(data, c); err != nil {
return err
}
if c.Graph == "" {
return errors.New("Graph config: invalid `graph`")
}
return nil
}
func ParseConfigFromFile(filename string) *GraphConfig {
data, err := ioutil.ReadFile(filename)
if err != nil {
log.Printf("Error: Config: ParseConfigFromFile: File: %v", err)
return nil
}
var config GraphConfig
if err := config.Parse(data); err != nil {
log.Printf("Error: Config: ParseConfigFromFile: Parse: %v", err)
return nil
}
return &config
}
// XXX: we need to fix this function so that it either fails without modifying
// the graph, passes successfully and modifies it, or basically panics i guess
// this way an invalid compilation can leave the old graph running, and we we
// don't modify a partial graph. so we really need to validate, and then perform
// whatever actions are necessary
// finding some way to do this on a copy of the graph, and then do a graph diff
// and merge the new data into the old graph would be more appropriate, in
// particular if we can ensure the graph merge can't fail. As for the putting
// of stuff into etcd, we should probably store the operations to complete in
// the new graph, and keep retrying until it succeeds, thus blocking any new
// etcd operations until that time.
func UpdateGraphFromConfig(config *GraphConfig, hostname string, g *Graph, etcdO *EtcdWObject) bool {
var NoopMap = make(map[string]*Vertex)
var FileMap = make(map[string]*Vertex)
var ServiceMap = make(map[string]*Vertex)
var ExecMap = make(map[string]*Vertex)
var lookup = make(map[string]map[string]*Vertex)
lookup["noop"] = NoopMap
lookup["file"] = FileMap
lookup["service"] = ServiceMap
lookup["exec"] = ExecMap
//log.Printf("%+v", config) // debug
g.SetName(config.Graph) // set graph name
var keep []*Vertex // list of vertex which are the same in new graph
for _, t := range config.Types.Noop {
obj := NewNoopType(t.Name)
v := g.GetVertexMatch(obj)
if v == nil { // no match found
v = NewVertex(obj)
g.AddVertex(v) // call standalone in case not part of an edge
}
NoopMap[obj.Name] = v // used for constructing edges
keep = append(keep, v) // append
}
for _, t := range config.Types.File {
// XXX: should we export based on a @@ prefix, or a metaparam
// like exported => true || exported => (host pattern)||(other pattern?)
if strings.HasPrefix(t.Name, "@@") { // exported resource
// add to etcd storage...
t.Name = t.Name[2:] //slice off @@
if !etcdO.EtcdPut(hostname, t.Name, "file", t) {
log.Printf("Problem exporting file resource %v.", t.Name)
continue
}
} else {
obj := NewFileType(t.Name, t.Path, t.Dirname, t.Basename, t.Content, t.State)
v := g.GetVertexMatch(obj)
if v == nil { // no match found
v = NewVertex(obj)
g.AddVertex(v) // call standalone in case not part of an edge
}
FileMap[obj.Name] = v // used for constructing edges
keep = append(keep, v) // append
}
}
for _, t := range config.Types.Service {
obj := NewServiceType(t.Name, t.State, t.Startup)
v := g.GetVertexMatch(obj)
if v == nil { // no match found
v = NewVertex(obj)
g.AddVertex(v) // call standalone in case not part of an edge
}
ServiceMap[obj.Name] = v // used for constructing edges
keep = append(keep, v) // append
}
for _, t := range config.Types.Exec {
obj := NewExecType(t.Name, t.Cmd, t.Shell, t.Timeout, t.WatchCmd, t.WatchShell, t.IfCmd, t.IfShell, t.PollInt, t.State)
v := g.GetVertexMatch(obj)
if v == nil { // no match found
v = NewVertex(obj)
g.AddVertex(v) // call standalone in case not part of an edge
}
ExecMap[obj.Name] = v // used for constructing edges
keep = append(keep, v) // append
}
// lookup from etcd graph
// do all the graph look ups in one single step, so that if the etcd
// database changes, we don't have a partial state of affairs...
nodes, ok := etcdO.EtcdGet()
if ok {
for _, t := range config.Collector {
// XXX: use t.Type and optionally t.Pattern to collect from etcd storage
log.Printf("Collect: %v; Pattern: %v", t.Type, t.Pattern)
for _, x := range etcdO.EtcdGetProcess(nodes, "file") {
var obj *FileType
if B64ToObj(x, &obj) != true {
log.Printf("Collect: File: %v not collected!", x)
continue
}
if t.Pattern != "" { // XXX: currently the pattern for files can only override the Dirname variable :P
obj.Dirname = t.Pattern
}
log.Printf("Collect: File: %v collected!", obj.GetName())
// XXX: similar to file add code:
v := g.GetVertexMatch(obj)
if v == nil { // no match found
obj.Init() // initialize go channels or things won't work!!!
v = NewVertex(obj)
g.AddVertex(v) // call standalone in case not part of an edge
}
FileMap[obj.GetName()] = v // used for constructing edges
keep = append(keep, v) // append
}
}
}
// get rid of any vertices we shouldn't "keep" (that aren't in new graph)
for _, v := range g.GetVertices() {
if !HasVertex(v, keep) {
// wait for exit before starting new graph!
v.Type.SendEvent(eventExit, true, false)
g.DeleteVertex(v)
}
}
for _, e := range config.Edges {
if _, ok := lookup[e.From.Type]; !ok {
return false
}
if _, ok := lookup[e.To.Type]; !ok {
return false
}
if _, ok := lookup[e.From.Type][e.From.Name]; !ok {
return false
}
if _, ok := lookup[e.To.Type][e.To.Name]; !ok {
return false
}
g.AddEdge(lookup[e.From.Type][e.From.Name], lookup[e.To.Type][e.To.Name], NewEdge(e.Name))
}
return true
}

View File

@@ -1,155 +0,0 @@
// Mgmt
// Copyright (C) 2013-2016+ James Shubin and the project contributors
// Written by James Shubin <james@shubin.ca> and the project contributors
//
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU Affero General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// This program is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU Affero General Public License for more details.
//
// You should have received a copy of the GNU Affero General Public License
// along with this program. If not, see <http://www.gnu.org/licenses/>.
package main
import (
"gopkg.in/fsnotify.v1"
//"github.com/go-fsnotify/fsnotify" // git master of "gopkg.in/fsnotify.v1"
"log"
"math"
"path"
"strings"
"syscall"
)
// XXX: it would be great if we could reuse code between this and the file type
// XXX: patch this to submit it as part of go-fsnotify if they're interested...
func ConfigWatch(file string) chan bool {
ch := make(chan bool)
go func() {
var safename = path.Clean(file) // no trailing slash
watcher, err := fsnotify.NewWatcher()
if err != nil {
log.Fatal(err)
}
defer watcher.Close()
patharray := PathSplit(safename) // tokenize the path
var index = len(patharray) // starting index
var current string // current "watcher" location
var deltaDepth int // depth delta between watcher and event
var send = false // send event?
for {
current = strings.Join(patharray[0:index], "/")
if current == "" { // the empty string top is the root dir ("/")
current = "/"
}
log.Printf("Watching: %v", current) // attempting to watch...
// initialize in the loop so that we can reset on rm-ed handles
err = watcher.Add(current)
if err != nil {
if err == syscall.ENOENT {
index-- // usually not found, move up one dir
} else if err == syscall.ENOSPC {
// XXX: occasionally: no space left on device,
// XXX: probably due to lack of inotify watches
log.Printf("Lack of watches for config(%v) error: %+v", file, err.Error) // 0x408da0
log.Fatal(err)
} else {
log.Printf("Unknown config(%v) error:", file)
log.Fatal(err)
}
index = int(math.Max(1, float64(index)))
continue
}
select {
case event := <-watcher.Events:
// the deeper you go, the bigger the deltaDepth is...
// this is the difference between what we're watching,
// and the event... doesn't mean we can't watch deeper
if current == event.Name {
deltaDepth = 0 // i was watching what i was looking for
} else if HasPathPrefix(event.Name, current) {
deltaDepth = len(PathSplit(current)) - len(PathSplit(event.Name)) // -1 or less
} else if HasPathPrefix(current, event.Name) {
deltaDepth = len(PathSplit(event.Name)) - len(PathSplit(current)) // +1 or more
} else {
// TODO different watchers get each others events!
// https://github.com/go-fsnotify/fsnotify/issues/95
// this happened with two values such as:
// event.Name: /tmp/mgmt/f3 and current: /tmp/mgmt/f2
continue
}
//log.Printf("The delta depth is: %v", deltaDepth)
// if we have what we wanted, awesome, send an event...
if event.Name == safename {
//log.Println("Event!")
send = true
// file removed, move the watch upwards
if deltaDepth >= 0 && (event.Op&fsnotify.Remove == fsnotify.Remove) {
//log.Println("Removal!")
watcher.Remove(current)
index--
}
// we must be a parent watcher, so descend in
if deltaDepth < 0 {
watcher.Remove(current)
index++
}
// if safename starts with event.Name, we're above, and no event should be sent
} else if HasPathPrefix(safename, event.Name) {
//log.Println("Above!")
if deltaDepth >= 0 && (event.Op&fsnotify.Remove == fsnotify.Remove) {
log.Println("Removal!")
watcher.Remove(current)
index--
}
if deltaDepth < 0 {
log.Println("Parent!")
if PathPrefixDelta(safename, event.Name) == 1 { // we're the parent dir
//send = true
}
watcher.Remove(current)
index++
}
// if event.Name startswith safename, send event, we're already deeper
} else if HasPathPrefix(event.Name, safename) {
//log.Println("Event2!")
//send = true
}
case err := <-watcher.Errors:
log.Println("error:", err)
log.Fatal(err)
}
// do our event sending all together to avoid duplicate msgs
if send {
send = false
ch <- true
}
}
//close(ch)
}()
return ch
}

386
converger/converger.go Normal file
View File

@@ -0,0 +1,386 @@
// Mgmt
// Copyright (C) 2013-2017+ James Shubin and the project contributors
// Written by James Shubin <james@shubin.ca> and the project contributors
//
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU Affero General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// This program is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU Affero General Public License for more details.
//
// You should have received a copy of the GNU Affero General Public License
// along with this program. If not, see <http://www.gnu.org/licenses/>.
// Package converger is a facility for reporting the converged state.
package converger
import (
"fmt"
"sync"
"time"
"github.com/purpleidea/mgmt/util"
)
// TODO: we could make a new function that masks out the state of certain
// UID's, but at the moment the new Timer code has obsoleted the need...
// Converger is the general interface for implementing a convergence watcher.
type Converger interface { // TODO: need a better name
Register() UID
IsConverged(UID) bool // is the UID converged ?
SetConverged(UID, bool) error // set the converged state of the UID
Unregister(UID)
Start()
Pause()
Loop(bool)
ConvergedTimer(UID) <-chan time.Time
Status() map[uint64]bool
Timeout() int // returns the timeout that this was created with
SetStateFn(func(bool) error) // sets the stateFn
}
// UID is the interface resources can use to notify with if converged. You'll
// need to use part of the Converger interface to Register initially too.
type UID interface {
ID() uint64 // get Id
Name() string // get a friendly name
SetName(string)
IsValid() bool // has Id been initialized ?
InvalidateID() // set Id to nil
IsConverged() bool
SetConverged(bool) error
Unregister()
ConvergedTimer() <-chan time.Time
StartTimer() (func() error, error) // cancellable is the same as StopTimer()
ResetTimer() error // resets counter to zero
StopTimer() error
}
// converger is an implementation of the Converger interface.
type converger struct {
timeout int // must be zero (instant) or greater seconds to run
stateFn func(bool) error // run on converged state changes with state bool
converged bool // did we converge (state changes of this run Fn)
channel chan struct{} // signal here to run an isConverged check
control chan bool // control channel for start/pause
mutex sync.RWMutex // used for controlling access to status and lastid
lastid uint64
status map[uint64]bool
}
// cuid is an implementation of the UID interface.
type cuid struct {
converger Converger
id uint64
name string // user defined, friendly name
mutex sync.Mutex
timer chan struct{}
running bool // is the above timer running?
wg sync.WaitGroup
}
// NewConverger builds a new converger struct.
func NewConverger(timeout int, stateFn func(bool) error) *converger {
return &converger{
timeout: timeout,
stateFn: stateFn,
channel: make(chan struct{}),
control: make(chan bool),
lastid: 0,
status: make(map[uint64]bool),
}
}
// Register assigns a UID to the caller.
func (obj *converger) Register() UID {
obj.mutex.Lock()
defer obj.mutex.Unlock()
obj.lastid++
obj.status[obj.lastid] = false // initialize as not converged
return &cuid{
converger: obj,
id: obj.lastid,
name: fmt.Sprintf("%d", obj.lastid), // some default
timer: nil,
running: false,
}
}
// IsConverged gets the converged status of a uid.
func (obj *converger) IsConverged(uid UID) bool {
if !uid.IsValid() {
panic(fmt.Sprintf("the ID of UID(%s) is nil", uid.Name()))
}
obj.mutex.RLock()
isConverged, found := obj.status[uid.ID()] // lookup
obj.mutex.RUnlock()
if !found {
panic("the ID of UID is unregistered")
}
return isConverged
}
// SetConverged updates the converger with the converged state of the UID.
func (obj *converger) SetConverged(uid UID, isConverged bool) error {
if !uid.IsValid() {
return fmt.Errorf("the ID of UID(%s) is nil", uid.Name())
}
obj.mutex.Lock()
if _, found := obj.status[uid.ID()]; !found {
panic("the ID of UID is unregistered")
}
obj.status[uid.ID()] = isConverged // set
obj.mutex.Unlock() // unlock *before* poke or deadlock!
if isConverged != obj.converged { // only poke if it would be helpful
// run in a go routine so that we never block... just queue up!
// this allows us to send events, even if we haven't started...
go func() { obj.channel <- struct{}{} }()
}
return nil
}
// isConverged returns true if *every* registered uid has converged.
func (obj *converger) isConverged() bool {
obj.mutex.RLock() // take a read lock
defer obj.mutex.RUnlock()
for _, v := range obj.status {
if !v { // everyone must be converged for this to be true
return false
}
}
return true
}
// Unregister dissociates the ConvergedUID from the converged checking.
func (obj *converger) Unregister(uid UID) {
if !uid.IsValid() {
panic(fmt.Sprintf("the ID of UID(%s) is nil", uid.Name()))
}
obj.mutex.Lock()
uid.StopTimer() // ignore any errors
delete(obj.status, uid.ID())
obj.mutex.Unlock()
uid.InvalidateID()
}
// Start causes a Converger object to start or resume running.
func (obj *converger) Start() {
obj.control <- true
}
// Pause causes a Converger object to stop running temporarily.
func (obj *converger) Pause() { // FIXME: add a sync ACK on pause before return
obj.control <- false
}
// Loop is the main loop for a Converger object. It usually runs in a goroutine.
// TODO: we could eventually have each resource tell us as soon as it converges,
// and then keep track of the time delays here, to avoid callers needing select.
// NOTE: when we have very short timeouts, if we start before all the resources
// have joined the map, then it might appear as if we converged before we did!
func (obj *converger) Loop(startPaused bool) {
if obj.control == nil {
panic("converger not initialized correctly")
}
if startPaused { // start paused without racing
select {
case e := <-obj.control:
if !e {
panic("converger expected true")
}
}
}
for {
select {
case e := <-obj.control: // expecting "false" which means pause!
if e {
panic("converger expected false")
}
// now i'm paused...
select {
case e := <-obj.control:
if !e {
panic("converger expected true")
}
// restart
// kick once to refresh the check...
go func() { obj.channel <- struct{}{} }()
continue
}
case <-obj.channel:
if !obj.isConverged() {
if obj.converged { // we're doing a state change
if obj.stateFn != nil {
// call an arbitrary function
if err := obj.stateFn(false); err != nil {
// FIXME: what to do on error ?
}
}
}
obj.converged = false
continue
}
// we have converged!
if obj.timeout >= 0 { // only run if timeout is valid
if !obj.converged { // we're doing a state change
if obj.stateFn != nil {
// call an arbitrary function
if err := obj.stateFn(true); err != nil {
// FIXME: what to do on error ?
}
}
}
}
obj.converged = true
// loop and wait again...
}
}
}
// ConvergedTimer adds a timeout to a select call and blocks until then.
// TODO: this means we could eventually have per resource converged timeouts
func (obj *converger) ConvergedTimer(uid UID) <-chan time.Time {
// be clever: if i'm already converged, this timeout should block which
// avoids unnecessary new signals being sent! this avoids fast loops if
// we have a low timeout, or in particular a timeout == 0
if uid.IsConverged() {
// blocks the case statement in select forever!
return util.TimeAfterOrBlock(-1)
}
return util.TimeAfterOrBlock(obj.timeout)
}
// Status returns a map of the converged status of each UID.
func (obj *converger) Status() map[uint64]bool {
status := make(map[uint64]bool)
obj.mutex.RLock() // take a read lock
defer obj.mutex.RUnlock()
for k, v := range obj.status { // make a copy to avoid the mutex
status[k] = v
}
return status
}
// Timeout returns the timeout in seconds that converger was created with. This
// is useful to avoid passing in the timeout value separately when you're
// already passing in the Converger struct.
func (obj *converger) Timeout() int {
return obj.timeout
}
// SetStateFn sets the state function to be run on change of converged state.
func (obj *converger) SetStateFn(stateFn func(bool) error) {
obj.stateFn = stateFn
}
// ID returns the unique id of this UID object.
func (obj *cuid) ID() uint64 {
return obj.id
}
// Name returns a user defined name for the specific cuid.
func (obj *cuid) Name() string {
return obj.name
}
// SetName sets a user defined name for the specific cuid.
func (obj *cuid) SetName(name string) {
obj.name = name
}
// IsValid tells us if the id is valid or has already been destroyed.
func (obj *cuid) IsValid() bool {
return obj.id != 0 // an id of 0 is invalid
}
// InvalidateID marks the id as no longer valid.
func (obj *cuid) InvalidateID() {
obj.id = 0 // an id of 0 is invalid
}
// IsConverged is a helper function to the regular IsConverged method.
func (obj *cuid) IsConverged() bool {
return obj.converger.IsConverged(obj)
}
// SetConverged is a helper function to the regular SetConverged notification.
func (obj *cuid) SetConverged(isConverged bool) error {
return obj.converger.SetConverged(obj, isConverged)
}
// Unregister is a helper function to unregister myself.
func (obj *cuid) Unregister() {
obj.converger.Unregister(obj)
}
// ConvergedTimer is a helper around the regular ConvergedTimer method.
func (obj *cuid) ConvergedTimer() <-chan time.Time {
return obj.converger.ConvergedTimer(obj)
}
// StartTimer runs an invisible timer that automatically converges on timeout.
func (obj *cuid) StartTimer() (func() error, error) {
obj.mutex.Lock()
if !obj.running {
obj.timer = make(chan struct{})
obj.running = true
} else {
obj.mutex.Unlock()
return obj.StopTimer, fmt.Errorf("timer already started")
}
obj.mutex.Unlock()
obj.wg.Add(1)
go func() {
defer obj.wg.Done()
for {
select {
case _, ok := <-obj.timer: // reset signal channel
if !ok { // channel is closed
return // false to exit
}
obj.SetConverged(false)
case <-obj.ConvergedTimer():
obj.SetConverged(true) // converged!
select {
case _, ok := <-obj.timer: // reset signal channel
if !ok { // channel is closed
return // false to exit
}
}
}
}
}()
return obj.StopTimer, nil
}
// ResetTimer resets the counter to zero if using a StartTimer internally.
func (obj *cuid) ResetTimer() error {
obj.mutex.Lock()
defer obj.mutex.Unlock()
if obj.running {
obj.timer <- struct{}{} // send the reset message
return nil
}
return fmt.Errorf("timer hasn't been started")
}
// StopTimer stops the running timer permanently until a StartTimer is run.
func (obj *cuid) StopTimer() error {
obj.mutex.Lock()
defer obj.mutex.Unlock()
if !obj.running {
return fmt.Errorf("timer isn't running")
}
close(obj.timer)
obj.wg.Wait()
obj.running = false
return nil
}

View File

@@ -1,5 +1,5 @@
// Mgmt // Mgmt
// Copyright (C) 2013-2016+ James Shubin and the project contributors // Copyright (C) 2013-2017+ James Shubin and the project contributors
// Written by James Shubin <james@shubin.ca> and the project contributors // Written by James Shubin <james@shubin.ca> and the project contributors
// //
// This program is free software: you can redistribute it and/or modify // This program is free software: you can redistribute it and/or modify
@@ -15,12 +15,5 @@
// You should have received a copy of the GNU Affero General Public License // You should have received a copy of the GNU Affero General Public License
// along with this program. If not, see <http://www.gnu.org/licenses/>. // along with this program. If not, see <http://www.gnu.org/licenses/>.
// Package main provides the main entrypoint for using the `mgmt` software.
package main package main
import (
//"testing"
)
//func TestT1(t *testing.T) {
//}

22
docker/Dockerfile Normal file
View File

@@ -0,0 +1,22 @@
FROM golang:1.6.2
MAINTAINER Michał Czeraszkiewicz <contact@czerasz.com>
# Set the reset cache variable
# Read more here: http://czerasz.com/2014/11/13/docker-tip-and-tricks/#use-refreshedat-variable-for-better-cache-control
ENV REFRESHED_AT 2016-05-10
# Update the package list to be able to use required packages
RUN apt-get update
# Change the working directory
WORKDIR /go/src/mgmt
# Copy all the files to the working directory
COPY . /go/src/mgmt
# Install dependencies
RUN make deps
# Build the binary
RUN make build

View File

@@ -0,0 +1,31 @@
FROM golang:1.6.2
MAINTAINER Michał Czeraszkiewicz <contact@czerasz.com>
# Set the reset cache variable
# Read more here: http://czerasz.com/2014/11/13/docker-tip-and-tricks/#use-refreshedat-variable-for-better-cache-control
ENV REFRESHED_AT 2016-05-14
RUN apt-get update
# Setup User to match Host User
# Give the nre user superuser permissions
ARG USER_ID=1000
ARG GROUP_ID=1000
ARG USER_NAME=mgmt
ARG GROUP_NAME=$USER_NAME
RUN groupadd --gid $GROUP_ID $GROUP_NAME && \
useradd --create-home --home /home/$USER_NAME --uid ${USER_ID} --gid $GROUP_NAME --groups sudo $USER_NAME && \
echo "$USER_NAME ALL=(ALL) NOPASSWD:ALL" >> /etc/sudoers
# Copy all the files to the working directory
COPY . /home/$USER_NAME/mgmt
# Change working directory
WORKDIR /home/$USER_NAME/mgmt
# Install dependencies
RUN make deps
# Change user
USER ${USER_NAME}

26
docker/scripts/build Executable file
View File

@@ -0,0 +1,26 @@
#!/bin/bash
script_directory="$( cd "$( dirname "$0" )" && pwd )"
project_directory=$script_directory/../..
# Specify the Docker image name
image_name='purpleidea/mgmt'
# Build the image which contains the compiled binary
docker build -t $image_name \
--file=$project_directory/docker/Dockerfile $project_directory
# Remove the container if it already exists
docker rm -f mgmt-export 2> /dev/null
# Start the container in background so we can "copy out" the binary
docker run -d --name=mgmt-export $image_name bash -c 'while true; sleep 1000; done'
# Remove the current binary
rm $project_directory/mgmt 2> /dev/null
# Get the binary from the container
docker cp mgmt-export:/go/src/mgmt/mgmt $project_directory/mgmt
# Remove the container
docker rm -f mgmt-export 2> /dev/null

View File

@@ -0,0 +1,14 @@
#!/bin/bash
# Stop on any error
set -e
script_directory="$( cd "$( dirname "$0" )" && pwd )"
project_directory=$script_directory/../..
# Specify the Docker image name
image_name='purpleidea/mgmt:development'
# Build the image
docker build -t $image_name \
--file=$project_directory/docker/Dockerfile.development $project_directory

15
docker/scripts/run-development Executable file
View File

@@ -0,0 +1,15 @@
#!/bin/bash
# Stop on any error
set -e
script_directory="$( cd "$( dirname "$0" )" && pwd )"
project_directory=$script_directory/../..
# Specify the Docker image name
image_name='purpleidea/mgmt:development'
# Run container in development mode
docker run --rm --name=mgm_development --user=mgmt \
-v $project_directory:/home/mgmt/mgmt \
-it $image_name bash

2
docs/.gitignore vendored Normal file
View File

@@ -0,0 +1,2 @@
mgmt-documentation.pdf
_build

20
docs/Makefile Normal file
View File

@@ -0,0 +1,20 @@
# Minimal makefile for Sphinx documentation
#
# You can set these variables from the command line.
SPHINXOPTS =
SPHINXBUILD = sphinx-build
SPHINXPROJ = mgmt
SOURCEDIR = .
BUILDDIR = _build
# Put it first so that "make" without argument is like "make help".
help:
@$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
.PHONY: help Makefile
# Catch-all target: route all unknown targets to Sphinx using the new
# "make mode" option. $(O) is meant as a shortcut for $(SPHINXOPTS).
%: Makefile
@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)

158
docs/conf.py Normal file
View File

@@ -0,0 +1,158 @@
# -*- coding: utf-8 -*-
#
# mgmt documentation build configuration file, created by
# sphinx-quickstart on Wed Feb 15 21:34:09 2017.
#
# This file is execfile()d with the current directory set to its
# containing dir.
#
# Note that not all possible configuration values are present in this
# autogenerated file.
#
# All configuration values have a default; values that are commented out
# serve to show the default.
# If extensions (or modules to document with autodoc) are in another directory,
# add these directories to sys.path here. If the directory is relative to the
# documentation root, use os.path.abspath to make it absolute, like shown here.
#
# import os
# import sys
# sys.path.insert(0, os.path.abspath('.'))
from recommonmark.parser import CommonMarkParser
# -- General configuration ------------------------------------------------
# If your documentation needs a minimal Sphinx version, state it here.
#
# needs_sphinx = '1.0'
# Add any Sphinx extension module names here, as strings. They can be
# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
# ones.
extensions = []
# Add any paths that contain templates here, relative to this directory.
templates_path = ['_templates']
# The suffix(es) of source filenames.
# You can specify multiple suffix as a list of string:
#
source_parsers = {
'.md': CommonMarkParser,
}
source_suffix = ['.rst', '.md']
# The master toctree document.
master_doc = 'index'
# General information about the project.
project = u'mgmt'
copyright = u'2013-2017+ James Shubin and the project contributors'
author = u'James Shubin'
# The version info for the project you're documenting, acts as replacement for
# |version| and |release|, also used in various other places throughout the
# built documents.
#
# The short X.Y version.
version = u''
# The full version, including alpha/beta/rc tags.
release = u''
# The language for content autogenerated by Sphinx. Refer to documentation
# for a list of supported languages.
#
# This is also used if you do content translation via gettext catalogs.
# Usually you set "language" from the command line for these cases.
language = None
# List of patterns, relative to source directory, that match files and
# directories to ignore when looking for source files.
# This patterns also effect to html_static_path and html_extra_path
exclude_patterns = ['_build', 'Thumbs.db', '.DS_Store', 'venv']
# The name of the Pygments (syntax highlighting) style to use.
pygments_style = 'sphinx'
# If true, `todo` and `todoList` produce output, else they produce nothing.
todo_include_todos = False
# -- Options for HTML output ----------------------------------------------
# The theme to use for HTML and HTML Help pages. See the documentation for
# a list of builtin themes.
#
#html_theme = 'alabaster'
# Theme options are theme-specific and customize the look and feel of a theme
# further. For a list of options available for each theme, see the
# documentation.
#
# html_theme_options = {}
# Add any paths that contain custom static files (such as style sheets) here,
# relative to this directory. They are copied after the builtin static files,
# so a file named "default.css" will overwrite the builtin "default.css".
html_static_path = ['_static']
# -- Options for HTMLHelp output ------------------------------------------
# Output file base name for HTML help builder.
htmlhelp_basename = 'mgmtdoc'
# -- Options for LaTeX output ---------------------------------------------
latex_elements = {
# The paper size ('letterpaper' or 'a4paper').
#
# 'papersize': 'letterpaper',
# The font size ('10pt', '11pt' or '12pt').
#
# 'pointsize': '10pt',
# Additional stuff for the LaTeX preamble.
#
# 'preamble': '',
# Latex figure (float) alignment
#
# 'figure_align': 'htbp',
}
# Grouping the document tree into LaTeX files. List of tuples
# (source start file, target name, title,
# author, documentclass [howto, manual, or own class]).
latex_documents = [
(master_doc, 'mgmt.tex', u'mgmt Documentation',
u'James Shubin', 'manual'),
]
# -- Options for manual page output ---------------------------------------
# One entry per manual page. List of tuples
# (source start file, name, description, authors, manual section).
man_pages = [
(master_doc, 'mgmt', u'mgmt Documentation',
[author], 1)
]
# -- Options for Texinfo output -------------------------------------------
# Grouping the document tree into Texinfo files. List of tuples
# (source start file, target name, title, author,
# dir menu entry, description, category)
texinfo_documents = [
(master_doc, 'mgmt', u'mgmt Documentation',
author, 'mgmt', 'A next generation config management prototype!',
'Miscellaneous'),
]

646
docs/documentation.md Normal file
View File

@@ -0,0 +1,646 @@
# mgmt
Available from:
[https://github.com/purpleidea/mgmt/](https://github.com/purpleidea/mgmt/)
This documentation is available in: [Markdown](https://github.com/purpleidea/mgmt/blob/master/docs/documentation.md) or [PDF](https://pdfdoc-purpleidea.rhcloud.com/pdf/https://github.com/purpleidea/mgmt/blob/master/docs/documentation.md) format.
## Overview
The `mgmt` tool is a next generation config management prototype. It's not yet
ready for production, but we hope to get there soon. Get involved today!
## Project Description
The mgmt tool is a distributed, event driven, config management tool, that
supports parallel execution, and librarification to be used as the management
foundation in and for, new and existing software.
For more information, you may like to read some blog posts from the author:
* [Next generation config mgmt](https://ttboj.wordpress.com/2016/01/18/next-generation-configuration-mgmt/)
* [Automatic edges in mgmt](https://ttboj.wordpress.com/2016/03/14/automatic-edges-in-mgmt/)
* [Automatic grouping in mgmt](https://ttboj.wordpress.com/2016/03/30/automatic-grouping-in-mgmt/)
* [Automatic clustering in mgmt](https://ttboj.wordpress.com/2016/06/20/automatic-clustering-in-mgmt/)
* [Remote execution in mgmt](https://ttboj.wordpress.com/2016/10/07/remote-execution-in-mgmt/)
* [Send/Recv in mgmt](https://ttboj.wordpress.com/2016/12/07/sendrecv-in-mgmt/)
* [Metaparameters in mgmt](https://ttboj.wordpress.com/2017/03/01/metaparameters-in-mgmt/)
There is also an [introductory video](http://meetings-archive.debian.net/pub/debian-meetings/2016/debconf16/Next_Generation_Config_Mgmt.webm) available.
Older videos and other material [is available](https://github.com/purpleidea/mgmt/#on-the-web).
## Setup
During this prototype phase, the tool can be run out of the source directory.
You'll probably want to use ```./run.sh run --yaml examples/graph1.yaml``` to
get started. Beware that this _can_ cause data loss. Understand what you're
doing first, or perform these actions in a virtual environment such as the one
provided by [Oh-My-Vagrant](https://github.com/purpleidea/oh-my-vagrant).
## Features
This section details the numerous features of mgmt and some caveats you might
need to be aware of.
### Autoedges
Automatic edges, or AutoEdges, is the mechanism in mgmt by which it will
automatically create dependencies for you between resources. For example,
since mgmt can discover which files are installed by a package it will
automatically ensure that any file resource you declare that matches a
file installed by your package resource will only be processed after the
package is installed.
#### Controlling autoedges
Though autoedges is likely to be very helpful and avoid you having to declare
all dependencies explicitly, there are cases where this behaviour is
undesirable.
Some distributions allow package installations to automatically start the
service they ship. This can be problematic in the case of packages like MySQL
as there are configuration options that need to be set before MySQL is ever
started for the first time (or you'll need to wipe the data directory). In
order to handle this situation you can disable autoedges per resource and
explicitly declare that you want `my.cnf` to be written to disk before the
installation of the `mysql-server` package.
You can disable autoedges for a resource by setting the `autoedge` key on
the meta attributes of that resource to `false`.
#### Blog post
You can read the introductory blog post about this topic here:
[https://ttboj.wordpress.com/2016/03/14/automatic-edges-in-mgmt/](https://ttboj.wordpress.com/2016/03/14/automatic-edges-in-mgmt/)
### Autogrouping
Automatic grouping or AutoGroup is the mechanism in mgmt by which it will
automatically group multiple resource vertices into a single one. This is
particularly useful for grouping multiple package resources into a single
resource, since the multiple installations can happen together in a single
transaction, which saves a lot of time because package resources typically have
a large fixed cost to running (downloading and verifying the package repo) and
if they are grouped they share this fixed cost. This grouping feature can be
used for other use cases too.
You can disable autogrouping for a resource by setting the `autogroup` key on
the meta attributes of that resource to `false`.
#### Blog post
You can read the introductory blog post about this topic here:
[https://ttboj.wordpress.com/2016/03/30/automatic-grouping-in-mgmt/](https://ttboj.wordpress.com/2016/03/30/automatic-grouping-in-mgmt/)
### Automatic clustering
Automatic clustering is a feature by which mgmt automatically builds, scales,
and manages the embedded etcd cluster which is compiled into mgmt itself. It is
quite helpful for rapidly bootstrapping clusters and avoiding the extra work to
setup etcd.
If you prefer to avoid this feature. you can always opt to use an existing etcd
cluster that is managed separately from mgmt by pointing your mgmt agents at it
with the `--seeds` variable.
#### Blog post
You can read the introductory blog post about this topic here:
[https://ttboj.wordpress.com/2016/06/20/automatic-clustering-in-mgmt/](https://ttboj.wordpress.com/2016/06/20/automatic-clustering-in-mgmt/)
### Remote ("agent-less") mode
Remote mode is a special mode that lets you kick off mgmt runs on one or more
remote machines which are only accessible via SSH. In this mode the initiating
host connects over SSH, copies over the `mgmt` binary, opens an SSH tunnel, and
runs the remote program while simultaneously passing the etcd traffic back
through the tunnel so that the initiators etcd cluster can be used to exchange
resource data.
The interesting benefit of this architecture is that multiple hosts which can't
connect directly use the initiator to pass the important traffic through to each
other. Once the cluster has converged all the remote programs can shutdown
leaving no residual agent.
This mode can also be useful for bootstrapping a new host where you'd like to
have the service run continuously and as part of an mgmt cluster normally.
In particular, when combined with the `--converged-timeout` parameter, the
entire set of running mgmt agents will need to all simultaneously converge for
the group to exit. This is particularly useful for bootstrapping new clusters
which need to exchange information that is only available at run time.
#### Blog post
You can read the introductory blog post about this topic here:
[https://ttboj.wordpress.com/2016/10/07/remote-execution-in-mgmt/](https://ttboj.wordpress.com/2016/10/07/remote-execution-in-mgmt/)
### Puppet support
You can supply a Puppet manifest instead of creating the (YAML) graph manually.
Puppet must be installed and in `mgmt`'s search path. You also need the
[ffrank-mgmtgraph Puppet module](https://forge.puppet.com/ffrank/mgmtgraph).
Invoke `mgmt` with the `--puppet` switch, which supports 3 variants:
1. Request the configuration from the Puppet Master (like `puppet agent` does)
mgmt run --puppet agent
2. Compile a local manifest file (like `puppet apply`)
mgmt run --puppet /path/to/my/manifest.pp
3. Compile an ad hoc manifest from the commandline (like `puppet apply -e`)
mgmt run --puppet 'file { "/etc/ntp.conf": ensure => file }'
For more details and caveats see [Puppet.md](Puppet.md).
#### Blog post
An introductory post on the Puppet support is on
[Felix's blog](http://ffrank.github.io/features/2016/06/19/puppet-powered-mgmt/).
## Resources
This section lists all the built-in resources and their properties. The
resource primitives in `mgmt` are typically more powerful than resources in
other configuration management systems because they can be event based which
lets them respond in real-time to converge to the desired state. This property
allows you to build more complex resources that you probably hadn't considered
in the past.
In addition to the resource specific properties, there are resource properties
(otherwise known as parameters) which can apply to every resource. These are
called [meta parameters](#meta-parameters) and are listed separately. Certain
meta parameters aren't very useful when combined with certain resources, but
in general, it should be fairly obvious, such as when combining the `noop` meta
parameter with the [Noop](#Noop) resource.
* [Augeas](#Augeas): Manipulate files using augeas.
* [Exec](#Exec): Execute shell commands on the system.
* [File](#File): Manage files and directories.
* [Hostname](#Hostname): Manages the hostname on the system.
* [KV](#KV): Set a key value pair in our shared world database.
* [Msg](#Msg): Send log messages.
* [Noop](#Noop): A simple resource that does nothing.
* [Nspawn](#Nspawn): Manage systemd-machined nspawn containers.
* [Password](#Password): Create random password strings.
* [Pkg](#Pkg): Manage system packages with PackageKit.
* [Svc](#Svc): Manage system systemd services.
* [Timer](#Timer): Manage system systemd services.
* [Virt](#Virt): Manage virtual machines with libvirt.
### Augeas
The augeas resource uses [augeas](http://augeas.net/) commands to manipulate
files.
### Exec
The exec resource can execute commands on your system.
### File
The file resource manages files and directories. In `mgmt`, directories are
identified by a trailing slash in their path name. File have no such slash.
It has the following properties:
- `path`: file path (directories have a trailing slash here)
- `content`: raw file content
- `state`: either `exists` (the default value) or `absent`
- `mode`: octal unix file permissions
- `owner`: username or uid for the file owner
- `group`: group name or gid for the file group
#### Path
The path property specifies the file or directory that we are managing.
#### Content
The content property is a string that specifies the desired file contents.
#### Source
The source property points to a source file or directory path that we wish to
copy over and use as the desired contents for our resource.
#### State
The state property describes the action we'd like to apply for the resource. The
possible values are: `exists` and `absent`.
#### Recurse
The recurse property limits whether file resource operations should recurse into
and monitor directory contents with a depth greater than one.
#### Force
The force property is required if we want the file resource to be able to change
a file into a directory or vice-versa. If such a change is needed, but the force
property is not set to `true`, then this file resource will error.
### Hostname
The hostname resource manages static, transient/dynamic and pretty hostnames
on the system and watches them for changes.
#### static_hostname
The static hostname is the one configured in /etc/hostname or a similar
file.
It is chosen by the local user. It is not always in sync with the current
host name as returned by the gethostname() system call.
#### transient_hostname
The transient / dynamic hostname is the one configured via the kernel's
sethostbyname().
It can be different from the static hostname in case DHCP or mDNS have been
configured to change the name based on network information.
#### pretty_hostname
The pretty hostname is a free-form UTF8 host name for presentation to the user.
#### hostname
Hostname is the fallback value for all 3 fields above, if only `hostname` is
specified, it will set all 3 fields to this value.
### KV
The KV resource sets a key and value pair in the global world database. This is
quite useful for setting a flag after a number of resources have run. It will
ignore database updates to the value that are greater in compare order than the
requested key if the `SkipLessThan` parameter is set to true. If we receive a
refresh, then the stored value will be reset to the requested value even if the
stored value is greater.
#### Key
The string key used to store the key.
#### Value
The string value to set. This can also be set via Send/Recv.
#### SkipLessThan
If this parameter is set to `true`, then it will ignore updating the value as
long as the database versions are greater than the requested value. The compare
operation used is based on the `SkipCmpStyle` parameter.
#### SkipCmpStyle
By default this converts the string values to integers and compares them as you
would expect.
### Msg
The msg resource sends messages to the main log, or an external service such
as systemd's journal.
### Noop
The noop resource does absolutely nothing. It does have some utility in testing
`mgmt` and also as a placeholder in the resource graph.
### Nspawn
The nspawn resource is used to manage systemd-machined style containers.
### Password
The password resource can generate a random string to be used as a password. It
will re-generate the password if it receives a refresh notification.
### Pkg
The pkg resource is used to manage system packages. This resource works on many
different distributions because it uses the underlying packagekit facility which
supports different backends for different environments. This ensures that we
have great Debian (deb/dpkg) and Fedora (rpm/dnf) support simultaneously.
### Svc
The service resource is still very WIP. Please help us my improving it!
### Timer
This resource needs better documentation. Please help us my improving it!
### Virt
The virt resource can manage virtual machines via libvirt.
## Usage and frequently asked questions
(Send your questions as a patch to this FAQ! I'll review it, merge it, and
respond by commit with the answer.)
### Why did you start this project?
I wanted a next generation config management solution that didn't have all of
the design flaws or limitations that the current generation of tools do, and no
tool existed!
### Why did you use etcd? What about consul?
Etcd and consul are both written in golang, which made them the top two
contenders for my prototype. Ultimately a choice had to be made, and etcd was
chosen, but it was also somewhat arbitrary. If there is available interest,
good reasoning, *and* patches, then we would consider either switching or
supporting both, but this is not a high priority at this time.
### Can I use an existing etcd cluster instead of the automatic embedded servers?
Yes, it's possible to use an existing etcd cluster instead of the automatic,
elastic embedded etcd servers. To do so, simply point to the cluster with the
`--seeds` variable, the same way you would if you were seeding a new member to
an existing mgmt cluster.
The downside to this approach is that you won't benefit from the automatic
elastic nature of the embedded etcd servers, and that you're responsible if you
accidentally break your etcd cluster, or if you use an unsupported version.
### What does the error message about an inconsistent dataDir mean?
If you get an error message similar to:
```
Etcd: Connect: CtxError...
Etcd: CtxError: Reason: CtxDelayErr(5s): No endpoints available yet!
Etcd: Connect: Endpoints: []
Etcd: The dataDir (/var/lib/mgmt/etcd) might be inconsistent or corrupt.
```
This happens when there are a series of fatal connect errors in a row. This can
happen when you start `mgmt` using a dataDir that doesn't correspond to the
current cluster view. As a result, the embedded etcd server never finishes
starting up, and as a result, a default endpoint never gets added. The solution
is to either reconcile the mistake, and if there is no important data saved, you
can remove the etcd dataDir. This is typically `/var/lib/mgmt/etcd/member/`.
### Why do resources have both a `Compare` method and an `IFF` (on the UID) method?
The `Compare()` methods are for determining if two resources are effectively the
same, which is used to make graph change delta's efficient. This is when we want
to change from the current running graph to a new graph, but preserve the common
vertices. Since we want to make this process efficient, we only update the parts
that are different, and leave everything else alone. This `Compare()` method can
tell us if two resources are the same.
The `IFF()` method is part of the whole UID system, which is for discerning if a
resource meets the requirements another expects for an automatic edge. This is
because the automatic edge system assumes a unified UID pattern to test for
equality. In the future it might be helpful or sane to merge the two similar
comparison functions although for now they are separate because they are
actually answer different questions.
### Did you know that there is a band named `MGMT`?
I didn't realize this when naming the project, and it is accidental. After much
anguishing, I chose the name because it was short and I thought it was
appropriately descriptive. If you need a less ambiguous search term or phrase,
you can try using `mgmtconfig` or `mgmt config`.
### You didn't answer my question, or I have a question!
It's best to ask on [IRC](https://webchat.freenode.net/?channels=#mgmtconfig)
to see if someone can help you. Once we get a big enough community going, we'll
add a mailing list. If you don't get any response from the above, you can
contact me through my [technical blog](https://ttboj.wordpress.com/contact/)
and I'll do my best to help. If you have a good question, please add it as a
patch to this documentation. I'll merge your question, and add a patch with the
answer!
## Reference
Please note that there are a number of undocumented options. For more
information on these options, please view the source at:
[https://github.com/purpleidea/mgmt/](https://github.com/purpleidea/mgmt/).
If you feel that a well used option needs documenting here, please patch it!
### Overview of reference
* [Meta parameters](#meta-parameters): List of available resource meta parameters.
* [Graph definition file](#graph-definition-file): Main graph definition file.
* [Command line](#command-line): Command line parameters.
* [Compilation options](#compilation-options): Compilation options.
### Meta parameters
These meta parameters are special parameters (or properties) which can apply to
any resource. The usefulness of doing so will depend on the particular meta
parameter and resource combination.
#### AutoEdge
Boolean. Should we generate auto edges for this resource?
#### AutoGroup
Boolean. Should we attempt to automatically group this resource with others?
#### Noop
Boolean. Should the Apply portion of the CheckApply method of the resource
make any changes? Noop is a concatenation of no-operation.
#### Retry
Integer. The number of times to retry running the resource on error. Use -1 for
infinite. This currently applies for both the Watch operation (which can fail)
and for the CheckApply operation. While they could have separate values, I've
decided to use the same ones for both until there's a proper reason to want to
do something differently for the Watch errors.
#### Delay
Integer. Number of milliseconds to wait between retries. The same value is
shared between the Watch and CheckApply retries. This currently applies for both
the Watch operation (which can fail) and for the CheckApply operation. While
they could have separate values, I've decided to use the same ones for both
until there's a proper reason to want to do something differently for the Watch
errors.
#### Poll
Integer. Number of seconds to wait between `CheckApply` checks. If this is
greater than zero, then the standard event based `Watch` mechanism for this
resource is replaced with a simple polling mechanism. In general, this is not
recommended, unless you have a very good reason for doing so.
Please keep in mind that if you have a resource which changes every `I` seconds,
and you poll it every `J` seconds, and you've asked for a converged timeout of
`K` seconds, and `I <= J <= K`, then your graph will likely never converge.
When polling, the system detects that a resource is not converged if its
`CheckApply` method returns false. This allows a resource which changes every
`I` seconds, and which is polled every `J` seconds, and with a converged timeout
of `K` seconds to still converge when `J <= K`, as long as `I > J || I > K`,
which is another way of saying that if the resource finally settles down to give
the graph enough time, it can probably converge.
#### Limit
Float. Maximum rate of `CheckApply` runs started per second. Useful to limit
an especially _eventful_ process from causing excessive checks to run. This
defaults to `+Infinity` which adds no limiting. If you change this value, you
will also need to change the `Burst` value to a non-zero value. Please see the
[rate](https://godoc.org/golang.org/x/time/rate) package for more information.
#### Burst
Integer. Burst is the maximum number of runs which can happen without invoking
the rate limiter as designated by the `Limit` value. If the `Limit` is not set
to `+Infinity`, this must be a non-zero value. Please see the
[rate](https://godoc.org/golang.org/x/time/rate) package for more information.
#### Sema
List of string ids. Sema is a P/V style counting semaphore which can be used to
limit parallelism during the CheckApply phase of resource execution. Each
resource can have `N` different semaphores which share a graph global namespace.
Each semaphore has a maximum count associated with it. The default value of the
size is 1 (one) if size is unspecified. Each string id is the unique id of the
semaphore. If the id contains a trailing colon (:) followed by a positive
integer, then that value is the max size for that semaphore. Valid semaphore
id's include: `some_id`, `hello:42`, `not:smart:4` and `:13`. It is expected
that the last bare example be only used by the engine to add a global semaphore.
### Graph definition file
graph.yaml is the compiled graph definition file. The format is currently
undocumented, but by looking through the [examples/](https://github.com/purpleidea/mgmt/tree/master/examples)
you can probably figure out most of it, as it's fairly intuitive.
### Command line
The main interface to the `mgmt` tool is the command line. For the most recent
documentation, please run `mgmt --help`.
#### `--yaml <graph.yaml>`
Point to a graph file to run.
#### `--converged-timeout <seconds>`
Exit if the machine has converged for approximately this many seconds.
#### `--max-runtime <seconds>`
Exit when the agent has run for approximately this many seconds. This is not
generally recommended, but may be useful for users who know what they're doing.
#### `--noop`
Globally force all resources into no-op mode. This also disables the export to
etcd functionality, but does not disable resource collection, however all
resources that are collected will have their individual noop settings set.
#### `--sema <size>`
Globally add a counting semaphore of this size to each resource in the graph.
The semaphore will get given an id of `:size`. In other words if you specify a
size of 42, you can expect a semaphore if named: `:42`. It is expected that
consumers of the semaphore metaparameter always include a prefix to avoid a
collision with this globally defined semaphore. The size value must be greater
than zero at this time. The traditional non-parallel execution found in config
management tools such as `Puppet` can be obtained with `--sema 1`.
#### `--remote <graph.yaml>`
Point to a graph file to run on the remote host specified within. This parameter
can be used multiple times if you'd like to remotely run on multiple hosts in
parallel.
#### `--allow-interactive`
Allow interactive prompting for SSH passwords if there is no authentication
method that works.
#### `--ssh-priv-id-rsa`
Specify the path for finding SSH keys. This defaults to `~/.ssh/id_rsa`. To
never use this method of authentication, set this to the empty string.
#### `--cconns`
The maximum number of concurrent remote ssh connections to run. This defaults
to `0`, which means unlimited.
#### `--no-caching`
Don't allow remote caching of the remote execution binary. This will require
the binary to be copied over for every remote execution, but it limits the
likelihood that there is leftover information from the configuration process.
#### `--prefix <path>`
Specify a path to a custom working directory prefix. This directory will get
created if it does not exist. This usually defaults to `/var/lib/mgmt/`. This
can't be combined with the `--tmp-prefix` option. It can be combined with the
`--allow-tmp-prefix` option.
#### `--tmp-prefix`
If this option is specified, a temporary prefix will be used instead of the
default prefix. This can't be combined with the `--prefix` option.
#### `--allow-tmp-prefix`
If this option is specified, we will attempt to fall back to a temporary prefix
if the primary prefix couldn't be created. This is useful for avoiding failures
in environments where the primary prefix may or may not be available, but you'd
like to try. The canonical example is when running `mgmt` with `--remote` there
might be a cached copy of the binary in the primary prefix, but in case there's
no binary available continue working in a temporary directory to avoid failure.
### Compilation options
You can control some compilation variables by using environment variables.
#### Disable libvirt support
If you wish to compile mgmt without libvirt, you can use the following command:
```
GOTAGS=novirt make build
```
#### Disable augeas support
If you wish to compile mgmt without augeas support, you can use the following command:
```
GOTAGS=noaugeas make build
```
#### Combining compile-time flags
You can combine multiple tags by using a space-separated list:
```
GOTAGS="noaugeas novirt" make build
```
## Examples
For example configurations, please consult the [examples/](https://github.com/purpleidea/mgmt/tree/master/examples) directory in the git
source repository. It is available from:
[https://github.com/purpleidea/mgmt/tree/master/examples](https://github.com/purpleidea/mgmt/tree/master/examples)
### Systemd:
See [`misc/mgmt.service`](misc/mgmt.service) for a sample systemd unit file.
This unit file is part of the RPM.
To specify your custom options for `mgmt` on a systemd distro:
```bash
sudo mkdir -p /etc/systemd/system/mgmt.service.d/
cat > /etc/systemd/system/mgmt.service.d/env.conf <<EOF
# Environment variables:
MGMT_SEEDS=http://127.0.0.1:2379
MGMT_CONVERGED_TIMEOUT=-1
MGMT_MAX_RUNTIME=0
# Other CLI options if necessary.
#OPTS="--max-runtime=0"
EOF
sudo systemctl daemon-reload
```
## Development
This is a project that I started in my free time in 2013. Development is driven
by all of our collective patches! Dive right in, and start hacking!
Please contact me if you'd like to invite me to speak about this at your event.
You can follow along [on my technical blog](https://ttboj.wordpress.com/).
To report any bugs, please file a ticket at: [https://github.com/purpleidea/mgmt/issues](https://github.com/purpleidea/mgmt/issues).
## Authors
Copyright (C) 2013-2017+ James Shubin and the project contributors
Please see the
[AUTHORS](https://github.com/purpleidea/mgmt/tree/master/AUTHORS) file
for more information.
* [github](https://github.com/purpleidea/)
* [&#64;purpleidea](https://twitter.com/#!/purpleidea)
* [https://ttboj.wordpress.com/](https://ttboj.wordpress.com/)

17
docs/index.rst Normal file
View File

@@ -0,0 +1,17 @@
.. mgmt documentation master file, created by
sphinx-quickstart on Wed Feb 15 21:34:09 2017.
You can adapt this file completely to your liking, but it should at least
contain the root `toctree` directive.
Welcome to mgmt's documentation!
================================
.. toctree::
:maxdepth: 2
:caption: Contents:
documentation
quick-start-guide
resource-guide
prometheus
puppet-guide

66
docs/prometheus.md Normal file
View File

@@ -0,0 +1,66 @@
# Prometheus support
Mgmt comes with a built-in prometheus support. It is disabled by default, and
can be enabled with the `--prometheus` command line switch.
By default, the prometheus instance will listen on [`127.0.0.1:9233`][pd]. You
can change this setting by using the `--prometheus-listen` cli option:
To have mgmt prometheus bind interface on 0.0.0.0:45001, use:
`./mgmt r --prometheus --prometheus-listen :45001`
## Metrics
Mgmt exposes three kinds of resources: _go_ metrics, _etcd_ metrics and _mgmt_
metrics.
### go metrics
We use the [prometheus go_collector][pgc] to expose go metrics. Those metrics
are mainly useful for debugging and perf testing.
### etcd metrics
mgmt exposes etcd metrics. Read more in the [upstream documentation][etcdm]
### mgmt metrics
Here is a list of the metrics we provide:
- `mgmt_resources_total`: The number of resources that mgmt is managing
- `mgmt_checkapply_total`: The number of CheckApply's that mgmt has run
- `mgmt_failures_total`: The number of resources that have failed
- `mgmt_failures_current`: The number of resources that have failed
- `mgmt_graph_start_time_seconds`: Start time of the current graph since unix epoch in seconds
For each metric, you will get some extra labels:
- `kind`: The kind of mgmt resource
For `mgmt_checkapply_total`, those extra labels are set:
- `eventful`: "true" or "false", if the CheckApply triggered some changes
- `errorful`: "true" or "false", if the CheckApply reported an error
- `apply`: "true" or "false", if the CheckApply ran in apply or noop mode
## Alerting
You can use prometheus to alert you upon changes or failures. We do not provide
such templates yet, but we plan to provide some examples in this repository.
Patches welcome!
## Grafana
We do not have grafana dashboards yet. Patches welcome!
## External resources
- [prometheus website](https://prometheus.io/)
- [prometheus documentation](https://prometheus.io/docs/introduction/overview/)
- [prometheus best practices regarding metrics
naming](https://prometheus.io/docs/practices/naming/)
- [grafana website](http://grafana.org/)
[pgc]: https://github.com/prometheus/client_golang/blob/master/prometheus/go_collector.go
[etcdm]: https://coreos.com/etcd/docs/latest/metrics.html
[pd]: https://github.com/prometheus/prometheus/wiki/Default-port-allocation

166
docs/puppet-guide.md Normal file
View File

@@ -0,0 +1,166 @@
# Puppet guide
`mgmt` can use Puppet as its source for the configuration graph.
This document goes into detail on how this works, and lists
some pitfalls and limitations.
For basic instructions on how to use the Puppet support, see
the [main documentation](documentation.md#puppet-support).
## Prerequisites
You need Puppet installed in your system. It is not important how you
get it. On the most common Linux distributions, you can use packages
from the OS maintainer, or upstream Puppet repositories. An alternative
that will also work on OSX is the `puppet` Ruby gem. It also has the
advantage that you can install any desired version in your home directory
or any other location.
Any release of Puppet's 3.x and 4.x series should be suitable for use with
`mgmt`. Most importantly, make sure to install the `ffrank-mgmtgraph` Puppet
module (referred to below as "the translator module").
```
puppet module install ffrank-mgmtgraph
```
Please note that the module is not required on your Puppet master (if you
use a master/agent setup). It's needed on the machine that runs `mgmt`.
You can install the module on the master anyway, so that it gets distributed
to your agents through Puppet's `pluginsync` mechanism.
### Testing the Puppet side
The following command should run successfully and print a YAML hash on your
terminal:
```puppet
puppet mgmtgraph print --code 'file { "/tmp/mgmt-test": ensure => present }'
```
You can use this CLI to test any manifests before handing them straight
to `mgmt`.
## Writing a suitable manifest
### Unsupported attributes
`mgmt` inherited its resource module from Puppet, so by and large, it's quite
possible to express `mgmt` graphs in terms of Puppet manifests. However,
there isn't (and likely never will be) full feature parity between the
respective resource types. In consequence, a manifest can have semantics that
cannot be transferred to `mgmt`.
For example, at the time of writing this, the `file` type in `mgmt` had no
notion of permissions (the file `mode`) yet. This lead to the following
warning (among others that will be discussed below):
```
$ puppet mgmtgraph print --code 'file { "/tmp/foo": mode => "0600" }'
Warning: cannot translate: File[/tmp/foo] { mode => "600" } (attribute is ignored)
```
This is a heads-up for the user, because the resulting `mgmt` graph will
in fact not pass this information to the `/tmp/foo` file resource, and
`mgmt` will ignore this file's permissions. Including such attributes in
manifests that are written expressly for `mgmt` is not sensible and should
be avoided.
### Unsupported resources
Puppet has a fairly large number of
[built-in types](https://docs.puppet.com/puppet/latest/reference/type.html),
and countless more are available through
[modules](https://forge.puppet.com/). It's unlikely that all of them will
eventually receive native counterparts in `mgmt`.
When encountering an unknown resource, the translator module will replace
it with an `exec` resource in its output. This resource will run the equivalent
of a `puppet resource` command to make Puppet apply the original resource
itself. This has quite abysmal performance, because processing such a
resource requires the forking of at least one Puppet process (two if it
is found to be out of sync). This comes with considerable overhead.
On most systems, starting up any Puppet command takes several seconds.
Compared to the split second that the actual work usually takes,
this overhead can amount to several orders of magnitude.
Avoid Puppet types that `mgmt` does not implement (yet).
### Avoiding common warnings
Many resource parameters in Puppet take default values. For the most part,
the translator module just ignores them. However, there are cases in which
Puppet will default to convenient behavior that `mgmt` cannot quite replicate.
For example, translating a plain `file` resource will lead to a warning message:
```
$ puppet mgmtgraph print --code 'file { "/tmp/mgmt-test": }'
Warning: File[/tmp/mgmt-test] uses the 'puppet' file bucket, which mgmt cannot do. There will be no backup copies!
```
The reason is that per default, Puppet assumes the following parameter value
(among others)
```puppet
file { "/tmp/mgmt-test":
backup => 'puppet',
}
```
To avoid this, specify the parameter explicitly:
```
$ puppet mgmtgraph print --code 'file { "/tmp/mgmt-test": backup => false }'
```
This is tedious in a more complex manifest. A good simplification is the
following [resource default](https://docs.puppet.com/puppet/latest/reference/lang_defaults.html)
anywhere on the top scope of your manifest:
```puppet
File { backup => false }
```
If you encounter similar warnings from other types and/or parameters,
use the same approach to silence them if possible.
## Configuring Puppet
Since `mgmt` uses an actual Puppet CLI behind the scenes, you might
need to tweak some of Puppet's runtime options in order to make it
do what you want. Reasons for this could be among the following:
* You use the `--puppet agent` variant and need to configure
`servername`, `certname` and other master/agent-related options.
* You don't want runtime information to end up in the `vardir`
that is used by your regular `puppet agent`.
* You install specific Puppet modules for `mgmt` in a non-standard
location.
`mgmt` exposes only one Puppet option in order to allow you to
control all of them, through its `--puppet-conf` option. It allows
you to specify which `puppet.conf` file should be used during
translation.
```
mgmt run --puppet /opt/my-manifest.pp --puppet-conf /etc/mgmt/puppet.conf
```
Within this file, you can just specify any needed options in the
`[main]` section:
```
[main]
server=mgmt-master.example.net
vardir=/var/lib/mgmt/puppet
```
## Caveats
Please see the [README](https://github.com/ffrank/puppet-mgmtgraph/blob/master/README.md)
of the translator module for the current state of supported and unsupported
language features.
You should probably make sure to always use the latest release of
both `ffrank-mgmtgraph` and `ffrank-yamlresource` (the latter is
getting pulled in as a dependency of the former).

93
docs/quick-start-guide.md Normal file
View File

@@ -0,0 +1,93 @@
# Quick start guide
## Introduction
This guide is intended for developers. Once `mgmt` is minimally viable, we'll
publish a quick start guide for users too. In the meantime, please contribute!
If you're brand new to `mgmt`, it's probably a good idea to start by reading the
[introductory article](https://ttboj.wordpress.com/2016/01/18/next-generation-configuration-mgmt/)
or to watch an [introductory video](https://github.com/purpleidea/mgmt/#on-the-web).
Once you're familiar with the general idea, please start hacking...
## Vagrant
If you would like to avoid doing the following steps manually, we have prepared
a [Vagrant](https://www.vagrantup.com/) environment for your convenience. From
the project directory, run a `vagrant up`, and then a `vagrant status`. From
there, you can `vagrant ssh` into the `mgmt` machine. The MOTD will explain the
rest.
## Dependencies
Software projects have a few different kinds of dependencies. There are _build_
dependencies, _runtime_ dependencies, and additionally, a few extra dependencies
required for running the _test_ suite.
### Build
* `golang` 1.6 or higher (required, available in most distros)
* golang libraries (required, available with `go get ./...`) a partial list includes:
```
github.com/coreos/etcd/client
gopkg.in/yaml.v2
gopkg.in/fsnotify.v1
github.com/urfave/cli
github.com/coreos/go-systemd/dbus
github.com/coreos/go-systemd/util
github.com/libvirt/libvirt-go
```
* `stringer` (optional), available as a package on some platforms, otherwise via `go get`
```
golang.org/x/tools/cmd/stringer
```
* `pandoc` (optional), for building a pdf of the documentation
### Runtime
A relatively modern GNU/Linux system should be able to run `mgmt` without any
problems. Since `mgmt` runs as a single statically compiled binary, all of the
library dependencies are included. It is expected, that certain advanced
resources require host specific facilities to work. These requirements are
listed below:
| Resource | Dependency | Version |
|----------|-------------------|---------|
| file | inotify | ? |
| hostname | systemd-hostnamed | ? |
| nspawn | systemd-nspawn | ? |
| pkg | packagekitd | ? |
| svc | systemd | ? |
| virt | libvirtd | ? |
For building a visual representation of the graph, `graphviz` is required.
### Testing
* golint `github.com/golang/lint/golint`
## Quick start
* Make sure you have golang version 1.6 or greater installed.
* If you do not have a GOPATH yet, create one and export it:
```
mkdir $HOME/gopath
export GOPATH=$HOME/gopath
```
* You might also want to add the GOPATH to your `~/.bashrc` or `~/.profile`.
* For more information you can read the [GOPATH documentation](https://golang.org/cmd/go/#hdr-GOPATH_environment_variable).
* Next download the mgmt code base, and switch to that directory:
```
mkdir -p $GOPATH/src/github.com/purpleidea/
cd $GOPATH/src/github.com/purpleidea/
git clone --recursive https://github.com/purpleidea/mgmt/
cd $GOPATH/src/github.com/purpleidea/mgmt
```
* Run `make deps` to install system and golang dependencies. Take a look at `misc/make-deps.sh` for details.
* Run `make build` to get a freshly built `mgmt` binary.
* Run `time ./mgmt run --yaml examples/graph0.yaml --converged-timeout=5 --tmp-prefix` to try out a very simple example!
* To run continuously in the default mode of operation, omit the `--converged-timeout` option.
* Have fun hacking on our future technology!
## Examples
Please look in the [examples/](../examples/) folder for some examples!
## Installation
Installation of `mgmt` from distribution packages currently needs improvement.
At the moment we have:
* [COPR](https://copr.fedoraproject.org/coprs/purpleidea/mgmt/)
* [Arch](https://aur.archlinux.org/packages/mgmt/)
Please contribute more! We'd especially like to see a Debian package!

575
docs/resource-guide.md Normal file
View File

@@ -0,0 +1,575 @@
# Resource guide
## Overview
The `mgmt` tool has built-in resource primitives which make up the building
blocks of any configuration. Each instance of a resource is mapped to a single
vertex in the resource [graph](https://en.wikipedia.org/wiki/Directed_acyclic_graph).
This guide is meant to instruct developers on how to write a brand new resource.
Since `mgmt` and the core resources are written in golang, some prior golang
knowledge is assumed.
## Theory
Resources in `mgmt` are similar to resources in other systems in that they are
[idempotent](https://en.wikipedia.org/wiki/Idempotence). Our resources are
uniquely different in that they can detect when their state has changed, and as
a result can run to revert or repair this change instantly. For some background
on this design, please read the
[original article](https://ttboj.wordpress.com/2016/01/18/next-generation-configuration-mgmt/)
on the subject.
## Resource API
To implement a resource in `mgmt` it must satisfy the
[`Res`](https://github.com/purpleidea/mgmt/blob/master/resources/resources.go)
interface. What follows are each of the method signatures and a description of
each.
### Default
```golang
Default() Res
```
This returns a populated resource struct as a `Res`. It shouldn't populate any
values which already have the correct default as the golang zero value. In
general it is preferable if the zero values make for the correct defaults.
#### Example
```golang
// Default returns some sensible defaults for this resource.
func (obj *FooRes) Default() Res {
return &FooRes{
Answer: 42, // sometimes, defaults shouldn't be the zero value
}
}
```
### Validate
```golang
Validate() error
```
This method is used to validate if the populated resource struct is a valid
representation of the resource kind. If it does not conform to the resource
specifications, it should generate an error. If you notice that this method is
quite large, it might be an indication that you should reconsider the parameter
list and interface to this resource. This method is called _before_ `Init`.
#### Example
```golang
// Validate reports any problems with the struct definition.
func (obj *FooRes) Validate() error {
if obj.Answer != 42 { // validate whatever you want
return fmt.Errorf("expected an answer of 42")
}
return obj.BaseRes.Validate() // remember to call the base method!
}
```
### Init
```golang
Init() error
```
This is called to initialize the resource. If something goes wrong, it should
return an error. It should set the resource `kind`, do any resource specific
work, and finish by calling the `Init` method of the base resource.
#### Example
```golang
// Init initializes the Foo resource.
func (obj *FooRes) Init() error {
obj.BaseRes.kind = "foo" // must lower case resource kind
// run the resource specific initialization, and error if anything fails
if some_error {
return err // something went wrong!
}
return obj.BaseRes.Init() // call the base resource init
}
```
This method is always called after `Validate` has run successfully, with the
exception that we can't prevent a malicious or buggy `libmgmt` user to not run
this. In other words, you should expect `Validate` to have run first, but you
shouldn't allow `Init` to dangerously `rm -rf /$the_world` if your code only
checks `$the_world` in `Validate`. Remember to always program safely!
### Close
```golang
Close() error
```
This is called to cleanup after the resource. It is usually not necessary, but
can be useful if you'd like to properly close a persistent connection that you
opened in the `Init` method and were using throughout the resource.
#### Example
```golang
// Close runs some cleanup code for this resource.
func (obj *FooRes) Close() error {
err := obj.conn.Close() // close some internal connection
// call base close, b/c we're overriding
if e := obj.BaseRes.Close(); err == nil {
err = e
} else if e != nil {
err = multierr.Append(err, e) // list of errors
}
return err
}
```
You should probably check the return errors of your internal methods, and pass
on an error if something went wrong. Remember to always call the base `Close`
method! If you plan to return early if you hit an internal error, then at least
call it with a defer!
### CheckApply
```golang
CheckApply(apply bool) (checkOK bool, err error)
```
`CheckApply` is where the real _work_ is done. Under normal circumstances, this
function should check if the state of this resource is correct, and if so, it
should return: `(true, nil)`. If the `apply` variable is set to `true`, then
this means that we should then proceed to run the changes required to bring the
resource into the correct state. If the `apply` variable is set to `false`, then
the resource is operating in _noop_ mode and _no operations_ should be executed!
After having executed the necessary operations to bring the resource back into
the desired state, or after having detected that the state was incorrect, but
that changes can't be made because `apply` is `false`, you should then return
`(false, nil)`.
You must cause the resource to converge during a single execution of this
function. If you cannot, then you must return an error! The exception to this
rule is that if an external force changes the state of the resource while it is
being remedied, it is possible to return from this function even though the
resource isn't now converged. This is not a bug, as the resources `Watch`
facility will detect the change, ultimately resulting in a subsequent call to
`CheckApply`.
#### Example
```golang
// CheckApply does the idempotent work of checking and applying resource state.
func (obj *FooRes) CheckApply(apply bool) (bool, error) {
// check the state
if state_is_okay { return true, nil } // done early! :)
// state was bad
if !apply { return false, nil } // don't apply; !stateok, nil
// do the apply!
return false, nil // after success applying
if any_error { return false, err } // anytime there's an err!
}
```
The `CheckApply` function is called by the `mgmt` engine when it believes a call
is necessary. Under certain conditions when a `Watch` call does not invalidate
the state of the resource, and no refresh call was sent, its execution might be
skipped. This is an engine optimization, and not a bug. It is mentioned here in
the documentation in case you are confused as to why a debug message you've
added to the code isn't always printed.
#### Refresh notifications
Some resources may choose to support receiving refresh notifications. In general
these should be avoided if possible, but nevertheless, they do make sense in
certain situations. Resources that support these need to verify if one was sent
during the CheckApply phase of execution. This is accomplished by calling the
`Refresh() bool` method of the resource, and inspecting the return value. This
is only necessary if you plan to perform a refresh action. Refresh actions
should still respect the `apply` variable, and no system changes should be made
if it is `false`. Refresh notifications are generated by any resource when an
action is applied by that resource and are transmitted through graph edges which
have enabled their propagation. Resources that currently perform some refresh
action include `svc`, `timer`, and `password`.
#### Paired execution
For many resources it is not uncommon to see `CheckApply` run twice in rapid
succession. This is usually not a pathological occurrence, but rather a healthy
pattern which is a consequence of the event system. When the state of the
resource is incorrect, `CheckApply` will run to remedy the state. In response to
having just changed the state, it is usually the case that this repair will
trigger the `Watch` code! In response, a second `CheckApply` is triggered, which
will likely find the state to now be correct.
#### Summary
* Anytime an error occurs during `CheckApply`, you should return `(false, err)`.
* If the state is correct and no changes are needed, return `(true, nil)`.
* You should only make changes to the system if `apply` is set to `true`.
* After checking the state and possibly applying the fix, return `(false, nil)`.
* Returning `(true, err)` is a programming error and will cause a `Fatal`.
### Watch
```golang
Watch(chan *Event) error
```
`Watch` is a main loop that runs and sends messages when it detects that the
state of the resource might have changed. To send a message you should write to
the input event channel using the `Event` helper method. The Watch function
should run continuously until a shutdown message is received. If at any time
something goes wrong, you should return an error, and the `mgmt` engine will
handle possibly restarting the main loop based on the `retry` meta parameters.
It is better to send an event notification which turns out to be spurious, than
to miss a possible event. Resources which can miss events are incorrect and need
to be re-engineered so that this isn't the case. If you have an idea for a
resource which would fit this criteria, but you can't find a solution, please
contact the `mgmt` maintainers so that this problem can be investigated and a
possible system level engineering fix can be found.
You may have trouble deciding how much resource state checking should happen in
the `Watch` loop versus deferring it all to the `CheckApply` method. You may
want to put some simple fast path checking in `Watch` to avoid generating
obviously spurious events, but in general it's best to keep the `Watch` method
as simple as possible. Contact the `mgmt` maintainers if you're not sure.
If the resource is activated in `polling` mode, the `Watch` method will not get
executed. As a result, the resource must still work even if the main loop is not
running.
#### Select
The lifetime of most resources `Watch` method should be spent in an infinite
loop that is bounded by a `select` call. The `select` call is the point where
our method hands back control to the engine (and the kernel) so that we can
sleep until something of interest wakes us up. In this loop we must process
events from the engine via the `<-obj.Events()` call, and receive events for our
resource itself!
#### Events
If we receive an internal event from the `<-obj.Events()` method, we can read it
with the ReadEvent helper function. This function tells us if we should shutdown
our resource, and if we should generate an event. When we want to send an event,
we use the `Event` helper function. It is also important to mark the resource
state as `dirty` if we believe it might have changed. We do this with the
`StateOK(false)` function.
#### Startup
Once the `Watch` function has finished starting up successfully, it is important
to generate one event to notify the `mgmt` engine that we're now listening
successfully, so that it can run an initial `CheckApply` to ensure we're safely
tracking a healthy state and that we didn't miss anything when `Watch` was down
or from before `mgmt` was running. It does this by calling the `Running` method.
#### Converged
The engine might be asked to shutdown when the entire state of the system has
not seen any changes for some duration of time. The engine can determine this
automatically, but each resource can block this if it is absolutely necessary.
To do this, the `Watch` method should get the `ConvergedUID` handle that has
been prepared for it by the engine. This is done by calling the `ConvergerUID`
method on the resource object. The result can be used to set the converged
status with `SetConverged`, and to notify when the particular timeout has been
reached by waiting on `ConvergedTimer`.
Instead of interacting with the `ConvergedUID` with these two methods, we can
instead use the `StartTimer` and `ResetTimer` methods which accomplish the same
thing, but provide a `select`-free interface for different coding situations.
This particular facility is most likely not required for most resources. It may
prove to be useful if a resource wants to start off a long operation, but avoid
sending out erroneous `Event` messages to keep things alive until it finishes.
#### Example
```golang
// Watch is the listener and main loop for this resource.
func (obj *FooRes) Watch() error {
// setup the Foo resource
var err error
if err, obj.foo = OpenFoo(); err != nil {
return err // we couldn't startup
}
defer obj.whatever.CloseFoo() // shutdown our
// notify engine that we're running
if err := obj.Running(); err != nil {
return err // bubble up a NACK...
}
var send = false // send event?
var exit *error
for {
select {
case event := <-obj.Events():
// we avoid sending events on unpause
if exit, send = obj.ReadEvent(event); exit != nil {
return *exit // exit
}
// the actual events!
case event := <-obj.foo.Events:
if is_an_event {
send = true // used below
obj.StateOK(false) // dirty
}
// event errors
case err := <-obj.foo.Errors:
return err // will cause a retry or permanent failure
}
// do all our event sending all together to avoid duplicate msgs
if send {
send = false
obj.Event() // send the event!
}
}
}
```
#### Summary
* Remember to call the appropriate `converger` methods throughout the resource.
* Remember to call `Startup` when the `Watch` is running successfully.
* Remember to process internal events and shutdown promptly if asked to.
* Ensure the design of your resource is well thought out.
* Have a look at the existing resources for a rough idea of how this all works.
### Compare
```golang
Compare(Res) bool
```
Each resource must have a `Compare` method. This takes as input another resource
and must return whether they are identical or not. This is used for identifying
if an existing resource can be used in place of a new one with a similar set of
parameters. In particular, when switching from one graph to a new (possibly
identical) graph, this avoids recomputing the state for resources which don't
change or that are sufficiently similar that they don't need to be swapped out.
In general if all the resource properties are identical, then they usually don't
need to be changed. On occasion, not all of them need to be compared, in
particular if they store some generated state, or if they aren't significant in
some way.
#### Example
```golang
// Compare two resources and return if they are equivalent.
func (obj *FooRes) Compare(res Res) bool {
switch res.(type) {
case *FooRes: // only compare to other resources of the Foo kind!
res := res.(*FileRes)
if !obj.BaseRes.Compare(res) { // call base Compare
return false
}
if obj.Name != res.Name {
return false
}
if obj.whatever != res.whatever {
return false
}
if obj.Flag != res.Flag {
return false
}
default:
return false // different kind of resource
}
return true // they must match!
}
```
### UIDs
```golang
UIDs() []ResUID
```
The `UIDs` method returns a list of `ResUID` interfaces that represent the
particular resource uniquely. This is used with the AutoEdges API to determine
if another resource can match a dependency to this one.
### AutoEdges
```golang
AutoEdges() AutoEdge
```
This returns a struct that implements the `AutoEdge` interface. This struct
is used to match other resources that might be relevant dependencies for this
resource.
### CollectPattern
```golang
CollectPattern() string
```
This is currently a stub and will be updated once the DSL is further along.
### UnmarshalYAML
```golang
UnmarshalYAML(unmarshal func(interface{}) error) error // optional
```
This is optional, but recommended for any resource that will have a YAML
accessible struct, and an entry in the `GraphConfig` struct. It is not required
because to do so would mean that third-party or custom resources (such as those
someone writes to use with `libmgmt`) would have to implement this needlessly.
The signature intentionally matches what is required to satisfy the `go-yaml`
[Unmarshaler](https://godoc.org/gopkg.in/yaml.v2#Unmarshaler) interface.
#### Example
```golang
// UnmarshalYAML is the custom unmarshal handler for this struct.
// It is primarily useful for setting the defaults.
func (obj *FooRes) UnmarshalYAML(unmarshal func(interface{}) error) error {
type rawRes FooRes // indirection to avoid infinite recursion
def := obj.Default() // get the default
res, ok := def.(*FooRes) // put in the right format
if !ok {
return fmt.Errorf("could not convert to FooRes")
}
raw := rawRes(*res) // convert; the defaults go here
if err := unmarshal(&raw); err != nil {
return err
}
*obj = FooRes(raw) // restore from indirection with type conversion!
return nil
}
```
## Further considerations
There is some additional information that any resource writer will need to know.
Each issue is listed separately below!
### Resource struct
Each resource will implement methods as pointer receivers on a resource struct.
The resource struct must include an anonymous reference to the `BaseRes` struct.
The naming convention for resources is that they end with a `Res` suffix. If
you'd like your resource to be accessible by the `YAML` graph API (GAPI), then
you'll need to include the appropriate YAML fields as shown below.
#### Example
```golang
type FooRes struct {
BaseRes `yaml:",inline"` // base properties
Whatever string `yaml:"whatever"` // you pick!
Bar int // no yaml, used as public output value for send/recv
Baz bool `yaml:"baz"` // something else
something string // some private field
}
```
### YAML
In addition to labelling your resource struct with YAML fields, you must also
add an entry to the internal `GraphConfig` struct. It is a fairly straight
forward one line patch.
```golang
type GraphConfig struct {
// [snip...]
Resources struct {
Noop []*resources.NoopRes `yaml:"noop"`
File []*resources.FileRes `yaml:"file"`
// [snip...]
Foo []*resources.FooRes `yaml:"foo"` // tada :)
}
}
```
It's also recommended that you add the [UnmarshalYAML](#unmarshalyaml) method to
your resources so that unspecified values are given sane defaults.
### Gob registration
All resources must be registered with the `golang` _gob_ module so that they can
be encoded and decoded. Make sure to include the following code snippet for this
to work.
```golang
import "encoding/gob"
func init() { // special golang method that runs once
gob.Register(&FooRes{}) // substitude your resource here
}
```
## Automatic edges
Automatic edges in `mgmt` are well described in [this article](https://ttboj.wordpress.com/2016/03/14/automatic-edges-in-mgmt/).
The best example of this technique can be seen in the `svc` resource.
Unfortunately no further documentation about this subject has been written. To
expand this section, please send a patch! Please contact us if you'd like to
work on a resource that uses this feature, or to add it to an existing one!
## Automatic grouping
Automatic grouping in `mgmt` is well described in [this article](https://ttboj.wordpress.com/2016/03/30/automatic-grouping-in-mgmt/).
The best example of this technique can be seen in the `pkg` resource.
Unfortunately no further documentation about this subject has been written. To
expand this section, please send a patch! Please contact us if you'd like to
work on a resource that uses this feature, or to add it to an existing one!
## Send/Recv
In `mgmt` there is a novel concept called _Send/Recv_. For some background,
please [read the introductory article](https://ttboj.wordpress.com/2016/12/07/sendrecv-in-mgmt/).
When using this feature, the engine will automatically send the user specified
value to the intended destination without requiring any resource specific code.
Any time that one of the destination values is changed, the engine automatically
marks the resource state as `dirty`. To detect if a particular value was
received, and if it changed (during this invocation of CheckApply) from the
previous value, you can query the Recv parameter. It will contain a `map` of all
the keys which can be received on, and the value has a `Changed` property which
will indicate whether the value was updated on this particular `CheckApply`
invocation. The type of the sending key must match that of the receiving one.
This can _only_ be done inside of the `CheckApply` function!
```golang
// inside CheckApply, probably near the top
if val, exists := obj.Recv["SomeKey"]; exists {
log.Printf("SomeKey was sent to us from: %s[%s].%s", val.Res.Kind(), val.Res.GetName(), val.Key)
if val.Changed {
log.Printf("SomeKey was just updated!")
// you may want to invalidate some local cache
}
}
```
Astute readers will note that there isn't anything that prevents a user from
sending an identically typed value to some arbitrary (public) key that the
resource author hadn't considered! While this is true, resources should probably
work within this problem space anyways. The rule of thumb is that any public
parameter which is normally used in a resource can be used safely.
One subtle scenario is that if a resource creates a local cache or stores a
computation that depends on the value of a public parameter and will require
invalidation should that public parameter change, then you must detect that
scenario and invalidate the cache when it occurs. This *must* be processed
before there is a possibility of failure in CheckApply, because if we fail (and
possibly run again) the subsequent send->recv transfer might not have a new
value to copy, and therefore we won't see this notification of change.
Therefore, it is important to process these promptly, if they must not be lost,
such as for cache invalidation.
Remember, `Send/Recv` only changes your resource code if you cache state.
## Composite resources
Composite resources are resources which embed one or more existing resources.
This is useful to prevent code duplication in higher level resource scenarios.
The best example of this technique can be seen in the `nspawn` resource which
can be seen to partially embed a `svc` resource, but without its `Watch`.
Unfortunately no further documentation about this subject has been written. To
expand this section, please send a patch! Please contact us if you'd like to
work on a resource that uses this feature, or to add it to an existing one!
## Frequently asked questions
(Send your questions as a patch to this FAQ! I'll review it, merge it, and
respond by commit with the answer.)
### Can I write resources in a different language?
Currently `golang` is the only supported language for built-in resources. We
might consider allowing external resources to be imported in the future. This
will likely require a language that can expose a C-like API, such as `python` or
`ruby`. Custom `golang` resources are already possible when using mgmt as a lib.
Higher level resource collections will be possible once the `mgmt` DSL is ready.
### What new resource primitives need writing?
There are still many ideas for new resources that haven't been written yet. If
you'd like to contribute one, please contact us and tell us about your idea!
### Where can I find more information about mgmt?
Additional blog posts, videos and other material [is available!](https://github.com/purpleidea/mgmt/#on-the-web).
## Suggestions
If you have any ideas for API changes or other improvements to resource writing,
please let us know! We're still pre 1.0 and pre 0.1 and happy to break API in
order to get it right!

300
etcd.go
View File

@@ -1,300 +0,0 @@
// Mgmt
// Copyright (C) 2013-2016+ James Shubin and the project contributors
// Written by James Shubin <james@shubin.ca> and the project contributors
//
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU Affero General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// This program is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU Affero General Public License for more details.
//
// You should have received a copy of the GNU Affero General Public License
// along with this program. If not, see <http://www.gnu.org/licenses/>.
package main
import (
"fmt"
etcd_context "github.com/coreos/etcd/Godeps/_workspace/src/golang.org/x/net/context"
etcd "github.com/coreos/etcd/client"
"log"
"math"
"strings"
"time"
)
//go:generate stringer -type=etcdMsg -output=etcdmsg_stringer.go
type etcdMsg int
const (
etcdStart etcdMsg = iota
etcdEvent
etcdFoo
etcdBar
)
//go:generate stringer -type=etcdConvergedState -output=etcdconvergedstate_stringer.go
type etcdConvergedState int
const (
etcdConvergedNil etcdConvergedState = iota
//etcdConverged
etcdConvergedTimeout
)
type EtcdWObject struct { // etcd wrapper object
seed string
ctimeout int
converged chan bool
kapi etcd.KeysAPI
convergedState etcdConvergedState
}
func (etcdO *EtcdWObject) GetConvergedState() etcdConvergedState {
return etcdO.convergedState
}
func (etcdO *EtcdWObject) SetConvergedState(state etcdConvergedState) {
etcdO.convergedState = state
}
func (etcdO *EtcdWObject) GetKAPI() etcd.KeysAPI {
if etcdO.kapi != nil { // memoize
return etcdO.kapi
}
cfg := etcd.Config{
Endpoints: []string{etcdO.seed},
Transport: etcd.DefaultTransport,
// set timeout per request to fail fast when the target endpoint is unavailable
HeaderTimeoutPerRequest: time.Second,
}
var c etcd.Client
var err error
c, err = etcd.New(cfg)
if err != nil {
// XXX: not sure if this ever errors
if cerr, ok := err.(*etcd.ClusterError); ok {
// XXX: not sure if this part ever matches
// not running or disconnected
if cerr == etcd.ErrClusterUnavailable {
log.Fatal("XXX: etcd: ErrClusterUnavailable")
} else {
log.Fatal("XXX: etcd: Unknown")
}
}
log.Fatal(err) // some unhandled error
}
etcdO.kapi = etcd.NewKeysAPI(c)
return etcdO.kapi
}
type EtcdChannelWatchResponse struct {
resp *etcd.Response
err error
}
// wrap the etcd watcher.Next blocking function inside of a channel
func (etcdO *EtcdWObject) EtcdChannelWatch(watcher etcd.Watcher, context etcd_context.Context) chan *EtcdChannelWatchResponse {
ch := make(chan *EtcdChannelWatchResponse)
go func() {
for {
resp, err := watcher.Next(context) // blocks here
ch <- &EtcdChannelWatchResponse{resp, err}
}
}()
return ch
}
func (etcdO *EtcdWObject) EtcdWatch() chan etcdMsg {
kapi := etcdO.GetKAPI()
ctimeout := etcdO.ctimeout
converged := etcdO.converged
// XXX: i think we need this buffered so that when we're hanging on the
// channel, which is inside the EtcdWatch main loop, we still want the
// calls to Get/Set on etcd to succeed, so blocking them here would
// kill the whole thing
ch := make(chan etcdMsg, 1) // XXX: buffer of at least 1 is required
go func(ch chan etcdMsg) {
tmin := 500 // initial (min) delay in ms
t := tmin // current time
tmult := 2 // multiplier for exponential delay
tmax := 16000 // max delay
watcher := kapi.Watcher("/exported/", &etcd.WatcherOptions{Recursive: true})
etcdch := etcdO.EtcdChannelWatch(watcher, etcd_context.Background())
for {
log.Printf("Etcd: Watching...")
var resp *etcd.Response // = nil by default
var err error
select {
case out := <-etcdch:
etcdO.SetConvergedState(etcdConvergedNil)
resp, err = out.resp, out.err
case _ = <-TimeAfterOrBlock(ctimeout):
etcdO.SetConvergedState(etcdConvergedTimeout)
converged <- true
continue
}
if err != nil {
if err == etcd_context.Canceled {
// ctx is canceled by another routine
log.Fatal("Canceled")
} else if err == etcd_context.DeadlineExceeded {
// ctx is attached with a deadline and it exceeded
log.Fatal("Deadline")
} else if cerr, ok := err.(*etcd.ClusterError); ok {
// not running or disconnected
// TODO: is there a better way to parse errors?
for _, e := range cerr.Errors {
if strings.HasSuffix(e.Error(), "getsockopt: connection refused") {
t = int(math.Min(float64(t*tmult), float64(tmax)))
log.Printf("Etcd: Waiting %d ms for connection...", t)
time.Sleep(time.Duration(t) * time.Millisecond) // sleep for t ms
} else if e.Error() == "unexpected EOF" {
log.Printf("Etcd: Unexpected disconnect...")
} else if e.Error() == "EOF" {
log.Printf("Etcd: Disconnected...")
} else if strings.HasPrefix(e.Error(), "unsupported protocol scheme") {
// usually a bad peer endpoint value
log.Fatal("Bad peer endpoint value?")
} else {
log.Fatal("Woops: ", e.Error())
}
}
} else {
// bad cluster endpoints, which are not etcd servers
log.Fatal("Woops: ", err)
}
} else {
//log.Print(resp)
//log.Printf("Watcher().Node.Value(%v): %+v", key, resp.Node.Value)
// FIXME: we should actually reset when the server comes back, not here on msg!
//XXX: can we fix this with one of these patterns?: https://blog.golang.org/go-concurrency-patterns-timing-out-and
t = tmin // reset timer
// don't trigger event if nothing changed
if n, p := resp.Node, resp.PrevNode; resp.Action == "set" && p != nil {
if n.Key == p.Key && n.Value == p.Value {
continue
}
}
// FIXME: we get events on key/type/value changes for
// each type directory... ignore the non final ones...
// IOW, ignore everything except for the value or some
// field which gets set last... this could be the max count field thing...
log.Printf("Etcd: Value: %v", resp.Node.Value) // event
ch <- etcdEvent // event
}
} // end for loop
//close(ch)
}(ch) // call go routine
return ch
}
// helper function to store our data in etcd
func (etcdO *EtcdWObject) EtcdPut(hostname, key, typ string, obj interface{}) bool {
kapi := etcdO.GetKAPI()
output, ok := ObjToB64(obj)
if !ok {
log.Printf("Etcd: Could not encode %v key.", key)
return false
}
path := fmt.Sprintf("/exported/%s/types/%s/type", hostname, key)
_, err := kapi.Set(etcd_context.Background(), path, typ, nil)
// XXX validate...
path = fmt.Sprintf("/exported/%s/types/%s/value", hostname, key)
resp, err := kapi.Set(etcd_context.Background(), path, output, nil)
if err != nil {
if cerr, ok := err.(*etcd.ClusterError); ok {
// not running or disconnected
for _, e := range cerr.Errors {
if strings.HasSuffix(e.Error(), "getsockopt: connection refused") {
}
//if e == etcd.ErrClusterUnavailable
}
}
log.Printf("Etcd: Could not store %v key.", key)
return false
}
log.Print("Etcd: ", resp) // w00t... bonus
return true
}
// lookup /exported/ node hierarchy
func (etcdO *EtcdWObject) EtcdGet() (etcd.Nodes, bool) {
kapi := etcdO.GetKAPI()
// key structure is /exported/<hostname>/types/...
resp, err := kapi.Get(etcd_context.Background(), "/exported/", &etcd.GetOptions{Recursive: true})
if err != nil {
return nil, false // not found
}
return resp.Node.Nodes, true
}
func (etcdO *EtcdWObject) EtcdGetProcess(nodes etcd.Nodes, typ string) []string {
//path := fmt.Sprintf("/exported/%s/types/", h)
top := "/exported/"
log.Printf("Etcd: Get: %+v", nodes) // Get().Nodes.Nodes
var output []string
for _, x := range nodes { // loop through hosts
if !strings.HasPrefix(x.Key, top) {
log.Fatal("Error!")
}
host := x.Key[len(top):]
//log.Printf("Get().Nodes[%v]: %+v ==> %+v", -1, host, x.Nodes)
//log.Printf("Get().Nodes[%v]: %+v ==> %+v", i, x.Key, x.Nodes)
types, ok := EtcdGetChildNodeByKey(x, "types")
if !ok {
continue
}
for _, y := range types.Nodes { // loop through types
//key := y.Key # UUID?
//log.Printf("Get(%v): TYPE[%v]", host, y.Key)
t, ok := EtcdGetChildNodeByKey(y, "type")
if !ok {
continue
}
if typ != "" && typ != t.Value {
continue
} // filter based on type
v, ok := EtcdGetChildNodeByKey(y, "value") // B64ToObj this
if !ok {
continue
}
log.Printf("Etcd: Hostname: %v; Get: %v", host, t.Value)
output = append(output, v.Value)
}
}
return output
}
// TODO: wrap this somehow so it's a method of *etcd.Node
// helper function that returns the node for a particular key under a node
func EtcdGetChildNodeByKey(node *etcd.Node, key string) (*etcd.Node, bool) {
for _, x := range node.Nodes {
if x.Key == fmt.Sprintf("%s/%s", node.Key, key) {
return x, true
}
}
return nil, false // not found
}

2298
etcd/etcd.go Normal file

File diff suppressed because it is too large Load Diff

115
etcd/str.go Normal file
View File

@@ -0,0 +1,115 @@
// Mgmt
// Copyright (C) 2013-2017+ James Shubin and the project contributors
// Written by James Shubin <james@shubin.ca> and the project contributors
//
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU Affero General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// This program is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU Affero General Public License for more details.
//
// You should have received a copy of the GNU Affero General Public License
// along with this program. If not, see <http://www.gnu.org/licenses/>.
package etcd
import (
"fmt"
"strings"
"github.com/purpleidea/mgmt/util"
etcd "github.com/coreos/etcd/clientv3"
errwrap "github.com/pkg/errors"
)
// WatchStr returns a channel which spits out events on key activity.
// FIXME: It should close the channel when it's done, and spit out errors when
// something goes wrong.
func WatchStr(obj *EmbdEtcd, key string) chan error {
// new key structure is /$NS/strings/$key/$hostname = $data
path := fmt.Sprintf("/%s/strings/%s", NS, key)
ch := make(chan error, 1)
// FIXME: fix our API so that we get a close event on shutdown.
callback := func(re *RE) error {
// TODO: is this even needed? it used to happen on conn errors
//log.Printf("Etcd: Watch: Path: %v", path) // event
if re == nil || re.response.Canceled {
return fmt.Errorf("watch is empty") // will cause a CtxError+retry
}
if len(ch) == 0 { // send event only if one isn't pending
ch <- nil // event
}
return nil
}
_, _ = obj.AddWatcher(path, callback, true, false, etcd.WithPrefix()) // no need to check errors
return ch
}
// GetStr collects all of the strings which match a namespace in etcd.
func GetStr(obj *EmbdEtcd, hostnameFilter []string, key string) (map[string]string, error) {
// old key structure is /$NS/strings/$hostname/$key = $data
// new key structure is /$NS/strings/$key/$hostname = $data
// FIXME: if we have the $key as the last token (old key structure), we
// can allow the key to contain the slash char, otherwise we need to
// verify that one isn't present in the input string.
path := fmt.Sprintf("/%s/strings/%s", NS, key)
keyMap, err := obj.Get(path, etcd.WithPrefix(), etcd.WithSort(etcd.SortByKey, etcd.SortAscend))
if err != nil {
return nil, errwrap.Wrapf(err, "could not get strings in: %s", key)
}
result := make(map[string]string)
for key, val := range keyMap {
if !strings.HasPrefix(key, path) { // sanity check
continue
}
str := strings.Split(key[len(path):], "/")
if len(str) != 2 {
return nil, fmt.Errorf("unexpected chunk count of %d", len(str))
}
_, hostname := str[0], str[1]
if hostname == "" {
return nil, fmt.Errorf("unexpected chunk length of %d", len(hostname))
}
// FIXME: ideally this would be a server side filter instead!
if len(hostnameFilter) > 0 && !util.StrInList(hostname, hostnameFilter) {
continue
}
//log.Printf("Etcd: GetStr(%s): (Hostname, Data): (%s, %s)", key, hostname, val)
result[hostname] = val
}
return result, nil
}
// SetStr sets a key and hostname pair to a certain value. If the value is nil,
// then it deletes the key. Otherwise the value should point to a string.
// TODO: TTL or delete disconnect?
func SetStr(obj *EmbdEtcd, hostname, key string, data *string) error {
// key structure is /$NS/strings/$key/$hostname = $data
path := fmt.Sprintf("/%s/strings/%s/%s", NS, key, hostname)
ifs := []etcd.Cmp{} // list matching the desired state
ops := []etcd.Op{} // list of ops in this transaction (then)
els := []etcd.Op{} // list of ops in this transaction (else)
if data == nil { // perform a delete
// TODO: use https://github.com/coreos/etcd/pull/7417 if merged
//ifs = append(ifs, etcd.KeyExists(path))
ifs = append(ifs, etcd.Compare(etcd.Version(path), ">", 0))
ops = append(ops, etcd.OpDelete(path))
} else {
data := *data // get the real value
ifs = append(ifs, etcd.Compare(etcd.Value(path), "=", data)) // desired state
els = append(els, etcd.OpPut(path, data))
}
// it's important to do this in one transaction, and atomically, because
// this way, we only generate one watch event, and only when it's needed
_, err := obj.Txn(ifs, ops, els) // TODO: do we need to look at response?
return errwrap.Wrapf(err, "could not set strings in: %s", key)
}

64
etcd/world.go Normal file
View File

@@ -0,0 +1,64 @@
// Mgmt
// Copyright (C) 2013-2017+ James Shubin and the project contributors
// Written by James Shubin <james@shubin.ca> and the project contributors
//
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU Affero General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// This program is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU Affero General Public License for more details.
//
// You should have received a copy of the GNU Affero General Public License
// along with this program. If not, see <http://www.gnu.org/licenses/>.
package etcd
import (
"github.com/purpleidea/mgmt/resources"
)
// World is an etcd backed implementation of the World interface.
type World struct {
Hostname string // uuid for the consumer of these
EmbdEtcd *EmbdEtcd
}
// ResExport exports a list of resources under our hostname namespace.
// Subsequent calls replace the previously set collection atomically.
func (obj *World) ResExport(resourceList []resources.Res) error {
return SetResources(obj.EmbdEtcd, obj.Hostname, resourceList)
}
// ResCollect gets the collection of exported resources which match the filter.
// It does this atomically so that a call always returns a complete collection.
func (obj *World) ResCollect(hostnameFilter, kindFilter []string) ([]resources.Res, error) {
// XXX: should we be restricted to retrieving resources that were
// exported with a tag that allows or restricts our hostname? We could
// enforce that here if the underlying API supported it... Add this?
return GetResources(obj.EmbdEtcd, hostnameFilter, kindFilter)
}
// SetWatch returns a channel which spits out events on possible string changes.
func (obj *World) StrWatch(namespace string) chan error {
return WatchStr(obj.EmbdEtcd, namespace)
}
// StrGet returns a map of hostnames to values in the given namespace.
func (obj *World) StrGet(namespace string) (map[string]string, error) {
return GetStr(obj.EmbdEtcd, []string{}, namespace)
}
// StrSet sets the namespace value to a particular string under the identity of
// its own hostname.
func (obj *World) StrSet(namespace, value string) error {
return SetStr(obj.EmbdEtcd, obj.Hostname, namespace, &value)
}
// StrDel deletes the value in a particular namespace.
func (obj *World) StrDel(namespace string) error {
return SetStr(obj.EmbdEtcd, obj.Hostname, namespace, nil)
}

View File

@@ -1,55 +0,0 @@
// Mgmt
// Copyright (C) 2013-2016+ James Shubin and the project contributors
// Written by James Shubin <james@shubin.ca> and the project contributors
//
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU Affero General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// This program is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU Affero General Public License for more details.
//
// You should have received a copy of the GNU Affero General Public License
// along with this program. If not, see <http://www.gnu.org/licenses/>.
package main
//go:generate stringer -type=eventName -output=eventname_stringer.go
type eventName int
const (
eventExit eventName = iota
eventStart
eventPause
eventPoke
eventBackPoke
)
type Event struct {
Name eventName
Resp chan bool // channel to send an ack response on, nil to skip
//Wg *sync.WaitGroup // receiver barrier to Wait() for everyone else on
Msg string // some words for fun
Activity bool // did something interesting happen?
}
// send a single acknowledgement on the channel if one was requested
func (event *Event) ACK() {
if event.Resp != nil { // if they've requested an ACK
event.Resp <- true // send ACK
}
}
func (event *Event) NACK() {
if event.Resp != nil { // if they've requested an ACK
event.Resp <- false // send NACK
}
}
// get the activity value
func (event *Event) GetActivity() bool {
return event.Activity
}

119
event/event.go Normal file
View File

@@ -0,0 +1,119 @@
// Mgmt
// Copyright (C) 2013-2017+ James Shubin and the project contributors
// Written by James Shubin <james@shubin.ca> and the project contributors
//
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU Affero General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// This program is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU Affero General Public License for more details.
//
// You should have received a copy of the GNU Affero General Public License
// along with this program. If not, see <http://www.gnu.org/licenses/>.
// Package event provides some primitives that are used for message passing.
package event
import (
"fmt"
)
//go:generate stringer -type=Kind -output=kind_stringer.go
// Kind represents the type of event being passed.
type Kind int
// The different event kinds are used in different contexts.
const (
EventNil Kind = iota
EventExit
EventStart
EventPause
EventPoke
EventBackPoke
)
// Resp is a channel to be used for boolean responses. A nil represents an ACK,
// and a non-nil represents a NACK (false). This also lets us use custom errors.
type Resp chan error
// Event is the main struct that stores event information and responses.
type Event struct {
Kind Kind
Resp Resp // channel to send an ack response on, nil to skip
//Wg *sync.WaitGroup // receiver barrier to Wait() for everyone else on
Err error // store an error in our event
}
// ACK sends a single acknowledgement on the channel if one was requested.
func (event *Event) ACK() {
if event.Resp != nil { // if they've requested an ACK
event.Resp.ACK()
}
}
// NACK sends a negative acknowledgement message on the channel if one was requested.
func (event *Event) NACK() {
if event.Resp != nil { // if they've requested a NACK
event.Resp.NACK()
}
}
// ACKNACK sends a custom ACK or NACK message on the channel if one was requested.
func (event *Event) ACKNACK(err error) {
if event.Resp != nil { // if they've requested a NACK
event.Resp.ACKNACK(err)
}
}
// NewResp is just a helper to return the right type of response channel.
func NewResp() Resp {
resp := make(chan error)
return resp
}
// ACK sends a true value to resp.
func (resp Resp) ACK() {
if resp != nil {
resp <- nil // TODO: close instead?
}
}
// NACK sends a false value to resp.
func (resp Resp) NACK() {
if resp != nil {
resp <- fmt.Errorf("NACK")
}
}
// ACKNACK sends a custom ACK or NACK. The ACK value is always nil, the NACK can
// be any non-nil error value.
func (resp Resp) ACKNACK(err error) {
if resp != nil {
resp <- err
}
}
// Wait waits for any response from a Resp channel and returns it.
func (resp Resp) Wait() error {
return <-resp
}
// ACKWait waits for a +ive Ack from a Resp channel.
func (resp Resp) ACKWait() {
for {
// wait until true value
if resp.Wait() == nil {
return
}
}
}
// Error returns the stored error value.
func (event *Event) Error() error {
return event.Err
}

11
examples/augeas1.yaml Normal file
View File

@@ -0,0 +1,11 @@
---
graph: mygraph
resources:
augeas:
- name: sshd_config
lens: Sshd.lns
file: "/etc/ssh/sshd_config"
sets:
- path: X11Forwarding
value: false
edges:

19
examples/autoedges1.yaml Normal file
View File

@@ -0,0 +1,19 @@
---
graph: mygraph
resources:
file:
- name: file1
meta:
autoedge: true
path: "/tmp/foo/bar/f1"
content: |
i am f1
state: exists
- name: file2
meta:
autoedge: true
path: "/tmp/foo/"
content: |
i am f2
state: exists
edges: []

24
examples/autoedges2.yaml Normal file
View File

@@ -0,0 +1,24 @@
---
graph: mygraph
resources:
file:
- name: file1
meta:
autoedge: true
path: "/etc/drbd.conf"
content: |
# this is an mgmt test
state: exists
- name: file2
meta:
autoedge: true
path: "/tmp/foo/"
content: |
i am f2
state: exists
pkg:
- name: drbd-utils
meta:
autoedge: true
state: installed
edges: []

32
examples/autoedges3.yaml Normal file
View File

@@ -0,0 +1,32 @@
---
graph: mygraph
resources:
pkg:
- name: drbd-utils
meta:
autoedge: true
noop: true
state: installed
file:
- name: file1
meta:
autoedge: true
noop: true
path: "/etc/drbd.conf"
content: |
# this is an mgmt test
state: exists
- name: file2
meta:
autoedge: true
noop: true
path: "/etc/drbd.d/"
source: /dev/null
state: exists
svc:
- name: drbd
meta:
autoedge: true
noop: true
state: stopped
edges: []

21
examples/autogroup1.yaml Normal file
View File

@@ -0,0 +1,21 @@
---
graph: mygraph
resources:
pkg:
- name: drbd-utils
meta:
autogroup: false
state: installed
- name: powertop
meta:
autogroup: true
state: installed
- name: sl
meta:
autogroup: true
state: installed
- name: cowsay
meta:
autogroup: true
state: installed
edges: []

17
examples/autogroup2.yaml Normal file
View File

@@ -0,0 +1,17 @@
---
graph: mygraph
resources:
pkg:
- name: powertop
meta:
autogroup: true
state: installed
- name: sl
meta:
autogroup: true
state: installed
- name: cowsay
meta:
autogroup: true
state: installed
edges: []

9
examples/deep-dirs.yaml Normal file
View File

@@ -0,0 +1,9 @@
---
graph: mygraph
resources:
file:
- name: file1
path: "/tmp/mgmt/a/b/c/f1"
content: |
i am f1
state: exists

13
examples/etcd1a.yaml Normal file
View File

@@ -0,0 +1,13 @@
---
graph: mygraph
resources:
file:
- name: "@@filea"
path: "/tmp/mgmtA/fA"
content: |
i am fA, exported from host A
state: exists
collect:
- kind: file
pattern: "/tmp/mgmtA/"
edges: []

13
examples/etcd1b.yaml Normal file
View File

@@ -0,0 +1,13 @@
---
graph: mygraph
resources:
file:
- name: "@@fileb"
path: "/tmp/mgmtB/fB"
content: |
i am fB, exported from host B
state: exists
collect:
- kind: file
pattern: "/tmp/mgmtB/"
edges: []

13
examples/etcd1c.yaml Normal file
View File

@@ -0,0 +1,13 @@
---
graph: mygraph
resources:
file:
- name: "@@filec"
path: "/tmp/mgmtC/fC"
content: |
i am fC, exported from host C
state: exists
collect:
- kind: file
pattern: "/tmp/mgmtC/"
edges: []

13
examples/etcd1d.yaml Normal file
View File

@@ -0,0 +1,13 @@
---
graph: mygraph
resources:
file:
- name: "@@filed"
path: "/tmp/mgmtD/fD"
content: |
i am fD, exported from host D
state: exists
collect:
- kind: file
pattern: "/tmp/mgmtD/"
edges: []

13
examples/etcd1e.yaml Normal file
View File

@@ -0,0 +1,13 @@
---
graph: mygraph
resources:
file:
- name: "@@filee"
path: "/tmp/mgmtE/fE"
content: |
i am fE, exported from host E
state: exists
collect:
- kind: file
pattern: "/tmp/mgmtE/"
edges: []

View File

@@ -1,6 +1,6 @@
--- ---
graph: mygraph graph: mygraph
types: resources:
exec: exec:
- name: exec1 - name: exec1
cmd: sleep 10s cmd: sleep 10s
@@ -45,15 +45,15 @@ types:
edges: edges:
- name: e1 - name: e1
from: from:
type: exec kind: exec
name: exec1 name: exec1
to: to:
type: exec kind: exec
name: exec2 name: exec2
- name: e2 - name: e2
from: from:
type: exec kind: exec
name: exec2 name: exec2
to: to:
type: exec kind: exec
name: exec3 name: exec3

View File

@@ -1,6 +1,6 @@
--- ---
graph: mygraph graph: mygraph
types: resources:
exec: exec:
- name: exec1 - name: exec1
cmd: sleep 10s cmd: sleep 10s
@@ -25,8 +25,8 @@ types:
edges: edges:
- name: e1 - name: e1
from: from:
type: exec kind: exec
name: exec1 name: exec1
to: to:
type: exec kind: exec
name: exec2 name: exec2

View File

@@ -1,6 +1,6 @@
--- ---
graph: mygraph graph: mygraph
types: resources:
exec: exec:
- name: exec1 - name: exec1
cmd: sleep 10s cmd: sleep 10s
@@ -25,8 +25,8 @@ types:
edges: edges:
- name: e1 - name: e1
from: from:
type: exec kind: exec
name: exec1 name: exec1
to: to:
type: exec kind: exec
name: exec2 name: exec2

View File

@@ -1,6 +1,6 @@
--- ---
graph: mygraph graph: mygraph
types: resources:
exec: exec:
- name: exec1 - name: exec1
cmd: echo hello from exec1 cmd: echo hello from exec1
@@ -25,8 +25,8 @@ types:
edges: edges:
- name: e1 - name: e1
from: from:
type: exec kind: exec
name: exec1 name: exec1
to: to:
type: exec kind: exec
name: exec2 name: exec2

View File

@@ -1,6 +1,6 @@
--- ---
graph: mygraph graph: mygraph
types: resources:
exec: exec:
- name: exec1 - name: exec1
cmd: echo hello from exec1 cmd: echo hello from exec1

83
examples/exec2.yaml Normal file
View File

@@ -0,0 +1,83 @@
---
graph: mygraph
resources:
exec:
- name: exec1
cmd: sleep 10s
shell: ''
timeout: 0
watchcmd: ''
watchshell: ''
ifcmd: ''
ifshell: ''
pollint: 0
state: present
- name: exec2
cmd: sleep 10s
shell: ''
timeout: 0
watchcmd: ''
watchshell: ''
ifcmd: ''
ifshell: ''
pollint: 0
state: present
- name: exec3
cmd: sleep 10s
shell: ''
timeout: 0
watchcmd: ''
watchshell: ''
ifcmd: ''
ifshell: ''
pollint: 0
state: present
- name: exec4
cmd: sleep 10s
shell: ''
timeout: 0
watchcmd: ''
watchshell: ''
ifcmd: ''
ifshell: ''
pollint: 0
state: present
- name: exec5
cmd: sleep 15s
shell: ''
timeout: 0
watchcmd: ''
watchshell: ''
ifcmd: ''
ifshell: ''
pollint: 0
state: present
edges:
- name: e1
from:
kind: exec
name: exec1
to:
kind: exec
name: exec2
- name: e2
from:
kind: exec
name: exec1
to:
kind: exec
name: exec3
- name: e3
from:
kind: exec
name: exec2
to:
kind: exec
name: exec4
- name: e4
from:
kind: exec
name: exec3
to:
kind: exec
name: exec4

67
examples/exec3-sema.yaml Normal file
View File

@@ -0,0 +1,67 @@
---
graph: parallel
resources:
exec:
- name: pkg10
meta:
sema: ['mylock:1', 'otherlock:42']
cmd: sleep 10s
shell: ''
timeout: 0
watchcmd: ''
watchshell: ''
ifcmd: ''
ifshell: ''
pollint: 0
state: present
- name: svc10
meta:
sema: ['mylock:1']
cmd: sleep 10s
shell: ''
timeout: 0
watchcmd: ''
watchshell: ''
ifcmd: ''
ifshell: ''
pollint: 0
state: present
- name: exec10
meta:
sema: ['mylock:1']
cmd: sleep 10s
shell: ''
timeout: 0
watchcmd: ''
watchshell: ''
ifcmd: ''
ifshell: ''
pollint: 0
state: present
- name: pkg15
meta:
sema: ['mylock:1', 'otherlock:42']
cmd: sleep 15s
shell: ''
timeout: 0
watchcmd: ''
watchshell: ''
ifcmd: ''
ifshell: ''
pollint: 0
state: present
edges:
- name: e1
from:
kind: exec
name: pkg10
to:
kind: exec
name: svc10
- name: e2
from:
kind: exec
name: svc10
to:
kind: exec
name: exec10

59
examples/exec3.yaml Normal file
View File

@@ -0,0 +1,59 @@
---
graph: parallel
resources:
exec:
- name: pkg10
cmd: sleep 10s
shell: ''
timeout: 0
watchcmd: ''
watchshell: ''
ifcmd: ''
ifshell: ''
pollint: 0
state: present
- name: svc10
cmd: sleep 10s
shell: ''
timeout: 0
watchcmd: ''
watchshell: ''
ifcmd: ''
ifshell: ''
pollint: 0
state: present
- name: exec10
cmd: sleep 10s
shell: ''
timeout: 0
watchcmd: ''
watchshell: ''
ifcmd: ''
ifshell: ''
pollint: 0
state: present
- name: pkg15
cmd: sleep 15s
shell: ''
timeout: 0
watchcmd: ''
watchshell: ''
ifcmd: ''
ifshell: ''
pollint: 0
state: present
edges:
- name: e1
from:
kind: exec
name: pkg10
to:
kind: exec
name: svc10
- name: e2
from:
kind: exec
name: svc10
to:
kind: exec
name: exec10

View File

@@ -1,6 +1,6 @@
--- ---
graph: mygraph graph: mygraph
types: resources:
noop: noop:
- name: noop1 - name: noop1
file: file:
@@ -27,15 +27,15 @@ types:
edges: edges:
- name: e1 - name: e1
from: from:
type: file kind: file
name: file1 name: file1
to: to:
type: file kind: file
name: file2 name: file2
- name: e2 - name: e2
from: from:
type: file kind: file
name: file2 name: file2
to: to:
type: file kind: file
name: file3 name: file3

13
examples/file2.yaml Normal file
View File

@@ -0,0 +1,13 @@
---
graph: mygraph
resources:
noop:
- name: noop1
file:
- name: file1
path: "/tmp/mgmt/hello/"
source: "/var/lib/mgmt/files/some_dir/"
recurse: true
force: true
state: exists
edges: []

13
examples/file3.yaml Normal file
View File

@@ -0,0 +1,13 @@
---
graph: mygraph
resources:
file:
- name: file1
meta:
limit: .inf
burst: 0
path: "/tmp/mgmt/hello"
content: |
i am a file
state: exists
edges: []

View File

@@ -1,7 +1,7 @@
--- ---
graph: mygraph graph: mygraph
comment: hello world example comment: hello world example
types: resources:
noop: noop:
- name: noop1 - name: noop1
file: file:
@@ -13,8 +13,8 @@ types:
edges: edges:
- name: e1 - name: e1
from: from:
type: noop kind: noop
name: noop1 name: noop1
to: to:
type: file kind: file
name: file1 name: file1

View File

@@ -1,7 +1,7 @@
--- ---
graph: mygraph graph: mygraph
comment: simple exec fan in to fan out example to demonstrate optimization comment: simple exec fan in to fan out example to demonstrate optimization
types: resources:
exec: exec:
- name: exec1 - name: exec1
cmd: sleep 10s cmd: sleep 10s
@@ -86,43 +86,43 @@ types:
edges: edges:
- name: e1 - name: e1
from: from:
type: exec kind: exec
name: exec1 name: exec1
to: to:
type: exec kind: exec
name: exec4 name: exec4
- name: e2 - name: e2
from: from:
type: exec kind: exec
name: exec2 name: exec2
to: to:
type: exec kind: exec
name: exec4 name: exec4
- name: e3 - name: e3
from: from:
type: exec kind: exec
name: exec3 name: exec3
to: to:
type: exec kind: exec
name: exec4 name: exec4
- name: e4 - name: e4
from: from:
type: exec kind: exec
name: exec4 name: exec4
to: to:
type: exec kind: exec
name: exec5 name: exec5
- name: e5 - name: e5
from: from:
type: exec kind: exec
name: exec4 name: exec4
to: to:
type: exec kind: exec
name: exec6 name: exec6
- name: e6 - name: e6
from: from:
type: exec kind: exec
name: exec4 name: exec4
to: to:
type: exec kind: exec
name: exec7 name: exec7

View File

@@ -1,6 +1,6 @@
--- ---
graph: mygraph graph: mygraph
types: resources:
file: file:
- name: file1 - name: file1
path: "/tmp/mgmt/f1" path: "/tmp/mgmt/f1"
@@ -15,8 +15,8 @@ types:
edges: edges:
- name: e1 - name: e1
from: from:
type: file kind: file
name: file1 name: file1
to: to:
type: file kind: file
name: file2 name: file2

View File

@@ -1,6 +1,6 @@
--- ---
graph: mygraph graph: mygraph
types: resources:
file: file:
- name: file2 - name: file2
path: "/tmp/mgmt/f2" path: "/tmp/mgmt/f2"
@@ -15,8 +15,8 @@ types:
edges: edges:
- name: e2 - name: e2
from: from:
type: file kind: file
name: file2 name: file2
to: to:
type: file kind: file
name: file3 name: file3

View File

@@ -1,6 +1,6 @@
--- ---
graph: mygraph graph: mygraph
types: resources:
file: file:
- name: file1a - name: file1a
path: "/tmp/mgmtA/f1a" path: "/tmp/mgmtA/f1a"
@@ -23,6 +23,6 @@ types:
i am f4, exported from host A i am f4, exported from host A
state: exists state: exists
collect: collect:
- type: file - kind: file
pattern: "/tmp/mgmtA/" pattern: "/tmp/mgmtA/"
edges: [] edges: []

View File

@@ -1,6 +1,6 @@
--- ---
graph: mygraph graph: mygraph
types: resources:
file: file:
- name: file1b - name: file1b
path: "/tmp/mgmtB/f1b" path: "/tmp/mgmtB/f1b"
@@ -23,6 +23,6 @@ types:
i am f4, exported from host B i am f4, exported from host B
state: exists state: exists
collect: collect:
- type: file - kind: file
pattern: "/tmp/mgmtB/" pattern: "/tmp/mgmtB/"
edges: [] edges: []

View File

@@ -1,6 +1,6 @@
--- ---
graph: mygraph graph: mygraph
types: resources:
file: file:
- name: file1c - name: file1c
path: "/tmp/mgmtC/f1c" path: "/tmp/mgmtC/f1c"
@@ -23,6 +23,6 @@ types:
i am f4, exported from host C i am f4, exported from host C
state: exists state: exists
collect: collect:
- type: file - kind: file
pattern: "/tmp/mgmtC/" pattern: "/tmp/mgmtC/"
edges: [] edges: []

View File

@@ -1,6 +1,6 @@
--- ---
graph: mygraph graph: mygraph
types: resources:
file: file:
- name: file1 - name: file1
path: "/tmp/mgmt/f1" path: "/tmp/mgmt/f1"
@@ -13,6 +13,6 @@ types:
i am f3, exported from host A i am f3, exported from host A
state: exists state: exists
collect: collect:
- type: file - kind: file
pattern: '' pattern: ''
edges: edges:

View File

@@ -1,6 +1,6 @@
--- ---
graph: mygraph graph: mygraph
types: resources:
file: file:
- name: file1 - name: file1
path: "/tmp/mgmt/f1" path: "/tmp/mgmt/f1"
@@ -8,6 +8,6 @@ types:
i am f1 i am f1
state: exists state: exists
collect: collect:
- type: file - kind: file
pattern: '' pattern: ''
edges: edges:

View File

@@ -1,6 +1,6 @@
--- ---
graph: mygraph graph: mygraph
types: resources:
noop: noop:
- name: noop1 - name: noop1
edges: edges:

View File

@@ -1,6 +1,6 @@
--- ---
graph: mygraph graph: mygraph
types: resources:
noop: noop:
- name: noop1 - name: noop1
exec: exec:

View File

@@ -1,7 +1,7 @@
--- ---
graph: mygraph graph: mygraph
comment: simple exec fan in example to demonstrate optimization comment: simple exec fan in example to demonstrate optimization
types: resources:
exec: exec:
- name: exec1 - name: exec1
cmd: sleep 10s cmd: sleep 10s
@@ -56,22 +56,22 @@ types:
edges: edges:
- name: e1 - name: e1
from: from:
type: exec kind: exec
name: exec1 name: exec1
to: to:
type: exec kind: exec
name: exec5 name: exec5
- name: e2 - name: e2
from: from:
type: exec kind: exec
name: exec2 name: exec2
to: to:
type: exec kind: exec
name: exec5 name: exec5
- name: e3 - name: e3
from: from:
type: exec kind: exec
name: exec3 name: exec3
to: to:
type: exec kind: exec
name: exec5 name: exec5

7
examples/hostname.yaml Normal file
View File

@@ -0,0 +1,7 @@
---
graph: mygraph
resources:
hostname:
- name: Hostname Watcher @ TestHost
hostname: test.hostname.example.com
edges: []

8
examples/kv1.yaml Normal file
View File

@@ -0,0 +1,8 @@
---
graph: mygraph
resources:
kv:
- name: kv1
key: "hello"
value: "world"
edges: []

7
examples/kv2.yaml Normal file
View File

@@ -0,0 +1,7 @@
---
graph: mygraph
resources:
kv:
- name: kv1
key: "iamdeleted"
edges: []

9
examples/kv3.yaml Normal file
View File

@@ -0,0 +1,9 @@
---
graph: mygraph
resources:
kv:
- name: kv1
key: "stage"
value: "3"
skiplessthan: true
edges: []

31
examples/kv4.yaml Normal file
View File

@@ -0,0 +1,31 @@
---
graph: mygraph
resources:
kv:
- name: kv1
key: "stage"
value: "1"
skiplessthan: true
- name: kv2
key: "stage"
value: "2"
skiplessthan: true
- name: kv3
key: "stage"
value: "3"
skiplessthan: true
edges:
- name: e1
from:
kind: kv
name: kv1
to:
kind: kv
name: kv2
- name: e2
from:
kind: kv
name: kv2
to:
kind: kv
name: kv3

192
examples/lib/libmgmt1.go Normal file
View File

@@ -0,0 +1,192 @@
// libmgmt example
package main
import (
"fmt"
"log"
"os"
"os/signal"
"sync"
"syscall"
"time"
"github.com/purpleidea/mgmt/gapi"
mgmt "github.com/purpleidea/mgmt/lib"
"github.com/purpleidea/mgmt/pgraph"
"github.com/purpleidea/mgmt/resources"
"github.com/purpleidea/mgmt/yamlgraph"
)
// MyGAPI implements the main GAPI interface.
type MyGAPI struct {
Name string // graph name
Interval uint // refresh interval, 0 to never refresh
data gapi.Data
initialized bool
closeChan chan struct{}
wg sync.WaitGroup // sync group for tunnel go routines
}
// NewMyGAPI creates a new MyGAPI struct and calls Init().
func NewMyGAPI(data gapi.Data, name string, interval uint) (*MyGAPI, error) {
obj := &MyGAPI{
Name: name,
Interval: interval,
}
return obj, obj.Init(data)
}
// Init initializes the MyGAPI struct.
func (obj *MyGAPI) Init(data gapi.Data) error {
if obj.initialized {
return fmt.Errorf("already initialized")
}
if obj.Name == "" {
return fmt.Errorf("the graph name must be specified")
}
obj.data = data // store for later
obj.closeChan = make(chan struct{})
obj.initialized = true
return nil
}
// Graph returns a current Graph.
func (obj *MyGAPI) Graph() (*pgraph.Graph, error) {
if !obj.initialized {
return nil, fmt.Errorf("libmgmt: MyGAPI is not initialized")
}
n1, err := resources.NewNoopRes("noop1")
if err != nil {
return nil, fmt.Errorf("can't create resource: %v", err)
}
// we can still build a graph via the yaml method
gc := &yamlgraph.GraphConfig{
Graph: obj.Name,
Resources: yamlgraph.Resources{ // must redefine anonymous struct :(
// in alphabetical order
Exec: []*resources.ExecRes{},
File: []*resources.FileRes{},
Msg: []*resources.MsgRes{},
Noop: []*resources.NoopRes{n1},
Pkg: []*resources.PkgRes{},
Svc: []*resources.SvcRes{},
Timer: []*resources.TimerRes{},
Virt: []*resources.VirtRes{},
},
//Collector: []collectorResConfig{},
//Edges: []Edge{},
Comment: "comment!",
}
g, err := gc.NewGraphFromConfig(obj.data.Hostname, obj.data.World, obj.data.Noop)
return g, err
}
// Next returns nil errors every time there could be a new graph.
func (obj *MyGAPI) Next() chan error {
if obj.data.NoWatch || obj.Interval <= 0 {
return nil
}
ch := make(chan error)
obj.wg.Add(1)
go func() {
defer obj.wg.Done()
defer close(ch) // this will run before the obj.wg.Done()
if !obj.initialized {
ch <- fmt.Errorf("libmgmt: MyGAPI is not initialized")
return
}
// arbitrarily change graph every interval seconds
ticker := time.NewTicker(time.Duration(obj.Interval) * time.Second)
defer ticker.Stop()
for {
select {
case <-ticker.C:
log.Printf("libmgmt: Generating new graph...")
select {
case ch <- nil: // trigger a run
case <-obj.closeChan:
return
}
case <-obj.closeChan:
return
}
}
}()
return ch
}
// Close shuts down the MyGAPI.
func (obj *MyGAPI) Close() error {
if !obj.initialized {
return fmt.Errorf("libmgmt: MyGAPI is not initialized")
}
close(obj.closeChan)
obj.wg.Wait()
obj.initialized = false // closed = true
return nil
}
// Run runs an embedded mgmt server.
func Run() error {
obj := &mgmt.Main{}
obj.Program = "libmgmt" // TODO: set on compilation
obj.Version = "0.0.1" // TODO: set on compilation
obj.TmpPrefix = true
obj.IdealClusterSize = -1
obj.ConvergedTimeout = -1
obj.Noop = true
obj.GAPI = &MyGAPI{ // graph API
Name: "libmgmt", // TODO: set on compilation
Interval: 15, // arbitrarily change graph every 15 seconds
}
if err := obj.Init(); err != nil {
return err
}
// install the exit signal handler
exit := make(chan struct{})
defer close(exit)
go func() {
signals := make(chan os.Signal, 1)
signal.Notify(signals, os.Interrupt) // catch ^C
//signal.Notify(signals, os.Kill) // catch signals
signal.Notify(signals, syscall.SIGTERM)
select {
case sig := <-signals: // any signal will do
if sig == os.Interrupt {
log.Println("Interrupted by ^C")
obj.Exit(nil)
return
}
log.Println("Interrupted by signal")
obj.Exit(fmt.Errorf("killed by %v", sig))
return
case <-exit:
return
}
}()
if err := obj.Run(); err != nil {
return err
}
return nil
}
func main() {
log.Printf("Hello!")
if err := Run(); err != nil {
fmt.Println(err)
os.Exit(1)
return
}
log.Printf("Goodbye!")
}

192
examples/lib/libmgmt2.go Normal file
View File

@@ -0,0 +1,192 @@
// libmgmt example
package main
import (
"fmt"
"log"
"os"
"os/signal"
"strconv"
"sync"
"syscall"
"time"
"github.com/purpleidea/mgmt/gapi"
mgmt "github.com/purpleidea/mgmt/lib"
"github.com/purpleidea/mgmt/pgraph"
"github.com/purpleidea/mgmt/resources"
)
// MyGAPI implements the main GAPI interface.
type MyGAPI struct {
Name string // graph name
Count uint // number of resources to create
Interval uint // refresh interval, 0 to never refresh
data gapi.Data
initialized bool
closeChan chan struct{}
wg sync.WaitGroup // sync group for tunnel go routines
}
// NewMyGAPI creates a new MyGAPI struct and calls Init().
func NewMyGAPI(data gapi.Data, name string, interval uint, count uint) (*MyGAPI, error) {
obj := &MyGAPI{
Name: name,
Count: count,
Interval: interval,
}
return obj, obj.Init(data)
}
// Init initializes the MyGAPI struct.
func (obj *MyGAPI) Init(data gapi.Data) error {
if obj.initialized {
return fmt.Errorf("already initialized")
}
if obj.Name == "" {
return fmt.Errorf("the graph name must be specified")
}
obj.data = data // store for later
obj.closeChan = make(chan struct{})
obj.initialized = true
return nil
}
// Graph returns a current Graph.
func (obj *MyGAPI) Graph() (*pgraph.Graph, error) {
if !obj.initialized {
return nil, fmt.Errorf("libmgmt: MyGAPI is not initialized")
}
g := pgraph.NewGraph(obj.Name)
var vertex *pgraph.Vertex
for i := uint(0); i < obj.Count; i++ {
n, err := resources.NewNoopRes(fmt.Sprintf("noop%d", i))
if err != nil {
return nil, fmt.Errorf("can't create resource: %v", err)
}
v := pgraph.NewVertex(n)
g.AddVertex(v)
if i > 0 {
g.AddEdge(vertex, v, pgraph.NewEdge(fmt.Sprintf("e%d", i)))
}
vertex = v // save
}
//g, err := config.NewGraphFromConfig(obj.data.Hostname, obj.data.World, obj.data.Noop)
return g, nil
}
// Next returns nil errors every time there could be a new graph.
func (obj *MyGAPI) Next() chan error {
if obj.data.NoWatch || obj.Interval <= 0 {
return nil
}
ch := make(chan error)
obj.wg.Add(1)
go func() {
defer obj.wg.Done()
defer close(ch) // this will run before the obj.wg.Done()
if !obj.initialized {
ch <- fmt.Errorf("libmgmt: MyGAPI is not initialized")
return
}
// arbitrarily change graph every interval seconds
ticker := time.NewTicker(time.Duration(obj.Interval) * time.Second)
defer ticker.Stop()
for {
select {
case <-ticker.C:
log.Printf("libmgmt: Generating new graph...")
select {
case ch <- nil: // trigger a run
case <-obj.closeChan:
return
}
case <-obj.closeChan:
return
}
}
}()
return ch
}
// Close shuts down the MyGAPI.
func (obj *MyGAPI) Close() error {
if !obj.initialized {
return fmt.Errorf("libmgmt: MyGAPI is not initialized")
}
close(obj.closeChan)
obj.wg.Wait()
obj.initialized = false // closed = true
return nil
}
// Run runs an embedded mgmt server.
func Run(count uint) error {
obj := &mgmt.Main{}
obj.Program = "libmgmt" // TODO: set on compilation
obj.Version = "0.0.1" // TODO: set on compilation
obj.TmpPrefix = true
obj.IdealClusterSize = -1
obj.ConvergedTimeout = -1
obj.Noop = true
obj.GAPI = &MyGAPI{ // graph API
Name: "libmgmt", // TODO: set on compilation
Count: count, // number of vertices to add
Interval: 15, // arbitrarily change graph every 15 seconds
}
if err := obj.Init(); err != nil {
return err
}
// install the exit signal handler
exit := make(chan struct{})
defer close(exit)
go func() {
signals := make(chan os.Signal, 1)
signal.Notify(signals, os.Interrupt) // catch ^C
//signal.Notify(signals, os.Kill) // catch signals
signal.Notify(signals, syscall.SIGTERM)
select {
case sig := <-signals: // any signal will do
if sig == os.Interrupt {
log.Println("Interrupted by ^C")
obj.Exit(nil)
return
}
log.Println("Interrupted by signal")
obj.Exit(fmt.Errorf("killed by %v", sig))
return
case <-exit:
return
}
}()
if err := obj.Run(); err != nil {
return err
}
return nil
}
func main() {
log.Printf("Hello!")
var count uint = 1 // default
if len(os.Args) == 2 {
if i, err := strconv.Atoi(os.Args[1]); err == nil && i > 0 {
count = uint(i)
}
}
if err := Run(count); err != nil {
fmt.Println(err)
os.Exit(1)
return
}
log.Printf("Goodbye!")
}

241
examples/lib/libmgmt3.go Normal file
View File

@@ -0,0 +1,241 @@
// libmgmt example of send->recv
package main
import (
"fmt"
"log"
"os"
"os/signal"
"sync"
"syscall"
"time"
"github.com/purpleidea/mgmt/gapi"
mgmt "github.com/purpleidea/mgmt/lib"
"github.com/purpleidea/mgmt/pgraph"
"github.com/purpleidea/mgmt/resources"
"golang.org/x/time/rate"
)
// MyGAPI implements the main GAPI interface.
type MyGAPI struct {
Name string // graph name
Interval uint // refresh interval, 0 to never refresh
data gapi.Data
initialized bool
closeChan chan struct{}
wg sync.WaitGroup // sync group for tunnel go routines
}
// NewMyGAPI creates a new MyGAPI struct and calls Init().
func NewMyGAPI(data gapi.Data, name string, interval uint) (*MyGAPI, error) {
obj := &MyGAPI{
Name: name,
Interval: interval,
}
return obj, obj.Init(data)
}
// Init initializes the MyGAPI struct.
func (obj *MyGAPI) Init(data gapi.Data) error {
if obj.initialized {
return fmt.Errorf("already initialized")
}
if obj.Name == "" {
return fmt.Errorf("the graph name must be specified")
}
obj.data = data // store for later
obj.closeChan = make(chan struct{})
obj.initialized = true
return nil
}
// Graph returns a current Graph.
func (obj *MyGAPI) Graph() (*pgraph.Graph, error) {
if !obj.initialized {
return nil, fmt.Errorf("libmgmt: MyGAPI is not initialized")
}
g := pgraph.NewGraph(obj.Name)
// FIXME: these are being specified temporarily until it's the default!
metaparams := resources.MetaParams{
Limit: rate.Inf,
Burst: 0,
}
content := "Delete me to trigger a notification!\n"
f0 := &resources.FileRes{
BaseRes: resources.BaseRes{
Name: "README",
MetaParams: metaparams,
},
Path: "/tmp/mgmt/README",
Content: &content,
State: "present",
}
v0 := pgraph.NewVertex(f0)
g.AddVertex(v0)
p1 := &resources.PasswordRes{
BaseRes: resources.BaseRes{
Name: "password1",
MetaParams: metaparams,
},
Length: 8, // generated string will have this many characters
Saved: true, // this causes passwords to be stored in plain text!
}
v1 := pgraph.NewVertex(p1)
g.AddVertex(v1)
f1 := &resources.FileRes{
BaseRes: resources.BaseRes{
Name: "file1",
MetaParams: metaparams,
// send->recv!
Recv: map[string]*resources.Send{
"Content": {Res: p1, Key: "Password"},
},
},
Path: "/tmp/mgmt/secret",
//Content: p1.Password, // won't work
State: "present",
}
v2 := pgraph.NewVertex(f1)
g.AddVertex(v2)
n1 := &resources.NoopRes{
BaseRes: resources.BaseRes{
Name: "noop1",
MetaParams: metaparams,
},
}
v3 := pgraph.NewVertex(n1)
g.AddVertex(v3)
e0 := pgraph.NewEdge("e0")
e0.Notify = true // send a notification from v0 to v1
g.AddEdge(v0, v1, e0)
g.AddEdge(v1, v2, pgraph.NewEdge("e1"))
e2 := pgraph.NewEdge("e2")
e2.Notify = true // send a notification from v2 to v3
g.AddEdge(v2, v3, e2)
//g, err := config.NewGraphFromConfig(obj.data.Hostname, obj.data.World, obj.data.Noop)
return g, nil
}
// Next returns nil errors every time there could be a new graph.
func (obj *MyGAPI) Next() chan error {
if obj.data.NoWatch || obj.Interval <= 0 {
return nil
}
ch := make(chan error)
obj.wg.Add(1)
go func() {
defer obj.wg.Done()
defer close(ch) // this will run before the obj.wg.Done()
if !obj.initialized {
ch <- fmt.Errorf("libmgmt: MyGAPI is not initialized")
return
}
// arbitrarily change graph every interval seconds
ticker := time.NewTicker(time.Duration(obj.Interval) * time.Second)
defer ticker.Stop()
for {
select {
case <-ticker.C:
log.Printf("libmgmt: Generating new graph...")
select {
case ch <- nil: // trigger a run
case <-obj.closeChan:
return
}
case <-obj.closeChan:
return
}
}
}()
return ch
}
// Close shuts down the MyGAPI.
func (obj *MyGAPI) Close() error {
if !obj.initialized {
return fmt.Errorf("libmgmt: MyGAPI is not initialized")
}
close(obj.closeChan)
obj.wg.Wait()
obj.initialized = false // closed = true
return nil
}
// Run runs an embedded mgmt server.
func Run() error {
obj := &mgmt.Main{}
obj.Program = "libmgmt" // TODO: set on compilation
obj.Version = "0.0.1" // TODO: set on compilation
obj.TmpPrefix = true // disable for easy debugging
//prefix := "/tmp/testprefix/"
//obj.Prefix = &p // enable for easy debugging
obj.IdealClusterSize = -1
obj.ConvergedTimeout = -1
obj.Noop = false // FIXME: careful!
obj.GAPI = &MyGAPI{ // graph API
Name: "libmgmt", // TODO: set on compilation
Interval: 60 * 10, // arbitrarily change graph every 15 seconds
}
if err := obj.Init(); err != nil {
return err
}
// install the exit signal handler
exit := make(chan struct{})
defer close(exit)
go func() {
signals := make(chan os.Signal, 1)
signal.Notify(signals, os.Interrupt) // catch ^C
//signal.Notify(signals, os.Kill) // catch signals
signal.Notify(signals, syscall.SIGTERM)
select {
case sig := <-signals: // any signal will do
if sig == os.Interrupt {
log.Println("Interrupted by ^C")
obj.Exit(nil)
return
}
log.Println("Interrupted by signal")
obj.Exit(fmt.Errorf("killed by %v", sig))
return
case <-exit:
return
}
}()
if err := obj.Run(); err != nil {
return err
}
return nil
}
func main() {
log.Printf("Hello!")
if err := Run(); err != nil {
fmt.Println(err)
os.Exit(1)
return
}
log.Printf("Goodbye!")
}

13
examples/limit1.yaml Normal file
View File

@@ -0,0 +1,13 @@
---
graph: mygraph
resources:
file:
- name: file1
meta:
limit: 0.2
burst: 5
path: "/tmp/mgmt/limit"
content: |
i am a normal file
state: exists
edges: []

19
examples/msg1.yaml Normal file
View File

@@ -0,0 +1,19 @@
---
graph: mygraph
comment: timer example
resources:
timer:
- name: timer1
interval: 30
msg:
- name: msg1
body: mgmt logged this message
journal: true
edges:
- name: e1
from:
kind: timer
name: timer1
to:
kind: msg
name: msg1

24
examples/noop1.yaml Normal file
View File

@@ -0,0 +1,24 @@
---
graph: mygraph
comment: noop example
resources:
noop:
- name: noop1
meta:
noop: true
file:
- name: file1
path: "/tmp/mgmt-hello-noop"
content: |
hello world from @purpleidea
state: exists
meta:
noop: true
edges:
- name: e1
from:
kind: noop
name: noop1
to:
kind: file
name: file1

30
examples/noop2.yaml Normal file
View File

@@ -0,0 +1,30 @@
---
graph: mygraph
comment: dangerous noop example
resources:
noop:
- name: noop1
meta:
noop: true
file:
- name: file1
path: "/tmp/mgmt/hello-noop"
content: |
hello world from @purpleidea
state: exists
meta:
noop: true
exec:
- name: exec1
meta:
noop: true
cmd: 'rm -rf /'
shell: '/bin/bash'
timeout: 0
watchcmd: ''
watchshell: ''
ifcmd: ''
ifshell: ''
pollint: 0
state: present
edges: []

7
examples/nspawn1.yaml Normal file
View File

@@ -0,0 +1,7 @@
---
graph: mygraph
resources:
nspawn:
- name: mgmt-nspawn1
state: running
edges: []

7
examples/nspawn2.yaml Normal file
View File

@@ -0,0 +1,7 @@
---
graph: mygraph
resources:
nspawn:
- name: mgmt-nspawn2
state: stopped
edges: []

7
examples/pkg1.yaml Normal file
View File

@@ -0,0 +1,7 @@
---
graph: mygraph
resources:
pkg:
- name: powertop
state: installed
edges: []

Some files were not shown because too many files have changed in this diff Show More