resources: Enhancements to user and group

This patch adds autoedges between users and groups, and extends users with additional fields for supplementary groups and a named primary group. Also, some small fixes to log and error messages.
resources: Add a group resource
2017-10-23 19:18:52 -04:00 · 2017-10-21 01:28:22 -04:00 · 2017-10-19 06:58:31 -04:00 · 2017-10-17 19:23:04 -04:00 · 2017-10-09 04:05:13 -04:00 · 2017-09-26 19:38:53 -04:00
167 changed files with 11973 additions and 3928 deletions
--- a/.github/PULL_REQUEST_TEMPLATE.md
+++ b/.github/PULL_REQUEST_TEMPLATE.md
@@ -0,0 +1,36 @@
+## Tips:
+
+* commit message titles must be in the form:
+```topic: Capitalized message with no trailing period```
+or:
+```topic, topic2: Capitalized message with no trailing period```
+
+* golang code must be formatted according to the standard, please run:
+```
+make gofmt		# formats the entire project correctly
+```
+or format a single golang file correctly:
+```
+gofmt -w yourcode.go
+```
+
+* please rebase your patch against current git master:
+```
+git checkout master
+git pull origin master
+git checkout your-feature
+git rebase master
+git push your-remote your-feature
+hub pull-request	# or submit with the github web ui
+```
+
+* after a patch review, please ping @purpleidea so we know to re-review:
+```
+# make changes based on reviews...
+git add -p		# add new changes
+git commit --amend	# combine with existing commit
+git push your-remote your-feature -f
+# now ping @purpleidea in the github PR since it doesn't notify us automatically
+```
+
+## Thanks for contributing to mgmt and welcome to the team!
--- a/.gitignore
+++ b/.gitignore
@@ -2,6 +2,7 @@
 .omv/
 .ssh/
 .vagrant/
+.envrc
 old/
 tmp/
 *_stringer.go
--- a/.gitmodules
+++ b/.gitmodules
@@ -16,3 +16,9 @@
 [submodule "vendor/honnef.co/go/augeas"]
 	path = vendor/honnef.co/go/augeas
 	url = https://github.com/dominikh/go-augeas/
+[submodule "vendor/github.com/grpc-ecosystem/go-grpc-prometheus"]
+	path = vendor/github.com/grpc-ecosystem/go-grpc-prometheus
+	url = https://github.com/grpc-ecosystem/go-grpc-prometheus
+[submodule "vendor/github.com/ugorji/go"]
+	path = vendor/github.com/ugorji/go
+	url = https://github.com/ugorji/go
--- a/.travis.yml
+++ b/.travis.yml
@@ -1,9 +1,9 @@
 language: go
 go:
-  - 1.6
-  - 1.7
-  - 1.8
+  - 1.8.x
+  - 1.9.x
  - tip
+go_import_path: github.com/purpleidea/mgmt
 sudo: true
 dist: trusty
 before_install:
@@ -15,7 +15,7 @@ matrix:
  fast_finish: true
  allow_failures:
    - go: tip
-    - go: 1.8
+    - go: 1.9.x
 notifications:
  irc:
    channels:
--- a/141
+++ b/141
@@ -1,5 +1,5 @@
-                    GNU AFFERO GENERAL PUBLIC LICENSE
-                       Version 3, 19 November 2007
+                    GNU GENERAL PUBLIC LICENSE
+                       Version 3, 29 June 2007

 Copyright (C) 2007 Free Software Foundation, Inc. <http://fsf.org/>
 Everyone is permitted to copy and distribute verbatim copies
@@ -7,15 +7,17 @@

                            Preamble

-  The GNU Affero General Public License is a free, copyleft license for
-software and other kinds of works, specifically designed to ensure
-cooperation with the community in the case of network server software.
+  The GNU General Public License is a free, copyleft license for
+software and other kinds of works.

  The licenses for most software and other practical works are designed
 to take away your freedom to share and change the works.  By contrast,
-our General Public Licenses are intended to guarantee your freedom to
+the GNU General Public License is intended to guarantee your freedom to
 share and change all versions of a program--to make sure it remains free
-software for all its users.
+software for all its users.  We, the Free Software Foundation, use the
+GNU General Public License for most of our software; it applies also to
+any other work released this way by its authors.  You can apply it to
+your programs, too.

  When we speak of free software, we are referring to freedom, not
 price.  Our General Public Licenses are designed to make sure that you
@@ -24,34 +26,44 @@ them if you wish), that you receive source code or can get it if you
 want it, that you can change the software or use pieces of it in new
 free programs, and that you know you can do these things.

-  Developers that use our General Public Licenses protect your rights
-with two steps: (1) assert copyright on the software, and (2) offer
-you this License which gives you legal permission to copy, distribute
-and/or modify the software.
+  To protect your rights, we need to prevent others from denying you
+these rights or asking you to surrender the rights.  Therefore, you have
+certain responsibilities if you distribute copies of the software, or if
+you modify it: responsibilities to respect the freedom of others.

-  A secondary benefit of defending all users' freedom is that
-improvements made in alternate versions of the program, if they
-receive widespread use, become available for other developers to
-incorporate.  Many developers of free software are heartened and
-encouraged by the resulting cooperation.  However, in the case of
-software used on network servers, this result may fail to come about.
-The GNU General Public License permits making a modified version and
-letting the public access it on a server without ever releasing its
-source code to the public.
+  For example, if you distribute copies of such a program, whether
+gratis or for a fee, you must pass on to the recipients the same
+freedoms that you received.  You must make sure that they, too, receive
+or can get the source code.  And you must show them these terms so they
+know their rights.

-  The GNU Affero General Public License is designed specifically to
-ensure that, in such cases, the modified source code becomes available
-to the community.  It requires the operator of a network server to
-provide the source code of the modified version running there to the
-users of that server.  Therefore, public use of a modified version, on
-a publicly accessible server, gives the public access to the source
-code of the modified version.
+  Developers that use the GNU GPL protect your rights with two steps:
+(1) assert copyright on the software, and (2) offer you this License
+giving you legal permission to copy, distribute and/or modify it.

-  An older license, called the Affero General Public License and
-published by Affero, was designed to accomplish similar goals.  This is
-a different license, not a version of the Affero GPL, but Affero has
-released a new version of the Affero GPL which permits relicensing under
-this license.
+  For the developers' and authors' protection, the GPL clearly explains
+that there is no warranty for this free software.  For both users' and
+authors' sake, the GPL requires that modified versions be marked as
+changed, so that their problems will not be attributed erroneously to
+authors of previous versions.
+
+  Some devices are designed to deny users access to install or run
+modified versions of the software inside them, although the manufacturer
+can do so.  This is fundamentally incompatible with the aim of
+protecting users' freedom to change the software.  The systematic
+pattern of such abuse occurs in the area of products for individuals to
+use, which is precisely where it is most unacceptable.  Therefore, we
+have designed this version of the GPL to prohibit the practice for those
+products.  If such problems arise substantially in other domains, we
+stand ready to extend this provision to those domains in future versions
+of the GPL, as needed to protect the freedom of users.
+
+  Finally, every program is threatened constantly by software patents.
+States should not allow patents to restrict development and use of
+software on general-purpose computers, but in those that do, we wish to
+avoid the special danger that patents applied to a free program could
+make it effectively proprietary.  To prevent this, the GPL assures that
+patents cannot be used to render the program non-free.

  The precise terms and conditions for copying, distribution and
 modification follow.
@@ -60,7 +72,7 @@ modification follow.

  0. Definitions.

-  "This License" refers to version 3 of the GNU Affero General Public License.
+  "This License" refers to version 3 of the GNU General Public License.

  "Copyright" also means copyright-like laws that apply to other kinds of
 works, such as semiconductor masks.
@@ -537,45 +549,35 @@ to collect a royalty for further conveying from those to whom you convey
 the Program, the only way you could satisfy both those terms and this
 License would be to refrain entirely from conveying the Program.

-  13. Remote Network Interaction; Use with the GNU General Public License.
-
-  Notwithstanding any other provision of this License, if you modify the
-Program, your modified version must prominently offer all users
-interacting with it remotely through a computer network (if your version
-supports such interaction) an opportunity to receive the Corresponding
-Source of your version by providing access to the Corresponding Source
-from a network server at no charge, through some standard or customary
-means of facilitating copying of software.  This Corresponding Source
-shall include the Corresponding Source for any work covered by version 3
-of the GNU General Public License that is incorporated pursuant to the
-following paragraph.
+  13. Use with the GNU Affero General Public License.

  Notwithstanding any other provision of this License, you have
 permission to link or combine any covered work with a work licensed
-under version 3 of the GNU General Public License into a single
+under version 3 of the GNU Affero General Public License into a single
 combined work, and to convey the resulting work.  The terms of this
 License will continue to apply to the part which is the covered work,
-but the work with which it is combined will remain governed by version
-3 of the GNU General Public License.
+but the special requirements of the GNU Affero General Public License,
+section 13, concerning interaction through a network will apply to the
+combination as such.

  14. Revised Versions of this License.

  The Free Software Foundation may publish revised and/or new versions of
-the GNU Affero General Public License from time to time.  Such new versions
-will be similar in spirit to the present version, but may differ in detail to
+the GNU General Public License from time to time.  Such new versions will
+be similar in spirit to the present version, but may differ in detail to
 address new problems or concerns.

  Each version is given a distinguishing version number.  If the
-Program specifies that a certain numbered version of the GNU Affero General
+Program specifies that a certain numbered version of the GNU General
 Public License "or any later version" applies to it, you have the
 option of following the terms and conditions either of that numbered
 version or of any later version published by the Free Software
 Foundation.  If the Program does not specify a version number of the
-GNU Affero General Public License, you may choose any version ever published
+GNU General Public License, you may choose any version ever published
 by the Free Software Foundation.

  If the Program specifies that a proxy can decide which future
-versions of the GNU Affero General Public License can be used, that proxy's
+versions of the GNU General Public License can be used, that proxy's
 public statement of acceptance of a version permanently authorizes you
 to choose that version for the Program.

@@ -633,29 +635,40 @@ the "copyright" line and a pointer to where the full notice is found.
    Copyright (C) <year>  <name of author>

    This program is free software: you can redistribute it and/or modify
-    it under the terms of the GNU Affero General Public License as published by
+    it under the terms of the GNU General Public License as published by
    the Free Software Foundation, either version 3 of the License, or
    (at your option) any later version.

    This program is distributed in the hope that it will be useful,
    but WITHOUT ANY WARRANTY; without even the implied warranty of
    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-    GNU Affero General Public License for more details.
+    GNU General Public License for more details.

-    You should have received a copy of the GNU Affero General Public License
+    You should have received a copy of the GNU General Public License
    along with this program.  If not, see <http://www.gnu.org/licenses/>.

 Also add information on how to contact you by electronic and paper mail.

-  If your software can interact with users remotely through a computer
-network, you should also make sure that it provides a way for users to
-get its source.  For example, if your program is a web application, its
-interface could display a "Source" link that leads users to an archive
-of the code.  There are many ways you could offer source, and different
-solutions will be better for different programs; see section 13 for the
-specific requirements.
+  If the program does terminal interaction, make it output a short
+notice like this when it starts in an interactive mode:
+
+    <program>  Copyright (C) <year>  <name of author>
+    This program comes with ABSOLUTELY NO WARRANTY; for details type `show w'.
+    This is free software, and you are welcome to redistribute it
+    under certain conditions; type `show c' for details.
+
+The hypothetical commands `show w' and `show c' should show the appropriate
+parts of the General Public License.  Of course, your program's commands
+might be different; for a GUI interface, you would use an "about box".

  You should also get your employer (if you work as a programmer) or school,
 if any, to sign a "copyright disclaimer" for the program, if necessary.
-For more information on this, and how to apply and follow the GNU AGPL, see
+For more information on this, and how to apply and follow the GNU GPL, see
 <http://www.gnu.org/licenses/>.
+
+  The GNU General Public License does not permit incorporating your program
+into proprietary programs.  If your program is a subroutine library, you
+may consider it more useful to permit linking proprietary applications with
+the library.  If this is what you want to do, use the GNU Lesser General
+Public License instead of this License.  But first, please read
+<http://www.gnu.org/philosophy/why-not-lgpl.html>.
--- a/6
+++ b/6
@@ -3,14 +3,14 @@ Copyright (C) 2013-2017+ James Shubin and the project contributors
 Written by James Shubin <james@shubin.ca> and the project contributors

 This program is free software: you can redistribute it and/or modify
-it under the terms of the GNU Affero General Public License as published by
+it under the terms of the GNU General Public License as published by
 the Free Software Foundation, either version 3 of the License, or
 (at your option) any later version.

 This program is distributed in the hope that it will be useful,
 but WITHOUT ANY WARRANTY; without even the implied warranty of
 MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-GNU Affero General Public License for more details.
+GNU General Public License for more details.

-You should have received a copy of the GNU Affero General Public License
+You should have received a copy of the GNU General Public License
 along with this program.  If not, see <http://www.gnu.org/licenses/>.
--- a/10
+++ b/10
@@ -3,16 +3,16 @@
 # Written by James Shubin <james@shubin.ca> and the project contributors
 #
 # This program is free software: you can redistribute it and/or modify
-# it under the terms of the GNU Affero General Public License as published by
+# it under the terms of the GNU General Public License as published by
 # the Free Software Foundation, either version 3 of the License, or
 # (at your option) any later version.
 #
 # This program is distributed in the hope that it will be useful,
 # but WITHOUT ANY WARRANTY; without even the implied warranty of
 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU Affero General Public License for more details.
+# GNU General Public License for more details.
 #
-# You should have received a copy of the GNU Affero General Public License
+# You should have received a copy of the GNU General Public License
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.

 SHELL = /usr/bin/env bash
@@ -41,6 +41,8 @@ ifneq ($(GOTAGS),)
    BUILD_FLAGS = -tags '$(GOTAGS)'
 endif

+default: build
+
 #
 #	art
 #
@@ -110,7 +112,7 @@ ifneq ($(OLDGOLANG),)
 	@# avoid equals sign in old golang versions eg in: -X foo=bar
 	time go build -ldflags "-X main.program $(PROGRAM) -X main.version $(SVERSION)" -o $(PROGRAM) $(BUILD_FLAGS);
 else
-	time go build -ldflags "-X main.program=$(PROGRAM) -X main.version=$(SVERSION)" -o $(PROGRAM) $(BUILD_FLAGS);
+	time go build -i -ldflags "-X main.program=$(PROGRAM) -X main.version=$(SVERSION)" -o $(PROGRAM) $(BUILD_FLAGS);
 endif

 $(PROGRAM).static: main.go
--- a/README.md
+++ b/README.md
@@ -12,7 +12,7 @@
 Come join us in the `mgmt` community!

 | Medium | Link |
-|---|---|---|
+|---|---|
 | IRC | [#mgmtconfig](https://webchat.freenode.net/?channels=#mgmtconfig) on Freenode |
 | Twitter | [@mgmtconfig](https://twitter.com/mgmtconfig) & [#mgmtconfig](https://twitter.com/hashtag/mgmtconfig) |
 | Mailing list | [mgmtconfig-list@redhat.com](https://www.redhat.com/mailman/listinfo/mgmtconfig-list) |
@@ -77,6 +77,10 @@ We'd love to have your patches! Please send them by email, or as a pull request.
 | James Shubin | video | [Recording from High Load Strategy 2016](https://vimeo.com/191493409) |
 | James Shubin | video | [Recording from NLUUG 2016](https://www.youtube.com/watch?v=MmpwOQAb_SE&html5=1) |
 | James Shubin | blog | [Send/Recv in mgmt](https://ttboj.wordpress.com/2016/12/07/sendrecv-in-mgmt/) |
+| James Shubin | blog | [Metaparameters in mgmt](https://ttboj.wordpress.com/2017/03/01/metaparameters-in-mgmt/) |
+| James Shubin | video | [Recording from Incontro DevOps 2017](https://vimeo.com/212241877) |
+| Yves Brissaud | blog | [mgmt aux HumanTalks Grenoble (french)](http://log.winsos.net/2017/04/12/mgmt-aux-human-talks-grenoble.html) |
+| James Shubin | video | [Recording from OSDC Berlin 2017](https://www.youtube.com/watch?v=LkEtBVLfygE&html5=1) |

 ##

--- a/TODO.md
+++ b/TODO.md
@@ -1,16 +1,16 @@
 # TODO
 If you're looking for something to do, look here!
 Let us know if you're working on one of the items.
+If you'd like something to work on, ping @purpleidea and I'll create an issue
+tailored especially for you! Just let me know your approximate golang skill
+level and how many hours you'd like to spend on the patch.

 ## Package resource
 - [ ] getfiles support on debian [bug](https://github.com/hughsie/PackageKit/issues/118)
 - [ ] directory info on fedora [bug](https://github.com/hughsie/PackageKit/issues/117)
 - [ ] dnf blocker [bug](https://github.com/hughsie/PackageKit/issues/110)
- [ ] install signal blocker [bug](https://github.com/hughsie/PackageKit/issues/109)

 ## File resource [bug](https://github.com/purpleidea/mgmt/issues/64) [:heart:](https://github.com/purpleidea/mgmt/labels/mgmtlove)
- [ ] chown/chmod support [:heart:](https://github.com/purpleidea/mgmt/labels/mgmtlove)
- [ ] user/group support [:heart:](https://github.com/purpleidea/mgmt/labels/mgmtlove)
 - [ ] recurse limit support [:heart:](https://github.com/purpleidea/mgmt/labels/mgmtlove)
 - [ ] fanotify support [bug](https://github.com/go-fsnotify/fsnotify/issues/114)

@@ -29,7 +29,6 @@ Let us know if you're working on one of the items.

 ## Virt (libvirt) resource
 - [ ] base resource improvements [:heart:](https://github.com/purpleidea/mgmt/labels/mgmtlove)
- [ ] port to upstream https://github.com/libvirt/libvirt-go [:heart:](https://github.com/purpleidea/mgmt/labels/mgmtlove)

 ## Net (systemd-networkd) resource
 - [ ] base resource
@@ -52,6 +51,9 @@ Let us know if you're working on one of the items.
 ## Torrent/dht file transfer
 - [ ] base plumbing

+## GPG/Auth improvements
+- [ ] base plumbing
+
 ## Language improvements
 - [ ] language design
 - [ ] lexer/parser
--- a/16
+++ b/16
@@ -6,13 +6,16 @@ Vagrant.configure(2) do |config|
 	config.vm.synced_folder ".", "/vagrant", disabled: true

 	config.vm.define "mgmt-dev" do |instance|
-		instance.vm.box = "fedora/24-cloud-base"
+		instance.vm.box = "fedora/26-cloud-base"
 	end

 	config.vm.provider "virtualbox" do |v|
 		v.memory = 1536
 		v.cpus = 2
 	end
+	config.vm.provider "libvirt" do |v|
+		v.memory = 2048
+	end

 	config.vm.provision "file", source: "vagrant/motd", destination: ".motd"
 	config.vm.provision "shell", inline: "cp ~vagrant/.motd /etc/motd"
@@ -21,7 +24,16 @@ Vagrant.configure(2) do |config|
 	config.vm.provision "file", source: "~/.gitconfig", destination: ".gitconfig"

 	# copied from make-deps.sh (with added git)
-	config.vm.provision "shell", inline: "dnf install -y libvirt-devel golang golang-googlecode-tools-stringer hg git"
+	config.vm.provision "shell", inline: "dnf install -y libvirt-devel golang golang-googlecode-tools-stringer hg git make"
+
+	# set up packagekit
+	config.vm.provision "shell" do |shell|
+		shell.inline = <<-SCRIPT
+			dnf install -y PackageKit
+			systemctl enable packagekit
+			systemctl start packagekit
+		SCRIPT
+	end

 	# set up vagrant home
 	script = <<-SCRIPT
--- a/converger/converger.go
+++ b/converger/converger.go
@@ -3,16 +3,16 @@
 // Written by James Shubin <james@shubin.ca> and the project contributors
 //
 // This program is free software: you can redistribute it and/or modify
-// it under the terms of the GNU Affero General Public License as published by
+// it under the terms of the GNU General Public License as published by
 // the Free Software Foundation, either version 3 of the License, or
 // (at your option) any later version.
 //
 // This program is distributed in the hope that it will be useful,
 // but WITHOUT ANY WARRANTY; without even the implied warranty of
 // MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-// GNU Affero General Public License for more details.
+// GNU General Public License for more details.
 //
-// You should have received a copy of the GNU Affero General Public License
+// You should have received a copy of the GNU General Public License
 // along with this program.  If not, see <http://www.gnu.org/licenses/>.

 // Package converger is a facility for reporting the converged state.
@@ -85,7 +85,7 @@ type cuid struct {
 }

 // NewConverger builds a new converger struct.
-func NewConverger(timeout int, stateFn func(bool) error) *converger {
+func NewConverger(timeout int, stateFn func(bool) error) Converger {
 	return &converger{
 		timeout: timeout,
 		stateFn: stateFn,
--- a/doc.go
+++ b/doc.go
@@ -3,16 +3,16 @@
 // Written by James Shubin <james@shubin.ca> and the project contributors
 //
 // This program is free software: you can redistribute it and/or modify
-// it under the terms of the GNU Affero General Public License as published by
+// it under the terms of the GNU General Public License as published by
 // the Free Software Foundation, either version 3 of the License, or
 // (at your option) any later version.
 //
 // This program is distributed in the hope that it will be useful,
 // but WITHOUT ANY WARRANTY; without even the implied warranty of
 // MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-// GNU Affero General Public License for more details.
+// GNU General Public License for more details.
 //
-// You should have received a copy of the GNU Affero General Public License
+// You should have received a copy of the GNU General Public License
 // along with this program.  If not, see <http://www.gnu.org/licenses/>.

 // Package main provides the main entrypoint for using the `mgmt` software.
--- a/docs/documentation.md
+++ b/docs/documentation.md
@@ -24,6 +24,7 @@ For more information, you may like to read some blog posts from the author:
 * [Automatic clustering in mgmt](https://ttboj.wordpress.com/2016/06/20/automatic-clustering-in-mgmt/)
 * [Remote execution in mgmt](https://ttboj.wordpress.com/2016/10/07/remote-execution-in-mgmt/)
 * [Send/Recv in mgmt](https://ttboj.wordpress.com/2016/12/07/sendrecv-in-mgmt/)
+* [Metaparameters in mgmt](https://ttboj.wordpress.com/2017/03/01/metaparameters-in-mgmt/)

 There is also an [introductory video](http://meetings-archive.debian.net/pub/debian-meetings/2016/debconf16/Next_Generation_Config_Mgmt.webm) available.
 Older videos and other material [is available](https://github.com/purpleidea/mgmt/#on-the-web).
@@ -181,6 +182,7 @@ parameter with the [Noop](#Noop) resource.
 * [Exec](#Exec): Execute shell commands on the system.
 * [File](#File): Manage files and directories.
 * [Hostname](#Hostname): Manages the hostname on the system.
+* [KV](#KV): Set a key value pair in our shared world database.
 * [Msg](#Msg): Send log messages.
 * [Noop](#Noop): A simple resource that does nothing.
 * [Nspawn](#Nspawn): Manage systemd-machined nspawn containers.
@@ -267,6 +269,30 @@ The pretty hostname is a free-form UTF8 host name for presentation to the user.
 Hostname is the fallback value for all 3 fields above, if only `hostname` is
 specified, it will set all 3 fields to this value.

+### KV
+
+The KV resource sets a key and value pair in the global world database. This is
+quite useful for setting a flag after a number of resources have run. It will
+ignore database updates to the value that are greater in compare order than the
+requested key if the `SkipLessThan` parameter is set to true. If we receive a
+refresh, then the stored value will be reset to the requested value even if the
+stored value is greater.
+
+#### Key
+The string key used to store the key.
+
+#### Value
+The string value to set. This can also be set via Send/Recv.
+
+#### SkipLessThan
+If this parameter is set to `true`, then it will ignore updating the value as
+long as the database versions are greater than the requested value. The compare
+operation used is based on the `SkipCmpStyle` parameter.
+
+#### SkipCmpStyle
+By default this converts the string values to integers and compares them as you
+would expect.
+
 ### Msg

 The msg resource sends messages to the main log, or an external service such
@@ -457,6 +483,17 @@ the rate limiter as designated by the `Limit` value. If the `Limit` is not set
 to `+Infinity`, this must be a non-zero value. Please see the
 [rate](https://godoc.org/golang.org/x/time/rate) package for more information.

+#### Sema
+List of string ids. Sema is a P/V style counting semaphore which can be used to
+limit parallelism during the CheckApply phase of resource execution. Each
+resource can have `N` different semaphores which share a graph global namespace.
+Each semaphore has a maximum count associated with it. The default value of the
+size is 1 (one) if size is unspecified. Each string id is the unique id of the
+semaphore. If the id contains a trailing colon (:) followed by a positive
+integer, then that value is the max size for that semaphore. Valid semaphore
+id's include: `some_id`, `hello:42`, `not:smart:4` and `:13`. It is expected
+that the last bare example be only used by the engine to add a global semaphore.
+
 ### Graph definition file
 graph.yaml is the compiled graph definition file. The format is currently
 undocumented, but by looking through the [examples/](https://github.com/purpleidea/mgmt/tree/master/examples)
@@ -481,6 +518,15 @@ Globally force all resources into no-op mode. This also disables the export to
 etcd functionality, but does not disable resource collection, however all
 resources that are collected will have their individual noop settings set.

+#### `--sema <size>`
+Globally add a counting semaphore of this size to each resource in the graph.
+The semaphore will get given an id of `:size`. In other words if you specify a
+size of 42, you can expect a semaphore if named: `:42`. It is expected that
+consumers of the semaphore metaparameter always include a prefix to avoid a
+collision with this globally defined semaphore. The size value must be greater
+than zero at this time. The traditional non-parallel execution found in config
+management tools such as `Puppet` can be obtained with `--sema 1`.
+
 #### `--remote <graph.yaml>`
 Point to a graph file to run on the remote host specified within. This parameter
 can be used multiple times if you'd like to remotely run on multiple hosts in
--- a/docs/prometheus.md
+++ b/docs/prometheus.md
@@ -31,6 +31,7 @@ Here is a list of the metrics we provide:
 - `mgmt_checkapply_total`: The number of CheckApply's that mgmt has run
 - `mgmt_failures_total`: The number of resources that have failed
 - `mgmt_failures_current`: The number of resources that have failed
+- `mgmt_graph_start_time_seconds`: Start time of the current graph since unix epoch in seconds

 For each metric, you will get some extra labels:

--- a/docs/quick-start-guide.md
+++ b/docs/quick-start-guide.md
@@ -21,7 +21,8 @@ dependencies, _runtime_ dependencies, and additionally, a few extra dependencies
 required for running the _test_ suite.

 ### Build
-* `golang` 1.6 or higher (required, available in most distros)
+* `golang` 1.8 or higher (required, available in some distros and distributed
+  as a binary officially by [golang.org](https://golang.org/dl/))
 * golang libraries (required, available with `go get ./...`) a partial list includes:
 ```
 github.com/coreos/etcd/client
@@ -60,7 +61,7 @@ For building a visual representation of the graph, `graphviz` is required.
 * golint `github.com/golang/lint/golint`

 ## Quick start
-* Make sure you have golang version 1.6 or greater installed.
+* Make sure you have golang version 1.8 or greater installed.
 * If you do not have a GOPATH yet, create one and export it:
 ```
 mkdir $HOME/gopath
--- a/docs/resource-guide.md
+++ b/docs/resource-guide.md
@@ -73,14 +73,13 @@ Init() error
 ```

 This is called to initialize the resource. If something goes wrong, it should
-return an error. It should set the resource `kind`, do any resource specific
-work, and finish by calling the `Init` method of the base resource.
+return an error. It should do any resource specific work, and finish by calling
+the `Init` method of the base resource.

 #### Example
 ```golang
 // Init initializes the Foo resource.
 func (obj *FooRes) Init() error {
-	obj.BaseRes.kind = "Foo" // must set capitalized resource kind
 	// run the resource specific initialization, and error if anything fails
 	if some_error {
 		return err // something went wrong!
@@ -202,7 +201,7 @@ will likely find the state to now be correct.

 ### Watch
 ```golang
-Watch(chan *Event) error
+Watch() error
 ```

 `Watch` is a main loop that runs and sends messages when it detects that the
@@ -344,25 +343,26 @@ some way.
 #### Example
 ```golang
 // Compare two resources and return if they are equivalent.
-func (obj *FooRes) Compare(res Res) bool {
-	switch res.(type) {
-	case *FooRes: // only compare to other resources of the Foo kind!
-		res := res.(*FileRes)
-		if !obj.BaseRes.Compare(res) { // call base Compare
-			return false
-		}
-		if obj.Name != res.Name {
-			return false
-		}
-		if obj.whatever != res.whatever {
-			return false
-		}
-		if obj.Flag != res.Flag {
-			return false
-		}
-	default:
-		return false // different kind of resource
+func (obj *FooRes) Compare(r Res) bool {
+	// we can only compare FooRes to others of the same resource kind
+	res, ok := r.(*FooRes)
+	if !ok {
+		return false
 	}
+	if !obj.BaseRes.Compare(res) { // call base Compare
+		return false
+	}
+	if obj.Name != res.Name {
+		return false
+	}
+
+	if obj.whatever != res.whatever {
+		return false
+	}
+	if obj.Flag != res.Flag {
+		return false
+	}
+
 	return true // they must match!
 }
 ```
@@ -378,7 +378,7 @@ if another resource can match a dependency to this one.

 ### AutoEdges
 ```golang
-AutoEdges() AutoEdge
+AutoEdges() (AutoEdge, error)
 ```

 This returns a struct that implements the `AutoEdge` interface. This struct
@@ -398,9 +398,9 @@ UnmarshalYAML(unmarshal func(interface{}) error) error // optional
 ```

 This is optional, but recommended for any resource that will have a YAML
-accessible struct, and an entry in the `GraphConfig` struct. It is not required
-because to do so would mean that third-party or custom resources (such as those
-someone writes to use with `libmgmt`) would have to implement this needlessly.
+accessible struct. It is not required because to do so would mean that
+third-party or custom resources (such as those someone writes to use with
+`libmgmt`) would have to implement this needlessly.

 The signature intentionally matches what is required to satisfy the `go-yaml`
 [Unmarshaler](https://godoc.org/gopkg.in/yaml.v2#Unmarshaler) interface.
@@ -452,35 +452,15 @@ type FooRes struct {
 }
 ```

-### YAML
-In addition to labelling your resource struct with YAML fields, you must also
-add an entry to the internal `GraphConfig` struct. It is a fairly straight
-forward one line patch.
+### Resource registration
+All resources must be registered with the engine so that they can be found. This
+also ensures they can be encoded and decoded. Make sure to include the following
+code snippet for this to work.

 ```golang
-type GraphConfig struct {
-// [snip...]
-	Resources struct {
-		Noop []*resources.NoopRes `yaml:"noop"`
-		File []*resources.FileRes `yaml:"file"`
-		// [snip...]
-		Foo []*resources.FooRes `yaml:"foo"` // tada :)
-	}
-}
-```
-
-It's also recommended that you add the [UnmarshalYAML](#unmarshalyaml) method to
-your resources so that unspecified values are given sane defaults.
-
-### Gob registration
-All resources must be registered with the `golang` _gob_ module so that they can
-be encoded and decoded. Make sure to include the following code snippet for this
-to work.
-
-```golang
-import "encoding/gob"
 func init() { // special golang method that runs once
-	gob.Register(&FooRes{}) // substitude your resource here
+	// set your resource kind and struct here (the kind must be lower case)
+	RegisterResource("foo", func() Res { return &FooRes{} })
 }
 ```

@@ -516,7 +496,7 @@ This can _only_ be done inside of the `CheckApply` function!
 ```golang
 // inside CheckApply, probably near the top
 if val, exists := obj.Recv["SomeKey"]; exists {
-	log.Printf("SomeKey was sent to us from: %s[%s].%s", val.Res.Kind(), val.Res.GetName(), val.Key)
+	log.Printf("SomeKey was sent to us from: %s.%s", val.Res, val.Key)
 	if val.Changed {
 		log.Printf("SomeKey was just updated!")
 		// you may want to invalidate some local cache
--- a/etcd/etcd.go
+++ b/etcd/etcd.go
@@ -3,16 +3,16 @@
 // Written by James Shubin <james@shubin.ca> and the project contributors
 //
 // This program is free software: you can redistribute it and/or modify
-// it under the terms of the GNU Affero General Public License as published by
+// it under the terms of the GNU General Public License as published by
 // the Free Software Foundation, either version 3 of the License, or
 // (at your option) any later version.
 //
 // This program is distributed in the hope that it will be useful,
 // but WITHOUT ANY WARRANTY; without even the implied warranty of
 // MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-// GNU Affero General Public License for more details.
+// GNU General Public License for more details.
 //
-// You should have received a copy of the GNU Affero General Public License
+// You should have received a copy of the GNU General Public License
 // along with this program.  If not, see <http://www.gnu.org/licenses/>.

 // TODO: Add TTL's (eg: volunteering)
@@ -65,7 +65,6 @@ import (

 	"github.com/purpleidea/mgmt/converger"
 	"github.com/purpleidea/mgmt/event"
-	"github.com/purpleidea/mgmt/resources"
 	"github.com/purpleidea/mgmt/util"

 	etcd "github.com/coreos/etcd/clientv3" // "clientv3"
@@ -82,8 +81,8 @@ import (
 const (
 	NS                      = "_mgmt" // root namespace for mgmt operations
 	seedSentinel            = "_seed" // you must not name your hostname this
-	maxStartServerTimeout   = 60      // max number of seconds to wait for server to start
-	maxStartServerRetries   = 3       // number of times to retry starting the etcd server
+	MaxStartServerTimeout   = 60      // max number of seconds to wait for server to start
+	MaxStartServerRetries   = 3       // number of times to retry starting the etcd server
 	maxClientConnectRetries = 5       // number of times to retry consecutive connect failures
 	selfRemoveTimeout       = 3       // give unnominated members a chance to self exit
 	exitDelay               = 3       // number of sec of inactivity after exit to clean up
@@ -96,7 +95,7 @@ var (
 	errApplyDeltaEventsInconsistent = errors.New("inconsistent key in ApplyDeltaEvents")
 )

-// AW is a struct for the AddWatcher queue
+// AW is a struct for the AddWatcher queue.
 type AW struct {
 	path       string
 	opts       []etcd.OpOption
@@ -107,8 +106,8 @@ type AW struct {
 	cancelFunc func() // data
 }

-// RE is a response + error struct since these two values often occur together
-// This is now called an event with the move to the etcd v3 API
+// RE is a response + error struct since these two values often occur together.
+// This is now called an event with the move to the etcd v3 API.
 type RE struct {
 	response  etcd.WatchResponse
 	path      string
@@ -120,7 +119,7 @@ type RE struct {
 	retries   uint // number of times we've retried on error
 }

-// KV is a key + value struct to hold the two items together
+// KV is a key + value struct to hold the two items together.
 type KV struct {
 	key   string
 	value string
@@ -128,7 +127,7 @@ type KV struct {
 	resp  event.Resp
 }

-// GQ is a struct for the get queue
+// GQ is a struct for the get queue.
 type GQ struct {
 	path     string
 	skipConv bool
@@ -137,7 +136,7 @@ type GQ struct {
 	data     map[string]string
 }

-// DL is a struct for the delete queue
+// DL is a struct for the delete queue.
 type DL struct {
 	path string
 	opts []etcd.OpOption
@@ -145,7 +144,7 @@ type DL struct {
 	data int64
 }

-// TN is a struct for the txn queue
+// TN is a struct for the txn queue.
 type TN struct {
 	ifcmps  []etcd.Cmp
 	thenops []etcd.Op
@@ -161,7 +160,7 @@ type Flags struct {
 	Verbose bool // add extra log message output
 }

-// EmbdEtcd provides the embedded server and client etcd functionality
+// EmbdEtcd provides the embedded server and client etcd functionality.
 type EmbdEtcd struct { // EMBeddeD etcd
 	// etcd client connection related
 	cLock  sync.Mutex   // client connect lock
@@ -202,12 +201,13 @@ type EmbdEtcd struct { // EMBeddeD etcd
 	converger converger.Converger // converged tracking

 	// etcd server related
-	serverwg sync.WaitGroup // wait for server to shutdown
-	server   *embed.Etcd    // technically this contains the server struct
-	dataDir  string         // our data dir, prefix + "etcd"
+	serverwg    sync.WaitGroup // wait for server to shutdown
+	server      *embed.Etcd    // technically this contains the server struct
+	dataDir     string         // our data dir, prefix + "etcd"
+	serverReady chan struct{}  // closes when ready
 }

-// NewEmbdEtcd creates the top level embedded etcd struct client and server obj
+// NewEmbdEtcd creates the top level embedded etcd struct client and server obj.
 func NewEmbdEtcd(hostname string, seeds, clientURLs, serverURLs etcdtypes.URLs, noServer bool, idealClusterSize uint16, flags Flags, prefix string, converger converger.Converger) *EmbdEtcd {
 	endpoints := make(etcdtypes.URLsMap)
 	if hostname == seedSentinel { // safety
@@ -240,6 +240,7 @@ func NewEmbdEtcd(hostname string, seeds, clientURLs, serverURLs etcdtypes.URLs,
 		flags:            flags,
 		prefix:           prefix,
 		dataDir:          path.Join(prefix, "etcd"),
+		serverReady:      make(chan struct{}),
 	}
 	// TODO: add some sort of auto assign method for picking these defaults
 	// add a default so that our local client can connect locally if needed
@@ -260,7 +261,7 @@ func NewEmbdEtcd(hostname string, seeds, clientURLs, serverURLs etcdtypes.URLs,
 	return obj
 }

-// GetConfig returns the config struct to be used for the etcd client connect
+// GetConfig returns the config struct to be used for the etcd client connect.
 func (obj *EmbdEtcd) GetConfig() etcd.Config {
 	endpoints := []string{}
 	// XXX: filter out any urls which wouldn't resolve here ?
@@ -342,7 +343,7 @@ func (obj *EmbdEtcd) Connect(reconnect bool) error {
 	return nil
 }

-// Startup is the main entry point to kick off the embedded etcd client & server
+// Startup is the main entry point to kick off the embedded etcd client & server.
 func (obj *EmbdEtcd) Startup() error {
 	bootstrapping := len(obj.endpoints) == 0 // because value changes after start

@@ -464,7 +465,7 @@ func (obj *EmbdEtcd) Destroy() error {
 	return nil
 }

-// CtxDelayErr requests a retry in Delta duration
+// CtxDelayErr requests a retry in Delta duration.
 type CtxDelayErr struct {
 	Delta   time.Duration
 	Message string
@@ -474,7 +475,7 @@ func (obj *CtxDelayErr) Error() string {
 	return fmt.Sprintf("CtxDelayErr(%v): %s", obj.Delta, obj.Message)
 }

-// CtxRetriesErr lets you retry as long as you have retries available
+// CtxRetriesErr lets you retry as long as you have retries available.
 // TODO: consider combining this with CtxDelayErr
 type CtxRetriesErr struct {
 	Retries uint
@@ -494,7 +495,7 @@ func (obj *CtxPermanentErr) Error() string {
 	return fmt.Sprintf("CtxPermanentErr: %s", obj.Message)
 }

-// CtxReconnectErr requests a client reconnect to the new endpoint list
+// CtxReconnectErr requests a client reconnect to the new endpoint list.
 type CtxReconnectErr struct {
 	Message string
 }
@@ -503,7 +504,7 @@ func (obj *CtxReconnectErr) Error() string {
 	return fmt.Sprintf("CtxReconnectErr: %s", obj.Message)
 }

-// CancelCtx adds a tracked cancel function around an existing context
+// CancelCtx adds a tracked cancel function around an existing context.
 func (obj *EmbdEtcd) CancelCtx(ctx context.Context) (context.Context, func()) {
 	cancelCtx, cancelFunc := context.WithCancel(ctx)
 	obj.cancelLock.Lock()
@@ -512,7 +513,7 @@ func (obj *EmbdEtcd) CancelCtx(ctx context.Context) (context.Context, func()) {
 	return cancelCtx, cancelFunc
 }

-// TimeoutCtx adds a tracked cancel function with timeout around an existing context
+// TimeoutCtx adds a tracked cancel function with timeout around an existing context.
 func (obj *EmbdEtcd) TimeoutCtx(ctx context.Context, t time.Duration) (context.Context, func()) {
 	timeoutCtx, cancelFunc := context.WithTimeout(ctx, t)
 	obj.cancelLock.Lock()
@@ -528,8 +529,9 @@ func (obj *EmbdEtcd) CtxError(ctx context.Context, err error) (context.Context,
 	if obj.ctxErr != nil { // stop on permanent error
 		return ctx, obj.ctxErr
 	}
-	const ctxErr = "ctxErr"
-	const ctxIter = "ctxIter"
+	type ctxKey string // use a non-basic type as ctx key (str can conflict)
+	const ctxErr ctxKey = "ctxErr"
+	const ctxIter ctxKey = "ctxIter"
 	expBackoff := func(tmin, texp, iter, tmax int) time.Duration {
 		// https://en.wikipedia.org/wiki/Exponential_backoff
 		// tmin <= texp^iter - 1 <= tmax // TODO: check my math
@@ -699,7 +701,7 @@ func (obj *EmbdEtcd) CtxError(ctx context.Context, err error) (context.Context,
 	return ctx, obj.ctxErr
 }

-// CbLoop is the loop where callback execution is serialized
+// CbLoop is the loop where callback execution is serialized.
 func (obj *EmbdEtcd) CbLoop() {
 	cuid := obj.converger.Register()
 	cuid.SetName("Etcd: CbLoop")
@@ -755,7 +757,7 @@ func (obj *EmbdEtcd) CbLoop() {
 	}
 }

-// Loop is the main loop where everything is serialized
+// Loop is the main loop where everything is serialized.
 func (obj *EmbdEtcd) Loop() {
 	cuid := obj.converger.Register()
 	cuid.SetName("Etcd: Loop")
@@ -933,7 +935,7 @@ func (obj *EmbdEtcd) loopProcessAW(ctx context.Context, aw *AW) {
 	}
 }

-// Set queues up a set operation to occur using our mainloop
+// Set queues up a set operation to occur using our mainloop.
 func (obj *EmbdEtcd) Set(key, value string, opts ...etcd.OpOption) error {
 	resp := event.NewResp()
 	obj.setq <- &KV{key: key, value: value, opts: opts, resp: resp}
@@ -943,7 +945,7 @@ func (obj *EmbdEtcd) Set(key, value string, opts ...etcd.OpOption) error {
 	return nil
 }

-// rawSet actually implements the key set operation
+// rawSet actually implements the key set operation.
 func (obj *EmbdEtcd) rawSet(ctx context.Context, kv *KV) error {
 	if obj.flags.Trace {
 		log.Printf("Trace: Etcd: rawSet()")
@@ -960,7 +962,7 @@ func (obj *EmbdEtcd) rawSet(ctx context.Context, kv *KV) error {
 	return err
 }

-// Get performs a get operation and waits for an ACK to continue
+// Get performs a get operation and waits for an ACK to continue.
 func (obj *EmbdEtcd) Get(path string, opts ...etcd.OpOption) (map[string]string, error) {
 	return obj.ComplexGet(path, false, opts...)
 }
@@ -1001,7 +1003,7 @@ func (obj *EmbdEtcd) rawGet(ctx context.Context, gq *GQ) (result map[string]stri
 	return
 }

-// Delete performs a delete operation and waits for an ACK to continue
+// Delete performs a delete operation and waits for an ACK to continue.
 func (obj *EmbdEtcd) Delete(path string, opts ...etcd.OpOption) (int64, error) {
 	resp := event.NewResp()
 	dl := &DL{path: path, opts: opts, resp: resp, data: -1}
@@ -1029,7 +1031,7 @@ func (obj *EmbdEtcd) rawDelete(ctx context.Context, dl *DL) (count int64, err er
 	return
 }

-// Txn performs a transaction and waits for an ACK to continue
+// Txn performs a transaction and waits for an ACK to continue.
 func (obj *EmbdEtcd) Txn(ifcmps []etcd.Cmp, thenops, elseops []etcd.Op) (*etcd.TxnResponse, error) {
 	resp := event.NewResp()
 	tn := &TN{ifcmps: ifcmps, thenops: thenops, elseops: elseops, resp: resp, data: nil}
@@ -1053,8 +1055,8 @@ func (obj *EmbdEtcd) rawTxn(ctx context.Context, tn *TN) (*etcd.TxnResponse, err
 	return response, err
 }

-// AddWatcher queues up an add watcher request and returns a cancel function
-// Remember to add the etcd.WithPrefix() option if you want to watch recursively
+// AddWatcher queues up an add watcher request and returns a cancel function.
+// Remember to add the etcd.WithPrefix() option if you want to watch recursively.
 func (obj *EmbdEtcd) AddWatcher(path string, callback func(re *RE) error, errCheck bool, skipConv bool, opts ...etcd.OpOption) (func(), error) {
 	resp := event.NewResp()
 	awq := &AW{path: path, opts: opts, callback: callback, errCheck: errCheck, skipConv: skipConv, cancelFunc: nil, resp: resp}
@@ -1065,7 +1067,7 @@ func (obj *EmbdEtcd) AddWatcher(path string, callback func(re *RE) error, errChe
 	return awq.cancelFunc, nil
 }

-// rawAddWatcher adds a watcher and returns a cancel function to call to end it
+// rawAddWatcher adds a watcher and returns a cancel function to call to end it.
 func (obj *EmbdEtcd) rawAddWatcher(ctx context.Context, aw *AW) (func(), error) {
 	cancelCtx, cancelFunc := obj.CancelCtx(ctx)
 	go func(ctx context.Context) {
@@ -1142,7 +1144,7 @@ func (obj *EmbdEtcd) rawAddWatcher(ctx context.Context, aw *AW) (func(), error)
 	return cancelFunc, nil
 }

-// rawCallback is the companion to AddWatcher which runs the callback processing
+// rawCallback is the companion to AddWatcher which runs the callback processing.
 func rawCallback(ctx context.Context, re *RE) error {
 	var err = re.err // the watch event itself might have had an error
 	if err == nil {
@@ -1161,8 +1163,8 @@ func rawCallback(ctx context.Context, re *RE) error {
 	return err
 }

-// volunteerCallback runs to respond to the volunteer list change events
-// functionally, it controls the adding and removing of members
+// volunteerCallback runs to respond to the volunteer list change events.
+// Functionally, it controls the adding and removing of members.
 // FIXME: we might need to respond to member change/disconnect/shutdown events,
 // see: https://github.com/coreos/etcd/issues/5277
 func (obj *EmbdEtcd) volunteerCallback(re *RE) error {
@@ -1351,8 +1353,8 @@ func (obj *EmbdEtcd) volunteerCallback(re *RE) error {
 	return nil
 }

-// nominateCallback runs to respond to the nomination list change events
-// functionally, it controls the starting and stopping of the server process
+// nominateCallback runs to respond to the nomination list change events.
+// Functionally, it controls the starting and stopping of the server process.
 func (obj *EmbdEtcd) nominateCallback(re *RE) error {
 	if obj.flags.Trace {
 		log.Printf("Trace: Etcd: nominateCallback()")
@@ -1419,8 +1421,8 @@ func (obj *EmbdEtcd) nominateCallback(re *RE) error {
 			if re != nil {
 				retries = re.retries
 			}
-			// retry maxStartServerRetries times, then permanently fail
-			return &CtxRetriesErr{maxStartServerRetries - retries, fmt.Sprintf("Etcd: StartServer: Error: %+v", err)}
+			// retry MaxStartServerRetries times, then permanently fail
+			return &CtxRetriesErr{MaxStartServerRetries - retries, fmt.Sprintf("Etcd: StartServer: Error: %+v", err)}
 		}

 		if len(obj.endpoints) == 0 {
@@ -1504,7 +1506,7 @@ func (obj *EmbdEtcd) nominateCallback(re *RE) error {
 	return nil
 }

-// endpointCallback runs to respond to the endpoint list change events
+// endpointCallback runs to respond to the endpoint list change events.
 func (obj *EmbdEtcd) endpointCallback(re *RE) error {
 	if obj.flags.Trace {
 		log.Printf("Trace: Etcd: endpointCallback()")
@@ -1570,7 +1572,7 @@ func (obj *EmbdEtcd) endpointCallback(re *RE) error {
 	return nil
 }

-// idealClusterSizeCallback runs to respond to the ideal cluster size changes
+// idealClusterSizeCallback runs to respond to the ideal cluster size changes.
 func (obj *EmbdEtcd) idealClusterSizeCallback(re *RE) error {
 	if obj.flags.Trace {
 		log.Printf("Trace: Etcd: idealClusterSizeCallback()")
@@ -1604,8 +1606,8 @@ func (obj *EmbdEtcd) idealClusterSizeCallback(re *RE) error {
 	return nil
 }

-// LocalhostClientURLs returns the most localhost like URLs for direct connection
-// this gets clients to talk to the local servers first before searching remotely
+// LocalhostClientURLs returns the most localhost like URLs for direct connection.
+// This gets clients to talk to the local servers first before searching remotely.
 func (obj *EmbdEtcd) LocalhostClientURLs() etcdtypes.URLs {
 	// look through obj.clientURLs and return the localhost ones
 	urls := etcdtypes.URLs{}
@@ -1671,8 +1673,8 @@ func (obj *EmbdEtcd) StartServer(newCluster bool, peerURLsMap etcdtypes.URLsMap)
 	select {
 	case <-obj.server.Server.ReadyNotify(): // we hang here if things are bad
 		log.Printf("Etcd: StartServer: Done starting server!") // it didn't hang!
-	case <-time.After(time.Duration(maxStartServerTimeout) * time.Second):
-		e := fmt.Errorf("timeout of %d seconds reached", maxStartServerTimeout)
+	case <-time.After(time.Duration(MaxStartServerTimeout) * time.Second):
+		e := fmt.Errorf("timeout of %d seconds reached", MaxStartServerTimeout)
 		log.Printf("Etcd: StartServer: %s", e.Error())
 		obj.server.Server.Stop() // trigger a shutdown
 		obj.serverwg.Add(1)      // add for the DestroyServer()
@@ -1690,12 +1692,16 @@ func (obj *EmbdEtcd) StartServer(newCluster bool, peerURLsMap etcdtypes.URLsMap)
 	//log.Fatal(<-obj.server.Err())	XXX
 	log.Printf("Etcd: StartServer: Server running...")
 	obj.memberID = uint64(obj.server.Server.ID()) // store member id for internal use
+	close(obj.serverReady)                        // send a signal

 	obj.serverwg.Add(1)
 	return nil
 }

-// DestroyServer shuts down the embedded etcd server portion
+// ServerReady returns on a channel when the server has started successfully.
+func (obj *EmbdEtcd) ServerReady() <-chan struct{} { return obj.serverReady }
+
+// DestroyServer shuts down the embedded etcd server portion.
 func (obj *EmbdEtcd) DestroyServer() error {
 	var err error
 	log.Printf("Etcd: DestroyServer: Destroying...")
@@ -1710,544 +1716,11 @@ func (obj *EmbdEtcd) DestroyServer() error {
 	}
 	obj.server = nil // important because this is used as an isRunning flag
 	log.Printf("Etcd: DestroyServer: Unlocking server...")
-	obj.serverwg.Done() // -1
+	obj.serverReady = make(chan struct{}) // reset the signal
+	obj.serverwg.Done()                   // -1
 	return err
 }

-// TODO: Could all these Etcd*(obj *EmbdEtcd, ...) functions which deal with the
-// interface between etcd paths and behaviour be grouped into a single struct ?
-
-// Nominate nominates a particular client to be a server (peer)
-func Nominate(obj *EmbdEtcd, hostname string, urls etcdtypes.URLs) error {
-	if obj.flags.Trace {
-		log.Printf("Trace: Etcd: Nominate(%v): %v", hostname, urls.String())
-		defer log.Printf("Trace: Etcd: Nominate(%v): Finished!", hostname)
-	}
-	// nominate someone to be a server
-	nominate := fmt.Sprintf("/%s/nominated/%s", NS, hostname)
-	ops := []etcd.Op{} // list of ops in this txn
-	if urls != nil {
-		ops = append(ops, etcd.OpPut(nominate, urls.String())) // TODO: add a TTL? (etcd.WithLease)
-
-	} else { // delete message if set to erase
-		ops = append(ops, etcd.OpDelete(nominate))
-	}
-
-	if _, err := obj.Txn(nil, ops, nil); err != nil {
-		return fmt.Errorf("nominate failed") // exit in progress?
-	}
-	return nil
-}
-
-// Nominated returns a urls map of nominated etcd server volunteers
-// NOTE: I know 'nominees' might be more correct, but is less consistent here
-func Nominated(obj *EmbdEtcd) (etcdtypes.URLsMap, error) {
-	path := fmt.Sprintf("/%s/nominated/", NS)
-	keyMap, err := obj.Get(path, etcd.WithPrefix()) // map[string]string, bool
-	if err != nil {
-		return nil, fmt.Errorf("nominated isn't available: %v", err)
-	}
-	nominated := make(etcdtypes.URLsMap)
-	for key, val := range keyMap { // loop through directory of nominated
-		if !strings.HasPrefix(key, path) {
-			continue
-		}
-		name := key[len(path):] // get name of nominee
-		if val == "" {          // skip "erased" values
-			continue
-		}
-		urls, err := etcdtypes.NewURLs(strings.Split(val, ","))
-		if err != nil {
-			return nil, fmt.Errorf("nominated data format error: %v", err)
-		}
-		nominated[name] = urls // add to map
-		if obj.flags.Debug {
-			log.Printf("Etcd: Nominated(%v): %v", name, val)
-		}
-	}
-	return nominated, nil
-}
-
-// Volunteer offers yourself up to be a server if needed
-func Volunteer(obj *EmbdEtcd, urls etcdtypes.URLs) error {
-	if obj.flags.Trace {
-		log.Printf("Trace: Etcd: Volunteer(%v): %v", obj.hostname, urls.String())
-		defer log.Printf("Trace: Etcd: Volunteer(%v): Finished!", obj.hostname)
-	}
-	// volunteer to be a server
-	volunteer := fmt.Sprintf("/%s/volunteers/%s", NS, obj.hostname)
-	ops := []etcd.Op{} // list of ops in this txn
-	if urls != nil {
-		// XXX: adding a TTL is crucial! (i think)
-		ops = append(ops, etcd.OpPut(volunteer, urls.String())) // value is usually a peer "serverURL"
-
-	} else { // delete message if set to erase
-		ops = append(ops, etcd.OpDelete(volunteer))
-	}
-
-	if _, err := obj.Txn(nil, ops, nil); err != nil {
-		return fmt.Errorf("volunteering failed") // exit in progress?
-	}
-	return nil
-}
-
-// Volunteers returns a urls map of available etcd server volunteers
-func Volunteers(obj *EmbdEtcd) (etcdtypes.URLsMap, error) {
-	if obj.flags.Trace {
-		log.Printf("Trace: Etcd: Volunteers()")
-		defer log.Printf("Trace: Etcd: Volunteers(): Finished!")
-	}
-	path := fmt.Sprintf("/%s/volunteers/", NS)
-	keyMap, err := obj.Get(path, etcd.WithPrefix())
-	if err != nil {
-		return nil, fmt.Errorf("volunteers aren't available: %v", err)
-	}
-	volunteers := make(etcdtypes.URLsMap)
-	for key, val := range keyMap { // loop through directory of volunteers
-		if !strings.HasPrefix(key, path) {
-			continue
-		}
-		name := key[len(path):] // get name of volunteer
-		if val == "" {          // skip "erased" values
-			continue
-		}
-		urls, err := etcdtypes.NewURLs(strings.Split(val, ","))
-		if err != nil {
-			return nil, fmt.Errorf("volunteers data format error: %v", err)
-		}
-		volunteers[name] = urls // add to map
-		if obj.flags.Debug {
-			log.Printf("Etcd: Volunteer(%v): %v", name, val)
-		}
-	}
-	return volunteers, nil
-}
-
-// AdvertiseEndpoints advertises the list of available client endpoints
-func AdvertiseEndpoints(obj *EmbdEtcd, urls etcdtypes.URLs) error {
-	if obj.flags.Trace {
-		log.Printf("Trace: Etcd: AdvertiseEndpoints(%v): %v", obj.hostname, urls.String())
-		defer log.Printf("Trace: Etcd: AdvertiseEndpoints(%v): Finished!", obj.hostname)
-	}
-	// advertise endpoints
-	endpoints := fmt.Sprintf("/%s/endpoints/%s", NS, obj.hostname)
-	ops := []etcd.Op{} // list of ops in this txn
-	if urls != nil {
-		// TODO: add a TTL? (etcd.WithLease)
-		ops = append(ops, etcd.OpPut(endpoints, urls.String())) // value is usually a "clientURL"
-
-	} else { // delete message if set to erase
-		ops = append(ops, etcd.OpDelete(endpoints))
-	}
-
-	if _, err := obj.Txn(nil, ops, nil); err != nil {
-		return fmt.Errorf("endpoint advertising failed") // exit in progress?
-	}
-	return nil
-}
-
-// Endpoints returns a urls map of available etcd server endpoints
-func Endpoints(obj *EmbdEtcd) (etcdtypes.URLsMap, error) {
-	if obj.flags.Trace {
-		log.Printf("Trace: Etcd: Endpoints()")
-		defer log.Printf("Trace: Etcd: Endpoints(): Finished!")
-	}
-	path := fmt.Sprintf("/%s/endpoints/", NS)
-	keyMap, err := obj.Get(path, etcd.WithPrefix())
-	if err != nil {
-		return nil, fmt.Errorf("endpoints aren't available: %v", err)
-	}
-	endpoints := make(etcdtypes.URLsMap)
-	for key, val := range keyMap { // loop through directory of endpoints
-		if !strings.HasPrefix(key, path) {
-			continue
-		}
-		name := key[len(path):] // get name of volunteer
-		if val == "" {          // skip "erased" values
-			continue
-		}
-		urls, err := etcdtypes.NewURLs(strings.Split(val, ","))
-		if err != nil {
-			return nil, fmt.Errorf("endpoints data format error: %v", err)
-		}
-		endpoints[name] = urls // add to map
-		if obj.flags.Debug {
-			log.Printf("Etcd: Endpoint(%v): %v", name, val)
-		}
-	}
-	return endpoints, nil
-}
-
-// SetHostnameConverged sets whether a specific hostname is converged.
-func SetHostnameConverged(obj *EmbdEtcd, hostname string, isConverged bool) error {
-	if obj.flags.Trace {
-		log.Printf("Trace: Etcd: SetHostnameConverged(%s): %v", hostname, isConverged)
-		defer log.Printf("Trace: Etcd: SetHostnameConverged(%v): Finished!", hostname)
-	}
-	converged := fmt.Sprintf("/%s/converged/%s", NS, hostname)
-	op := []etcd.Op{etcd.OpPut(converged, fmt.Sprintf("%t", isConverged))}
-	if _, err := obj.Txn(nil, op, nil); err != nil { // TODO: do we need a skipConv flag here too?
-		return fmt.Errorf("set converged failed") // exit in progress?
-	}
-	return nil
-}
-
-// HostnameConverged returns a map of every hostname's converged state.
-func HostnameConverged(obj *EmbdEtcd) (map[string]bool, error) {
-	if obj.flags.Trace {
-		log.Printf("Trace: Etcd: HostnameConverged()")
-		defer log.Printf("Trace: Etcd: HostnameConverged(): Finished!")
-	}
-	path := fmt.Sprintf("/%s/converged/", NS)
-	keyMap, err := obj.ComplexGet(path, true, etcd.WithPrefix()) // don't un-converge
-	if err != nil {
-		return nil, fmt.Errorf("converged values aren't available: %v", err)
-	}
-	converged := make(map[string]bool)
-	for key, val := range keyMap { // loop through directory...
-		if !strings.HasPrefix(key, path) {
-			continue
-		}
-		name := key[len(path):] // get name of key
-		if val == "" {          // skip "erased" values
-			continue
-		}
-		b, err := strconv.ParseBool(val)
-		if err != nil {
-			return nil, fmt.Errorf("converged data format error: %v", err)
-		}
-		converged[name] = b // add to map
-	}
-	return converged, nil
-}
-
-// AddHostnameConvergedWatcher adds a watcher with a callback that runs on
-// hostname state changes.
-func AddHostnameConvergedWatcher(obj *EmbdEtcd, callbackFn func(map[string]bool) error) (func(), error) {
-	path := fmt.Sprintf("/%s/converged/", NS)
-	internalCbFn := func(re *RE) error {
-		// TODO: get the value from the response, and apply delta...
-		// for now, just run a get operation which is easier to code!
-		m, err := HostnameConverged(obj)
-		if err != nil {
-			return err
-		}
-		return callbackFn(m) // call my function
-	}
-	return obj.AddWatcher(path, internalCbFn, true, true, etcd.WithPrefix()) // no block and no converger reset
-}
-
-// SetClusterSize sets the ideal target cluster size of etcd peers
-func SetClusterSize(obj *EmbdEtcd, value uint16) error {
-	if obj.flags.Trace {
-		log.Printf("Trace: Etcd: SetClusterSize(): %v", value)
-		defer log.Printf("Trace: Etcd: SetClusterSize(): Finished!")
-	}
-	key := fmt.Sprintf("/%s/idealClusterSize", NS)
-
-	if err := obj.Set(key, strconv.FormatUint(uint64(value), 10)); err != nil {
-		return fmt.Errorf("function SetClusterSize failed: %v", err) // exit in progress?
-	}
-	return nil
-}
-
-// GetClusterSize gets the ideal target cluster size of etcd peers
-func GetClusterSize(obj *EmbdEtcd) (uint16, error) {
-	key := fmt.Sprintf("/%s/idealClusterSize", NS)
-	keyMap, err := obj.Get(key)
-	if err != nil {
-		return 0, fmt.Errorf("function GetClusterSize failed: %v", err)
-	}
-
-	val, exists := keyMap[key]
-	if !exists || val == "" {
-		return 0, fmt.Errorf("function GetClusterSize failed: %v", err)
-	}
-
-	v, err := strconv.ParseUint(val, 10, 16)
-	if err != nil {
-		return 0, fmt.Errorf("function GetClusterSize failed: %v", err)
-	}
-	return uint16(v), nil
-}
-
-// MemberAdd adds a member to the cluster.
-func MemberAdd(obj *EmbdEtcd, peerURLs etcdtypes.URLs) (*etcd.MemberAddResponse, error) {
-	//obj.Connect(false) // TODO: ?
-	ctx := context.Background()
-	var response *etcd.MemberAddResponse
-	var err error
-	for {
-		if obj.exiting { // the exit signal has been sent!
-			return nil, fmt.Errorf("exiting etcd")
-		}
-		obj.rLock.RLock()
-		response, err = obj.client.MemberAdd(ctx, peerURLs.StringSlice())
-		obj.rLock.RUnlock()
-		if err == nil {
-			break
-		}
-		if ctx, err = obj.CtxError(ctx, err); err != nil {
-			return nil, err
-		}
-	}
-	return response, nil
-}
-
-// MemberRemove removes a member by mID and returns if it worked, and also
-// if there was an error. This is because it might have run without error, but
-// the member wasn't found, for example.
-func MemberRemove(obj *EmbdEtcd, mID uint64) (bool, error) {
-	//obj.Connect(false) // TODO: ?
-	ctx := context.Background()
-	for {
-		if obj.exiting { // the exit signal has been sent!
-			return false, fmt.Errorf("exiting etcd")
-		}
-		obj.rLock.RLock()
-		_, err := obj.client.MemberRemove(ctx, mID)
-		obj.rLock.RUnlock()
-		if err == nil {
-			break
-		} else if err == rpctypes.ErrMemberNotFound {
-			// if we get this, member already shut itself down :)
-			return false, nil
-		}
-		if ctx, err = obj.CtxError(ctx, err); err != nil {
-			return false, err
-		}
-	}
-	return true, nil
-}
-
-// Members returns information on cluster membership.
-// The member ID's are the keys, because an empty names means unstarted!
-// TODO: consider queueing this through the main loop with CtxError(ctx, err)
-func Members(obj *EmbdEtcd) (map[uint64]string, error) {
-	//obj.Connect(false) // TODO: ?
-	ctx := context.Background()
-	var response *etcd.MemberListResponse
-	var err error
-	for {
-		if obj.exiting { // the exit signal has been sent!
-			return nil, fmt.Errorf("exiting etcd")
-		}
-		obj.rLock.RLock()
-		if obj.flags.Trace {
-			log.Printf("Trace: Etcd: Members(): Endpoints are: %v", obj.client.Endpoints())
-		}
-		response, err = obj.client.MemberList(ctx)
-		obj.rLock.RUnlock()
-		if err == nil {
-			break
-		}
-		if ctx, err = obj.CtxError(ctx, err); err != nil {
-			return nil, err
-		}
-	}
-
-	members := make(map[uint64]string)
-	for _, x := range response.Members {
-		members[x.ID] = x.Name // x.Name will be "" if unstarted!
-	}
-	return members, nil
-}
-
-// Leader returns the current leader of the etcd server cluster
-func Leader(obj *EmbdEtcd) (string, error) {
-	//obj.Connect(false) // TODO: ?
-	var err error
-	membersMap := make(map[uint64]string)
-	if membersMap, err = Members(obj); err != nil {
-		return "", err
-	}
-	addresses := obj.LocalhostClientURLs() // heuristic, but probably correct
-	if len(addresses) == 0 {
-		// probably a programming error...
-		return "", fmt.Errorf("programming error")
-	}
-	endpoint := addresses[0].Host // FIXME: arbitrarily picked the first one
-
-	// part two
-	ctx := context.Background()
-	var response *etcd.StatusResponse
-	for {
-		if obj.exiting { // the exit signal has been sent!
-			return "", fmt.Errorf("exiting etcd")
-		}
-
-		obj.rLock.RLock()
-		response, err = obj.client.Maintenance.Status(ctx, endpoint)
-		obj.rLock.RUnlock()
-		if err == nil {
-			break
-		}
-		if ctx, err = obj.CtxError(ctx, err); err != nil {
-			return "", err
-		}
-	}
-
-	// isLeader: response.Header.MemberId == response.Leader
-	for id, name := range membersMap {
-		if id == response.Leader {
-			return name, nil
-		}
-	}
-	return "", fmt.Errorf("members map is not current") // not found
-}
-
-// WatchAll returns a channel that outputs a true bool when activity occurs
-// TODO: Filter our watch (on the server side if possible) based on the
-// collection prefixes and filters that we care about...
-func WatchAll(obj *EmbdEtcd) chan bool {
-	ch := make(chan bool, 1) // buffer it so we can measure it
-	path := fmt.Sprintf("/%s/exported/", NS)
-	callback := func(re *RE) error {
-		// TODO: is this even needed? it used to happen on conn errors
-		log.Printf("Etcd: Watch: Path: %v", path) // event
-		if re == nil || re.response.Canceled {
-			return fmt.Errorf("watch is empty") // will cause a CtxError+retry
-		}
-		// we normally need to check if anything changed since the last
-		// event, since a set (export) with no changes still causes the
-		// watcher to trigger and this would cause an infinite loop. we
-		// don't need to do this check anymore because we do the export
-		// transactionally, and only if a change is needed. since it is
-		// atomic, all the changes arrive together which avoids dupes!!
-		if len(ch) == 0 { // send event only if one isn't pending
-			// this check avoids multiple events all queueing up and then
-			// being released continuously long after the changes stopped
-			// do not block!
-			ch <- true // event
-		}
-		return nil
-	}
-	_, _ = obj.AddWatcher(path, callback, true, false, etcd.WithPrefix()) // no need to check errors
-	return ch
-}
-
-// SetResources exports all of the resources which we pass in to etcd
-func SetResources(obj *EmbdEtcd, hostname string, resourceList []resources.Res) error {
-	// key structure is /$NS/exported/$hostname/resources/$uid = $data
-
-	var kindFilter []string // empty to get from everyone
-	hostnameFilter := []string{hostname}
-	// this is not a race because we should only be reading keys which we
-	// set, and there should not be any contention with other hosts here!
-	originals, err := GetResources(obj, hostnameFilter, kindFilter)
-	if err != nil {
-		return err
-	}
-
-	if len(originals) == 0 && len(resourceList) == 0 { // special case of no add or del
-		return nil
-	}
-
-	ifs := []etcd.Cmp{} // list matching the desired state
-	ops := []etcd.Op{}  // list of ops in this transaction
-	for _, res := range resourceList {
-		if res.Kind() == "" {
-			log.Fatalf("Etcd: SetResources: Error: Empty kind: %v", res.GetName())
-		}
-		uid := fmt.Sprintf("%s/%s", res.Kind(), res.GetName())
-		path := fmt.Sprintf("/%s/exported/%s/resources/%s", NS, hostname, uid)
-		if data, err := resources.ResToB64(res); err == nil {
-			ifs = append(ifs, etcd.Compare(etcd.Value(path), "=", data)) // desired state
-			ops = append(ops, etcd.OpPut(path, data))
-		} else {
-			return fmt.Errorf("can't convert to B64: %v", err)
-		}
-	}
-
-	match := func(res resources.Res, resourceList []resources.Res) bool { // helper lambda
-		for _, x := range resourceList {
-			if res.Kind() == x.Kind() && res.GetName() == x.GetName() {
-				return true
-			}
-		}
-		return false
-	}
-
-	hasDeletes := false
-	// delete old, now unused resources here...
-	for _, res := range originals {
-		if res.Kind() == "" {
-			log.Fatalf("Etcd: SetResources: Error: Empty kind: %v", res.GetName())
-		}
-		uid := fmt.Sprintf("%s/%s", res.Kind(), res.GetName())
-		path := fmt.Sprintf("/%s/exported/%s/resources/%s", NS, hostname, uid)
-
-		if match(res, resourceList) { // if we match, no need to delete!
-			continue
-		}
-
-		ops = append(ops, etcd.OpDelete(path))
-
-		hasDeletes = true
-	}
-
-	// if everything is already correct, do nothing, otherwise, run the ops!
-	// it's important to do this in one transaction, and atomically, because
-	// this way, we only generate one watch event, and only when it's needed
-	if hasDeletes { // always run, ifs don't matter
-		_, err = obj.Txn(nil, ops, nil) // TODO: does this run? it should!
-	} else {
-		_, err = obj.Txn(ifs, nil, ops) // TODO: do we need to look at response?
-	}
-	return err
-}
-
-// GetResources collects all of the resources which match a filter from etcd
-// If the kindfilter or hostnameFilter is empty, then it assumes no filtering...
-// TODO: Expand this with a more powerful filter based on what we eventually
-// support in our collect DSL. Ideally a server side filter like WithFilter()
-// We could do this if the pattern was /$NS/exported/$kind/$hostname/$uid = $data
-func GetResources(obj *EmbdEtcd, hostnameFilter, kindFilter []string) ([]resources.Res, error) {
-	// key structure is /$NS/exported/$hostname/resources/$uid = $data
-	path := fmt.Sprintf("/%s/exported/", NS)
-	resourceList := []resources.Res{}
-	keyMap, err := obj.Get(path, etcd.WithPrefix(), etcd.WithSort(etcd.SortByKey, etcd.SortAscend))
-	if err != nil {
-		return nil, fmt.Errorf("could not get resources: %v", err)
-	}
-	for key, val := range keyMap {
-		if !strings.HasPrefix(key, path) { // sanity check
-			continue
-		}
-
-		str := strings.Split(key[len(path):], "/")
-		if len(str) != 4 {
-			return nil, fmt.Errorf("unexpected chunk count")
-		}
-		hostname, r, kind, name := str[0], str[1], str[2], str[3]
-		if r != "resources" {
-			return nil, fmt.Errorf("unexpected chunk pattern")
-		}
-		if kind == "" {
-			return nil, fmt.Errorf("unexpected kind chunk")
-		}
-
-		// FIXME: ideally this would be a server side filter instead!
-		if len(hostnameFilter) > 0 && !util.StrInList(hostname, hostnameFilter) {
-			continue
-		}
-
-		// FIXME: ideally this would be a server side filter instead!
-		if len(kindFilter) > 0 && !util.StrInList(kind, kindFilter) {
-			continue
-		}
-
-		if obj, err := resources.B64ToRes(val); err == nil {
-			obj.SetKind(kind) // cheap init
-			log.Printf("Etcd: Get: (Hostname, Kind, Name): (%s, %s, %s)", hostname, kind, name)
-			resourceList = append(resourceList, obj)
-		} else {
-			return nil, fmt.Errorf("can't convert from B64: %v", err)
-		}
-	}
-	return resourceList, nil
-}
-
 //func UrlRemoveScheme(urls etcdtypes.URLs) []string {
 //	strs := []string{}
 //	for _, u := range urls {
@@ -2256,7 +1729,7 @@ func GetResources(obj *EmbdEtcd, hostnameFilter, kindFilter []string) ([]resourc
 //	return strs
 //}

-// ApplyDeltaEvents modifies a URLsMap with the deltas from a WatchResponse
+// ApplyDeltaEvents modifies a URLsMap with the deltas from a WatchResponse.
 func ApplyDeltaEvents(re *RE, urlsmap etcdtypes.URLsMap) (etcdtypes.URLsMap, error) {
 	if re == nil { // passthrough
 		return urlsmap, nil
--- a/etcd/methods.go
+++ b/etcd/methods.go
@@ -0,0 +1,412 @@
+// Mgmt
+// Copyright (C) 2013-2017+ James Shubin and the project contributors
+// Written by James Shubin <james@shubin.ca> and the project contributors
+//
+// This program is free software: you can redistribute it and/or modify
+// it under the terms of the GNU General Public License as published by
+// the Free Software Foundation, either version 3 of the License, or
+// (at your option) any later version.
+//
+// This program is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+// GNU General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License
+// along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+package etcd
+
+import (
+	"fmt"
+	"log"
+	"strconv"
+	"strings"
+
+	etcd "github.com/coreos/etcd/clientv3"
+	rpctypes "github.com/coreos/etcd/etcdserver/api/v3rpc/rpctypes"
+	etcdtypes "github.com/coreos/etcd/pkg/types"
+	context "golang.org/x/net/context"
+)
+
+// TODO: Could all these Etcd*(obj *EmbdEtcd, ...) functions which deal with the
+// interface between etcd paths and behaviour be grouped into a single struct ?
+
+// Nominate nominates a particular client to be a server (peer).
+func Nominate(obj *EmbdEtcd, hostname string, urls etcdtypes.URLs) error {
+	if obj.flags.Trace {
+		log.Printf("Trace: Etcd: Nominate(%v): %v", hostname, urls.String())
+		defer log.Printf("Trace: Etcd: Nominate(%v): Finished!", hostname)
+	}
+	// nominate someone to be a server
+	nominate := fmt.Sprintf("/%s/nominated/%s", NS, hostname)
+	ops := []etcd.Op{} // list of ops in this txn
+	if urls != nil {
+		ops = append(ops, etcd.OpPut(nominate, urls.String())) // TODO: add a TTL? (etcd.WithLease)
+
+	} else { // delete message if set to erase
+		ops = append(ops, etcd.OpDelete(nominate))
+	}
+
+	if _, err := obj.Txn(nil, ops, nil); err != nil {
+		return fmt.Errorf("nominate failed") // exit in progress?
+	}
+	return nil
+}
+
+// Nominated returns a urls map of nominated etcd server volunteers.
+// NOTE: I know 'nominees' might be more correct, but is less consistent here
+func Nominated(obj *EmbdEtcd) (etcdtypes.URLsMap, error) {
+	path := fmt.Sprintf("/%s/nominated/", NS)
+	keyMap, err := obj.Get(path, etcd.WithPrefix()) // map[string]string, bool
+	if err != nil {
+		return nil, fmt.Errorf("nominated isn't available: %v", err)
+	}
+	nominated := make(etcdtypes.URLsMap)
+	for key, val := range keyMap { // loop through directory of nominated
+		if !strings.HasPrefix(key, path) {
+			continue
+		}
+		name := key[len(path):] // get name of nominee
+		if val == "" {          // skip "erased" values
+			continue
+		}
+		urls, err := etcdtypes.NewURLs(strings.Split(val, ","))
+		if err != nil {
+			return nil, fmt.Errorf("nominated data format error: %v", err)
+		}
+		nominated[name] = urls // add to map
+		if obj.flags.Debug {
+			log.Printf("Etcd: Nominated(%v): %v", name, val)
+		}
+	}
+	return nominated, nil
+}
+
+// Volunteer offers yourself up to be a server if needed.
+func Volunteer(obj *EmbdEtcd, urls etcdtypes.URLs) error {
+	if obj.flags.Trace {
+		log.Printf("Trace: Etcd: Volunteer(%v): %v", obj.hostname, urls.String())
+		defer log.Printf("Trace: Etcd: Volunteer(%v): Finished!", obj.hostname)
+	}
+	// volunteer to be a server
+	volunteer := fmt.Sprintf("/%s/volunteers/%s", NS, obj.hostname)
+	ops := []etcd.Op{} // list of ops in this txn
+	if urls != nil {
+		// XXX: adding a TTL is crucial! (i think)
+		ops = append(ops, etcd.OpPut(volunteer, urls.String())) // value is usually a peer "serverURL"
+
+	} else { // delete message if set to erase
+		ops = append(ops, etcd.OpDelete(volunteer))
+	}
+
+	if _, err := obj.Txn(nil, ops, nil); err != nil {
+		return fmt.Errorf("volunteering failed") // exit in progress?
+	}
+	return nil
+}
+
+// Volunteers returns a urls map of available etcd server volunteers.
+func Volunteers(obj *EmbdEtcd) (etcdtypes.URLsMap, error) {
+	if obj.flags.Trace {
+		log.Printf("Trace: Etcd: Volunteers()")
+		defer log.Printf("Trace: Etcd: Volunteers(): Finished!")
+	}
+	path := fmt.Sprintf("/%s/volunteers/", NS)
+	keyMap, err := obj.Get(path, etcd.WithPrefix())
+	if err != nil {
+		return nil, fmt.Errorf("volunteers aren't available: %v", err)
+	}
+	volunteers := make(etcdtypes.URLsMap)
+	for key, val := range keyMap { // loop through directory of volunteers
+		if !strings.HasPrefix(key, path) {
+			continue
+		}
+		name := key[len(path):] // get name of volunteer
+		if val == "" {          // skip "erased" values
+			continue
+		}
+		urls, err := etcdtypes.NewURLs(strings.Split(val, ","))
+		if err != nil {
+			return nil, fmt.Errorf("volunteers data format error: %v", err)
+		}
+		volunteers[name] = urls // add to map
+		if obj.flags.Debug {
+			log.Printf("Etcd: Volunteer(%v): %v", name, val)
+		}
+	}
+	return volunteers, nil
+}
+
+// AdvertiseEndpoints advertises the list of available client endpoints.
+func AdvertiseEndpoints(obj *EmbdEtcd, urls etcdtypes.URLs) error {
+	if obj.flags.Trace {
+		log.Printf("Trace: Etcd: AdvertiseEndpoints(%v): %v", obj.hostname, urls.String())
+		defer log.Printf("Trace: Etcd: AdvertiseEndpoints(%v): Finished!", obj.hostname)
+	}
+	// advertise endpoints
+	endpoints := fmt.Sprintf("/%s/endpoints/%s", NS, obj.hostname)
+	ops := []etcd.Op{} // list of ops in this txn
+	if urls != nil {
+		// TODO: add a TTL? (etcd.WithLease)
+		ops = append(ops, etcd.OpPut(endpoints, urls.String())) // value is usually a "clientURL"
+
+	} else { // delete message if set to erase
+		ops = append(ops, etcd.OpDelete(endpoints))
+	}
+
+	if _, err := obj.Txn(nil, ops, nil); err != nil {
+		return fmt.Errorf("endpoint advertising failed") // exit in progress?
+	}
+	return nil
+}
+
+// Endpoints returns a urls map of available etcd server endpoints.
+func Endpoints(obj *EmbdEtcd) (etcdtypes.URLsMap, error) {
+	if obj.flags.Trace {
+		log.Printf("Trace: Etcd: Endpoints()")
+		defer log.Printf("Trace: Etcd: Endpoints(): Finished!")
+	}
+	path := fmt.Sprintf("/%s/endpoints/", NS)
+	keyMap, err := obj.Get(path, etcd.WithPrefix())
+	if err != nil {
+		return nil, fmt.Errorf("endpoints aren't available: %v", err)
+	}
+	endpoints := make(etcdtypes.URLsMap)
+	for key, val := range keyMap { // loop through directory of endpoints
+		if !strings.HasPrefix(key, path) {
+			continue
+		}
+		name := key[len(path):] // get name of volunteer
+		if val == "" {          // skip "erased" values
+			continue
+		}
+		urls, err := etcdtypes.NewURLs(strings.Split(val, ","))
+		if err != nil {
+			return nil, fmt.Errorf("endpoints data format error: %v", err)
+		}
+		endpoints[name] = urls // add to map
+		if obj.flags.Debug {
+			log.Printf("Etcd: Endpoint(%v): %v", name, val)
+		}
+	}
+	return endpoints, nil
+}
+
+// SetHostnameConverged sets whether a specific hostname is converged.
+func SetHostnameConverged(obj *EmbdEtcd, hostname string, isConverged bool) error {
+	if obj.flags.Trace {
+		log.Printf("Trace: Etcd: SetHostnameConverged(%s): %v", hostname, isConverged)
+		defer log.Printf("Trace: Etcd: SetHostnameConverged(%v): Finished!", hostname)
+	}
+	converged := fmt.Sprintf("/%s/converged/%s", NS, hostname)
+	op := []etcd.Op{etcd.OpPut(converged, fmt.Sprintf("%t", isConverged))}
+	if _, err := obj.Txn(nil, op, nil); err != nil { // TODO: do we need a skipConv flag here too?
+		return fmt.Errorf("set converged failed") // exit in progress?
+	}
+	return nil
+}
+
+// HostnameConverged returns a map of every hostname's converged state.
+func HostnameConverged(obj *EmbdEtcd) (map[string]bool, error) {
+	if obj.flags.Trace {
+		log.Printf("Trace: Etcd: HostnameConverged()")
+		defer log.Printf("Trace: Etcd: HostnameConverged(): Finished!")
+	}
+	path := fmt.Sprintf("/%s/converged/", NS)
+	keyMap, err := obj.ComplexGet(path, true, etcd.WithPrefix()) // don't un-converge
+	if err != nil {
+		return nil, fmt.Errorf("converged values aren't available: %v", err)
+	}
+	converged := make(map[string]bool)
+	for key, val := range keyMap { // loop through directory...
+		if !strings.HasPrefix(key, path) {
+			continue
+		}
+		name := key[len(path):] // get name of key
+		if val == "" {          // skip "erased" values
+			continue
+		}
+		b, err := strconv.ParseBool(val)
+		if err != nil {
+			return nil, fmt.Errorf("converged data format error: %v", err)
+		}
+		converged[name] = b // add to map
+	}
+	return converged, nil
+}
+
+// AddHostnameConvergedWatcher adds a watcher with a callback that runs on
+// hostname state changes.
+func AddHostnameConvergedWatcher(obj *EmbdEtcd, callbackFn func(map[string]bool) error) (func(), error) {
+	path := fmt.Sprintf("/%s/converged/", NS)
+	internalCbFn := func(re *RE) error {
+		// TODO: get the value from the response, and apply delta...
+		// for now, just run a get operation which is easier to code!
+		m, err := HostnameConverged(obj)
+		if err != nil {
+			return err
+		}
+		return callbackFn(m) // call my function
+	}
+	return obj.AddWatcher(path, internalCbFn, true, true, etcd.WithPrefix()) // no block and no converger reset
+}
+
+// SetClusterSize sets the ideal target cluster size of etcd peers.
+func SetClusterSize(obj *EmbdEtcd, value uint16) error {
+	if obj.flags.Trace {
+		log.Printf("Trace: Etcd: SetClusterSize(): %v", value)
+		defer log.Printf("Trace: Etcd: SetClusterSize(): Finished!")
+	}
+	key := fmt.Sprintf("/%s/idealClusterSize", NS)
+
+	if err := obj.Set(key, strconv.FormatUint(uint64(value), 10)); err != nil {
+		return fmt.Errorf("function SetClusterSize failed: %v", err) // exit in progress?
+	}
+	return nil
+}
+
+// GetClusterSize gets the ideal target cluster size of etcd peers.
+func GetClusterSize(obj *EmbdEtcd) (uint16, error) {
+	key := fmt.Sprintf("/%s/idealClusterSize", NS)
+	keyMap, err := obj.Get(key)
+	if err != nil {
+		return 0, fmt.Errorf("function GetClusterSize failed: %v", err)
+	}
+
+	val, exists := keyMap[key]
+	if !exists || val == "" {
+		return 0, fmt.Errorf("function GetClusterSize failed: %v", err)
+	}
+
+	v, err := strconv.ParseUint(val, 10, 16)
+	if err != nil {
+		return 0, fmt.Errorf("function GetClusterSize failed: %v", err)
+	}
+	return uint16(v), nil
+}
+
+// MemberAdd adds a member to the cluster.
+func MemberAdd(obj *EmbdEtcd, peerURLs etcdtypes.URLs) (*etcd.MemberAddResponse, error) {
+	//obj.Connect(false) // TODO: ?
+	ctx := context.Background()
+	var response *etcd.MemberAddResponse
+	var err error
+	for {
+		if obj.exiting { // the exit signal has been sent!
+			return nil, fmt.Errorf("exiting etcd")
+		}
+		obj.rLock.RLock()
+		response, err = obj.client.MemberAdd(ctx, peerURLs.StringSlice())
+		obj.rLock.RUnlock()
+		if err == nil {
+			break
+		}
+		if ctx, err = obj.CtxError(ctx, err); err != nil {
+			return nil, err
+		}
+	}
+	return response, nil
+}
+
+// MemberRemove removes a member by mID and returns if it worked, and also
+// if there was an error. This is because it might have run without error, but
+// the member wasn't found, for example.
+func MemberRemove(obj *EmbdEtcd, mID uint64) (bool, error) {
+	//obj.Connect(false) // TODO: ?
+	ctx := context.Background()
+	for {
+		if obj.exiting { // the exit signal has been sent!
+			return false, fmt.Errorf("exiting etcd")
+		}
+		obj.rLock.RLock()
+		_, err := obj.client.MemberRemove(ctx, mID)
+		obj.rLock.RUnlock()
+		if err == nil {
+			break
+		} else if err == rpctypes.ErrMemberNotFound {
+			// if we get this, member already shut itself down :)
+			return false, nil
+		}
+		if ctx, err = obj.CtxError(ctx, err); err != nil {
+			return false, err
+		}
+	}
+	return true, nil
+}
+
+// Members returns information on cluster membership.
+// The member ID's are the keys, because an empty names means unstarted!
+// TODO: consider queueing this through the main loop with CtxError(ctx, err)
+func Members(obj *EmbdEtcd) (map[uint64]string, error) {
+	//obj.Connect(false) // TODO: ?
+	ctx := context.Background()
+	var response *etcd.MemberListResponse
+	var err error
+	for {
+		if obj.exiting { // the exit signal has been sent!
+			return nil, fmt.Errorf("exiting etcd")
+		}
+		obj.rLock.RLock()
+		if obj.flags.Trace {
+			log.Printf("Trace: Etcd: Members(): Endpoints are: %v", obj.client.Endpoints())
+		}
+		response, err = obj.client.MemberList(ctx)
+		obj.rLock.RUnlock()
+		if err == nil {
+			break
+		}
+		if ctx, err = obj.CtxError(ctx, err); err != nil {
+			return nil, err
+		}
+	}
+
+	members := make(map[uint64]string)
+	for _, x := range response.Members {
+		members[x.ID] = x.Name // x.Name will be "" if unstarted!
+	}
+	return members, nil
+}
+
+// Leader returns the current leader of the etcd server cluster.
+func Leader(obj *EmbdEtcd) (string, error) {
+	//obj.Connect(false) // TODO: ?
+	var err error
+	membersMap := make(map[uint64]string)
+	if membersMap, err = Members(obj); err != nil {
+		return "", err
+	}
+	addresses := obj.LocalhostClientURLs() // heuristic, but probably correct
+	if len(addresses) == 0 {
+		// probably a programming error...
+		return "", fmt.Errorf("programming error")
+	}
+	endpoint := addresses[0].Host // FIXME: arbitrarily picked the first one
+
+	// part two
+	ctx := context.Background()
+	var response *etcd.StatusResponse
+	for {
+		if obj.exiting { // the exit signal has been sent!
+			return "", fmt.Errorf("exiting etcd")
+		}
+
+		obj.rLock.RLock()
+		response, err = obj.client.Maintenance.Status(ctx, endpoint)
+		obj.rLock.RUnlock()
+		if err == nil {
+			break
+		}
+		if ctx, err = obj.CtxError(ctx, err); err != nil {
+			return "", err
+		}
+	}
+
+	// isLeader: response.Header.MemberId == response.Leader
+	for id, name := range membersMap {
+		if id == response.Leader {
+			return name, nil
+		}
+	}
+	return "", fmt.Errorf("members map is not current") // not found
+}
--- a/etcd/resources.go
+++ b/etcd/resources.go
@@ -0,0 +1,181 @@
+// Mgmt
+// Copyright (C) 2013-2017+ James Shubin and the project contributors
+// Written by James Shubin <james@shubin.ca> and the project contributors
+//
+// This program is free software: you can redistribute it and/or modify
+// it under the terms of the GNU General Public License as published by
+// the Free Software Foundation, either version 3 of the License, or
+// (at your option) any later version.
+//
+// This program is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+// GNU General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License
+// along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+package etcd
+
+import (
+	"fmt"
+	"log"
+	"strings"
+
+	"github.com/purpleidea/mgmt/resources"
+	"github.com/purpleidea/mgmt/util"
+
+	etcd "github.com/coreos/etcd/clientv3"
+)
+
+// WatchResources returns a channel that outputs events when exported resources
+// change.
+// TODO: Filter our watch (on the server side if possible) based on the
+// collection prefixes and filters that we care about...
+func WatchResources(obj *EmbdEtcd) chan error {
+	ch := make(chan error, 1) // buffer it so we can measure it
+	path := fmt.Sprintf("/%s/exported/", NS)
+	callback := func(re *RE) error {
+		// TODO: is this even needed? it used to happen on conn errors
+		log.Printf("Etcd: Watch: Path: %v", path) // event
+		if re == nil || re.response.Canceled {
+			return fmt.Errorf("watch is empty") // will cause a CtxError+retry
+		}
+		// we normally need to check if anything changed since the last
+		// event, since a set (export) with no changes still causes the
+		// watcher to trigger and this would cause an infinite loop. we
+		// don't need to do this check anymore because we do the export
+		// transactionally, and only if a change is needed. since it is
+		// atomic, all the changes arrive together which avoids dupes!!
+		if len(ch) == 0 { // send event only if one isn't pending
+			// this check avoids multiple events all queueing up and then
+			// being released continuously long after the changes stopped
+			// do not block!
+			ch <- nil // event
+		}
+		return nil
+	}
+	_, _ = obj.AddWatcher(path, callback, true, false, etcd.WithPrefix()) // no need to check errors
+	return ch
+}
+
+// SetResources exports all of the resources which we pass in to etcd.
+func SetResources(obj *EmbdEtcd, hostname string, resourceList []resources.Res) error {
+	// key structure is /$NS/exported/$hostname/resources/$uid = $data
+
+	var kindFilter []string // empty to get from everyone
+	hostnameFilter := []string{hostname}
+	// this is not a race because we should only be reading keys which we
+	// set, and there should not be any contention with other hosts here!
+	originals, err := GetResources(obj, hostnameFilter, kindFilter)
+	if err != nil {
+		return err
+	}
+
+	if len(originals) == 0 && len(resourceList) == 0 { // special case of no add or del
+		return nil
+	}
+
+	ifs := []etcd.Cmp{} // list matching the desired state
+	ops := []etcd.Op{}  // list of ops in this transaction
+	for _, res := range resourceList {
+		if res.GetKind() == "" {
+			log.Fatalf("Etcd: SetResources: Error: Empty kind: %v", res.GetName())
+		}
+		uid := fmt.Sprintf("%s/%s", res.GetKind(), res.GetName())
+		path := fmt.Sprintf("/%s/exported/%s/resources/%s", NS, hostname, uid)
+		if data, err := resources.ResToB64(res); err == nil {
+			ifs = append(ifs, etcd.Compare(etcd.Value(path), "=", data)) // desired state
+			ops = append(ops, etcd.OpPut(path, data))
+		} else {
+			return fmt.Errorf("can't convert to B64: %v", err)
+		}
+	}
+
+	match := func(res resources.Res, resourceList []resources.Res) bool { // helper lambda
+		for _, x := range resourceList {
+			if res.GetKind() == x.GetKind() && res.GetName() == x.GetName() {
+				return true
+			}
+		}
+		return false
+	}
+
+	hasDeletes := false
+	// delete old, now unused resources here...
+	for _, res := range originals {
+		if res.GetKind() == "" {
+			log.Fatalf("Etcd: SetResources: Error: Empty kind: %v", res.GetName())
+		}
+		uid := fmt.Sprintf("%s/%s", res.GetKind(), res.GetName())
+		path := fmt.Sprintf("/%s/exported/%s/resources/%s", NS, hostname, uid)
+
+		if match(res, resourceList) { // if we match, no need to delete!
+			continue
+		}
+
+		ops = append(ops, etcd.OpDelete(path))
+
+		hasDeletes = true
+	}
+
+	// if everything is already correct, do nothing, otherwise, run the ops!
+	// it's important to do this in one transaction, and atomically, because
+	// this way, we only generate one watch event, and only when it's needed
+	if hasDeletes { // always run, ifs don't matter
+		_, err = obj.Txn(nil, ops, nil) // TODO: does this run? it should!
+	} else {
+		_, err = obj.Txn(ifs, nil, ops) // TODO: do we need to look at response?
+	}
+	return err
+}
+
+// GetResources collects all of the resources which match a filter from etcd.
+// If the kindfilter or hostnameFilter is empty, then it assumes no filtering...
+// TODO: Expand this with a more powerful filter based on what we eventually
+// support in our collect DSL. Ideally a server side filter like WithFilter()
+// We could do this if the pattern was /$NS/exported/$kind/$hostname/$uid = $data.
+func GetResources(obj *EmbdEtcd, hostnameFilter, kindFilter []string) ([]resources.Res, error) {
+	// key structure is /$NS/exported/$hostname/resources/$uid = $data
+	path := fmt.Sprintf("/%s/exported/", NS)
+	resourceList := []resources.Res{}
+	keyMap, err := obj.Get(path, etcd.WithPrefix(), etcd.WithSort(etcd.SortByKey, etcd.SortAscend))
+	if err != nil {
+		return nil, fmt.Errorf("could not get resources: %v", err)
+	}
+	for key, val := range keyMap {
+		if !strings.HasPrefix(key, path) { // sanity check
+			continue
+		}
+
+		str := strings.Split(key[len(path):], "/")
+		if len(str) != 4 {
+			return nil, fmt.Errorf("unexpected chunk count")
+		}
+		hostname, r, kind, name := str[0], str[1], str[2], str[3]
+		if r != "resources" {
+			return nil, fmt.Errorf("unexpected chunk pattern")
+		}
+		if kind == "" {
+			return nil, fmt.Errorf("unexpected kind chunk")
+		}
+
+		// FIXME: ideally this would be a server side filter instead!
+		if len(hostnameFilter) > 0 && !util.StrInList(hostname, hostnameFilter) {
+			continue
+		}
+
+		// FIXME: ideally this would be a server side filter instead!
+		if len(kindFilter) > 0 && !util.StrInList(kind, kindFilter) {
+			continue
+		}
+
+		if obj, err := resources.B64ToRes(val); err == nil {
+			log.Printf("Etcd: Get: (Hostname, Kind, Name): (%s, %s, %s)", hostname, kind, name)
+			resourceList = append(resourceList, obj)
+		} else {
+			return nil, fmt.Errorf("can't convert from B64: %v", err)
+		}
+	}
+	return resourceList, nil
+}
--- a/etcd/str.go
+++ b/etcd/str.go
@@ -0,0 +1,105 @@
+// Mgmt
+// Copyright (C) 2013-2017+ James Shubin and the project contributors
+// Written by James Shubin <james@shubin.ca> and the project contributors
+//
+// This program is free software: you can redistribute it and/or modify
+// it under the terms of the GNU General Public License as published by
+// the Free Software Foundation, either version 3 of the License, or
+// (at your option) any later version.
+//
+// This program is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+// GNU General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License
+// along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+package etcd
+
+import (
+	"errors"
+	"fmt"
+
+	etcd "github.com/coreos/etcd/clientv3"
+	errwrap "github.com/pkg/errors"
+)
+
+// ErrNotExist is returned when GetStr can not find the requested key.
+// TODO: https://dave.cheney.net/2016/04/07/constant-errors
+var ErrNotExist = errors.New("errNotExist")
+
+// WatchStr returns a channel which spits out events on key activity.
+// FIXME: It should close the channel when it's done, and spit out errors when
+// something goes wrong.
+func WatchStr(obj *EmbdEtcd, key string) chan error {
+	// new key structure is /$NS/strings/$key = $data
+	path := fmt.Sprintf("/%s/strings/%s", NS, key)
+	ch := make(chan error, 1)
+	// FIXME: fix our API so that we get a close event on shutdown.
+	callback := func(re *RE) error {
+		// TODO: is this even needed? it used to happen on conn errors
+		//log.Printf("Etcd: Watch: Path: %v", path) // event
+		if re == nil || re.response.Canceled {
+			return fmt.Errorf("watch is empty") // will cause a CtxError+retry
+		}
+		if len(ch) == 0 { // send event only if one isn't pending
+			ch <- nil // event
+		}
+		return nil
+	}
+	_, _ = obj.AddWatcher(path, callback, true, false, etcd.WithPrefix()) // no need to check errors
+	return ch
+}
+
+// GetStr collects the string which matches a global namespace in etcd.
+func GetStr(obj *EmbdEtcd, key string) (string, error) {
+	// new key structure is /$NS/strings/$key = $data
+	path := fmt.Sprintf("/%s/strings/%s", NS, key)
+	keyMap, err := obj.Get(path, etcd.WithPrefix())
+	if err != nil {
+		return "", errwrap.Wrapf(err, "could not get strings in: %s", key)
+	}
+
+	if len(keyMap) == 0 {
+		return "", ErrNotExist
+	}
+
+	if count := len(keyMap); count != 1 {
+		return "", fmt.Errorf("returned %d entries", count)
+	}
+
+	val, exists := keyMap[path]
+	if !exists {
+		return "", fmt.Errorf("path `%s` is missing", path)
+	}
+
+	//log.Printf("Etcd: GetStr(%s): %s", key, val)
+	return val, nil
+}
+
+// SetStr sets a key and hostname pair to a certain value. If the value is
+// nil, then it deletes the key. Otherwise the value should point to a string.
+// TODO: TTL or delete disconnect?
+func SetStr(obj *EmbdEtcd, key string, data *string) error {
+	// key structure is /$NS/strings/$key = $data
+	path := fmt.Sprintf("/%s/strings/%s", NS, key)
+	ifs := []etcd.Cmp{} // list matching the desired state
+	ops := []etcd.Op{}  // list of ops in this transaction (then)
+	els := []etcd.Op{}  // list of ops in this transaction (else)
+	if data == nil {    // perform a delete
+		// TODO: use https://github.com/coreos/etcd/pull/7417 if merged
+		//ifs = append(ifs, etcd.KeyExists(path))
+		ifs = append(ifs, etcd.Compare(etcd.Version(path), ">", 0))
+		ops = append(ops, etcd.OpDelete(path))
+	} else {
+		data := *data                                                // get the real value
+		ifs = append(ifs, etcd.Compare(etcd.Value(path), "=", data)) // desired state
+		els = append(els, etcd.OpPut(path, data))
+	}
+
+	// it's important to do this in one transaction, and atomically, because
+	// this way, we only generate one watch event, and only when it's needed
+	_, err := obj.Txn(ifs, ops, els) // TODO: do we need to look at response?
+	return errwrap.Wrapf(err, "could not set strings in: %s", key)
+}
--- a/etcd/strmap.go
+++ b/etcd/strmap.go
@@ -0,0 +1,115 @@
+// Mgmt
+// Copyright (C) 2013-2017+ James Shubin and the project contributors
+// Written by James Shubin <james@shubin.ca> and the project contributors
+//
+// This program is free software: you can redistribute it and/or modify
+// it under the terms of the GNU General Public License as published by
+// the Free Software Foundation, either version 3 of the License, or
+// (at your option) any later version.
+//
+// This program is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+// GNU General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License
+// along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+package etcd
+
+import (
+	"fmt"
+	"strings"
+
+	"github.com/purpleidea/mgmt/util"
+
+	etcd "github.com/coreos/etcd/clientv3"
+	errwrap "github.com/pkg/errors"
+)
+
+// WatchStrMap returns a channel which spits out events on key activity.
+// FIXME: It should close the channel when it's done, and spit out errors when
+// something goes wrong.
+func WatchStrMap(obj *EmbdEtcd, key string) chan error {
+	// new key structure is /$NS/strings/$key/$hostname = $data
+	path := fmt.Sprintf("/%s/strings/%s", NS, key)
+	ch := make(chan error, 1)
+	// FIXME: fix our API so that we get a close event on shutdown.
+	callback := func(re *RE) error {
+		// TODO: is this even needed? it used to happen on conn errors
+		//log.Printf("Etcd: Watch: Path: %v", path) // event
+		if re == nil || re.response.Canceled {
+			return fmt.Errorf("watch is empty") // will cause a CtxError+retry
+		}
+		if len(ch) == 0 { // send event only if one isn't pending
+			ch <- nil // event
+		}
+		return nil
+	}
+	_, _ = obj.AddWatcher(path, callback, true, false, etcd.WithPrefix()) // no need to check errors
+	return ch
+}
+
+// GetStrMap collects all of the strings which match a namespace in etcd.
+func GetStrMap(obj *EmbdEtcd, hostnameFilter []string, key string) (map[string]string, error) {
+	// old key structure is /$NS/strings/$hostname/$key = $data
+	// new key structure is /$NS/strings/$key/$hostname = $data
+	// FIXME: if we have the $key as the last token (old key structure), we
+	// can allow the key to contain the slash char, otherwise we need to
+	// verify that one isn't present in the input string.
+	path := fmt.Sprintf("/%s/strings/%s", NS, key)
+	keyMap, err := obj.Get(path, etcd.WithPrefix(), etcd.WithSort(etcd.SortByKey, etcd.SortAscend))
+	if err != nil {
+		return nil, errwrap.Wrapf(err, "could not get strings in: %s", key)
+	}
+	result := make(map[string]string)
+	for key, val := range keyMap {
+		if !strings.HasPrefix(key, path) { // sanity check
+			continue
+		}
+
+		str := strings.Split(key[len(path):], "/")
+		if len(str) != 2 {
+			return nil, fmt.Errorf("unexpected chunk count of %d", len(str))
+		}
+		_, hostname := str[0], str[1]
+
+		if hostname == "" {
+			return nil, fmt.Errorf("unexpected chunk length of %d", len(hostname))
+		}
+
+		// FIXME: ideally this would be a server side filter instead!
+		if len(hostnameFilter) > 0 && !util.StrInList(hostname, hostnameFilter) {
+			continue
+		}
+		//log.Printf("Etcd: GetStr(%s): (Hostname, Data): (%s, %s)", key, hostname, val)
+		result[hostname] = val
+	}
+	return result, nil
+}
+
+// SetStrMap sets a key and hostname pair to a certain value. If the value is
+// nil, then it deletes the key. Otherwise the value should point to a string.
+// TODO: TTL or delete disconnect?
+func SetStrMap(obj *EmbdEtcd, hostname, key string, data *string) error {
+	// key structure is /$NS/strings/$key/$hostname = $data
+	path := fmt.Sprintf("/%s/strings/%s/%s", NS, key, hostname)
+	ifs := []etcd.Cmp{} // list matching the desired state
+	ops := []etcd.Op{}  // list of ops in this transaction (then)
+	els := []etcd.Op{}  // list of ops in this transaction (else)
+	if data == nil {    // perform a delete
+		// TODO: use https://github.com/coreos/etcd/pull/7417 if merged
+		//ifs = append(ifs, etcd.KeyExists(path))
+		ifs = append(ifs, etcd.Compare(etcd.Version(path), ">", 0))
+		ops = append(ops, etcd.OpDelete(path))
+	} else {
+		data := *data                                                // get the real value
+		ifs = append(ifs, etcd.Compare(etcd.Value(path), "=", data)) // desired state
+		els = append(els, etcd.OpPut(path, data))
+	}
+
+	// it's important to do this in one transaction, and atomically, because
+	// this way, we only generate one watch event, and only when it's needed
+	_, err := obj.Txn(ifs, ops, els) // TODO: do we need to look at response?
+	return errwrap.Wrapf(err, "could not set strings in: %s", key)
+}
--- a/etcd/world.go
+++ b/etcd/world.go
@@ -3,16 +3,16 @@
 // Written by James Shubin <james@shubin.ca> and the project contributors
 //
 // This program is free software: you can redistribute it and/or modify
-// it under the terms of the GNU Affero General Public License as published by
+// it under the terms of the GNU General Public License as published by
 // the Free Software Foundation, either version 3 of the License, or
 // (at your option) any later version.
 //
 // This program is distributed in the hope that it will be useful,
 // but WITHOUT ANY WARRANTY; without even the implied warranty of
 // MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-// GNU Affero General Public License for more details.
+// GNU General Public License for more details.
 //
-// You should have received a copy of the GNU Affero General Public License
+// You should have received a copy of the GNU General Public License
 // along with this program.  If not, see <http://www.gnu.org/licenses/>.

 package etcd
@@ -27,6 +27,12 @@ type World struct {
 	EmbdEtcd *EmbdEtcd
 }

+// ResWatch returns a channel which spits out events on possible exported
+// resource changes.
+func (obj *World) ResWatch() chan error {
+	return WatchResources(obj.EmbdEtcd)
+}
+
 // ResExport exports a list of resources under our hostname namespace.
 // Subsequent calls replace the previously set collection atomically.
 func (obj *World) ResExport(resourceList []resources.Res) error {
@@ -41,3 +47,49 @@ func (obj *World) ResCollect(hostnameFilter, kindFilter []string) ([]resources.R
 	// enforce that here if the underlying API supported it... Add this?
 	return GetResources(obj.EmbdEtcd, hostnameFilter, kindFilter)
 }
+
+// StrWatch returns a channel which spits out events on possible string changes.
+func (obj *World) StrWatch(namespace string) chan error {
+	return WatchStr(obj.EmbdEtcd, namespace)
+}
+
+// StrIsNotExist returns whether the error from StrGet is a key missing error.
+func (obj *World) StrIsNotExist(err error) bool {
+	return err == ErrNotExist
+}
+
+// StrGet returns the value for the the given namespace.
+func (obj *World) StrGet(namespace string) (string, error) {
+	return GetStr(obj.EmbdEtcd, namespace)
+}
+
+// StrSet sets the namespace value to a particular string.
+func (obj *World) StrSet(namespace, value string) error {
+	return SetStr(obj.EmbdEtcd, namespace, &value)
+}
+
+// StrDel deletes the value in a particular namespace.
+func (obj *World) StrDel(namespace string) error {
+	return SetStr(obj.EmbdEtcd, namespace, nil)
+}
+
+// StrMapWatch returns a channel which spits out events on possible string changes.
+func (obj *World) StrMapWatch(namespace string) chan error {
+	return WatchStrMap(obj.EmbdEtcd, namespace)
+}
+
+// StrMapGet returns a map of hostnames to values in the given namespace.
+func (obj *World) StrMapGet(namespace string) (map[string]string, error) {
+	return GetStrMap(obj.EmbdEtcd, []string{}, namespace)
+}
+
+// StrMapSet sets the namespace value to a particular string under the identity
+// of its own hostname.
+func (obj *World) StrMapSet(namespace, value string) error {
+	return SetStrMap(obj.EmbdEtcd, obj.Hostname, namespace, &value)
+}
+
+// StrMapDel deletes the value in a particular namespace.
+func (obj *World) StrMapDel(namespace string) error {
+	return SetStrMap(obj.EmbdEtcd, obj.Hostname, namespace, nil)
+}
--- a/event/event.go
+++ b/event/event.go
@@ -3,16 +3,16 @@
 // Written by James Shubin <james@shubin.ca> and the project contributors
 //
 // This program is free software: you can redistribute it and/or modify
-// it under the terms of the GNU Affero General Public License as published by
+// it under the terms of the GNU General Public License as published by
 // the Free Software Foundation, either version 3 of the License, or
 // (at your option) any later version.
 //
 // This program is distributed in the hope that it will be useful,
 // but WITHOUT ANY WARRANTY; without even the implied warranty of
 // MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-// GNU Affero General Public License for more details.
+// GNU General Public License for more details.
 //
-// You should have received a copy of the GNU Affero General Public License
+// You should have received a copy of the GNU General Public License
 // along with this program.  If not, see <http://www.gnu.org/licenses/>.

 // Package event provides some primitives that are used for message passing.
--- a/examples/autoedges4.yaml
+++ b/examples/autoedges4.yaml
@@ -0,0 +1,19 @@
+---
+graph: mygraph
+resources:
+  user:
+  - name: edgeuser
+    state: absent
+    gid: 10000
+  - name: edgeuser2
+    state: exists
+    group: edgegroup
+    groups: [edgegroup2, edgegroup3]
+  group:
+  - name: edgegroup
+    state: exists
+    gid: 10000
+  - name: edgegroup2
+    state: exists
+  - name: edgegroup3
+    state: exists
--- a/examples/aws_ec2_1.yaml
+++ b/examples/aws_ec2_1.yaml
@@ -0,0 +1,10 @@
+---
+graph: mygraph
+resources:
+  aws:ec2:
+  - name: ec2example
+    region: ca-central-1
+    type: t2.micro
+    imageid: ami-5ac17f3e
+    state: running
+edges: []
--- a/examples/exec3-sema.yaml
+++ b/examples/exec3-sema.yaml
@@ -0,0 +1,67 @@
+---
+graph: parallel
+resources:
+  exec:
+  - name: pkg10
+    meta:
+      sema: ['mylock:1', 'otherlock:42']
+    cmd: sleep 10s
+    shell: ''
+    timeout: 0
+    watchcmd: ''
+    watchshell: ''
+    ifcmd: ''
+    ifshell: ''
+    pollint: 0
+    state: present
+  - name: svc10
+    meta:
+      sema: ['mylock:1']
+    cmd: sleep 10s
+    shell: ''
+    timeout: 0
+    watchcmd: ''
+    watchshell: ''
+    ifcmd: ''
+    ifshell: ''
+    pollint: 0
+    state: present
+  - name: exec10
+    meta:
+      sema: ['mylock:1']
+    cmd: sleep 10s
+    shell: ''
+    timeout: 0
+    watchcmd: ''
+    watchshell: ''
+    ifcmd: ''
+    ifshell: ''
+    pollint: 0
+    state: present
+  - name: pkg15
+    meta:
+      sema: ['mylock:1', 'otherlock:42']
+    cmd: sleep 15s
+    shell: ''
+    timeout: 0
+    watchcmd: ''
+    watchshell: ''
+    ifcmd: ''
+    ifshell: ''
+    pollint: 0
+    state: present
+edges:
+- name: e1
+  from:
+    kind: exec
+    name: pkg10
+  to:
+    kind: exec
+    name: svc10
+- name: e2
+  from:
+    kind: exec
+    name: svc10
+  to:
+    kind: exec
+    name: exec10
--- a/examples/file0.yaml
+++ b/examples/file0.yaml
@@ -0,0 +1,10 @@
+---
+graph: mygraph
+resources:
+  file:
+  - name: file0
+    path: "/tmp/mgmt/f1"
+    content: |
+      i am f0
+    state: exists
+edges: []
--- a/examples/file4.yaml
+++ b/examples/file4.yaml
@@ -0,0 +1,10 @@
+---
+graph: mygraph
+resources:
+  file:
+  - name: file1
+    path: "/tmp/mgmt/hello"
+    content: |
+      i am a file
+    state: exists
+edges: []
--- a/examples/graph0.hcl
+++ b/examples/graph0.hcl
@@ -0,0 +1,14 @@
+resource "file" "file1" {
+  path = "/tmp/mgmt-hello-world"
+  content = "hello, world"
+  state = "exists"
+  depends_on = ["noop.noop1", "exec.sleep"]
+}
+
+resource "noop" "noop1" {
+  test = "nil"
+}
+
+resource "exec" "sleep" {
+ cmd = "sleep 10s"
+}
--- a/examples/graph1.hcl
+++ b/examples/graph1.hcl
@@ -0,0 +1,4 @@
+resource "exec" "exec1" {
+  cmd = "cat /tmp/mgmt-hello-world"
+  state =  "present"
+}
--- a/examples/group1.yaml
+++ b/examples/group1.yaml
@@ -0,0 +1,8 @@
+---
+graph: mygraph
+resources:
+  group:
+  - name: testgroup
+    state: exists
+    gid: 10000
+edges: []
--- a/examples/hil.hcl
+++ b/examples/hil.hcl
@@ -0,0 +1,9 @@
+resource "file" "file1" {
+  path = "/tmp/mgmt-hello-world"
+  content = "${exec.sleep.Output}"
+  state = "exists"
+}
+
+resource "exec" "sleep" {
+ cmd = "echo hello"
+}
--- a/examples/kv1.yaml
+++ b/examples/kv1.yaml
@@ -0,0 +1,8 @@
+---
+graph: mygraph
+resources:
+  kv:
+  - name: kv1
+    key: "hello"
+    value: "world"
+edges: []
--- a/examples/kv2.yaml
+++ b/examples/kv2.yaml
@@ -0,0 +1,7 @@
+---
+graph: mygraph
+resources:
+  kv:
+  - name: kv1
+    key: "iamdeleted"
+edges: []
--- a/examples/kv3.yaml
+++ b/examples/kv3.yaml
@@ -0,0 +1,9 @@
+---
+graph: mygraph
+resources:
+  kv:
+  - name: kv1
+    key: "stage"
+    value: "3"
+    skiplessthan: true
+edges: []
--- a/examples/kv4.yaml
+++ b/examples/kv4.yaml
@@ -0,0 +1,31 @@
+---
+graph: mygraph
+resources:
+  kv:
+  - name: kv1
+    key: "stage"
+    value: "1"
+    skiplessthan: true
+  - name: kv2
+    key: "stage"
+    value: "2"
+    skiplessthan: true
+  - name: kv3
+    key: "stage"
+    value: "3"
+    skiplessthan: true
+edges:
+- name: e1
+  from:
+    kind: kv
+    name: kv1
+  to:
+    kind: kv
+    name: kv2
+- name: e2
+  from:
+    kind: kv
+    name: kv2
+  to:
+    kind: kv
+    name: kv3
--- a/examples/lib/exec-send-recv.go
+++ b/examples/lib/exec-send-recv.go
@@ -0,0 +1,246 @@
+// libmgmt example of send->recv
+package main
+
+import (
+	"fmt"
+	"log"
+	"os"
+	"os/signal"
+	"sync"
+	"syscall"
+	"time"
+
+	"github.com/purpleidea/mgmt/gapi"
+	mgmt "github.com/purpleidea/mgmt/lib"
+	"github.com/purpleidea/mgmt/pgraph"
+	"github.com/purpleidea/mgmt/resources"
+)
+
+// MyGAPI implements the main GAPI interface.
+type MyGAPI struct {
+	Name     string // graph name
+	Interval uint   // refresh interval, 0 to never refresh
+
+	data        gapi.Data
+	initialized bool
+	closeChan   chan struct{}
+	wg          sync.WaitGroup // sync group for tunnel go routines
+}
+
+// NewMyGAPI creates a new MyGAPI struct and calls Init().
+func NewMyGAPI(data gapi.Data, name string, interval uint) (*MyGAPI, error) {
+	obj := &MyGAPI{
+		Name:     name,
+		Interval: interval,
+	}
+	return obj, obj.Init(data)
+}
+
+// Init initializes the MyGAPI struct.
+func (obj *MyGAPI) Init(data gapi.Data) error {
+	if obj.initialized {
+		return fmt.Errorf("already initialized")
+	}
+	if obj.Name == "" {
+		return fmt.Errorf("the graph name must be specified")
+	}
+	obj.data = data // store for later
+	obj.closeChan = make(chan struct{})
+	obj.initialized = true
+	return nil
+}
+
+// Graph returns a current Graph.
+func (obj *MyGAPI) Graph() (*pgraph.Graph, error) {
+	if !obj.initialized {
+		return nil, fmt.Errorf("libmgmt: MyGAPI is not initialized")
+	}
+
+	g, err := pgraph.NewGraph(obj.Name)
+	if err != nil {
+		return nil, err
+	}
+
+	metaparams := resources.DefaultMetaParams
+
+	exec1 := &resources.ExecRes{
+		BaseRes: resources.BaseRes{
+			Name:       "exec1",
+			Kind:       "exec",
+			MetaParams: metaparams,
+		},
+		Cmd:   "echo hello world && echo goodbye world 1>&2", // to stdout && stderr
+		Shell: "/bin/bash",
+	}
+	g.AddVertex(exec1)
+
+	output := &resources.FileRes{
+		BaseRes: resources.BaseRes{
+			Name:       "output",
+			Kind:       "file",
+			MetaParams: metaparams,
+			// send->recv!
+			Recv: map[string]*resources.Send{
+				"Content": {Res: exec1, Key: "Output"},
+			},
+		},
+		Path:  "/tmp/mgmt/output",
+		State: "present",
+	}
+	g.AddVertex(output)
+	g.AddEdge(exec1, output, &resources.Edge{Name: "e0"})
+
+	stdout := &resources.FileRes{
+		BaseRes: resources.BaseRes{
+			Name:       "stdout",
+			Kind:       "file",
+			MetaParams: metaparams,
+			// send->recv!
+			Recv: map[string]*resources.Send{
+				"Content": {Res: exec1, Key: "Stdout"},
+			},
+		},
+		Path:  "/tmp/mgmt/stdout",
+		State: "present",
+	}
+	g.AddVertex(stdout)
+	g.AddEdge(exec1, stdout, &resources.Edge{Name: "e1"})
+
+	stderr := &resources.FileRes{
+		BaseRes: resources.BaseRes{
+			Name:       "stderr",
+			Kind:       "file",
+			MetaParams: metaparams,
+			// send->recv!
+			Recv: map[string]*resources.Send{
+				"Content": {Res: exec1, Key: "Stderr"},
+			},
+		},
+		Path:  "/tmp/mgmt/stderr",
+		State: "present",
+	}
+	g.AddVertex(stderr)
+	g.AddEdge(exec1, stderr, &resources.Edge{Name: "e2"})
+
+	//g, err := config.NewGraphFromConfig(obj.data.Hostname, obj.data.World, obj.data.Noop)
+	return g, nil
+}
+
+// Next returns nil errors every time there could be a new graph.
+func (obj *MyGAPI) Next() chan gapi.Next {
+	ch := make(chan gapi.Next)
+	obj.wg.Add(1)
+	go func() {
+		defer obj.wg.Done()
+		defer close(ch) // this will run before the obj.wg.Done()
+		if !obj.initialized {
+			next := gapi.Next{
+				Err:  fmt.Errorf("libmgmt: MyGAPI is not initialized"),
+				Exit: true, // exit, b/c programming error?
+			}
+			ch <- next
+			return
+		}
+		startChan := make(chan struct{}) // start signal
+		close(startChan)                 // kick it off!
+
+		ticker := make(<-chan time.Time)
+		if obj.data.NoStreamWatch || obj.Interval <= 0 {
+			ticker = nil
+		} else {
+			// arbitrarily change graph every interval seconds
+			t := time.NewTicker(time.Duration(obj.Interval) * time.Second)
+			defer t.Stop()
+			ticker = t.C
+		}
+		for {
+			select {
+			case <-startChan: // kick the loop once at start
+				startChan = nil // disable
+				// pass
+			case <-ticker:
+				// pass
+			case <-obj.closeChan:
+				return
+			}
+
+			log.Printf("libmgmt: Generating new graph...")
+			select {
+			case ch <- gapi.Next{}: // trigger a run
+			case <-obj.closeChan:
+				return
+			}
+		}
+	}()
+	return ch
+}
+
+// Close shuts down the MyGAPI.
+func (obj *MyGAPI) Close() error {
+	if !obj.initialized {
+		return fmt.Errorf("libmgmt: MyGAPI is not initialized")
+	}
+	close(obj.closeChan)
+	obj.wg.Wait()
+	obj.initialized = false // closed = true
+	return nil
+}
+
+// Run runs an embedded mgmt server.
+func Run() error {
+
+	obj := &mgmt.Main{}
+	obj.Program = "libmgmt" // TODO: set on compilation
+	obj.Version = "0.0.1"   // TODO: set on compilation
+	obj.TmpPrefix = true    // disable for easy debugging
+	//prefix := "/tmp/testprefix/"
+	//obj.Prefix = &p // enable for easy debugging
+	obj.IdealClusterSize = -1
+	obj.ConvergedTimeout = -1
+	obj.Noop = false // FIXME: careful!
+
+	obj.GAPI = &MyGAPI{ // graph API
+		Name:     "libmgmt", // TODO: set on compilation
+		Interval: 60 * 10,   // arbitrarily change graph every 15 seconds
+	}
+
+	if err := obj.Init(); err != nil {
+		return err
+	}
+
+	// install the exit signal handler
+	exit := make(chan struct{})
+	defer close(exit)
+	go func() {
+		signals := make(chan os.Signal, 1)
+		signal.Notify(signals, os.Interrupt) // catch ^C
+		//signal.Notify(signals, os.Kill) // catch signals
+		signal.Notify(signals, syscall.SIGTERM)
+
+		select {
+		case sig := <-signals: // any signal will do
+			if sig == os.Interrupt {
+				log.Println("Interrupted by ^C")
+				obj.Exit(nil)
+				return
+			}
+			log.Println("Interrupted by signal")
+			obj.Exit(fmt.Errorf("killed by %v", sig))
+			return
+		case <-exit:
+			return
+		}
+	}()
+
+	return obj.Run()
+}
+
+func main() {
+	log.Printf("Hello!")
+	if err := Run(); err != nil {
+		fmt.Println(err)
+		os.Exit(1)
+		return
+	}
+	log.Printf("Goodbye!")
+}
--- a/examples/lib/libmgmt-subgraph0.go
+++ b/examples/lib/libmgmt-subgraph0.go
@@ -0,0 +1,253 @@
+// libmgmt example of flattened subgraph
+package main
+
+import (
+	"fmt"
+	"log"
+	"os"
+	"os/signal"
+	"sync"
+	"syscall"
+	"time"
+
+	"github.com/purpleidea/mgmt/gapi"
+	mgmt "github.com/purpleidea/mgmt/lib"
+	"github.com/purpleidea/mgmt/pgraph"
+	"github.com/purpleidea/mgmt/resources"
+
+	errwrap "github.com/pkg/errors"
+)
+
+// MyGAPI implements the main GAPI interface.
+type MyGAPI struct {
+	Name     string // graph name
+	Interval uint   // refresh interval, 0 to never refresh
+
+	data        gapi.Data
+	initialized bool
+	closeChan   chan struct{}
+	wg          sync.WaitGroup // sync group for tunnel go routines
+}
+
+// NewMyGAPI creates a new MyGAPI struct and calls Init().
+func NewMyGAPI(data gapi.Data, name string, interval uint) (*MyGAPI, error) {
+	obj := &MyGAPI{
+		Name:     name,
+		Interval: interval,
+	}
+	return obj, obj.Init(data)
+}
+
+// Init initializes the MyGAPI struct.
+func (obj *MyGAPI) Init(data gapi.Data) error {
+	if obj.initialized {
+		return fmt.Errorf("already initialized")
+	}
+	if obj.Name == "" {
+		return fmt.Errorf("the graph name must be specified")
+	}
+	obj.data = data // store for later
+	obj.closeChan = make(chan struct{})
+	obj.initialized = true
+	return nil
+}
+
+func (obj *MyGAPI) subGraph() (*pgraph.Graph, error) {
+	g, err := pgraph.NewGraph(obj.Name)
+	if err != nil {
+		return nil, err
+	}
+
+	metaparams := resources.DefaultMetaParams
+
+	f1 := &resources.FileRes{
+		BaseRes: resources.BaseRes{
+			Name:       "file1",
+			Kind:       "file",
+			MetaParams: metaparams,
+		},
+		Path: "/tmp/mgmt/sub1",
+
+		State: "present",
+	}
+	g.AddVertex(f1)
+
+	n1 := &resources.NoopRes{
+		BaseRes: resources.BaseRes{
+			Name:       "noop1",
+			Kind:       "noop",
+			MetaParams: metaparams,
+		},
+	}
+	g.AddVertex(n1)
+
+	return g, nil
+}
+
+// Graph returns a current Graph.
+func (obj *MyGAPI) Graph() (*pgraph.Graph, error) {
+	if !obj.initialized {
+		return nil, fmt.Errorf("libmgmt: MyGAPI is not initialized")
+	}
+
+	g, err := pgraph.NewGraph(obj.Name)
+	if err != nil {
+		return nil, err
+	}
+
+	metaparams := resources.DefaultMetaParams
+
+	content := "I created a subgraph!\n"
+	f0 := &resources.FileRes{
+		BaseRes: resources.BaseRes{
+			Name:       "README",
+			MetaParams: metaparams,
+		},
+		Path:    "/tmp/mgmt/README",
+		Content: &content,
+		State:   "present",
+	}
+	g.AddVertex(f0)
+
+	subGraph, err := obj.subGraph()
+	if err != nil {
+		return nil, errwrap.Wrapf(err, "running subGraph() failed")
+	}
+
+	edgeGenFn := func(v1, v2 pgraph.Vertex) pgraph.Edge {
+		edge := &resources.Edge{
+			Name: fmt.Sprintf("edge: %s->%s", v1, v2),
+		}
+
+		// if we want to do something specific based on input
+		_, v2IsFile := v2.(*resources.FileRes)
+		if v1 == f0 && v2IsFile {
+			edge.Notify = true
+		}
+
+		return edge
+	}
+	g.AddEdgeVertexGraph(f0, subGraph, edgeGenFn)
+
+	//g, err := config.NewGraphFromConfig(obj.data.Hostname, obj.data.World, obj.data.Noop)
+	return g, nil
+}
+
+// Next returns nil errors every time there could be a new graph.
+func (obj *MyGAPI) Next() chan gapi.Next {
+	ch := make(chan gapi.Next)
+	obj.wg.Add(1)
+	go func() {
+		defer obj.wg.Done()
+		defer close(ch) // this will run before the obj.wg.Done()
+		if !obj.initialized {
+			next := gapi.Next{
+				Err:  fmt.Errorf("libmgmt: MyGAPI is not initialized"),
+				Exit: true, // exit, b/c programming error?
+			}
+			ch <- next
+			return
+		}
+		startChan := make(chan struct{}) // start signal
+		close(startChan)                 // kick it off!
+
+		ticker := make(<-chan time.Time)
+		if obj.data.NoStreamWatch || obj.Interval <= 0 {
+			ticker = nil
+		} else {
+			// arbitrarily change graph every interval seconds
+			t := time.NewTicker(time.Duration(obj.Interval) * time.Second)
+			defer t.Stop()
+			ticker = t.C
+		}
+		for {
+			select {
+			case <-startChan: // kick the loop once at start
+				startChan = nil // disable
+				// pass
+			case <-ticker:
+				// pass
+			case <-obj.closeChan:
+				return
+			}
+
+			log.Printf("libmgmt: Generating new graph...")
+			select {
+			case ch <- gapi.Next{}: // trigger a run
+			case <-obj.closeChan:
+				return
+			}
+		}
+	}()
+	return ch
+}
+
+// Close shuts down the MyGAPI.
+func (obj *MyGAPI) Close() error {
+	if !obj.initialized {
+		return fmt.Errorf("libmgmt: MyGAPI is not initialized")
+	}
+	close(obj.closeChan)
+	obj.wg.Wait()
+	obj.initialized = false // closed = true
+	return nil
+}
+
+// Run runs an embedded mgmt server.
+func Run() error {
+
+	obj := &mgmt.Main{}
+	obj.Program = "libmgmt" // TODO: set on compilation
+	obj.Version = "0.0.1"   // TODO: set on compilation
+	obj.TmpPrefix = true    // disable for easy debugging
+	//prefix := "/tmp/testprefix/"
+	//obj.Prefix = &p // enable for easy debugging
+	obj.IdealClusterSize = -1
+	obj.ConvergedTimeout = -1
+	obj.Noop = false // FIXME: careful!
+
+	obj.GAPI = &MyGAPI{ // graph API
+		Name:     "libmgmt", // TODO: set on compilation
+		Interval: 60 * 10,   // arbitrarily change graph every 15 seconds
+	}
+
+	if err := obj.Init(); err != nil {
+		return err
+	}
+
+	// install the exit signal handler
+	exit := make(chan struct{})
+	defer close(exit)
+	go func() {
+		signals := make(chan os.Signal, 1)
+		signal.Notify(signals, os.Interrupt) // catch ^C
+		//signal.Notify(signals, os.Kill) // catch signals
+		signal.Notify(signals, syscall.SIGTERM)
+
+		select {
+		case sig := <-signals: // any signal will do
+			if sig == os.Interrupt {
+				log.Println("Interrupted by ^C")
+				obj.Exit(nil)
+				return
+			}
+			log.Println("Interrupted by signal")
+			obj.Exit(fmt.Errorf("killed by %v", sig))
+			return
+		case <-exit:
+			return
+		}
+	}()
+
+	return obj.Run()
+}
+
+func main() {
+	log.Printf("Hello!")
+	if err := Run(); err != nil {
+		fmt.Println(err)
+		os.Exit(1)
+		return
+	}
+	log.Printf("Goodbye!")
+}
--- a/examples/lib/libmgmt-subgraph1.go
+++ b/examples/lib/libmgmt-subgraph1.go
@@ -0,0 +1,243 @@
+// libmgmt example of graph resource
+package main
+
+import (
+	"fmt"
+	"log"
+	"os"
+	"os/signal"
+	"sync"
+	"syscall"
+	"time"
+
+	"github.com/purpleidea/mgmt/gapi"
+	mgmt "github.com/purpleidea/mgmt/lib"
+	"github.com/purpleidea/mgmt/pgraph"
+	"github.com/purpleidea/mgmt/resources"
+)
+
+// MyGAPI implements the main GAPI interface.
+type MyGAPI struct {
+	Name     string // graph name
+	Interval uint   // refresh interval, 0 to never refresh
+
+	data        gapi.Data
+	initialized bool
+	closeChan   chan struct{}
+	wg          sync.WaitGroup // sync group for tunnel go routines
+}
+
+// NewMyGAPI creates a new MyGAPI struct and calls Init().
+func NewMyGAPI(data gapi.Data, name string, interval uint) (*MyGAPI, error) {
+	obj := &MyGAPI{
+		Name:     name,
+		Interval: interval,
+	}
+	return obj, obj.Init(data)
+}
+
+// Init initializes the MyGAPI struct.
+func (obj *MyGAPI) Init(data gapi.Data) error {
+	if obj.initialized {
+		return fmt.Errorf("already initialized")
+	}
+	if obj.Name == "" {
+		return fmt.Errorf("the graph name must be specified")
+	}
+	obj.data = data // store for later
+	obj.closeChan = make(chan struct{})
+	obj.initialized = true
+	return nil
+}
+
+// Graph returns a current Graph.
+func (obj *MyGAPI) Graph() (*pgraph.Graph, error) {
+	if !obj.initialized {
+		return nil, fmt.Errorf("libmgmt: MyGAPI is not initialized")
+	}
+
+	g, err := pgraph.NewGraph(obj.Name)
+	if err != nil {
+		return nil, err
+	}
+
+	metaparams := resources.DefaultMetaParams
+
+	content := "I created a subgraph!\n"
+	f0 := &resources.FileRes{
+		BaseRes: resources.BaseRes{
+			Name:       "README",
+			Kind:       "file",
+			MetaParams: metaparams,
+		},
+		Path:    "/tmp/mgmt/README",
+		Content: &content,
+		State:   "present",
+	}
+	g.AddVertex(f0)
+
+	// create a subgraph to add *into* a graph resource
+	subGraph, err := pgraph.NewGraph(fmt.Sprintf("%s->subgraph", obj.Name))
+	if err != nil {
+		return nil, err
+	}
+
+	// add elements into the sub graph
+	f1 := &resources.FileRes{
+		BaseRes: resources.BaseRes{
+			Name:       "file1",
+			Kind:       "file",
+			MetaParams: metaparams,
+		},
+		Path: "/tmp/mgmt/sub1",
+
+		State: "present",
+	}
+	subGraph.AddVertex(f1)
+
+	n1 := &resources.NoopRes{
+		BaseRes: resources.BaseRes{
+			Name:       "noop1",
+			Kind:       "noop",
+			MetaParams: metaparams,
+		},
+	}
+	subGraph.AddVertex(n1)
+
+	e0 := &resources.Edge{Name: "e0"}
+	e0.Notify = true // send a notification from v0 to v1
+	subGraph.AddEdge(f1, n1, e0)
+
+	// create the actual resource to hold the sub graph
+	subGraphRes0 := &resources.GraphRes{ // TODO: should we name this SubGraphRes ?
+		BaseRes: resources.BaseRes{
+			Name:       "subgraph1",
+			Kind:       "graph",
+			MetaParams: metaparams,
+		},
+		Graph: subGraph,
+	}
+	g.AddVertex(subGraphRes0) // add it to the main graph
+
+	//g, err := config.NewGraphFromConfig(obj.data.Hostname, obj.data.World, obj.data.Noop)
+	return g, nil
+}
+
+// Next returns nil errors every time there could be a new graph.
+func (obj *MyGAPI) Next() chan gapi.Next {
+	ch := make(chan gapi.Next)
+	obj.wg.Add(1)
+	go func() {
+		defer obj.wg.Done()
+		defer close(ch) // this will run before the obj.wg.Done()
+		if !obj.initialized {
+			next := gapi.Next{
+				Err:  fmt.Errorf("libmgmt: MyGAPI is not initialized"),
+				Exit: true, // exit, b/c programming error?
+			}
+			ch <- next
+			return
+		}
+		startChan := make(chan struct{}) // start signal
+		close(startChan)                 // kick it off!
+
+		ticker := make(<-chan time.Time)
+		if obj.data.NoStreamWatch || obj.Interval <= 0 {
+			ticker = nil
+		} else {
+			// arbitrarily change graph every interval seconds
+			t := time.NewTicker(time.Duration(obj.Interval) * time.Second)
+			defer t.Stop()
+			ticker = t.C
+		}
+		for {
+			select {
+			case <-startChan: // kick the loop once at start
+				startChan = nil // disable
+				// pass
+			case <-ticker:
+				// pass
+			case <-obj.closeChan:
+				return
+			}
+
+			log.Printf("libmgmt: Generating new graph...")
+			select {
+			case ch <- gapi.Next{}: // trigger a run
+			case <-obj.closeChan:
+				return
+			}
+		}
+	}()
+	return ch
+}
+
+// Close shuts down the MyGAPI.
+func (obj *MyGAPI) Close() error {
+	if !obj.initialized {
+		return fmt.Errorf("libmgmt: MyGAPI is not initialized")
+	}
+	close(obj.closeChan)
+	obj.wg.Wait()
+	obj.initialized = false // closed = true
+	return nil
+}
+
+// Run runs an embedded mgmt server.
+func Run() error {
+
+	obj := &mgmt.Main{}
+	obj.Program = "libmgmt" // TODO: set on compilation
+	obj.Version = "0.0.1"   // TODO: set on compilation
+	obj.TmpPrefix = true    // disable for easy debugging
+	//prefix := "/tmp/testprefix/"
+	//obj.Prefix = &p // enable for easy debugging
+	obj.IdealClusterSize = -1
+	obj.ConvergedTimeout = -1
+	obj.Noop = false // FIXME: careful!
+
+	obj.GAPI = &MyGAPI{ // graph API
+		Name:     "libmgmt", // TODO: set on compilation
+		Interval: 60 * 10,   // arbitrarily change graph every 15 seconds
+	}
+
+	if err := obj.Init(); err != nil {
+		return err
+	}
+
+	// install the exit signal handler
+	exit := make(chan struct{})
+	defer close(exit)
+	go func() {
+		signals := make(chan os.Signal, 1)
+		signal.Notify(signals, os.Interrupt) // catch ^C
+		//signal.Notify(signals, os.Kill) // catch signals
+		signal.Notify(signals, syscall.SIGTERM)
+
+		select {
+		case sig := <-signals: // any signal will do
+			if sig == os.Interrupt {
+				log.Println("Interrupted by ^C")
+				obj.Exit(nil)
+				return
+			}
+			log.Println("Interrupted by signal")
+			obj.Exit(fmt.Errorf("killed by %v", sig))
+			return
+		case <-exit:
+			return
+		}
+	}()
+
+	return obj.Run()
+}
+
+func main() {
+	log.Printf("Hello!")
+	if err := Run(); err != nil {
+		fmt.Println(err)
+		os.Exit(1)
+		return
+	}
+	log.Printf("Goodbye!")
+}
--- a/examples/lib/libmgmt1.go
+++ b/examples/lib/libmgmt1.go
@@ -57,11 +57,13 @@ func (obj *MyGAPI) Graph() (*pgraph.Graph, error) {
 		return nil, fmt.Errorf("libmgmt: MyGAPI is not initialized")
 	}

-	n1, err := resources.NewNoopRes("noop1")
+	n1, err := resources.NewNamedResource("noop", "noop1")
 	if err != nil {
-		return nil, fmt.Errorf("can't create resource: %v", err)
+		return nil, err
 	}

+	// NOTE: This is considered the legacy method to build graphs. Avoid
+	// importing the legacy `yamlgraph` lib if possible for custom graphs.
 	// we can still build a graph via the yaml method
 	gc := &yamlgraph.GraphConfig{
 		Graph: obj.Name,
@@ -70,7 +72,7 @@ func (obj *MyGAPI) Graph() (*pgraph.Graph, error) {
 			Exec:  []*resources.ExecRes{},
 			File:  []*resources.FileRes{},
 			Msg:   []*resources.MsgRes{},
-			Noop:  []*resources.NoopRes{n1},
+			Noop:  []*resources.NoopRes{n1.(*resources.NoopRes)},
 			Pkg:   []*resources.PkgRes{},
 			Svc:   []*resources.SvcRes{},
 			Timer: []*resources.TimerRes{},
@@ -86,32 +88,45 @@ func (obj *MyGAPI) Graph() (*pgraph.Graph, error) {
 }

 // Next returns nil errors every time there could be a new graph.
-func (obj *MyGAPI) Next() chan error {
-	if obj.data.NoWatch || obj.Interval <= 0 {
-		return nil
-	}
-	ch := make(chan error)
+func (obj *MyGAPI) Next() chan gapi.Next {
+	ch := make(chan gapi.Next)
 	obj.wg.Add(1)
 	go func() {
 		defer obj.wg.Done()
 		defer close(ch) // this will run before the obj.wg.Done()
 		if !obj.initialized {
-			ch <- fmt.Errorf("libmgmt: MyGAPI is not initialized")
-			return
+			next := gapi.Next{
+				Err:  fmt.Errorf("libmgmt: MyGAPI is not initialized"),
+				Exit: true, // exit, b/c programming error?
+			}
+			ch <- next
 		}
+		startChan := make(chan struct{}) // start signal
+		close(startChan)                 // kick it off!

-		// arbitrarily change graph every interval seconds
-		ticker := time.NewTicker(time.Duration(obj.Interval) * time.Second)
-		defer ticker.Stop()
+		ticker := make(<-chan time.Time)
+		if obj.data.NoStreamWatch || obj.Interval <= 0 {
+			ticker = nil
+		} else {
+			// arbitrarily change graph every interval seconds
+			t := time.NewTicker(time.Duration(obj.Interval) * time.Second)
+			defer t.Stop()
+			ticker = t.C
+		}
 		for {
 			select {
-			case <-ticker.C:
-				log.Printf("libmgmt: Generating new graph...")
-				select {
-				case ch <- nil: // trigger a run
-				case <-obj.closeChan:
-					return
-				}
+			case <-startChan: // kick the loop once at start
+				startChan = nil // disable
+				// pass
+			case <-ticker:
+				// pass
+			case <-obj.closeChan:
+				return
+			}
+
+			log.Printf("libmgmt: Generating new graph...")
+			select {
+			case ch <- gapi.Next{}: // trigger a run
 			case <-obj.closeChan:
 				return
 			}
@@ -175,10 +190,7 @@ func Run() error {
 		}
 	}()

-	if err := obj.Run(); err != nil {
-		return err
-	}
-	return nil
+	return obj.Run()
 }

 func main() {
--- a/examples/lib/libmgmt2.go
+++ b/examples/lib/libmgmt2.go
@@ -59,19 +59,21 @@ func (obj *MyGAPI) Graph() (*pgraph.Graph, error) {
 		return nil, fmt.Errorf("libmgmt: MyGAPI is not initialized")
 	}

-	g := pgraph.NewGraph(obj.Name)
-	var vertex *pgraph.Vertex
+	g, err := pgraph.NewGraph(obj.Name)
+	if err != nil {
+		return nil, err
+	}
+	var vertex pgraph.Vertex
 	for i := uint(0); i < obj.Count; i++ {
-		n, err := resources.NewNoopRes(fmt.Sprintf("noop%d", i))
+		n, err := resources.NewNamedResource("noop", fmt.Sprintf("noop%d", i))
 		if err != nil {
-			return nil, fmt.Errorf("can't create resource: %v", err)
+			return nil, err
 		}
-		v := pgraph.NewVertex(n)
-		g.AddVertex(v)
+		g.AddVertex(n)
 		if i > 0 {
-			g.AddEdge(vertex, v, pgraph.NewEdge(fmt.Sprintf("e%d", i)))
+			g.AddEdge(vertex, n, &resources.Edge{Name: fmt.Sprintf("e%d", i)})
 		}
-		vertex = v // save
+		vertex = n // save
 	}

 	//g, err := config.NewGraphFromConfig(obj.data.Hostname, obj.data.World, obj.data.Noop)
@@ -79,32 +81,45 @@ func (obj *MyGAPI) Graph() (*pgraph.Graph, error) {
 }

 // Next returns nil errors every time there could be a new graph.
-func (obj *MyGAPI) Next() chan error {
-	if obj.data.NoWatch || obj.Interval <= 0 {
-		return nil
-	}
-	ch := make(chan error)
+func (obj *MyGAPI) Next() chan gapi.Next {
+	ch := make(chan gapi.Next)
 	obj.wg.Add(1)
 	go func() {
 		defer obj.wg.Done()
 		defer close(ch) // this will run before the obj.wg.Done()
 		if !obj.initialized {
-			ch <- fmt.Errorf("libmgmt: MyGAPI is not initialized")
-			return
+			next := gapi.Next{
+				Err:  fmt.Errorf("libmgmt: MyGAPI is not initialized"),
+				Exit: true, // exit, b/c programming error?
+			}
+			ch <- next
 		}
+		startChan := make(chan struct{}) // start signal
+		close(startChan)                 // kick it off!

-		// arbitrarily change graph every interval seconds
-		ticker := time.NewTicker(time.Duration(obj.Interval) * time.Second)
-		defer ticker.Stop()
+		ticker := make(<-chan time.Time)
+		if obj.data.NoStreamWatch || obj.Interval <= 0 {
+			ticker = nil
+		} else {
+			// arbitrarily change graph every interval seconds
+			t := time.NewTicker(time.Duration(obj.Interval) * time.Second)
+			defer t.Stop()
+			ticker = t.C
+		}
 		for {
 			select {
-			case <-ticker.C:
-				log.Printf("libmgmt: Generating new graph...")
-				select {
-				case ch <- nil: // trigger a run
-				case <-obj.closeChan:
-					return
-				}
+			case <-startChan: // kick the loop once at start
+				startChan = nil // disable
+				// pass
+			case <-ticker:
+				// pass
+			case <-obj.closeChan:
+				return
+			}
+
+			log.Printf("libmgmt: Generating new graph...")
+			select {
+			case ch <- gapi.Next{}: // trigger a run
 			case <-obj.closeChan:
 				return
 			}
@@ -169,10 +184,7 @@ func Run(count uint) error {
 		}
 	}()

-	if err := obj.Run(); err != nil {
-		return err
-	}
-	return nil
+	return obj.Run()
 }

 func main() {
--- a/examples/lib/libmgmt3.go
+++ b/examples/lib/libmgmt3.go
@@ -14,8 +14,6 @@ import (
 	mgmt "github.com/purpleidea/mgmt/lib"
 	"github.com/purpleidea/mgmt/pgraph"
 	"github.com/purpleidea/mgmt/resources"
-
-	"golang.org/x/time/rate"
 )

 // MyGAPI implements the main GAPI interface.
@@ -58,18 +56,18 @@ func (obj *MyGAPI) Graph() (*pgraph.Graph, error) {
 		return nil, fmt.Errorf("libmgmt: MyGAPI is not initialized")
 	}

-	g := pgraph.NewGraph(obj.Name)
-
-	// FIXME: these are being specified temporarily until it's the default!
-	metaparams := resources.MetaParams{
-		Limit: rate.Inf,
-		Burst: 0,
+	g, err := pgraph.NewGraph(obj.Name)
+	if err != nil {
+		return nil, err
 	}

+	metaparams := resources.DefaultMetaParams
+
 	content := "Delete me to trigger a notification!\n"
 	f0 := &resources.FileRes{
 		BaseRes: resources.BaseRes{
 			Name:       "README",
+			Kind:       "file",
 			MetaParams: metaparams,
 		},
 		Path:    "/tmp/mgmt/README",
@@ -77,23 +75,23 @@ func (obj *MyGAPI) Graph() (*pgraph.Graph, error) {
 		State:   "present",
 	}

-	v0 := pgraph.NewVertex(f0)
-	g.AddVertex(v0)
+	g.AddVertex(f0)

 	p1 := &resources.PasswordRes{
 		BaseRes: resources.BaseRes{
 			Name:       "password1",
+			Kind:       "password",
 			MetaParams: metaparams,
 		},
 		Length: 8,    // generated string will have this many characters
 		Saved:  true, // this causes passwords to be stored in plain text!
 	}
-	v1 := pgraph.NewVertex(p1)
-	g.AddVertex(v1)
+	g.AddVertex(p1)

 	f1 := &resources.FileRes{
 		BaseRes: resources.BaseRes{
 			Name:       "file1",
+			Kind:       "file",
 			MetaParams: metaparams,
 			// send->recv!
 			Recv: map[string]*resources.Send{
@@ -105,60 +103,72 @@ func (obj *MyGAPI) Graph() (*pgraph.Graph, error) {
 		State: "present",
 	}

-	v2 := pgraph.NewVertex(f1)
-	g.AddVertex(v2)
+	g.AddVertex(f1)

 	n1 := &resources.NoopRes{
 		BaseRes: resources.BaseRes{
 			Name:       "noop1",
+			Kind:       "noop",
 			MetaParams: metaparams,
 		},
 	}

-	v3 := pgraph.NewVertex(n1)
-	g.AddVertex(v3)
+	g.AddVertex(n1)

-	e0 := pgraph.NewEdge("e0")
-	e0.Notify = true // send a notification from v0 to v1
-	g.AddEdge(v0, v1, e0)
+	e0 := &resources.Edge{Name: "e0"}
+	e0.Notify = true // send a notification from f0 to p1
+	g.AddEdge(f0, p1, e0)

-	g.AddEdge(v1, v2, pgraph.NewEdge("e1"))
+	g.AddEdge(p1, f1, &resources.Edge{Name: "e1"})

-	e2 := pgraph.NewEdge("e2")
-	e2.Notify = true // send a notification from v2 to v3
-	g.AddEdge(v2, v3, e2)
+	e2 := &resources.Edge{Name: "e2"}
+	e2.Notify = true // send a notification from f1 to n1
+	g.AddEdge(f1, n1, e2)

 	//g, err := config.NewGraphFromConfig(obj.data.Hostname, obj.data.World, obj.data.Noop)
 	return g, nil
 }

 // Next returns nil errors every time there could be a new graph.
-func (obj *MyGAPI) Next() chan error {
-	if obj.data.NoWatch || obj.Interval <= 0 {
-		return nil
-	}
-	ch := make(chan error)
+func (obj *MyGAPI) Next() chan gapi.Next {
+	ch := make(chan gapi.Next)
 	obj.wg.Add(1)
 	go func() {
 		defer obj.wg.Done()
 		defer close(ch) // this will run before the obj.wg.Done()
 		if !obj.initialized {
-			ch <- fmt.Errorf("libmgmt: MyGAPI is not initialized")
-			return
+			next := gapi.Next{
+				Err:  fmt.Errorf("libmgmt: MyGAPI is not initialized"),
+				Exit: true, // exit, b/c programming error?
+			}
+			ch <- next
 		}
+		startChan := make(chan struct{}) // start signal
+		close(startChan)                 // kick it off!

-		// arbitrarily change graph every interval seconds
-		ticker := time.NewTicker(time.Duration(obj.Interval) * time.Second)
-		defer ticker.Stop()
+		ticker := make(<-chan time.Time)
+		if obj.data.NoStreamWatch || obj.Interval <= 0 {
+			ticker = nil
+		} else {
+			// arbitrarily change graph every interval seconds
+			t := time.NewTicker(time.Duration(obj.Interval) * time.Second)
+			defer t.Stop()
+			ticker = t.C
+		}
 		for {
 			select {
-			case <-ticker.C:
-				log.Printf("libmgmt: Generating new graph...")
-				select {
-				case ch <- nil: // trigger a run
-				case <-obj.closeChan:
-					return
-				}
+			case <-startChan: // kick the loop once at start
+				startChan = nil // disable
+				// pass
+			case <-ticker:
+				// pass
+			case <-obj.closeChan:
+				return
+			}
+
+			log.Printf("libmgmt: Generating new graph...")
+			select {
+			case ch <- gapi.Next{}: // trigger a run
 			case <-obj.closeChan:
 				return
 			}
@@ -224,10 +234,7 @@ func Run() error {
 		}
 	}()

-	if err := obj.Run(); err != nil {
-		return err
-	}
-	return nil
+	return obj.Run()
 }

 func main() {
--- a/examples/noop0.yaml
+++ b/examples/noop0.yaml
@@ -0,0 +1,7 @@
+---
+graph: mygraph
+comment: simple noop example
+resources:
+  noop:
+  - name: noop0
+edges: []
--- a/examples/nspawn1.yaml
+++ b/examples/nspawn1.yaml
@@ -2,6 +2,6 @@
 graph: mygraph
 resources:
  nspawn:
-  - name: mgmt-nspawn1
+  - name: nspawn1
    state: running
 edges: []
--- a/examples/nspawn2.yaml
+++ b/examples/nspawn2.yaml
@@ -1,7 +0,0 @@
---
-graph: mygraph
-resources:
-  nspawn:
-  - name: mgmt-nspawn2
-    state: stopped
-edges: []
--- a/examples/svc2.yaml
+++ b/examples/svc2.yaml
@@ -0,0 +1,8 @@
+---
+graph: mygraph
+resources:
+  svc:
+  - name: purpleidea
+    state: running
+    session: true
+edges: []
--- a/examples/user1.yaml
+++ b/examples/user1.yaml
@@ -0,0 +1,9 @@
+---
+graph: mygraph
+resources:
+  user:
+  - name: testuser
+    uid: 1002
+    gid: 100
+    state: exists
+edges: []
--- a/gapi/gapi.go
+++ b/gapi/gapi.go
@@ -3,16 +3,16 @@
 // Written by James Shubin <james@shubin.ca> and the project contributors
 //
 // This program is free software: you can redistribute it and/or modify
-// it under the terms of the GNU Affero General Public License as published by
+// it under the terms of the GNU General Public License as published by
 // the Free Software Foundation, either version 3 of the License, or
 // (at your option) any later version.
 //
 // This program is distributed in the hope that it will be useful,
 // but WITHOUT ANY WARRANTY; without even the implied warranty of
 // MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-// GNU Affero General Public License for more details.
+// GNU General Public License for more details.
 //
-// You should have received a copy of the GNU Affero General Public License
+// You should have received a copy of the GNU General Public License
 // along with this program.  If not, see <http://www.gnu.org/licenses/>.

 // Package gapi defines the interface that graph API generators must meet.
@@ -23,28 +23,33 @@ import (
 	"github.com/purpleidea/mgmt/resources"
 )

-// World is an interface to the rest of the different graph state. It allows
-// the GAPI to store state and exchange information throughout the cluster. It
-// is the interface each machine uses to communicate with the rest of the world.
-type World interface { // TODO: is there a better name for this interface?
-	ResExport([]resources.Res) error
-	// FIXME: should this method take a "filter" data struct instead of many args?
-	ResCollect(hostnameFilter, kindFilter []string) ([]resources.Res, error)
-}
-
 // Data is the set of input values passed into the GAPI structs via Init.
 type Data struct {
-	Hostname string // uuid for the host, required for GAPI
-	World    World
-	Noop     bool
-	NoWatch  bool
+	Hostname      string // uuid for the host, required for GAPI
+	World         resources.World
+	Noop          bool
+	NoConfigWatch bool
+	NoStreamWatch bool
 	// NOTE: we can add more fields here if needed by GAPI endpoints
 }

+// Next describes the particular response the GAPI implementer wishes to emit.
+type Next struct {
+	// FIXME: the Fast pause parameter should eventually get replaced with a
+	// "SwitchMethod" parameter or similar that instead lets the implementer
+	// choose between fast pause, slow pause, and interrupt. Interrupt could
+	// be a future extension to the Resource API that lets an Interrupt() be
+	// called if we want to exit immediately from the CheckApply part of the
+	// resource for some reason. For now we'll keep this simple with a bool.
+	Fast bool  // run a fast pause to switch?
+	Exit bool  // should we cause the program to exit? (specify err or not)
+	Err  error // if something goes wrong (use with or without exit!)
+}
+
 // GAPI is a Graph API that represents incoming graphs and change streams.
 type GAPI interface {
 	Init(Data) error               // initializes the GAPI and passes in useful data
 	Graph() (*pgraph.Graph, error) // returns the most recent pgraph
-	Next() chan error              // returns a stream of switch events
+	Next() chan Next               // returns a stream of switch events
 	Close() error                  // shutdown the GAPI
 }
--- a/hcl/gapi.go
+++ b/hcl/gapi.go
@@ -0,0 +1,155 @@
+// Mgmt
+// Copyright (C) 2013-2017+ James Shubin and the project contributors
+// Written by James Shubin <james@shubin.ca> and the project contributors
+//
+// This program is free software: you can redistribute it and/or modify
+// it under the terms of the GNU General Public License as published by
+// the Free Software Foundation, either version 3 of the License, or
+// (at your option) any later version.
+//
+// This program is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+// GNU General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License
+// along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+package hcl
+
+import (
+	"fmt"
+	"log"
+	"sync"
+
+	"github.com/purpleidea/mgmt/gapi"
+	"github.com/purpleidea/mgmt/pgraph"
+	"github.com/purpleidea/mgmt/recwatch"
+)
+
+// GAPI ...
+type GAPI struct {
+	File *string
+
+	initialized   bool
+	data          gapi.Data
+	wg            sync.WaitGroup
+	closeChan     chan struct{}
+	configWatcher *recwatch.ConfigWatcher
+}
+
+// NewGAPI ...
+func NewGAPI(data gapi.Data, file *string) (*GAPI, error) {
+	if file == nil {
+		return nil, fmt.Errorf("empty file given")
+	}
+
+	obj := &GAPI{
+		File: file,
+	}
+	return obj, obj.Init(data)
+}
+
+// Init ...
+func (obj *GAPI) Init(d gapi.Data) error {
+	if obj.initialized {
+		return fmt.Errorf("already initialized")
+	}
+
+	if obj.File == nil {
+		return fmt.Errorf("file cannot be nil")
+	}
+
+	obj.data = d
+	obj.closeChan = make(chan struct{})
+	obj.initialized = true
+	obj.configWatcher = recwatch.NewConfigWatcher()
+
+	return nil
+}
+
+// Graph ...
+func (obj *GAPI) Graph() (*pgraph.Graph, error) {
+	config, err := loadHcl(obj.File)
+	if err != nil {
+		return nil, fmt.Errorf("unable to parse graph: %s", err)
+	}
+
+	return graphFromConfig(config, obj.data)
+}
+
+// Next ...
+func (obj *GAPI) Next() chan gapi.Next {
+	ch := make(chan gapi.Next)
+	obj.wg.Add(1)
+
+	go func() {
+		defer obj.wg.Done()
+		defer close(ch)
+		if !obj.initialized {
+			next := gapi.Next{
+				Err:  fmt.Errorf("hcl: GAPI is not initialized"),
+				Exit: true,
+			}
+			ch <- next
+			return
+		}
+		startChan := make(chan struct{}) // start signal
+		close(startChan)                 // kick it off!
+
+		watchChan, configChan := make(chan error), make(chan error)
+		if obj.data.NoConfigWatch {
+			configChan = nil
+		} else {
+			configChan = obj.configWatcher.ConfigWatch(*obj.File) // simple
+		}
+		if obj.data.NoStreamWatch {
+			watchChan = nil
+		} else {
+			watchChan = obj.data.World.ResWatch()
+		}
+
+		for {
+			var err error
+			var ok bool
+
+			select {
+			case <-startChan:
+				startChan = nil
+			case err, ok = <-watchChan:
+			case err, ok = <-configChan:
+				if !ok {
+					return
+				}
+			case <-obj.closeChan:
+				return
+			}
+
+			log.Printf("hcl: generating new graph")
+			next := gapi.Next{
+				Err: err,
+			}
+
+			select {
+			case ch <- next:
+			case <-obj.closeChan:
+				return
+			}
+		}
+	}()
+
+	return ch
+}
+
+// Close ...
+func (obj *GAPI) Close() error {
+	if !obj.initialized {
+		return fmt.Errorf("hcl: GAPI is not initialized")
+	}
+
+	obj.configWatcher.Close()
+	close(obj.closeChan)
+	obj.wg.Wait()
+	obj.initialized = false
+	return nil
+}
--- a/hcl/parse.go
+++ b/hcl/parse.go
@@ -0,0 +1,387 @@
+// Mgmt
+// Copyright (C) 2013-2017+ James Shubin and the project contributors
+// Written by James Shubin <james@shubin.ca> and the project contributors
+//
+// This program is free software: you can redistribute it and/or modify
+// it under the terms of the GNU General Public License as published by
+// the Free Software Foundation, either version 3 of the License, or
+// (at your option) any later version.
+//
+// This program is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+// GNU General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License
+// along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+package hcl
+
+import (
+	"fmt"
+	"io/ioutil"
+	"log"
+	"strings"
+
+	"github.com/hashicorp/hcl"
+	"github.com/hashicorp/hcl/hcl/ast"
+	"github.com/hashicorp/hil"
+	"github.com/purpleidea/mgmt/gapi"
+	hv "github.com/purpleidea/mgmt/hil"
+	"github.com/purpleidea/mgmt/pgraph"
+	"github.com/purpleidea/mgmt/resources"
+)
+
+type collectorResConfig struct {
+	Kind    string
+	Pattern string
+}
+
+// Config defines the structure of the hcl config.
+type Config struct {
+	Resources []*Resource
+	Edges     []*Edge
+	Collector []collectorResConfig
+}
+
+// vertex is the data structure of a vertex.
+type vertex struct {
+	Kind string `hcl:"kind"`
+	Name string `hcl:"name"`
+}
+
+// Edge defines an edge in hcl.
+type Edge struct {
+	Name   string
+	From   vertex
+	To     vertex
+	Notify bool
+}
+
+// Resources define the state for resources.
+type Resources struct {
+	Resources []resources.Res
+}
+
+// Resource ...
+type Resource struct {
+	Name     string
+	Kind     string
+	resource resources.Res
+	Meta     resources.MetaParams
+	deps     []*Edge
+	rcv      map[string]*hv.ResourceVariable
+}
+
+type key struct {
+	kind, name string
+}
+
+func graphFromConfig(c *Config, data gapi.Data) (*pgraph.Graph, error) {
+	var graph *pgraph.Graph
+	var err error
+
+	graph, err = pgraph.NewGraph("Graph")
+	if err != nil {
+		return nil, fmt.Errorf("unable to create graph from config: %s", err)
+	}
+
+	lookup := make(map[key]pgraph.Vertex)
+
+	var keep []pgraph.Vertex
+	var resourceList []resources.Res
+
+	log.Printf("hcl: parsing %d resources", len(c.Resources))
+	for _, r := range c.Resources {
+		res := r.resource
+		kind := r.resource.GetKind()
+
+		log.Printf("hcl: resource \"%s\" \"%s\"", kind, r.Name)
+		if !strings.HasPrefix(res.GetName(), "@@") {
+			fn := func(v pgraph.Vertex) (bool, error) {
+				return resources.VtoR(v).Compare(res), nil
+			}
+			v, err := graph.VertexMatchFn(fn)
+			if err != nil {
+				return nil, fmt.Errorf("could not match vertex: %s", err)
+			}
+			if v == nil {
+				v = res
+				graph.AddVertex(v)
+			}
+			lookup[key{kind, res.GetName()}] = v
+			keep = append(keep, v)
+		} else if !data.Noop {
+			res.SetName(res.GetName()[2:])
+			res.SetKind(kind)
+			resourceList = append(resourceList, res)
+		}
+	}
+
+	// store in backend (usually etcd)
+	if err := data.World.ResExport(resourceList); err != nil {
+		return nil, fmt.Errorf("Config: Could not export resources: %v", err)
+	}
+
+	// lookup from backend (usually etcd)
+	var hostnameFilter []string // empty to get from everyone
+	kindFilter := []string{}
+	for _, t := range c.Collector {
+		kind := strings.ToLower(t.Kind)
+		kindFilter = append(kindFilter, kind)
+	}
+	// do all the graph look ups in one single step, so that if the backend
+	// database changes, we don't have a partial state of affairs...
+	if len(kindFilter) > 0 { // if kindFilter is empty, don't need to do lookups!
+		var err error
+		resourceList, err = data.World.ResCollect(hostnameFilter, kindFilter)
+		if err != nil {
+			return nil, fmt.Errorf("Config: Could not collect resources: %v", err)
+		}
+	}
+	for _, res := range resourceList {
+		matched := false
+		// see if we find a collect pattern that matches
+		for _, t := range c.Collector {
+			kind := strings.ToLower(t.Kind)
+			// use t.Kind and optionally t.Pattern to collect from storage
+			log.Printf("Collect: %v; Pattern: %v", kind, t.Pattern)
+
+			// XXX: expand to more complex pattern matching here...
+			if res.GetKind() != kind {
+				continue
+			}
+
+			if matched {
+				// we've already matched this resource, should we match again?
+				log.Printf("Config: Warning: Matching %s again!", res)
+			}
+			matched = true
+
+			// collect resources but add the noop metaparam
+			//if noop { // now done in mgmtmain
+			//	res.Meta().Noop = noop
+			//}
+
+			if t.Pattern != "" { // XXX: simplistic for now
+				res.CollectPattern(t.Pattern) // res.Dirname = t.Pattern
+			}
+
+			log.Printf("Collect: %s: collected!", res)
+
+			// XXX: similar to other resource add code:
+			// if _, exists := lookup[kind]; !exists {
+			// 	lookup[kind] = make(map[string]pgraph.Vertex)
+			// }
+
+			fn := func(v pgraph.Vertex) (bool, error) {
+				return resources.VtoR(v).Compare(res), nil
+			}
+			v, err := graph.VertexMatchFn(fn)
+			if err != nil {
+				return nil, fmt.Errorf("could not VertexMatchFn() resource: %s", err)
+			}
+			if v == nil { // no match found
+				v = res            // a standalone res can be a vertex
+				graph.AddVertex(v) // call standalone in case not part of an edge
+			}
+			lookup[key{kind, res.GetName()}] = v // used for constructing edges
+			keep = append(keep, v)               // append
+
+			//break // let's see if another resource even matches
+		}
+	}
+
+	for _, r := range c.Resources {
+		for _, e := range r.deps {
+			if _, ok := lookup[key{strings.ToLower(e.From.Kind), e.From.Name}]; !ok {
+				return nil, fmt.Errorf("can't find 'from' name")
+			}
+			if _, ok := lookup[key{strings.ToLower(e.To.Kind), e.To.Name}]; !ok {
+				return nil, fmt.Errorf("can't find 'to' name")
+			}
+			from := lookup[key{strings.ToLower(e.From.Kind), e.From.Name}]
+			to := lookup[key{strings.ToLower(e.To.Kind), e.To.Name}]
+			edge := &resources.Edge{
+				Name:   e.Name,
+				Notify: e.Notify,
+			}
+			graph.AddEdge(from, to, edge)
+		}
+
+		recv := make(map[string]*resources.Send)
+		// build Rcv's from resource variables
+		for k, v := range r.rcv {
+			send, ok := lookup[key{strings.ToLower(v.Kind), v.Name}]
+			if !ok {
+				return nil, fmt.Errorf("resource not found")
+			}
+
+			recv[strings.ToUpper(string(k[0]))+k[1:]] = &resources.Send{
+				Res: resources.VtoR(send),
+				Key: v.Field,
+			}
+
+			to := lookup[key{strings.ToLower(r.Kind), r.Name}]
+			edge := &resources.Edge{
+				Name:   v.Name,
+				Notify: true,
+			}
+			graph.AddEdge(send, to, edge)
+		}
+
+		r.resource.SetRecv(recv)
+	}
+
+	return graph, nil
+}
+
+func loadHcl(f *string) (*Config, error) {
+	if f == nil {
+		return nil, fmt.Errorf("empty file given")
+	}
+
+	data, err := ioutil.ReadFile(*f)
+	if err != nil {
+		return nil, fmt.Errorf("unable to read file: %v", err)
+	}
+
+	file, err := hcl.ParseBytes(data)
+	if err != nil {
+		return nil, fmt.Errorf("unable to parse file: %s", err)
+	}
+
+	config := new(Config)
+
+	list, ok := file.Node.(*ast.ObjectList)
+	if !ok {
+		return nil, fmt.Errorf("unable to parse file: file does not contain root node object")
+	}
+
+	if resources := list.Filter("resource"); len(resources.Items) > 0 {
+		var err error
+		config.Resources, err = loadResourcesHcl(resources)
+		if err != nil {
+			return nil, fmt.Errorf("unable to parse: %s", err)
+		}
+	}
+
+	return config, nil
+}
+
+func loadResourcesHcl(list *ast.ObjectList) ([]*Resource, error) {
+	list = list.Children()
+	if len(list.Items) == 0 {
+		return nil, nil
+	}
+
+	var result []*Resource
+
+	for _, item := range list.Items {
+		kind := item.Keys[0].Token.Value().(string)
+		name := item.Keys[1].Token.Value().(string)
+
+		var listVal *ast.ObjectList
+		if ot, ok := item.Val.(*ast.ObjectType); ok {
+			listVal = ot.List
+		} else {
+			return nil, fmt.Errorf("module '%s': should be an object", name)
+		}
+
+		var params = resources.DefaultMetaParams
+		if o := listVal.Filter("meta"); len(o.Items) > 0 {
+			err := hcl.DecodeObject(&params, o)
+			if err != nil {
+				return nil, fmt.Errorf(
+					"Error parsing meta for %s: %s",
+					name,
+					err)
+			}
+		}
+
+		var deps []string
+		if edges := listVal.Filter("depends_on"); len(edges.Items) > 0 {
+			err := hcl.DecodeObject(&deps, edges.Items[0].Val)
+			if err != nil {
+				return nil, fmt.Errorf("unable to parse: %s", err)
+			}
+		}
+
+		var edges []*Edge
+		for _, dep := range deps {
+			vertices := strings.Split(dep, ".")
+			edges = append(edges, &Edge{
+				To: vertex{
+					Kind: kind,
+					Name: name,
+				},
+				From: vertex{
+					Kind: vertices[0],
+					Name: vertices[1],
+				},
+			})
+		}
+
+		var config map[string]interface{}
+		if err := hcl.DecodeObject(&config, item.Val); err != nil {
+			log.Printf("hcl: unable to decode body: %v", err)
+			return nil, fmt.Errorf(
+				"Error reading config for %s: %s",
+				name,
+				err)
+		}
+
+		delete(config, "meta")
+		delete(config, "depends_on")
+
+		rcv := make(map[string]*hv.ResourceVariable)
+		// parse strings for hil
+		for k, v := range config {
+			n, err := hil.Parse(v.(string))
+			if err != nil {
+				return nil, fmt.Errorf("unable to parse fields: %v", err)
+			}
+
+			variables, err := hv.ParseVariables(n)
+			if err != nil {
+				return nil, fmt.Errorf("unable to parse variables: %v", err)
+			}
+
+			for _, v := range variables {
+				val, ok := v.(*hv.ResourceVariable)
+				if !ok {
+					continue
+				}
+
+				rcv[k] = val
+			}
+		}
+
+		res, err := resources.NewNamedResource(kind, name)
+		if err != nil {
+			log.Printf("hcl: unable to parse resource: %v", err)
+			return nil, err
+		}
+
+		if err := hcl.DecodeObject(res, item.Val); err != nil {
+			log.Printf("hcl: unable to decode body: %v", err)
+			return nil, fmt.Errorf(
+				"Error reading config for %s: %s",
+				name,
+				err)
+		}
+
+		meta := res.Meta()
+		*meta = params
+
+		result = append(result, &Resource{
+			Name:     name,
+			Kind:     kind,
+			resource: res,
+			deps:     edges,
+			rcv:      rcv,
+		})
+	}
+
+	return result, nil
+}
--- a/hil/interpolate.go
+++ b/hil/interpolate.go
@@ -0,0 +1,89 @@
+// Mgmt
+// Copyright (C) 2013-2017+ James Shubin and the project contributors
+// Written by James Shubin <james@shubin.ca> and the project contributors
+//
+// This program is free software: you can redistribute it and/or modify
+// it under the terms of the GNU General Public License as published by
+// the Free Software Foundation, either version 3 of the License, or
+// (at your option) any later version.
+//
+// This program is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+// GNU General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License
+// along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+package hil
+
+import (
+	"fmt"
+	"strings"
+
+	"github.com/hashicorp/hil/ast"
+)
+
+// Variable defines an interpolated variable.
+type Variable interface {
+	Key() string
+}
+
+// ResourceVariable defines a variable type used to reference fields of a resource
+// e.g.  ${file.file1.Content}
+type ResourceVariable struct {
+	Kind, Name, Field string
+}
+
+// Key returns a string representation of the variable key.
+func (r *ResourceVariable) Key() string {
+	return fmt.Sprintf("%s.%s.%s", r.Kind, r.Name, r.Field)
+}
+
+// NewInterpolatedVariable takes a variable key and return the interpolated variable
+// of the required type.
+func NewInterpolatedVariable(k string) (Variable, error) {
+	// for now resource variables are the only thing.
+	parts := strings.SplitN(k, ".", 3)
+
+	return &ResourceVariable{
+		Kind:  parts[0],
+		Name:  parts[1],
+		Field: parts[2],
+	}, nil
+}
+
+// ParseVariables will traverse a HIL tree looking for variables and returns a
+// list of them.
+func ParseVariables(tree ast.Node) ([]Variable, error) {
+	var result []Variable
+	var finalErr error
+
+	visitor := func(n ast.Node) ast.Node {
+		if finalErr != nil {
+			return n
+		}
+
+		switch nt := n.(type) {
+		case *ast.VariableAccess:
+			v, err := NewInterpolatedVariable(nt.Name)
+			if err != nil {
+				finalErr = err
+				return n
+			}
+			result = append(result, v)
+		default:
+			return n
+		}
+
+		return n
+	}
+
+	tree.Accept(visitor)
+
+	if finalErr != nil {
+		return nil, finalErr
+	}
+
+	return result, nil
+}
--- a/lib/cli.go
+++ b/lib/cli.go
@@ -3,16 +3,16 @@
 // Written by James Shubin <james@shubin.ca> and the project contributors
 //
 // This program is free software: you can redistribute it and/or modify
-// it under the terms of the GNU Affero General Public License as published by
+// it under the terms of the GNU General Public License as published by
 // the Free Software Foundation, either version 3 of the License, or
 // (at your option) any later version.
 //
 // This program is distributed in the hope that it will be useful,
 // but WITHOUT ANY WARRANTY; without even the implied warranty of
 // MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-// GNU Affero General Public License for more details.
+// GNU General Public License for more details.
 //
-// You should have received a copy of the GNU Affero General Public License
+// You should have received a copy of the GNU General Public License
 // along with this program.  If not, see <http://www.gnu.org/licenses/>.

 package lib
@@ -24,8 +24,10 @@ import (
 	"os/signal"
 	"syscall"

+	"github.com/purpleidea/mgmt/hcl"
 	"github.com/purpleidea/mgmt/puppet"
 	"github.com/purpleidea/mgmt/yamlgraph"
+	"github.com/purpleidea/mgmt/yamlgraph2"

 	"github.com/urfave/cli"
 )
@@ -71,6 +73,14 @@ func run(c *cli.Context) error {
 			File: &y,
 		}
 	}
+	if y := c.String("yaml2"); c.IsSet("yaml2") {
+		if obj.GAPI != nil {
+			return fmt.Errorf("can't combine YAMLv2 GAPI with existing GAPI")
+		}
+		obj.GAPI = &yamlgraph2.GAPI{
+			File: &y,
+		}
+	}
 	if p := c.String("puppet"); c.IsSet("puppet") {
 		if obj.GAPI != nil {
 			return fmt.Errorf("can't combine puppet GAPI with existing GAPI")
@@ -80,10 +90,22 @@ func run(c *cli.Context) error {
 			PuppetConf:  c.String("puppet-conf"),
 		}
 	}
+	if h := c.String("hcl"); c.IsSet("hcl") {
+		if obj.GAPI != nil {
+			return fmt.Errorf("can't combine hcl GAPI with existing GAPI")
+		}
+		obj.GAPI = &hcl.GAPI{
+			File: &h,
+		}
+	}
 	obj.Remotes = c.StringSlice("remote") // FIXME: GAPI-ify somehow?

 	obj.NoWatch = c.Bool("no-watch")
+	obj.NoConfigWatch = c.Bool("no-config-watch")
+	obj.NoStreamWatch = c.Bool("no-stream-watch")
+
 	obj.Noop = c.Bool("noop")
+	obj.Sema = c.Int("sema")
 	obj.Graphviz = c.String("graphviz")
 	obj.GraphvizFilter = c.String("graphviz-filter")
 	obj.ConvergedTimeout = c.Int("converged-timeout")
@@ -204,6 +226,16 @@ func CLI(program, version string, flags Flags) error {
 					Value: "",
 					Usage: "yaml graph definition to run",
 				},
+				cli.StringFlag{
+					Name:  "yaml2",
+					Value: "",
+					Usage: "yaml graph definition to run (parser v2)",
+				},
+				cli.StringFlag{
+					Name:  "hcl",
+					Value: "",
+					Usage: "hcl graph definition to run",
+				},
 				cli.StringFlag{
 					Name:  "puppet, p",
 					Value: "",
@@ -222,12 +254,26 @@ func CLI(program, version string, flags Flags) error {

 				cli.BoolFlag{
 					Name:  "no-watch",
+					Usage: "do not update graph under any switch events",
+				},
+				cli.BoolFlag{
+					Name:  "no-config-watch",
+					Usage: "do not update graph on config switch events",
+				},
+				cli.BoolFlag{
+					Name:  "no-stream-watch",
 					Usage: "do not update graph on stream switch events",
 				},
+
 				cli.BoolFlag{
 					Name:  "noop",
 					Usage: "globally force all resources into no-op mode",
 				},
+				cli.IntFlag{
+					Name:  "sema",
+					Value: -1,
+					Usage: "globally add a semaphore to all resources with this lock count",
+				},
 				cli.StringFlag{
 					Name:  "graphviz, g",
 					Value: "",
@@ -235,7 +281,7 @@ func CLI(program, version string, flags Flags) error {
 				},
 				cli.StringFlag{
 					Name:  "graphviz-filter, gf",
-					Value: "dot", // directed graph default
+					Value: "",
 					Usage: "graphviz filter to use",
 				},
 				cli.IntFlag{
--- a/lib/main.go
+++ b/lib/main.go
@@ -3,16 +3,16 @@
 // Written by James Shubin <james@shubin.ca> and the project contributors
 //
 // This program is free software: you can redistribute it and/or modify
-// it under the terms of the GNU Affero General Public License as published by
+// it under the terms of the GNU General Public License as published by
 // the Free Software Foundation, either version 3 of the License, or
 // (at your option) any later version.
 //
 // This program is distributed in the hope that it will be useful,
 // but WITHOUT ANY WARRANTY; without even the implied warranty of
 // MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-// GNU Affero General Public License for more details.
+// GNU General Public License for more details.
 //
-// You should have received a copy of the GNU Affero General Public License
+// You should have received a copy of the GNU General Public License
 // along with this program.  If not, see <http://www.gnu.org/licenses/>.

 package lib
@@ -65,8 +65,12 @@ type Main struct {
 	GAPI    gapi.GAPI // graph API interface struct
 	Remotes []string  // list of remote graph definitions to run

-	NoWatch          bool   // do not update graph on watched graph definition file changes
+	NoWatch       bool // do not change graph under any circumstances
+	NoConfigWatch bool // do not update graph due to config changes
+	NoStreamWatch bool // do not update graph due to stream changes
+
 	Noop             bool   // globally force all resources into no-op mode
+	Sema             int    // add a semaphore with this lock count to each resource
 	Graphviz         string // output file for graphviz data
 	GraphvizFilter   string // graphviz filter to use
 	ConvergedTimeout int    // exit after approximately this many seconds in a converged state; -1 to disable
@@ -111,6 +115,15 @@ func (obj *Main) Init() error {
 		return fmt.Errorf("choosing a prefix and the request for a tmp prefix is illogical")
 	}

+	// if we've turned off watching, then be explicit and disable them all!
+	// if all the watches are disabled, then it's equivalent to no watching
+	if obj.NoWatch {
+		obj.NoConfigWatch = true
+		obj.NoStreamWatch = true
+	} else if obj.NoConfigWatch && obj.NoStreamWatch {
+		obj.NoWatch = true
+	}
+
 	obj.idealClusterSize = uint16(obj.IdealClusterSize)
 	if obj.IdealClusterSize < 0 { // value is undefined, set to the default
 		obj.idealClusterSize = etcd.DefaultIdealClusterSize
@@ -285,7 +298,11 @@ func (obj *Main) Run() error {
 		// TODO: Import admin key
 	}

-	var G, oldGraph *pgraph.Graph
+	oldGraph := &pgraph.Graph{}
+	graph := &resources.MGraph{}
+	// pass in the information we need
+	graph.Debug = obj.Flags.Debug
+	graph.Init()

 	// exit after `max-runtime` seconds for no reason at all...
 	if i := obj.MaxRuntime; i > 0 {
@@ -329,6 +346,16 @@ func (obj *Main) Run() error {
 	} else if err := EmbdEtcd.Startup(); err != nil { // startup (returns when etcd main loop is running)
 		obj.Exit(fmt.Errorf("Main: Etcd: Startup failed: %v", err))
 	}
+
+	// wait for etcd server to be ready before continuing...
+	select {
+	case <-EmbdEtcd.ServerReady():
+		log.Printf("Main: Etcd: Server: Ready!")
+		// pass
+	case <-time.After(((etcd.MaxStartServerTimeout * etcd.MaxStartServerRetries) + 1) * time.Second):
+		obj.Exit(fmt.Errorf("Main: Etcd: Startup timeout"))
+	}
+
 	convergerStateFn := func(b bool) error {
 		// exit if we are using the converged timeout and we are the
 		// root node. otherwise, if we are a child node in a remote
@@ -336,7 +363,7 @@ func (obj *Main) Run() error {
 		// state and wait for the parent to trigger the exit.
 		if t := obj.ConvergedTimeout; obj.Depth == 0 && t >= 0 {
 			if b {
-				log.Printf("Converged for %d seconds, exiting!", t)
+				log.Printf("Main: Converged for %d seconds, exiting!", t)
 				obj.Exit(nil) // trigger an exit!
 			}
 			return nil
@@ -348,47 +375,49 @@ func (obj *Main) Run() error {
 		converger.SetStateFn(convergerStateFn)
 	}

-	var gapiChan chan error // stream events are nil errors
+	// implementation of the World API (alternates can be substituted in)
+	world := &etcd.World{
+		Hostname: hostname,
+		EmbdEtcd: EmbdEtcd,
+	}
+
+	graph.Data = &resources.ResData{
+		Hostname:   hostname,
+		Converger:  converger,
+		Prometheus: prom,
+		World:      world,
+		Prefix:     pgraphPrefix,
+		Debug:      obj.Flags.Debug,
+	}
+
+	var gapiChan chan gapi.Next // stream events contain some instructions!
 	if obj.GAPI != nil {
 		data := gapi.Data{
 			Hostname: hostname,
-			// NOTE: alternate implementations can be substituted in
-			World: &etcd.World{
-				Hostname: hostname,
-				EmbdEtcd: EmbdEtcd,
-			},
-			Noop:    obj.Noop,
-			NoWatch: obj.NoWatch,
+			World:    world,
+			Noop:     obj.Noop,
+			//NoWatch:  obj.NoWatch,
+			NoConfigWatch: obj.NoConfigWatch,
+			NoStreamWatch: obj.NoStreamWatch,
 		}
 		if err := obj.GAPI.Init(data); err != nil {
 			obj.Exit(fmt.Errorf("Main: GAPI: Init failed: %v", err))
-		} else if !obj.NoWatch {
+		} else {
+			// this must generate at least one event for it to work
 			gapiChan = obj.GAPI.Next() // stream of graph switch events!
 		}
 	}

 	exitchan := make(chan struct{}) // exit on close
 	go func() {
-		startChan := make(chan struct{}) // start signal
-		close(startChan)                 // kick it off!
-
-		log.Println("Etcd: Starting...")
-		etcdChan := etcd.WatchAll(EmbdEtcd)
 		first := true // first loop or not
 		for {
 			log.Println("Main: Waiting...")
+			// The GAPI should always kick off an event on Next() at
+			// startup when (and if) it indeed has a graph to share!
+			fastPause := false
 			select {
-			case <-startChan: // kick the loop once at start
-				startChan = nil // disable
-				// pass
-
-			case b := <-etcdChan:
-				if !b { // ignore the message
-					continue
-				}
-				// everything else passes through to cause a compile!
-
-			case err, ok := <-gapiChan:
+			case next, ok := <-gapiChan:
 				if !ok { // channel closed
 					if obj.Flags.Debug {
 						log.Printf("Main: GAPI exited")
@@ -396,97 +425,159 @@ func (obj *Main) Run() error {
 					gapiChan = nil // disable it
 					continue
 				}
-				if err != nil {
-					obj.Exit(err) // trigger exit
-					continue      // wait for exitchan
+
+				// if we've been asked to exit...
+				if next.Exit {
+					obj.Exit(next.Err) // trigger exit
+					continue           // wait for exitchan
 				}
-				if obj.NoWatch { // extra safety for bad GAPI's
-					log.Printf("Main: GAPI stream should be quiet with NoWatch!") // fix the GAPI!
-					continue                                                      // no stream events should be sent
+
+				// the gapi lets us send an error to the channel
+				// this means there was a failure, but not fatal
+				if err := next.Err; err != nil {
+					log.Printf("Main: Error with graph stream: %v", err)
+					continue // wait for another event
 				}
+				// everything else passes through to cause a compile!
+
+				fastPause = next.Fast // should we pause fast?

 			case <-exitchan:
 				return
 			}

 			if obj.GAPI == nil {
-				log.Printf("Config: GAPI is empty!")
+				log.Printf("Main: GAPI is empty!")
 				continue
 			}

 			// we need the vertices to be paused to work on them, so
 			// run graph vertex LOCK...
 			if !first { // TODO: we can flatten this check out I think
-				converger.Pause() // FIXME: add sync wait?
-				G.Pause()         // sync
+				converger.Pause()      // FIXME: add sync wait?
+				graph.Pause(fastPause) // sync

-				//G.UnGroup() // FIXME: implement me if needed!
+				//graph.UnGroup() // FIXME: implement me if needed!
 			}

 			// make the graph from yaml, lib, puppet->yaml, or dsl!
 			newGraph, err := obj.GAPI.Graph() // generate graph!
 			if err != nil {
-				log.Printf("Config: Error creating new graph: %v", err)
+				log.Printf("Main: Error creating new graph: %v", err)
 				// unpause!
 				if !first {
-					G.Start(first)    // sync
-					converger.Start() // after G.Start()
+					graph.Start(first) // sync
+					converger.Start()  // after Start()
 				}
 				continue
 			}
-			newGraph.Flags = pgraph.Flags{Debug: obj.Flags.Debug}
-			// pass in the information we need
-			newGraph.AssociateData(&resources.Data{
-				Converger:  converger,
-				Prometheus: prom,
-				Prefix:     pgraphPrefix,
-				Debug:      obj.Flags.Debug,
-			})
+			if obj.Flags.Debug {
+				log.Printf("Main: New Graph: %v", newGraph)
+			}

-			// apply the global noop parameter if requested
-			if obj.Noop {
-				for _, m := range newGraph.GraphMetas() {
+			// this edits the paused vertices, but it is safe to do
+			// so even if we don't use this new graph, since those
+			// value should be the same for existing vertices...
+			for _, v := range newGraph.Vertices() {
+				m := resources.VtoR(v).Meta()
+				// apply the global noop parameter if requested
+				if obj.Noop {
 					m.Noop = obj.Noop
 				}
+
+				// append the semaphore to each resource
+				if obj.Sema > 0 { // NOTE: size == 0 would block
+					// a semaphore with an empty id is valid
+					m.Sema = append(m.Sema, fmt.Sprintf(":%d", obj.Sema))
+				}
 			}

-			// FIXME: make sure we "UnGroup()" any semi-destructive
-			// changes to the resources so our efficient GraphSync
-			// will be able to re-use and cmp to the old graph.
+			// We don't have to "UnGroup()" to compare, since we
+			// save the old graph to use when we compare.
+			// TODO: Does this hurt performance or graph changes ?
 			log.Printf("Main: GraphSync...")
-			newFullGraph, err := newGraph.GraphSync(oldGraph)
-			if err != nil {
-				log.Printf("Config: Error running graph sync: %v", err)
+			vertexCmpFn := func(v1, v2 pgraph.Vertex) (bool, error) {
+				return resources.VtoR(v1).Compare(resources.VtoR(v2)), nil
+			}
+			vertexAddFn := func(v pgraph.Vertex) error {
+				err := resources.VtoR(v).Validate()
+				return errwrap.Wrapf(err, "could not Validate() resource")
+			}
+			vertexRemoveFn := func(v pgraph.Vertex) error {
+				// wait for exit before starting new graph!
+				resources.VtoR(v).Exit() // sync
+				return nil
+			}
+			edgeCmpFn := func(e1, e2 pgraph.Edge) (bool, error) {
+				edge1 := e1.(*resources.Edge) // panic if wrong
+				edge2 := e2.(*resources.Edge) // panic if wrong
+				return edge1.Compare(edge2), nil
+			}
+			// on success, this updates the receiver graph...
+			if err := oldGraph.GraphSync(newGraph, vertexCmpFn, vertexAddFn, vertexRemoveFn, edgeCmpFn); err != nil {
+				log.Printf("Main: Error running graph sync: %v", err)
 				// unpause!
 				if !first {
-					G.Start(first)    // sync
-					converger.Start() // after G.Start()
+					graph.Start(first) // sync
+					converger.Start()  // after Start()
 				}
 				continue
 			}
-			oldGraph = newFullGraph // save old graph
-			G = oldGraph.Copy()     // copy to active graph

-			G.AutoEdges() // add autoedges; modifies the graph
-			G.AutoGroup() // run autogroup; modifies the graph
+			//savedGraph := oldGraph.Copy() // save a copy for errors
+
+			// TODO: should we call each Res.Setup() here instead?
+
+			// add autoedges; modifies the graph only if no error
+			if err := resources.AutoEdges(oldGraph); err != nil {
+				log.Printf("Main: Error running auto edges: %v", err)
+				// unpause!
+				if !first {
+					graph.Start(first) // sync
+					converger.Start()  // after Start()
+				}
+				continue
+			}
+
+			// at this point, any time we error after a destructive
+			// modification of the graph we need to restore the old
+			// graph that was previously running, eg:
+			//
+			//	oldGraph = savedGraph.Copy()
+			//
+			// which we are (luckily) able to avoid testing for now
+
+			resources.AutoGroup(oldGraph, &resources.NonReachabilityGrouper{}) // run autogroup; modifies the graph
 			// TODO: do we want to do a transitive reduction?
 			// FIXME: run a type checker that verifies all the send->recv relationships

-			log.Printf("Graph: %v", G) // show graph
-			if obj.GraphvizFilter != "" {
-				if err := G.ExecGraphviz(obj.GraphvizFilter, obj.Graphviz); err != nil {
-					log.Printf("Graphviz: %v", err)
-				} else {
-					log.Printf("Graphviz: Successfully generated graph!")
-				}
+			graph.Update(oldGraph) // copy in structure of new graph
+
+			// Call this here because at this point the graph does
+			// not know anything about the prometheus instance.
+			if err := prom.UpdatePgraphStartTime(); err != nil {
+				log.Printf("Main: Prometheus.UpdatePgraphStartTime() errored: %v", err)
 			}
-			// G.Start(...) needs to be synchronous or wait,
+			// Start() needs to be synchronous or wait,
 			// because if half of the nodes are started and
 			// some are not ready yet and the EtcdWatch
-			// loops, we'll cause G.Pause(...) before we
+			// loops, we'll cause Pause() before we
 			// even got going, thus causing nil pointer errors
-			G.Start(first)    // sync
-			converger.Start() // after G.Start()
+			graph.Start(first) // sync
+			converger.Start()  // after Start()
+
+			log.Printf("Main: Graph: %v", graph) // show graph
+			if obj.Graphviz != "" {
+				filter := obj.GraphvizFilter
+				if filter == "" {
+					filter = "dot" // directed graph default
+				}
+				if err := graph.ExecGraphviz(filter, obj.Graphviz, hostname); err != nil {
+					log.Printf("Main: Graphviz: %v", err)
+				} else {
+					log.Printf("Main: Graphviz: Successfully generated graph!")
+				}
+			}
 			first = false
 		}
 	}()
@@ -494,7 +585,7 @@ func (obj *Main) Run() error {
 	configWatcher := recwatch.NewConfigWatcher()
 	configWatcher.Flags = recwatch.Flags{Debug: obj.Flags.Debug}
 	events := configWatcher.Events()
-	if !obj.NoWatch {
+	if !obj.NoWatch { // FIXME: fit this into a clean GAPI?
 		configWatcher.Add(obj.Remotes...) // add all the files...
 	} else {
 		events = nil // signal that no-watch is true
@@ -538,6 +629,14 @@ func (obj *Main) Run() error {
 	// TODO: is there any benefit to running the remotes above in the loop?
 	// wait for etcd to be running before we remote in, which we do above!
 	go remotes.Run()
+	// wait for remotes to be ready before continuing...
+	select {
+	case <-remotes.Ready():
+		log.Printf("Main: Remotes: Run: Ready!")
+		// pass
+		//case <-time.After( ? * time.Second):
+		//	obj.Exit(fmt.Errorf("Main: Remotes: Run timeout"))
+	}

 	if obj.GAPI == nil {
 		converger.Start() // better start this for empty graphs
@@ -546,7 +645,7 @@ func (obj *Main) Run() error {

 	reterr := <-obj.exit // wait for exit signal

-	log.Println("Destroy...")
+	log.Println("Main: Destroy...")

 	if obj.GAPI != nil {
 		if err := obj.GAPI.Close(); err != nil {
@@ -564,7 +663,7 @@ func (obj *Main) Run() error {
 	// tell inner main loop to exit
 	close(exitchan)

-	G.Exit() // tells all the children to exit, and waits for them to do so
+	graph.Exit() // tells all the children to exit, and waits for them to do so

 	// cleanup etcd main loop last so it can process everything first
 	if err := EmbdEtcd.Destroy(); err != nil { // shutdown and cleanup etcd
@@ -581,7 +680,7 @@ func (obj *Main) Run() error {
 	}

 	if obj.Flags.Debug {
-		log.Printf("Main: Graph: %v", G)
+		log.Printf("Main: Graph: %v", graph)
 	}

 	// TODO: wait for each vertex to exit...
--- a/main.go
+++ b/main.go
@@ -3,16 +3,16 @@
 // Written by James Shubin <james@shubin.ca> and the project contributors
 //
 // This program is free software: you can redistribute it and/or modify
-// it under the terms of the GNU Affero General Public License as published by
+// it under the terms of the GNU General Public License as published by
 // the Free Software Foundation, either version 3 of the License, or
 // (at your option) any later version.
 //
 // This program is distributed in the hope that it will be useful,
 // but WITHOUT ANY WARRANTY; without even the implied warranty of
 // MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-// GNU Affero General Public License for more details.
+// GNU General Public License for more details.
 //
-// You should have received a copy of the GNU Affero General Public License
+// You should have received a copy of the GNU General Public License
 // along with this program.  If not, see <http://www.gnu.org/licenses/>.

 package main
--- a/misc/make-deps.sh
+++ b/misc/make-deps.sh
@@ -15,13 +15,14 @@ YUM=`which yum 2>/dev/null`
 DNF=`which dnf 2>/dev/null`
 APT=`which apt-get 2>/dev/null`
 BREW=`which brew 2>/dev/null`
+PACMAN=`which pacman 2>/dev/null`

 # if DNF is available use it
 if [ -x "$DNF" ]; then
 	YUM=$DNF
 fi

-if [ -z "$YUM" -a -z "$APT" -a -z "$BREW" ]; then
+if [ -z "$YUM" -a -z "$APT" -a -z "$BREW" -a -z "$PACMAN" ]; then
 	echo "The package managers can't be found."
 	exit 1
 fi
@@ -41,6 +42,10 @@ if [ ! -z "$BREW" ]; then
 	$BREW install libvirt || true
 fi

+if [ ! -z "$PACMAN" ]; then
+	$sudo_command $PACMAN -S --noconfirm libvirt augeas libpcap
+fi
+
 if [ $travis -eq 0 ]; then
 	if [ ! -z "$YUM" ]; then
 		# some go dependencies are stored in mercurial
@@ -54,6 +59,9 @@ if [ $travis -eq 0 ]; then
 		$sudo_command $APT install -y golang-golang-x-tools || true
 		$sudo_command $APT install -y golang-go.tools || true
 	fi
+	if [ ! -z "$PACMAN" ]; then
+		$sudo_command $PACMAN -S --noconfirm go
+	fi
 fi

 # if golang is too old, we don't want to fail with an obscure error later
@@ -72,4 +80,5 @@ if [[ $ret != 0 ]]; then
 fi
 go get golang.org/x/tools/cmd/stringer	# for automatic stringer-ing
 go get github.com/golang/lint/golint	# for `golint`-ing
+go get -u gopkg.in/alecthomas/gometalinter.v1 && mv "$(dirname $(which gometalinter.v1))/gometalinter.v1" "$(dirname $(which gometalinter.v1))/gometalinter" && gometalinter --install	# bonus
 cd "$XPWD" >/dev/null
--- a/pgp/pgp.go
+++ b/pgp/pgp.go
@@ -3,16 +3,16 @@
 // Written by James Shubin <james@shubin.ca> and the project contributors
 //
 // This program is free software: you can redistribute it and/or modify
-// it under the terms of the GNU Affero General Public License as published by
+// it under the terms of the GNU General Public License as published by
 // the Free Software Foundation, either version 3 of the License, or
 // (at your option) any later version.
 //
 // This program is distributed in the hope that it will be useful,
 // but WITHOUT ANY WARRANTY; without even the implied warranty of
 // MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-// GNU Affero General Public License for more details.
+// GNU General Public License for more details.
 //
-// You should have received a copy of the GNU Affero General Public License
+// You should have received a copy of the GNU General Public License
 // along with this program.  If not, see <http://www.gnu.org/licenses/>.

 package pgp
--- a/pgraph/autoedge.go
+++ b/pgraph/autoedge.go
@@ -1,103 +0,0 @@
-// Mgmt
-// Copyright (C) 2013-2017+ James Shubin and the project contributors
-// Written by James Shubin <james@shubin.ca> and the project contributors
-//
-// This program is free software: you can redistribute it and/or modify
-// it under the terms of the GNU Affero General Public License as published by
-// the Free Software Foundation, either version 3 of the License, or
-// (at your option) any later version.
-//
-// This program is distributed in the hope that it will be useful,
-// but WITHOUT ANY WARRANTY; without even the implied warranty of
-// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-// GNU Affero General Public License for more details.
-//
-// You should have received a copy of the GNU Affero General Public License
-// along with this program.  If not, see <http://www.gnu.org/licenses/>.
-
-// Package pgraph represents the internal "pointer graph" that we use.
-package pgraph
-
-import (
-	"fmt"
-	"log"
-
-	"github.com/purpleidea/mgmt/resources"
-)
-
-// add edges to the vertex in a graph based on if it matches a uid list
-func (g *Graph) addEdgesByMatchingUIDS(v *Vertex, uids []resources.ResUID) []bool {
-	// search for edges and see what matches!
-	var result []bool
-
-	// loop through each uid, and see if it matches any vertex
-	for _, uid := range uids {
-		var found = false
-		// uid is a ResUID object
-		for _, vv := range g.GetVertices() { // search
-			if v == vv { // skip self
-				continue
-			}
-			if g.Flags.Debug {
-				log.Printf("Compile: AutoEdge: Match: %v[%v] with UID: %v[%v]", vv.Kind(), vv.GetName(), uid.Kind(), uid.GetName())
-			}
-			// we must match to an effective UID for the resource,
-			// that is to say, the name value of a res is a helpful
-			// handle, but it is not necessarily a unique identity!
-			// remember, resources can return multiple UID's each!
-			if resources.UIDExistsInUIDs(uid, vv.UIDs()) {
-				// add edge from: vv -> v
-				if uid.Reversed() {
-					txt := fmt.Sprintf("AutoEdge: %v[%v] -> %v[%v]", vv.Kind(), vv.GetName(), v.Kind(), v.GetName())
-					log.Printf("Compile: Adding %v", txt)
-					g.AddEdge(vv, v, NewEdge(txt))
-				} else { // edges go the "normal" way, eg: pkg resource
-					txt := fmt.Sprintf("AutoEdge: %v[%v] -> %v[%v]", v.Kind(), v.GetName(), vv.Kind(), vv.GetName())
-					log.Printf("Compile: Adding %v", txt)
-					g.AddEdge(v, vv, NewEdge(txt))
-				}
-				found = true
-				break
-			}
-		}
-		result = append(result, found)
-	}
-	return result
-}
-
-// AutoEdges adds the automatic edges to the graph.
-func (g *Graph) AutoEdges() {
-	log.Println("Compile: Adding AutoEdges...")
-	for _, v := range g.GetVertices() { // for each vertexes autoedges
-		if !v.Meta().AutoEdge { // is the metaparam true?
-			continue
-		}
-		autoEdgeObj := v.AutoEdges()
-		if autoEdgeObj == nil {
-			log.Printf("%v[%v]: Config: No auto edges were found!", v.Kind(), v.GetName())
-			continue // next vertex
-		}
-
-		for { // while the autoEdgeObj has more uids to add...
-			uids := autoEdgeObj.Next() // get some!
-			if uids == nil {
-				log.Printf("%v[%v]: Config: The auto edge list is empty!", v.Kind(), v.GetName())
-				break // inner loop
-			}
-			if g.Flags.Debug {
-				log.Println("Compile: AutoEdge: UIDS:")
-				for i, u := range uids {
-					log.Printf("Compile: AutoEdge: UID%d: %v", i, u)
-				}
-			}
-
-			// match and add edges
-			result := g.addEdgesByMatchingUIDS(v, uids)
-
-			// report back, and find out if we should continue
-			if !autoEdgeObj.Test(result) {
-				break
-			}
-		}
-	}
-}
--- a/pgraph/graphsync.go
+++ b/pgraph/graphsync.go
@@ -0,0 +1,155 @@
+// Mgmt
+// Copyright (C) 2013-2017+ James Shubin and the project contributors
+// Written by James Shubin <james@shubin.ca> and the project contributors
+//
+// This program is free software: you can redistribute it and/or modify
+// it under the terms of the GNU General Public License as published by
+// the Free Software Foundation, either version 3 of the License, or
+// (at your option) any later version.
+//
+// This program is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+// GNU General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License
+// along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+package pgraph
+
+import (
+	"fmt"
+
+	errwrap "github.com/pkg/errors"
+)
+
+func strVertexCmpFn(v1, v2 Vertex) (bool, error) {
+	if v1.String() == "" || v2.String() == "" {
+		return false, fmt.Errorf("empty vertex")
+	}
+	return v1.String() == v2.String(), nil
+}
+
+func strEdgeCmpFn(e1, e2 Edge) (bool, error) {
+	if e1.String() == "" || e2.String() == "" {
+		return false, fmt.Errorf("empty edge")
+	}
+	return e1.String() == e2.String(), nil
+}
+
+// GraphSync updates the Graph so that it matches the newGraph. It leaves
+// identical elements alone so that they don't need to be refreshed.
+// It tries to mutate existing elements into new ones, if they support this.
+// This updates the Graph on success only.
+// FIXME: should we do this with copies of the vertex resources?
+func (obj *Graph) GraphSync(newGraph *Graph, vertexCmpFn func(Vertex, Vertex) (bool, error), vertexAddFn func(Vertex) error, vertexRemoveFn func(Vertex) error, edgeCmpFn func(Edge, Edge) (bool, error)) error {
+
+	oldGraph := obj.Copy() // work on a copy of the old graph
+	if oldGraph == nil {
+		var err error
+		oldGraph, err = NewGraph(newGraph.GetName()) // copy over the name
+		if err != nil {
+			return errwrap.Wrapf(err, "GraphSync failed")
+		}
+	}
+	oldGraph.SetName(newGraph.GetName()) // overwrite the name
+
+	if vertexCmpFn == nil {
+		vertexCmpFn = strVertexCmpFn // use simple string cmp version
+	}
+	if vertexAddFn == nil {
+		vertexAddFn = func(Vertex) error { return nil } // noop
+	}
+	if vertexRemoveFn == nil {
+		vertexRemoveFn = func(Vertex) error { return nil } // noop
+	}
+	if edgeCmpFn == nil {
+		edgeCmpFn = strEdgeCmpFn // use simple string cmp version
+	}
+
+	var lookup = make(map[Vertex]Vertex)
+	var vertexKeep []Vertex // list of vertices which are the same in new graph
+	var edgeKeep []Edge     // list of vertices which are the same in new graph
+
+	for v := range newGraph.Adjacency() { // loop through the vertices (resources)
+		var vertex Vertex
+		// step one, direct compare with res.Compare
+		if vertex == nil { // redundant guard for consistency
+			fn := func(vv Vertex) (bool, error) {
+				b, err := vertexCmpFn(vv, v)
+				return b, errwrap.Wrapf(err, "vertexCmpFn failed")
+			}
+			var err error
+			vertex, err = oldGraph.VertexMatchFn(fn)
+			if err != nil {
+				return errwrap.Wrapf(err, "VertexMatchFn failed")
+			}
+		}
+
+		// TODO: consider adding a mutate API.
+		// step two, try and mutate with res.Mutate
+		//if vertex == nil { // not found yet...
+		//	vertex = oldGraph.MutateMatch(res)
+		//}
+
+		if vertex == nil { // no match found yet
+			if err := vertexAddFn(v); err != nil {
+				return errwrap.Wrapf(err, "vertexAddFn failed")
+			}
+			vertex = v
+			oldGraph.AddVertex(vertex) // call standalone in case not part of an edge
+		}
+		lookup[v] = vertex                      // used for constructing edges
+		vertexKeep = append(vertexKeep, vertex) // append
+	}
+
+	// get rid of any vertices we shouldn't keep (that aren't in new graph)
+	for v := range oldGraph.Adjacency() {
+		if !VertexContains(v, vertexKeep) {
+			if err := vertexRemoveFn(v); err != nil {
+				return errwrap.Wrapf(err, "vertexRemoveFn failed")
+			}
+			oldGraph.DeleteVertex(v)
+		}
+	}
+
+	// compare edges
+	for v1 := range newGraph.Adjacency() { // loop through the vertices (resources)
+		for v2, e := range newGraph.Adjacency()[v1] {
+			// we have an edge!
+			// lookup vertices (these should exist now)
+			vertex1, exists1 := lookup[v1]
+			vertex2, exists2 := lookup[v2]
+			if !exists1 || !exists2 { // no match found, bug?
+				//if vertex1 == nil || vertex2 == nil { // no match found
+				return fmt.Errorf("new vertices weren't found") // programming error
+			}
+
+			edge, exists := oldGraph.Adjacency()[vertex1][vertex2]
+			if !exists {
+				edge = e // use edge
+			} else if b, err := edgeCmpFn(edge, e); err != nil {
+				return errwrap.Wrapf(err, "edgeCmpFn failed")
+			} else if !b {
+				edge = e // overwrite edge
+			}
+
+			oldGraph.Adjacency()[vertex1][vertex2] = edge // store it (AddEdge)
+			edgeKeep = append(edgeKeep, edge)             // mark as saved
+		}
+	}
+
+	// delete unused edges
+	for v1 := range oldGraph.Adjacency() {
+		for _, e := range oldGraph.Adjacency()[v1] {
+			// we have an edge!
+			if !EdgeContains(e, edgeKeep) {
+				oldGraph.DeleteEdge(e)
+			}
+		}
+	}
+
+	// success
+	*obj = *oldGraph // save old graph
+	return nil
+}
--- a/pgraph/graphsync_test.go
+++ b/pgraph/graphsync_test.go
@@ -0,0 +1,92 @@
+// Mgmt
+// Copyright (C) 2013-2017+ James Shubin and the project contributors
+// Written by James Shubin <james@shubin.ca> and the project contributors
+//
+// This program is free software: you can redistribute it and/or modify
+// it under the terms of the GNU General Public License as published by
+// the Free Software Foundation, either version 3 of the License, or
+// (at your option) any later version.
+//
+// This program is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+// GNU General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License
+// along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+package pgraph
+
+import (
+	"testing"
+)
+
+func TestGraphSync1(t *testing.T) {
+	g := &Graph{}
+	v1 := NV("v1")
+	v2 := NV("v2")
+	v3 := NV("v3")
+
+	e1 := NE("e1")
+	e2 := NE("e2")
+	g.AddEdge(v1, v3, e1)
+	g.AddEdge(v2, v3, e2)
+
+	// new graph
+	newGraph := &Graph{}
+	v4 := NV("v4")
+	v5 := NV("v5")
+	e3 := NE("e3")
+	newGraph.AddEdge(v4, v5, e3)
+
+	err := g.GraphSync(newGraph, nil, nil, nil, nil)
+	if err != nil {
+		t.Errorf("GraphSync failed: %v", err)
+		return
+	}
+
+	// g should change and become the same
+	if s := runGraphCmp(t, g, newGraph); s != "" {
+		t.Errorf("%s", s)
+	}
+}
+
+func TestGraphSync2(t *testing.T) {
+	v1 := NV("v1")
+	v2 := NV("v2")
+	v3 := NV("v3")
+	v4 := NV("v4")
+	v5 := NV("v5")
+	e1 := NE("e1")
+	e2 := NE("e2")
+	e3 := NE("e3")
+
+	g := &Graph{}
+	g.AddEdge(v1, v3, e1)
+	g.AddEdge(v2, v3, e2)
+
+	// new graph
+	newGraph := &Graph{}
+	newGraph.AddEdge(v1, v3, e1)
+	newGraph.AddEdge(v2, v3, e2)
+	newGraph.AddEdge(v4, v5, e3)
+	//newGraph.AddEdge(v3, v4, NE("v3,v4"))
+	//newGraph.AddEdge(v3, v5, NE("v3,v5"))
+
+	// graphs should differ!
+	if runGraphCmp(t, g, newGraph) == "" {
+		t.Errorf("graphs should differ initially")
+		return
+	}
+
+	err := g.GraphSync(newGraph, strVertexCmpFn, vertexAddFn, vertexRemoveFn, strEdgeCmpFn)
+	if err != nil {
+		t.Errorf("GraphSync failed: %v", err)
+		return
+	}
+
+	// g should change and become the same
+	if s := runGraphCmp(t, g, newGraph); s != "" {
+		t.Errorf("%s", s)
+	}
+}
--- a/pgraph/graphviz.go
+++ b/pgraph/graphviz.go
@@ -3,19 +3,19 @@
 // Written by James Shubin <james@shubin.ca> and the project contributors
 //
 // This program is free software: you can redistribute it and/or modify
-// it under the terms of the GNU Affero General Public License as published by
+// it under the terms of the GNU General Public License as published by
 // the Free Software Foundation, either version 3 of the License, or
 // (at your option) any later version.
 //
 // This program is distributed in the hope that it will be useful,
 // but WITHOUT ANY WARRANTY; without even the implied warranty of
 // MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-// GNU Affero General Public License for more details.
+// GNU General Public License for more details.
 //
-// You should have received a copy of the GNU Affero General Public License
+// You should have received a copy of the GNU General Public License
 // along with this program.  If not, see <http://www.gnu.org/licenses/>.

-package pgraph
+package pgraph // TODO: this should be a subpackage

 import (
 	"fmt"
@@ -45,16 +45,16 @@ func (g *Graph) Graphviz() (out string) {
 	out += fmt.Sprintf("\tlabel=\"%s\";\n", g.GetName())
 	//out += "\tnode [shape=box];\n"
 	str := ""
-	for i := range g.Adjacency { // reverse paths
-		out += fmt.Sprintf("\t%s [label=\"%s[%s]\"];\n", i.GetName(), i.Kind(), i.GetName())
-		for j := range g.Adjacency[i] {
-			k := g.Adjacency[i][j]
+	for i := range g.Adjacency() { // reverse paths
+		out += fmt.Sprintf("\t\"%s\" [label=\"%s\"];\n", i, i)
+		for j := range g.Adjacency()[i] {
+			k := g.Adjacency()[i][j]
 			// use str for clearer output ordering
-			if k.Notify {
-				str += fmt.Sprintf("\t%s -> %s [label=%s,style=bold];\n", i.GetName(), j.GetName(), k.Name)
-			} else {
-				str += fmt.Sprintf("\t%s -> %s [label=%s];\n", i.GetName(), j.GetName(), k.Name)
-			}
+			//if fmtBoldFn(k) { // TODO: add this sort of formatting
+			//	str += fmt.Sprintf("\t\"%s\" -> \"%s\" [label=\"%s\",style=bold];\n", i, j, k)
+			//} else {
+			str += fmt.Sprintf("\t\"%s\" -> \"%s\" [label=\"%s\"];\n", i, j, k)
+			//}
 		}
 	}
 	out += str
@@ -64,7 +64,7 @@ func (g *Graph) Graphviz() (out string) {

 // ExecGraphviz writes out the graphviz data and runs the correct graphviz
 // filter command.
-func (g *Graph) ExecGraphviz(program, filename string) error {
+func (g *Graph) ExecGraphviz(program, filename, hostname string) error {

 	switch program {
 	case "dot", "neato", "twopi", "circo", "fdp":
@@ -76,6 +76,10 @@ func (g *Graph) ExecGraphviz(program, filename string) error {
 		return fmt.Errorf("no filename given")
 	}

+	if hostname != "" {
+		filename = fmt.Sprintf("%s@%s", filename, hostname)
+	}
+
 	// run as a normal user if possible when run with sudo
 	uid, err1 := strconv.Atoi(os.Getenv("SUDO_UID"))
 	gid, err2 := strconv.Atoi(os.Getenv("SUDO_GID"))
--- a/pgraph/pgraph.go
+++ b/pgraph/pgraph.go
@@ -3,16 +3,16 @@
 // Written by James Shubin <james@shubin.ca> and the project contributors
 //
 // This program is free software: you can redistribute it and/or modify
-// it under the terms of the GNU Affero General Public License as published by
+// it under the terms of the GNU General Public License as published by
 // the Free Software Foundation, either version 3 of the License, or
 // (at your option) any later version.
 //
 // This program is distributed in the hope that it will be useful,
 // but WITHOUT ANY WARRANTY; without even the implied warranty of
 // MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-// GNU Affero General Public License for more details.
+// GNU General Public License for more details.
 //
-// You should have received a copy of the GNU Affero General Public License
+// You should have received a copy of the GNU General Public License
 // along with this program.  If not, see <http://www.gnu.org/licenses/>.

 // Package pgraph represents the internal "pointer graph" that we use.
@@ -21,30 +21,10 @@ package pgraph
 import (
 	"fmt"
 	"sort"
-	"sync"
-
-	"github.com/purpleidea/mgmt/event"
-	"github.com/purpleidea/mgmt/resources"

 	errwrap "github.com/pkg/errors"
 )

-//go:generate stringer -type=graphState -output=graphstate_stringer.go
-type graphState int
-
-const (
-	graphStateNil graphState = iota
-	graphStateStarting
-	graphStateStarted
-	graphStatePausing
-	graphStatePaused
-)
-
-// Flags contains specific constants used by the graph.
-type Flags struct {
-	Debug bool
-}
-
 // Graph is the graph structure in this library.
 // The graph abstract data type (ADT) is defined as follows:
 // * the directed graph arrows point from left to right ( -> )
@@ -52,76 +32,72 @@ type Flags struct {
 // * IOW, you might see package -> file -> service (where package runs first)
 // * This is also the direction that the notify should happen in...
 type Graph struct {
-	Name      string
-	Adjacency map[*Vertex]map[*Vertex]*Edge // *Vertex -> *Vertex (edge)
-	Flags     Flags
-	state     graphState
-	mutex     *sync.Mutex // used when modifying graph State variable
-	wg        *sync.WaitGroup
+	Name string
+
+	adjacency map[Vertex]map[Vertex]Edge // Vertex -> Vertex (edge)
+	kv        map[string]interface{}     // some values associated with the graph
 }

-// Vertex is the primary vertex struct in this library.
-type Vertex struct {
-	resources.Res       // anonymous field
-	timestamp     int64 // last updated timestamp ?
+// Vertex is the primary vertex struct in this library. It can be anything that
+// implements Stringer. The string output must be stable and unique in a graph.
+type Vertex interface {
+	fmt.Stringer // String() string
 }

-// Edge is the primary edge struct in this library.
-type Edge struct {
-	Name   string
-	Notify bool // should we send a refresh notification along this edge?
+// Edge is the primary edge struct in this library. It can be anything that
+// implements Stringer. The string output must be stable and unique in a graph.
+type Edge interface {
+	fmt.Stringer // String() string
+}

-	refresh bool // is there a notify pending for the dest vertex ?
+// Init initializes the graph which populates all the internal structures.
+func (g *Graph) Init() error {
+	if g.Name == "" { // FIXME: is this really a good requirement?
+		return fmt.Errorf("can't initialize graph with empty name")
+	}
+
+	//g.adjacency = make(map[Vertex]map[Vertex]Edge) // not required
+	//g.kv = make(map[string]interface{}) // not required
+	return nil
 }

 // NewGraph builds a new graph.
-func NewGraph(name string) *Graph {
-	return &Graph{
-		Name:      name,
-		Adjacency: make(map[*Vertex]map[*Vertex]*Edge),
-		state:     graphStateNil,
-		// ptr b/c: Mutex/WaitGroup must not be copied after first use
-		mutex: &sync.Mutex{},
-		wg:    &sync.WaitGroup{},
-	}
-}
-
-// NewVertex returns a new graph vertex struct with a contained resource.
-func NewVertex(r resources.Res) *Vertex {
-	return &Vertex{
-		Res: r,
-	}
-}
-
-// NewEdge returns a new graph edge struct.
-func NewEdge(name string) *Edge {
-	return &Edge{
+func NewGraph(name string) (*Graph, error) {
+	g := &Graph{
 		Name: name,
 	}
+	if err := g.Init(); err != nil {
+		return nil, err
+	}
+	return g, nil
 }

-// Refresh returns the pending refresh status of this edge.
-func (obj *Edge) Refresh() bool {
-	return obj.refresh
+// Value returns a value stored alongside the graph in a particular key.
+func (g *Graph) Value(key string) (interface{}, bool) {
+	val, exists := g.kv[key]
+	return val, exists
 }

-// SetRefresh sets the pending refresh status of this edge.
-func (obj *Edge) SetRefresh(b bool) {
-	obj.refresh = b
+// SetValue sets a value to be stored alongside the graph in a particular key.
+func (g *Graph) SetValue(key string, val interface{}) {
+	if g.kv == nil { // initialize on first use
+		g.kv = make(map[string]interface{})
+	}
+	g.kv[key] = val
 }

-// Copy makes a copy of the graph struct
+// Copy makes a copy of the graph struct.
 func (g *Graph) Copy() *Graph {
+	if g == nil { // allow nil graphs through
+		return g
+	}
 	newGraph := &Graph{
 		Name:      g.Name,
-		Adjacency: make(map[*Vertex]map[*Vertex]*Edge, len(g.Adjacency)),
-		Flags:     g.Flags,
-		state:     g.state,
-		mutex:     g.mutex,
-		wg:        g.wg,
+		adjacency: make(map[Vertex]map[Vertex]Edge, len(g.adjacency)),
+		kv:        g.kv,
 	}
-	for k, v := range g.Adjacency {
-		newGraph.Adjacency[k] = v // copy
+	for k, v := range g.adjacency {
+		newGraph.adjacency[k] = v // copy
 	}
 	return newGraph
 }
@@ -136,87 +112,49 @@ func (g *Graph) SetName(name string) {
 	g.Name = name
 }

-// getState returns the state of the graph. This state is used for optimizing
-// certain algorithms by knowing what part of processing the graph is currently
-// undergoing.
-func (g *Graph) getState() graphState {
-	//g.mutex.Lock()
-	//defer g.mutex.Unlock()
-	return g.state
-}
-
-// setState sets the graph state and returns the previous state.
-func (g *Graph) setState(state graphState) graphState {
-	g.mutex.Lock()
-	defer g.mutex.Unlock()
-	prev := g.getState()
-	g.state = state
-	return prev
-}
-
-// AddVertex uses variadic input to add all listed vertices to the graph
-func (g *Graph) AddVertex(xv ...*Vertex) {
+// AddVertex uses variadic input to add all listed vertices to the graph.
+func (g *Graph) AddVertex(xv ...Vertex) {
+	if g.adjacency == nil { // initialize on first use
+		g.adjacency = make(map[Vertex]map[Vertex]Edge)
+	}
 	for _, v := range xv {
-		if _, exists := g.Adjacency[v]; !exists {
-			g.Adjacency[v] = make(map[*Vertex]*Edge)
+		if _, exists := g.adjacency[v]; !exists {
+			g.adjacency[v] = make(map[Vertex]Edge)
 		}
 	}
 }

 // DeleteVertex deletes a particular vertex from the graph.
-func (g *Graph) DeleteVertex(v *Vertex) {
-	delete(g.Adjacency, v)
-	for k := range g.Adjacency {
-		delete(g.Adjacency[k], v)
+func (g *Graph) DeleteVertex(v Vertex) {
+	delete(g.adjacency, v)
+	for k := range g.adjacency {
+		delete(g.adjacency[k], v)
 	}
 }

 // AddEdge adds a directed edge to the graph from v1 to v2.
-func (g *Graph) AddEdge(v1, v2 *Vertex, e *Edge) {
+func (g *Graph) AddEdge(v1, v2 Vertex, e Edge) {
 	// NOTE: this doesn't allow more than one edge between two vertexes...
 	g.AddVertex(v1, v2) // supports adding N vertices now
 	// TODO: check if an edge exists to avoid overwriting it!
 	// NOTE: VertexMerge() depends on overwriting it at the moment...
-	g.Adjacency[v1][v2] = e
+	g.adjacency[v1][v2] = e
 }

 // DeleteEdge deletes a particular edge from the graph.
-// FIXME: add test cases
-func (g *Graph) DeleteEdge(e *Edge) {
-	for v1 := range g.Adjacency {
-		for v2, edge := range g.Adjacency[v1] {
+func (g *Graph) DeleteEdge(e Edge) {
+	for v1 := range g.adjacency {
+		for v2, edge := range g.adjacency[v1] {
 			if e == edge {
-				delete(g.Adjacency[v1], v2)
+				delete(g.adjacency[v1], v2)
 			}
 		}
 	}
 }

-// CompareMatch searches for an equivalent resource in the graph and returns the
-// vertex it is found in, or nil if not found.
-func (g *Graph) CompareMatch(obj resources.Res) *Vertex {
-	for v := range g.Adjacency {
-		if v.Res.Compare(obj) {
-			return v
-		}
-	}
-	return nil
-}
-
-// TODO: consider adding a mutate API.
-//func (g *Graph) MutateMatch(obj resources.Res) *Vertex {
-//	for v := range g.Adjacency {
-//		if err := v.Res.Mutate(obj); err == nil {
-//			// transmogrified!
-//			return v
-//		}
-//	}
-//	return nil
-//}
-
 // HasVertex returns if the input vertex exists in the graph.
-func (g *Graph) HasVertex(v *Vertex) bool {
-	if _, exists := g.Adjacency[v]; exists {
+func (g *Graph) HasVertex(v Vertex) bool {
+	if _, exists := g.adjacency[v]; exists {
 		return true
 	}
 	return false
@@ -224,33 +162,40 @@ func (g *Graph) HasVertex(v *Vertex) bool {

 // NumVertices returns the number of vertices in the graph.
 func (g *Graph) NumVertices() int {
-	return len(g.Adjacency)
+	return len(g.adjacency)
 }

 // NumEdges returns the number of edges in the graph.
 func (g *Graph) NumEdges() int {
 	count := 0
-	for k := range g.Adjacency {
-		count += len(g.Adjacency[k])
+	for k := range g.adjacency {
+		count += len(g.adjacency[k])
 	}
 	return count
 }

-// GetVertices returns a randomly sorted slice of all vertices in the graph
+// Adjacency returns the adjacency map representing this graph. This is useful
+// for users who which to operate on the raw data structure more efficiently.
+// This works because maps are reference types so we can edit this at will.
+func (g *Graph) Adjacency() map[Vertex]map[Vertex]Edge {
+	return g.adjacency
+}
+
+// Vertices returns a randomly sorted slice of all vertices in the graph.
 // The order is random, because the map implementation is intentionally so!
-func (g *Graph) GetVertices() []*Vertex {
-	var vertices []*Vertex
-	for k := range g.Adjacency {
+func (g *Graph) Vertices() []Vertex {
+	var vertices []Vertex
+	for k := range g.adjacency {
 		vertices = append(vertices, k)
 	}
 	return vertices
 }

-// GetVerticesChan returns a channel of all vertices in the graph.
-func (g *Graph) GetVerticesChan() chan *Vertex {
-	ch := make(chan *Vertex)
-	go func(ch chan *Vertex) {
-		for k := range g.Adjacency {
+// VerticesChan returns a channel of all vertices in the graph.
+func (g *Graph) VerticesChan() chan Vertex {
+	ch := make(chan Vertex)
+	go func(ch chan Vertex) {
+		for k := range g.adjacency {
 			ch <- k
 		}
 		close(ch)
@@ -259,17 +204,17 @@ func (g *Graph) GetVerticesChan() chan *Vertex {
 }

 // VertexSlice is a linear list of vertices. It can be sorted.
-type VertexSlice []*Vertex
+type VertexSlice []Vertex

 func (vs VertexSlice) Len() int           { return len(vs) }
 func (vs VertexSlice) Swap(i, j int)      { vs[i], vs[j] = vs[j], vs[i] }
 func (vs VertexSlice) Less(i, j int) bool { return vs[i].String() < vs[j].String() }

-// GetVerticesSorted returns a sorted slice of all vertices in the graph
-// The order is sorted by String() to avoid the non-determinism in the map type
-func (g *Graph) GetVerticesSorted() []*Vertex {
-	var vertices []*Vertex
-	for k := range g.Adjacency {
+// VerticesSorted returns a sorted slice of all vertices in the graph.
+// The order is sorted by String() to avoid the non-determinism in the map type.
+func (g *Graph) VerticesSorted() []Vertex {
+	var vertices []Vertex
+	for k := range g.adjacency {
 		vertices = append(vertices, k)
 	}
 	sort.Sort(VertexSlice(vertices)) // add determinism
@@ -281,19 +226,14 @@ func (g *Graph) String() string {
 	return fmt.Sprintf("Vertices(%d), Edges(%d)", g.NumVertices(), g.NumEdges())
 }

-// String returns the canonical form for a vertex
-func (v *Vertex) String() string {
-	return fmt.Sprintf("%s[%s]", v.Res.Kind(), v.Res.GetName())
-}
-
 // IncomingGraphVertices returns an array (slice) of all directed vertices to
 // vertex v (??? -> v). OKTimestamp should probably use this.
-func (g *Graph) IncomingGraphVertices(v *Vertex) []*Vertex {
+func (g *Graph) IncomingGraphVertices(v Vertex) []Vertex {
 	// TODO: we might be able to implement this differently by reversing
 	// the Adjacency graph and then looping through it again...
-	var s []*Vertex
-	for k := range g.Adjacency { // reverse paths
-		for w := range g.Adjacency[k] {
+	var s []Vertex
+	for k := range g.adjacency { // reverse paths
+		for w := range g.adjacency[k] {
 			if w == v {
 				s = append(s, k)
 			}
@@ -304,9 +244,9 @@ func (g *Graph) IncomingGraphVertices(v *Vertex) []*Vertex {

 // OutgoingGraphVertices returns an array (slice) of all vertices that vertex v
 // points to (v -> ???). Poke should probably use this.
-func (g *Graph) OutgoingGraphVertices(v *Vertex) []*Vertex {
-	var s []*Vertex
-	for k := range g.Adjacency[v] { // forward paths
+func (g *Graph) OutgoingGraphVertices(v Vertex) []Vertex {
+	var s []Vertex
+	for k := range g.adjacency[v] { // forward paths
 		s = append(s, k)
 	}
 	return s
@@ -314,18 +254,18 @@ func (g *Graph) OutgoingGraphVertices(v *Vertex) []*Vertex {

 // GraphVertices returns an array (slice) of all vertices that connect to vertex v.
 // This is the union of IncomingGraphVertices and OutgoingGraphVertices.
-func (g *Graph) GraphVertices(v *Vertex) []*Vertex {
-	var s []*Vertex
+func (g *Graph) GraphVertices(v Vertex) []Vertex {
+	var s []Vertex
 	s = append(s, g.IncomingGraphVertices(v)...)
 	s = append(s, g.OutgoingGraphVertices(v)...)
 	return s
 }

 // IncomingGraphEdges returns all of the edges that point to vertex v (??? -> v).
-func (g *Graph) IncomingGraphEdges(v *Vertex) []*Edge {
-	var edges []*Edge
-	for v1 := range g.Adjacency { // reverse paths
-		for v2, e := range g.Adjacency[v1] {
+func (g *Graph) IncomingGraphEdges(v Vertex) []Edge {
+	var edges []Edge
+	for v1 := range g.adjacency { // reverse paths
+		for v2, e := range g.adjacency[v1] {
 			if v2 == v {
 				edges = append(edges, e)
 			}
@@ -335,9 +275,9 @@ func (g *Graph) IncomingGraphEdges(v *Vertex) []*Edge {
 }

 // OutgoingGraphEdges returns all of the edges that point from vertex v (v -> ???).
-func (g *Graph) OutgoingGraphEdges(v *Vertex) []*Edge {
-	var edges []*Edge
-	for _, e := range g.Adjacency[v] { // forward paths
+func (g *Graph) OutgoingGraphEdges(v Vertex) []Edge {
+	var edges []Edge
+	for _, e := range g.adjacency[v] { // forward paths
 		edges = append(edges, e)
 	}
 	return edges
@@ -345,18 +285,18 @@ func (g *Graph) OutgoingGraphEdges(v *Vertex) []*Edge {

 // GraphEdges returns an array (slice) of all edges that connect to vertex v.
 // This is the union of IncomingGraphEdges and OutgoingGraphEdges.
-func (g *Graph) GraphEdges(v *Vertex) []*Edge {
-	var edges []*Edge
+func (g *Graph) GraphEdges(v Vertex) []Edge {
+	var edges []Edge
 	edges = append(edges, g.IncomingGraphEdges(v)...)
 	edges = append(edges, g.OutgoingGraphEdges(v)...)
 	return edges
 }

 // DFS returns a depth first search for the graph, starting at the input vertex.
-func (g *Graph) DFS(start *Vertex) []*Vertex {
-	var d []*Vertex // discovered
-	var s []*Vertex // stack
-	if _, exists := g.Adjacency[start]; !exists {
+func (g *Graph) DFS(start Vertex) []Vertex {
+	var d []Vertex // discovered
+	var s []Vertex // stack
+	if _, exists := g.adjacency[start]; !exists {
 		return nil // TODO: error
 	}
 	v := start
@@ -376,64 +316,65 @@ func (g *Graph) DFS(start *Vertex) []*Vertex {
 }

 // FilterGraph builds a new graph containing only vertices from the list.
-func (g *Graph) FilterGraph(name string, vertices []*Vertex) *Graph {
-	newgraph := NewGraph(name)
-	for k1, x := range g.Adjacency {
+func (g *Graph) FilterGraph(name string, vertices []Vertex) (*Graph, error) {
+	newGraph := &Graph{Name: name}
+	if err := newGraph.Init(); err != nil {
+		return nil, errwrap.Wrapf(err, "could not run FilterGraph() properly")
+	}
+	for k1, x := range g.adjacency {
 		for k2, e := range x {
 			//log.Printf("Filter: %s -> %s # %s", k1.Name, k2.Name, e.Name)
 			if VertexContains(k1, vertices) || VertexContains(k2, vertices) {
-				newgraph.AddEdge(k1, k2, e)
+				newGraph.AddEdge(k1, k2, e)
 			}
 		}
 	}
-	return newgraph
+	return newGraph, nil
 }

-// GetDisconnectedGraphs returns a channel containing the N disconnected graphs
-// in our main graph. We can then process each of these in parallel.
-func (g *Graph) GetDisconnectedGraphs() chan *Graph {
-	ch := make(chan *Graph)
-	go func() {
-		var start *Vertex
-		var d []*Vertex // discovered
-		c := g.NumVertices()
-		for len(d) < c {
+// DisconnectedGraphs returns a list containing the N disconnected graphs.
+func (g *Graph) DisconnectedGraphs() ([]*Graph, error) {
+	graphs := []*Graph{}
+	var start Vertex
+	var d []Vertex // discovered
+	c := g.NumVertices()
+	for len(d) < c {

-			// get an undiscovered vertex to start from
-			for _, s := range g.GetVertices() {
-				if !VertexContains(s, d) {
-					start = s
-				}
+		// get an undiscovered vertex to start from
+		for _, s := range g.Vertices() {
+			if !VertexContains(s, d) {
+				start = s
 			}
-
-			// dfs through the graph
-			dfs := g.DFS(start)
-			// filter all the collected elements into a new graph
-			newgraph := g.FilterGraph(g.Name, dfs)
-
-			// add number of elements found to found variable
-			d = append(d, dfs...) // extend
-
-			// return this new graph to the channel
-			ch <- newgraph
-
-			// if we've found all the elements, then we're done
-			// otherwise loop through to continue...
 		}
-		close(ch)
-	}()
-	return ch
+
+		// dfs through the graph
+		dfs := g.DFS(start)
+		// filter all the collected elements into a new graph
+		newgraph, err := g.FilterGraph(g.Name, dfs)
+		if err != nil {
+			return nil, errwrap.Wrapf(err, "could not run DisconnectedGraphs() properly")
+		}
+		// add number of elements found to found variable
+		d = append(d, dfs...) // extend
+
+		// append this new graph to the list
+		graphs = append(graphs, newgraph)
+
+		// if we've found all the elements, then we're done
+		// otherwise loop through to continue...
+	}
+	return graphs, nil
 }

 // InDegree returns the count of vertices that point to me in one big lookup map.
-func (g *Graph) InDegree() map[*Vertex]int {
-	result := make(map[*Vertex]int)
-	for k := range g.Adjacency {
+func (g *Graph) InDegree() map[Vertex]int {
+	result := make(map[Vertex]int)
+	for k := range g.adjacency {
 		result[k] = 0 // initialize
 	}

-	for k := range g.Adjacency {
-		for z := range g.Adjacency[k] {
+	for k := range g.adjacency {
+		for z := range g.adjacency[k] {
 			result[z]++
 		}
 	}
@@ -441,12 +382,12 @@ func (g *Graph) InDegree() map[*Vertex]int {
 }

 // OutDegree returns the count of vertices that point away in one big lookup map.
-func (g *Graph) OutDegree() map[*Vertex]int {
-	result := make(map[*Vertex]int)
+func (g *Graph) OutDegree() map[Vertex]int {
+	result := make(map[Vertex]int)

-	for k := range g.Adjacency {
+	for k := range g.adjacency {
 		result[k] = 0 // initialize
-		for range g.Adjacency[k] {
+		for range g.adjacency[k] {
 			result[k]++
 		}
 	}
@@ -454,12 +395,12 @@ func (g *Graph) OutDegree() map[*Vertex]int {
 }

 // TopologicalSort returns the sort of graph vertices in that order.
-// based on descriptions and code from wikipedia and rosetta code
+// It is based on descriptions and code from wikipedia and rosetta code.
 // TODO: add memoization, and cache invalidation to speed this up :)
-func (g *Graph) TopologicalSort() ([]*Vertex, error) { // kahn's algorithm
-	var L []*Vertex                    // empty list that will contain the sorted elements
-	var S []*Vertex                    // set of all nodes with no incoming edges
-	remaining := make(map[*Vertex]int) // amount of edges remaining
+func (g *Graph) TopologicalSort() ([]Vertex, error) { // kahn's algorithm
+	var L []Vertex                    // empty list that will contain the sorted elements
+	var S []Vertex                    // set of all nodes with no incoming edges
+	remaining := make(map[Vertex]int) // amount of edges remaining

 	for v, d := range g.InDegree() {
 		if d == 0 {
@@ -476,7 +417,7 @@ func (g *Graph) TopologicalSort() ([]*Vertex, error) { // kahn's algorithm
 		v := S[last]
 		S = S[:last]
 		L = append(L, v) // add v to tail of L
-		for n := range g.Adjacency[v] {
+		for n := range g.adjacency[v] {
 			// for each node n remaining in the graph, consume from
 			// remaining, so for remaining[n] > 0
 			if remaining[n] > 0 {
@@ -491,7 +432,7 @@ func (g *Graph) TopologicalSort() ([]*Vertex, error) { // kahn's algorithm
 	// if graph has edges, eg if any value in rem is > 0
 	for c, in := range remaining {
 		if in > 0 {
-			for n := range g.Adjacency[c] {
+			for n := range g.adjacency[c] {
 				if remaining[n] > 0 {
 					return nil, fmt.Errorf("not a dag")
 				}
@@ -510,19 +451,19 @@ func (g *Graph) TopologicalSort() ([]*Vertex, error) { // kahn's algorithm
 // actually return a tree if we cared about correctness.
 // This operates by a recursive algorithm; a more efficient version is likely.
 // If you don't give this function a DAG, you might cause infinite recursion!
-func (g *Graph) Reachability(a, b *Vertex) []*Vertex {
+func (g *Graph) Reachability(a, b Vertex) []Vertex {
 	if a == nil || b == nil {
 		return nil
 	}
 	vertices := g.OutgoingGraphVertices(a) // what points away from a ?
 	if len(vertices) == 0 {
-		return []*Vertex{} // nope
+		return []Vertex{} // nope
 	}
 	if VertexContains(b, vertices) {
-		return []*Vertex{a, b} // found
+		return []Vertex{a, b} // found
 	}
 	// TODO: parallelize this with go routines?
-	var collected = make([][]*Vertex, len(vertices))
+	var collected = make([][]Vertex, len(vertices))
 	pick := -1
 	for i, v := range vertices {
 		collected[i] = g.Reachability(v, b) // find b by recursion
@@ -535,123 +476,111 @@ func (g *Graph) Reachability(a, b *Vertex) []*Vertex {
 		}
 	}
 	if pick < 0 {
-		return []*Vertex{} // nope
+		return []Vertex{} // nope
 	}
-	result := []*Vertex{a} // tack on a
+	result := []Vertex{a} // tack on a
 	result = append(result, collected[pick]...)
 	return result
 }

-// GraphSync updates the oldGraph so that it matches the newGraph receiver. It
-// leaves identical elements alone so that they don't need to be refreshed. It
-// tries to mutate existing elements into new ones, if they support this.
-// FIXME: add test cases
-func (g *Graph) GraphSync(oldGraph *Graph) (*Graph, error) {
-
-	if oldGraph == nil {
-		oldGraph = NewGraph(g.GetName()) // copy over the name
-	}
-	oldGraph.SetName(g.GetName()) // overwrite the name
-
-	var lookup = make(map[*Vertex]*Vertex)
-	var vertexKeep []*Vertex // list of vertices which are the same in new graph
-	var edgeKeep []*Edge     // list of vertices which are the same in new graph
-
-	for v := range g.Adjacency { // loop through the vertices (resources)
-		res := v.Res // resource
-		var vertex *Vertex
-
-		// step one, direct compare with res.Compare
-		if vertex == nil { // redundant guard for consistency
-			vertex = oldGraph.CompareMatch(res)
-		}
-
-		// TODO: consider adding a mutate API.
-		// step two, try and mutate with res.Mutate
-		//if vertex == nil { // not found yet...
-		//	vertex = oldGraph.MutateMatch(res)
-		//}
-
-		if vertex == nil { // no match found yet
-			if err := res.Validate(); err != nil {
-				return nil, errwrap.Wrapf(err, "could not Validate() resource")
-			}
-			vertex = v
-			oldGraph.AddVertex(vertex) // call standalone in case not part of an edge
-		}
-		lookup[v] = vertex                      // used for constructing edges
-		vertexKeep = append(vertexKeep, vertex) // append
-	}
-
-	// get rid of any vertices we shouldn't keep (that aren't in new graph)
-	for v := range oldGraph.Adjacency {
-		if !VertexContains(v, vertexKeep) {
-			// wait for exit before starting new graph!
-			v.SendEvent(event.EventExit, nil) // sync
-			v.Res.WaitGroup().Wait()
-			oldGraph.DeleteVertex(v)
+// VertexMatchFn searches for a vertex in the graph and returns the vertex if
+// one matches. It uses a user defined function to match. That function must
+// return true on match, and an error if anything goes wrong.
+func (g *Graph) VertexMatchFn(fn func(Vertex) (bool, error)) (Vertex, error) {
+	for v := range g.adjacency {
+		if b, err := fn(v); err != nil {
+			return nil, errwrap.Wrapf(err, "fn in VertexMatchFn() errored")
+		} else if b {
+			return v, nil
 		}
 	}
-
-	// compare edges
-	for v1 := range g.Adjacency { // loop through the vertices (resources)
-		for v2, e := range g.Adjacency[v1] {
-			// we have an edge!
-
-			// lookup vertices (these should exist now)
-			//res1 := v1.Res // resource
-			//res2 := v2.Res
-			//vertex1 := oldGraph.CompareMatch(res1)
-			//vertex2 := oldGraph.CompareMatch(res2)
-			vertex1, exists1 := lookup[v1]
-			vertex2, exists2 := lookup[v2]
-			if !exists1 || !exists2 { // no match found, bug?
-				//if vertex1 == nil || vertex2 == nil { // no match found
-				return nil, fmt.Errorf("new vertices weren't found") // programming error
-			}
-
-			edge, exists := oldGraph.Adjacency[vertex1][vertex2]
-			if !exists || edge.Name != e.Name { // TODO: edgeCmp
-				edge = e // use or overwrite edge
-			}
-			oldGraph.Adjacency[vertex1][vertex2] = edge // store it (AddEdge)
-			edgeKeep = append(edgeKeep, edge)           // mark as saved
-		}
-	}
-
-	// delete unused edges
-	for v1 := range oldGraph.Adjacency {
-		for _, e := range oldGraph.Adjacency[v1] {
-			// we have an edge!
-			if !EdgeContains(e, edgeKeep) {
-				oldGraph.DeleteEdge(e)
-			}
-		}
-	}
-
-	return oldGraph, nil
+	return nil, nil // nothing found
 }

-// GraphMetas returns a list of pointers to each of the resource MetaParams.
-func (g *Graph) GraphMetas() []*resources.MetaParams {
-	metas := []*resources.MetaParams{}
-	for v := range g.Adjacency { // loop through the vertices (resources))
-		res := v.Res // resource
-		meta := res.Meta()
-		metas = append(metas, meta)
+// GraphCmp compares the topology of this graph to another and returns nil if
+// they're equal. It uses a user defined function to compare topologically
+// equivalent vertices, and edges.
+// FIXME: add more test cases
+func (g *Graph) GraphCmp(graph *Graph, vertexCmpFn func(Vertex, Vertex) (bool, error), edgeCmpFn func(Edge, Edge) (bool, error)) error {
+	n1, n2 := g.NumVertices(), graph.NumVertices()
+	if n1 != n2 {
+		return fmt.Errorf("base graph has %d vertices, while input graph has %d", n1, n2)
+	}
+	if e1, e2 := g.NumEdges(), graph.NumEdges(); e1 != e2 {
+		return fmt.Errorf("base graph has %d edges, while input graph has %d", e1, e2)
 	}
-	return metas
-}

-// AssociateData associates some data with the object in the graph in question.
-func (g *Graph) AssociateData(data *resources.Data) {
-	for k := range g.Adjacency {
-		k.Res.AssociateData(data)
+	var m = make(map[Vertex]Vertex) // g to graph vertex correspondence
+Loop:
+	// check vertices
+	for v1 := range g.Adjacency() { // for each vertex in g
+		for v2 := range graph.Adjacency() { // does it match in graph ?
+			b, err := vertexCmpFn(v1, v2)
+			if err != nil {
+				return errwrap.Wrapf(err, "could not run vertexCmpFn() properly")
+			}
+			// does it match ?
+			if b {
+				m[v1] = v2 // store the mapping
+				continue Loop
+			}
+		}
+		return fmt.Errorf("base graph, has no match in input graph for: %s", v1)
 	}
+	// vertices match :)
+
+	// is the mapping the right length?
+	if n1 := len(m); n1 != n2 {
+		return fmt.Errorf("mapping only has correspondence of %d, when it should have %d", n1, n2)
+	}
+
+	// check if mapping is unique (are there duplicates?)
+	m1 := []Vertex{}
+	m2 := []Vertex{}
+	for k, v := range m {
+		if VertexContains(k, m1) {
+			return fmt.Errorf("mapping from %s is used more than once to: %s", k, m1)
+		}
+		if VertexContains(v, m2) {
+			return fmt.Errorf("mapping to %s is used more than once from: %s", v, m2)
+		}
+		m1 = append(m1, k)
+		m2 = append(m2, v)
+	}
+
+	// check edges
+	for v1 := range g.Adjacency() { // for each vertex in g
+		v2 := m[v1] // lookup in map to get correspondance
+		// g.Adjacency()[v1] corresponds to graph.Adjacency()[v2]
+		if e1, e2 := len(g.Adjacency()[v1]), len(graph.Adjacency()[v2]); e1 != e2 {
+			return fmt.Errorf("base graph, vertex(%s) has %d edges, while input graph, vertex(%s) has %d", v1, e1, v2, e2)
+		}
+
+		for vv1, ee1 := range g.Adjacency()[v1] {
+			vv2 := m[vv1]
+			ee2 := graph.Adjacency()[v2][vv2]
+
+			// these are edges from v1 -> vv1 via ee1 (graph 1)
+			// to cmp to edges from v2 -> vv2 via ee2 (graph 2)
+
+			// check: (1) vv1 == vv2 ? (we've already checked this!)
+
+			// check: (2) ee1 == ee2
+			b, err := edgeCmpFn(ee1, ee2)
+			if err != nil {
+				return errwrap.Wrapf(err, "could not run edgeCmpFn() properly")
+			}
+			if !b {
+				return fmt.Errorf("base graph edge(%s) doesn't match input graph edge(%s)", ee1, ee2)
+			}
+		}
+	}
+
+	return nil // success!
 }

 // VertexContains is an "in array" function to test for a vertex in a slice of vertices.
-func VertexContains(needle *Vertex, haystack []*Vertex) bool {
+func VertexContains(needle Vertex, haystack []Vertex) bool {
 	for _, v := range haystack {
 		if needle == v {
 			return true
@@ -661,7 +590,7 @@ func VertexContains(needle *Vertex, haystack []*Vertex) bool {
 }

 // EdgeContains is an "in array" function to test for an edge in a slice of edges.
-func EdgeContains(needle *Edge, haystack []*Edge) bool {
+func EdgeContains(needle Edge, haystack []Edge) bool {
 	for _, v := range haystack {
 		if needle == v {
 			return true
@@ -671,12 +600,23 @@ func EdgeContains(needle *Edge, haystack []*Edge) bool {
 }

 // Reverse reverses a list of vertices.
-func Reverse(vs []*Vertex) []*Vertex {
-	//var out []*Vertex       // XXX: golint suggests, but it fails testing
-	out := make([]*Vertex, 0) // empty list
+func Reverse(vs []Vertex) []Vertex {
+	out := []Vertex{}
 	l := len(vs)
 	for i := range vs {
 		out = append(out, vs[l-i-1])
 	}
 	return out
 }
+
+// Sort the list of vertices and return a copy without modifying the input.
+func Sort(vs []Vertex) []Vertex {
+	vertices := []Vertex{}
+	for _, v := range vs { // copy
+		vertices = append(vertices, v)
+	}
+	sort.Sort(VertexSlice(vertices))
+	return vertices
+	// sort.Sort(VertexSlice(vs)) // this is wrong, it would modify input!
+	//return vs
+}
--- a/pgraph/pgraph_test.go
+++ b/pgraph/pgraph_test.go
--- a/pgraph/subgraph.go
+++ b/pgraph/subgraph.go
@@ -0,0 +1,106 @@
+// Mgmt
+// Copyright (C) 2013-2017+ James Shubin and the project contributors
+// Written by James Shubin <james@shubin.ca> and the project contributors
+//
+// This program is free software: you can redistribute it and/or modify
+// it under the terms of the GNU General Public License as published by
+// the Free Software Foundation, either version 3 of the License, or
+// (at your option) any later version.
+//
+// This program is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+// GNU General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License
+// along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+package pgraph
+
+// AddGraph adds the set of edges and vertices of a graph to the existing graph.
+func (g *Graph) AddGraph(graph *Graph) {
+	g.addEdgeVertexGraphHelper(nil, graph, nil, false, false)
+}
+
+// AddEdgeVertexGraph adds a directed edge to the graph from a vertex.
+// This is useful for flattening the relationship between a subgraph and an
+// existing graph, without having to run the subgraph recursively. It adds the
+// maximum number of edges, creating a relationship to every vertex.
+func (g *Graph) AddEdgeVertexGraph(vertex Vertex, graph *Graph, edgeGenFn func(v1, v2 Vertex) Edge) {
+	g.addEdgeVertexGraphHelper(vertex, graph, edgeGenFn, false, false)
+}
+
+// AddEdgeVertexGraphLight adds a directed edge to the graph from a vertex.
+// This is useful for flattening the relationship between a subgraph and an
+// existing graph, without having to run the subgraph recursively. It adds the
+// minimum number of edges, creating a relationship to the vertices with
+// indegree equal to zero.
+func (g *Graph) AddEdgeVertexGraphLight(vertex Vertex, graph *Graph, edgeGenFn func(v1, v2 Vertex) Edge) {
+	g.addEdgeVertexGraphHelper(vertex, graph, edgeGenFn, false, true)
+}
+
+// AddEdgeGraphVertex adds a directed edge to the vertex from a graph.
+// This is useful for flattening the relationship between a subgraph and an
+// existing graph, without having to run the subgraph recursively. It adds the
+// maximum number of edges, creating a relationship from every vertex.
+func (g *Graph) AddEdgeGraphVertex(graph *Graph, vertex Vertex, edgeGenFn func(v1, v2 Vertex) Edge) {
+	g.addEdgeVertexGraphHelper(vertex, graph, edgeGenFn, true, false)
+}
+
+// AddEdgeGraphVertexLight adds a directed edge to the vertex from a graph.
+// This is useful for flattening the relationship between a subgraph and an
+// existing graph, without having to run the subgraph recursively. It adds the
+// minimum number of edges, creating a relationship from the vertices with
+// outdegree equal to zero.
+func (g *Graph) AddEdgeGraphVertexLight(graph *Graph, vertex Vertex, edgeGenFn func(v1, v2 Vertex) Edge) {
+	g.addEdgeVertexGraphHelper(vertex, graph, edgeGenFn, true, true)
+}
+
+// addEdgeVertexGraphHelper is a helper function to add a directed edges to the
+// graph from a vertex, or vice-versa. It operates in this reverse direction by
+// specifying the reverse argument as true. It is useful for flattening the
+// relationship between a subgraph and an existing graph, without having to run
+// the subgraph recursively. It adds the maximum number of edges, creating a
+// relationship to or from every vertex if the light argument is false, and if
+// it is true, it adds the minimum number of edges, creating a relationship to
+// or from the vertices with an indegree or outdegree equal to zero depending on
+// if we specified reverse or not.
+func (g *Graph) addEdgeVertexGraphHelper(vertex Vertex, graph *Graph, edgeGenFn func(v1, v2 Vertex) Edge, reverse, light bool) {
+
+	var degree map[Vertex]int // compute all of the in/outdegree's if needed
+	if light && reverse {
+		degree = graph.OutDegree()
+	} else if light { // && !reverse
+		degree = graph.InDegree()
+	}
+	for _, v := range graph.VerticesSorted() { // sort to help out edgeGenFn
+
+		// forward:
+		// we only want to add edges to indegree == 0, because every
+		// other vertex is a dependency of at least one of those
+
+		// reverse:
+		// we only want to add edges to outdegree == 0, because every
+		// other vertex is a pre-requisite to at least one of these
+		if light && degree[v] != 0 {
+			continue
+		}
+
+		g.AddVertex(v) // ensure vertex is part of the graph
+
+		if vertex != nil && reverse {
+			edge := edgeGenFn(v, vertex) // generate a new unique edge
+			g.AddEdge(v, vertex, edge)
+		} else if vertex != nil { // && !reverse
+			edge := edgeGenFn(vertex, v)
+			g.AddEdge(vertex, v, edge)
+		}
+	}
+
+	// also remember to suck in all of the graph's edges too!
+	for v1 := range graph.Adjacency() {
+		for v2, e := range graph.Adjacency()[v1] {
+			g.AddEdge(v1, v2, e)
+		}
+	}
+}
--- a/pgraph/subgraph_test.go
+++ b/pgraph/subgraph_test.go
@@ -0,0 +1,187 @@
+// Mgmt
+// Copyright (C) 2013-2017+ James Shubin and the project contributors
+// Written by James Shubin <james@shubin.ca> and the project contributors
+//
+// This program is free software: you can redistribute it and/or modify
+// it under the terms of the GNU General Public License as published by
+// the Free Software Foundation, either version 3 of the License, or
+// (at your option) any later version.
+//
+// This program is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+// GNU General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License
+// along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+package pgraph
+
+import (
+	"testing"
+)
+
+func TestAddEdgeGraph1(t *testing.T) {
+	v1 := NV("v1")
+	v2 := NV("v2")
+	v3 := NV("v3")
+	v4 := NV("v4")
+	v5 := NV("v5")
+	e1 := NE("e1")
+	e2 := NE("e2")
+	e3 := NE("e3")
+
+	g := &Graph{}
+	g.AddEdge(v1, v3, e1)
+	g.AddEdge(v2, v3, e2)
+
+	sub := &Graph{}
+	sub.AddEdge(v4, v5, e3)
+
+	g.AddGraph(sub)
+
+	// expected (can re-use the same vertices)
+	expected := &Graph{}
+	expected.AddEdge(v1, v3, e1)
+	expected.AddEdge(v2, v3, e2)
+	expected.AddEdge(v4, v5, e3)
+
+	//expected.AddEdge(v3, v4, NE("v3,v4"))
+	//expected.AddEdge(v3, v5, NE("v3,v5"))
+
+	if s := runGraphCmp(t, g, expected); s != "" {
+		t.Errorf("%s", s)
+	}
+}
+
+func TestAddEdgeVertexGraph1(t *testing.T) {
+	v1 := NV("v1")
+	v2 := NV("v2")
+	v3 := NV("v3")
+	v4 := NV("v4")
+	v5 := NV("v5")
+	e1 := NE("e1")
+	e2 := NE("e2")
+	e3 := NE("e3")
+
+	g := &Graph{}
+	g.AddEdge(v1, v3, e1)
+	g.AddEdge(v2, v3, e2)
+
+	sub := &Graph{}
+	sub.AddEdge(v4, v5, e3)
+
+	g.AddEdgeVertexGraph(v3, sub, edgeGenFn)
+
+	// expected (can re-use the same vertices)
+	expected := &Graph{}
+	expected.AddEdge(v1, v3, e1)
+	expected.AddEdge(v2, v3, e2)
+	expected.AddEdge(v4, v5, e3)
+
+	expected.AddEdge(v3, v4, NE("v3,v4"))
+	expected.AddEdge(v3, v5, NE("v3,v5"))
+
+	if s := runGraphCmp(t, g, expected); s != "" {
+		t.Errorf("%s", s)
+	}
+}
+
+func TestAddEdgeGraphVertex1(t *testing.T) {
+	v1 := NV("v1")
+	v2 := NV("v2")
+	v3 := NV("v3")
+	v4 := NV("v4")
+	v5 := NV("v5")
+	e1 := NE("e1")
+	e2 := NE("e2")
+	e3 := NE("e3")
+
+	g := &Graph{}
+	g.AddEdge(v1, v3, e1)
+	g.AddEdge(v2, v3, e2)
+
+	sub := &Graph{}
+	sub.AddEdge(v4, v5, e3)
+
+	g.AddEdgeGraphVertex(sub, v3, edgeGenFn)
+
+	// expected (can re-use the same vertices)
+	expected := &Graph{}
+	expected.AddEdge(v1, v3, e1)
+	expected.AddEdge(v2, v3, e2)
+	expected.AddEdge(v4, v5, e3)
+
+	expected.AddEdge(v4, v3, NE("v4,v3"))
+	expected.AddEdge(v5, v3, NE("v5,v3"))
+
+	if s := runGraphCmp(t, g, expected); s != "" {
+		t.Errorf("%s", s)
+	}
+}
+
+func TestAddEdgeVertexGraphLight1(t *testing.T) {
+	v1 := NV("v1")
+	v2 := NV("v2")
+	v3 := NV("v3")
+	v4 := NV("v4")
+	v5 := NV("v5")
+	e1 := NE("e1")
+	e2 := NE("e2")
+	e3 := NE("e3")
+
+	g := &Graph{}
+	g.AddEdge(v1, v3, e1)
+	g.AddEdge(v2, v3, e2)
+
+	sub := &Graph{}
+	sub.AddEdge(v4, v5, e3)
+
+	g.AddEdgeVertexGraphLight(v3, sub, edgeGenFn)
+
+	// expected (can re-use the same vertices)
+	expected := &Graph{}
+	expected.AddEdge(v1, v3, e1)
+	expected.AddEdge(v2, v3, e2)
+	expected.AddEdge(v4, v5, e3)
+
+	expected.AddEdge(v3, v4, NE("v3,v4"))
+	//expected.AddEdge(v3, v5, NE("v3,v5")) // not needed with light
+
+	if s := runGraphCmp(t, g, expected); s != "" {
+		t.Errorf("%s", s)
+	}
+}
+
+func TestAddEdgeGraphVertexLight1(t *testing.T) {
+	v1 := NV("v1")
+	v2 := NV("v2")
+	v3 := NV("v3")
+	v4 := NV("v4")
+	v5 := NV("v5")
+	e1 := NE("e1")
+	e2 := NE("e2")
+	e3 := NE("e3")
+
+	g := &Graph{}
+	g.AddEdge(v1, v3, e1)
+	g.AddEdge(v2, v3, e2)
+
+	sub := &Graph{}
+	sub.AddEdge(v4, v5, e3)
+
+	g.AddEdgeGraphVertexLight(sub, v3, edgeGenFn)
+
+	// expected (can re-use the same vertices)
+	expected := &Graph{}
+	expected.AddEdge(v1, v3, e1)
+	expected.AddEdge(v2, v3, e2)
+	expected.AddEdge(v4, v5, e3)
+
+	//expected.AddEdge(v4, v3, NE("v4,v3")) // not needed with light
+	expected.AddEdge(v5, v3, NE("v5,v3"))
+
+	if s := runGraphCmp(t, g, expected); s != "" {
+		t.Errorf("%s", s)
+	}
+}
--- a/pgraph/util_test.go
+++ b/pgraph/util_test.go
@@ -0,0 +1,93 @@
+// Mgmt
+// Copyright (C) 2013-2017+ James Shubin and the project contributors
+// Written by James Shubin <james@shubin.ca> and the project contributors
+//
+// This program is free software: you can redistribute it and/or modify
+// it under the terms of the GNU General Public License as published by
+// the Free Software Foundation, either version 3 of the License, or
+// (at your option) any later version.
+//
+// This program is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+// GNU General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License
+// along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+package pgraph
+
+import (
+	"fmt"
+	"testing"
+)
+
+// vertex is a test struct to test the library.
+type vertex struct {
+	name string
+}
+
+// String is a required method of the Vertex interface that we must fulfill.
+func (v *vertex) String() string {
+	return v.name
+}
+
+// NV is a helper function to make testing easier. It creates a new noop vertex.
+func NV(s string) Vertex {
+	return &vertex{s}
+}
+
+// edge is a test struct to test the library.
+type edge struct {
+	name string
+}
+
+// String is a required method of the Edge interface that we must fulfill.
+func (e *edge) String() string {
+	return e.name
+}
+
+// NE is a helper function to make testing easier. It creates a new noop edge.
+func NE(s string) Edge {
+	return &edge{s}
+}
+
+// edgeGenFn generates unique edges for each vertex pair, assuming unique
+// vertices.
+func edgeGenFn(v1, v2 Vertex) Edge {
+	return NE(fmt.Sprintf("%s,%s", v1, v2))
+}
+
+func vertexAddFn(v Vertex) error {
+	return nil
+}
+
+func vertexRemoveFn(v Vertex) error {
+	return nil
+}
+
+func runGraphCmp(t *testing.T, g1, g2 *Graph) string {
+	err := g1.GraphCmp(g2, strVertexCmpFn, strEdgeCmpFn)
+	if err != nil {
+		str := ""
+		str += fmt.Sprintf("  actual (g1): %v%s", g1, fullPrint(g1))
+		str += fmt.Sprintf("expected (g2): %v%s", g2, fullPrint(g2))
+		str += fmt.Sprintf("cmp error:")
+		str += fmt.Sprintf("%v", err)
+		return str
+	}
+	return ""
+}
+
+func fullPrint(g *Graph) (str string) {
+	str += "\n"
+	for v := range g.Adjacency() {
+		str += fmt.Sprintf("* v: %s\n", v)
+	}
+	for v1 := range g.Adjacency() {
+		for v2, e := range g.Adjacency()[v1] {
+			str += fmt.Sprintf("* e: %s -> %s # %s\n", v1, v2, e)
+		}
+	}
+	return
+}
--- a/prometheus/prometheus.go
+++ b/prometheus/prometheus.go
@@ -3,16 +3,16 @@
 // Written by James Shubin <james@shubin.ca> and the project contributors
 //
 // This program is free software: you can redistribute it and/or modify
-// it under the terms of the GNU Affero General Public License as published by
+// it under the terms of the GNU General Public License as published by
 // the Free Software Foundation, either version 3 of the License, or
 // (at your option) any later version.
 //
 // This program is distributed in the hope that it will be useful,
 // but WITHOUT ANY WARRANTY; without even the implied warranty of
 // MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-// GNU Affero General Public License for more details.
+// GNU General Public License for more details.
 //
-// You should have received a copy of the GNU Affero General Public License
+// You should have received a copy of the GNU General Public License
 // along with this program.  If not, see <http://www.gnu.org/licenses/>.

 // Package prometheus provides functions that are useful to control and manage
@@ -20,24 +20,52 @@
 package prometheus

 import (
+	"errors"
 	"net/http"
 	"strconv"
+	"sync"

 	"github.com/prometheus/client_golang/prometheus"
 	"github.com/prometheus/client_golang/prometheus/promhttp"
+
+	errwrap "github.com/pkg/errors"
 )

 // DefaultPrometheusListen is registered in
 // https://github.com/prometheus/prometheus/wiki/Default-port-allocations
 const DefaultPrometheusListen = "127.0.0.1:9233"

+// ResState represents the status of a resource.
+type ResState int
+
+const (
+	// ResStateOK represents a working resource
+	ResStateOK ResState = iota
+	// ResStateSoftFail represents a resource in soft fail (will be retried)
+	ResStateSoftFail
+	// ResStateHardFail represents a resource in hard fail (will NOT be retried)
+	ResStateHardFail
+)
+
 // Prometheus is the struct that contains information about the
 // prometheus instance. Run Init() on it.
 type Prometheus struct {
 	Listen string // the listen specification for the net/http server

-	checkApplyTotal *prometheus.CounterVec // total of CheckApplies that have been triggered
+	checkApplyTotal        *prometheus.CounterVec // total of CheckApplies that have been triggered
+	pgraphStartTimeSeconds prometheus.Gauge       // process start time in seconds since unix epoch
+	managedResources       *prometheus.GaugeVec   // Resources we manage now
+	failedResourcesTotal   *prometheus.CounterVec // Total of failures since mgmt has started
+	failedResources        *prometheus.GaugeVec   // Number of current resources

+	resourcesState map[string]resStateWithKind // Maps the resources with their current kind/state
+	mutex          *sync.Mutex                 // Mutex used to update resourcesState
+}
+
+// resStateWithKind is used to count the failures by kind
+type resStateWithKind struct {
+	state ResState
+	kind  string
 }

 // Init some parameters - currently the Listen address.
@@ -45,6 +73,10 @@ func (obj *Prometheus) Init() error {
 	if len(obj.Listen) == 0 {
 		obj.Listen = DefaultPrometheusListen
 	}
+
+	obj.mutex = &sync.Mutex{}
+	obj.resourcesState = make(map[string]resStateWithKind)
+
 	obj.checkApplyTotal = prometheus.NewCounterVec(
 		prometheus.CounterOpts{
 			Name: "mgmt_checkapply_total",
@@ -59,6 +91,46 @@ func (obj *Prometheus) Init() error {
 	)
 	prometheus.MustRegister(obj.checkApplyTotal)

+	obj.pgraphStartTimeSeconds = prometheus.NewGauge(
+		prometheus.GaugeOpts{
+			Name: "mgmt_graph_start_time_seconds",
+			Help: "Start time of the current graph since unix epoch in seconds.",
+		},
+	)
+	prometheus.MustRegister(obj.pgraphStartTimeSeconds)
+
+	obj.managedResources = prometheus.NewGaugeVec(
+		prometheus.GaugeOpts{
+			Name: "mgmt_resources",
+			Help: "Number of managed resources.",
+		},
+		// kind: resource type: Svc, File, ...
+		[]string{"kind"},
+	)
+	prometheus.MustRegister(obj.managedResources)
+
+	obj.failedResourcesTotal = prometheus.NewCounterVec(
+		prometheus.CounterOpts{
+			Name: "mgmt_failures_total",
+			Help: "Total of failed resources.",
+		},
+		// kind: resource type: Svc, File, ...
+		// failure: soft or hard
+		[]string{"kind", "failure"},
+	)
+	prometheus.MustRegister(obj.failedResourcesTotal)
+
+	obj.failedResources = prometheus.NewGaugeVec(
+		prometheus.GaugeOpts{
+			Name: "mgmt_failures",
+			Help: "Number of failing resources.",
+		},
+		// kind: resource type: Svc, File, ...
+		// failure: soft or hard
+		[]string{"kind", "failure"},
+	)
+	prometheus.MustRegister(obj.failedResources)
+
 	return nil
 }

@@ -80,8 +152,112 @@ func (obj *Prometheus) Stop() error {
 // UpdateCheckApplyTotal refreshes the failing gauge by parsing the internal
 // state map.
 func (obj *Prometheus) UpdateCheckApplyTotal(kind string, apply, eventful, errorful bool) error {
+	if obj == nil {
+		return nil // happens when mgmt is launched without --prometheus
+	}
 	labels := prometheus.Labels{"kind": kind, "apply": strconv.FormatBool(apply), "eventful": strconv.FormatBool(eventful), "errorful": strconv.FormatBool(errorful)}
 	metric := obj.checkApplyTotal.With(labels)
 	metric.Inc()
 	return nil
 }
+
+// UpdatePgraphStartTime updates the mgmt_graph_start_time_seconds metric
+// to the current timestamp.
+func (obj *Prometheus) UpdatePgraphStartTime() error {
+	if obj == nil {
+		return nil // happens when mgmt is launched without --prometheus
+	}
+	obj.pgraphStartTimeSeconds.SetToCurrentTime()
+	return nil
+}
+
+// AddManagedResource increments the Managed Resource counter and updates the resource status.
+func (obj *Prometheus) AddManagedResource(resUUID string, rtype string) error {
+	if obj == nil {
+		return nil // happens when mgmt is launched without --prometheus
+	}
+	obj.managedResources.With(prometheus.Labels{"kind": rtype}).Inc()
+	if err := obj.UpdateState(resUUID, rtype, ResStateOK); err != nil {
+		return errwrap.Wrapf(err, "can't update the resource status in the map")
+	}
+	return nil
+}
+
+// RemoveManagedResource decrements the Managed Resource counter and updates the resource status.
+func (obj *Prometheus) RemoveManagedResource(resUUID string, rtype string) error {
+	if obj == nil {
+		return nil // happens when mgmt is launched without --prometheus
+	}
+	obj.managedResources.With(prometheus.Labels{"kind": rtype}).Dec()
+	if err := obj.deleteState(resUUID); err != nil {
+		return errwrap.Wrapf(err, "can't remove the resource status from the map")
+	}
+	return nil
+}
+
+// deleteState removes the resources for the state map and re-populates the failing gauge.
+func (obj *Prometheus) deleteState(resUUID string) error {
+	if obj == nil {
+		return nil // happens when mgmt is launched without --prometheus
+	}
+	obj.mutex.Lock()
+	delete(obj.resourcesState, resUUID)
+	obj.mutex.Unlock()
+	if err := obj.updateFailingGauge(); err != nil {
+		return errwrap.Wrapf(err, "can't update the failing gauge")
+	}
+	return nil
+}
+
+// UpdateState updates the state of the resources in our internal state map
+// then triggers a refresh of the failing gauge.
+func (obj *Prometheus) UpdateState(resUUID string, rtype string, newState ResState) error {
+	defer obj.updateFailingGauge()
+	if obj == nil {
+		return nil // happens when mgmt is launched without --prometheus
+	}
+	obj.mutex.Lock()
+	obj.resourcesState[resUUID] = resStateWithKind{state: newState, kind: rtype}
+	obj.mutex.Unlock()
+	if newState != ResStateOK {
+		var strState string
+		if newState == ResStateSoftFail {
+			strState = "soft"
+		} else if newState == ResStateHardFail {
+			strState = "hard"
+		} else {
+			return errors.New("state should be soft or hard failure")
+		}
+		obj.failedResourcesTotal.With(prometheus.Labels{"kind": rtype, "failure": strState}).Inc()
+	}
+	return nil
+}
+
+// updateFailingGauge refreshes the failing gauge by parsking the internal
+// state map.
+func (obj *Prometheus) updateFailingGauge() error {
+	if obj == nil {
+		return nil // happens when mgmt is launched without --prometheus
+	}
+	var softFails, hardFails map[string]float64
+	softFails = make(map[string]float64)
+	hardFails = make(map[string]float64)
+	for _, v := range obj.resourcesState {
+		if v.state == ResStateSoftFail {
+			softFails[v.kind]++
+		} else if v.state == ResStateHardFail {
+			hardFails[v.kind]++
+		}
+	}
+	// TODO: we might want to Zero the metrics we are not using
+	// because in prometheus design the metrics keep living for some time
+	// even after they are removed.
+	obj.failedResources.Reset()
+	for k, v := range softFails {
+		obj.failedResources.With(prometheus.Labels{"kind": k, "failure": "soft"}).Set(v)
+	}
+	for k, v := range hardFails {
+		obj.failedResources.With(prometheus.Labels{"kind": k, "failure": "hard"}).Set(v)
+	}
+	return nil
+}
--- a/puppet/gapi.go
+++ b/puppet/gapi.go
@@ -3,16 +3,16 @@
 // Written by James Shubin <james@shubin.ca> and the project contributors
 //
 // This program is free software: you can redistribute it and/or modify
-// it under the terms of the GNU Affero General Public License as published by
+// it under the terms of the GNU General Public License as published by
 // the Free Software Foundation, either version 3 of the License, or
 // (at your option) any later version.
 //
 // This program is distributed in the hope that it will be useful,
 // but WITHOUT ANY WARRANTY; without even the implied warranty of
 // MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-// GNU Affero General Public License for more details.
+// GNU General Public License for more details.
 //
-// You should have received a copy of the GNU Affero General Public License
+// You should have received a copy of the GNU General Public License
 // along with this program.  If not, see <http://www.gnu.org/licenses/>.

 package puppet
@@ -75,37 +75,61 @@ func (obj *GAPI) Graph() (*pgraph.Graph, error) {
 }

 // Next returns nil errors every time there could be a new graph.
-func (obj *GAPI) Next() chan error {
-	if obj.data.NoWatch {
-		return nil
-	}
+func (obj *GAPI) Next() chan gapi.Next {
 	puppetChan := func() <-chan time.Time { // helper function
 		return time.Tick(time.Duration(RefreshInterval(obj.PuppetConf)) * time.Second)
 	}
-	ch := make(chan error)
+	ch := make(chan gapi.Next)
 	obj.wg.Add(1)
 	go func() {
 		defer obj.wg.Done()
 		defer close(ch) // this will run before the obj.wg.Done()
 		if !obj.initialized {
-			ch <- fmt.Errorf("the puppet GAPI is not initialized")
+			next := gapi.Next{
+				Err:  fmt.Errorf("the puppet GAPI is not initialized"),
+				Exit: true, // exit, b/c programming error?
+			}
+			ch <- next
 			return
 		}
-		pChan := puppetChan()
+		startChan := make(chan struct{}) // start signal
+		close(startChan)                 // kick it off!
+
+		pChan := make(<-chan time.Time)
+		// NOTE: we don't look at obj.data.NoConfigWatch since emulating
+		// puppet means we do not switch graphs on code changes anyways.
+		if obj.data.NoStreamWatch {
+			pChan = nil
+		} else {
+			pChan = puppetChan()
+		}
+
 		for {
 			select {
+			case <-startChan: // kick the loop once at start
+				startChan = nil // disable
+				// pass
 			case _, ok := <-pChan:
 				if !ok { // the channel closed!
 					return
 				}
-				log.Printf("Puppet: Generating new graph...")
+			case <-obj.closeChan:
+				return
+			}
+
+			log.Printf("Puppet: Generating new graph...")
+			if obj.data.NoStreamWatch {
+				pChan = nil
+			} else {
 				pChan = puppetChan() // TODO: okay to update interval in case it changed?
-				select {
-				case ch <- nil: // trigger a run (send a msg)
-				// unblock if we exit while waiting to send!
-				case <-obj.closeChan:
-					return
-				}
+			}
+			next := gapi.Next{
+				//Exit: true, // TODO: for permanent shutdown!
+				Err: nil,
+			}
+			select {
+			case ch <- next: // trigger a run (send a msg)
+			// unblock if we exit while waiting to send!
 			case <-obj.closeChan:
 				return
 			}
--- a/puppet/puppet.go
+++ b/puppet/puppet.go
@@ -3,16 +3,16 @@
 // Written by James Shubin <james@shubin.ca> and the project contributors
 //
 // This program is free software: you can redistribute it and/or modify
-// it under the terms of the GNU Affero General Public License as published by
+// it under the terms of the GNU General Public License as published by
 // the Free Software Foundation, either version 3 of the License, or
 // (at your option) any later version.
 //
 // This program is distributed in the hope that it will be useful,
 // but WITHOUT ANY WARRANTY; without even the implied warranty of
 // MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-// GNU Affero General Public License for more details.
+// GNU General Public License for more details.
 //
-// You should have received a copy of the GNU Affero General Public License
+// You should have received a copy of the GNU General Public License
 // along with this program.  If not, see <http://www.gnu.org/licenses/>.

 // Package puppet provides the integration entrypoint for the puppet language.
--- a/recwatch/configwatch.go
+++ b/recwatch/configwatch.go
@@ -3,16 +3,16 @@
 // Written by James Shubin <james@shubin.ca> and the project contributors
 //
 // This program is free software: you can redistribute it and/or modify
-// it under the terms of the GNU Affero General Public License as published by
+// it under the terms of the GNU General Public License as published by
 // the Free Software Foundation, either version 3 of the License, or
 // (at your option) any later version.
 //
 // This program is distributed in the hope that it will be useful,
 // but WITHOUT ANY WARRANTY; without even the implied warranty of
 // MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-// GNU Affero General Public License for more details.
+// GNU General Public License for more details.
 //
-// You should have received a copy of the GNU Affero General Public License
+// You should have received a copy of the GNU General Public License
 // along with this program.  If not, see <http://www.gnu.org/licenses/>.

 package recwatch
--- a/recwatch/flags.go
+++ b/recwatch/flags.go
@@ -3,16 +3,16 @@
 // Written by James Shubin <james@shubin.ca> and the project contributors
 //
 // This program is free software: you can redistribute it and/or modify
-// it under the terms of the GNU Affero General Public License as published by
+// it under the terms of the GNU General Public License as published by
 // the Free Software Foundation, either version 3 of the License, or
 // (at your option) any later version.
 //
 // This program is distributed in the hope that it will be useful,
 // but WITHOUT ANY WARRANTY; without even the implied warranty of
 // MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-// GNU Affero General Public License for more details.
+// GNU General Public License for more details.
 //
-// You should have received a copy of the GNU Affero General Public License
+// You should have received a copy of the GNU General Public License
 // along with this program.  If not, see <http://www.gnu.org/licenses/>.

 package recwatch
--- a/recwatch/recwatch.go
+++ b/recwatch/recwatch.go
@@ -3,16 +3,16 @@
 // Written by James Shubin <james@shubin.ca> and the project contributors
 //
 // This program is free software: you can redistribute it and/or modify
-// it under the terms of the GNU Affero General Public License as published by
+// it under the terms of the GNU General Public License as published by
 // the Free Software Foundation, either version 3 of the License, or
 // (at your option) any later version.
 //
 // This program is distributed in the hope that it will be useful,
 // but WITHOUT ANY WARRANTY; without even the implied warranty of
 // MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-// GNU Affero General Public License for more details.
+// GNU General Public License for more details.
 //
-// You should have received a copy of the GNU Affero General Public License
+// You should have received a copy of the GNU General Public License
 // along with this program.  If not, see <http://www.gnu.org/licenses/>.

 // Package recwatch provides recursive file watching events via fsnotify.
--- a/remote/remote.go
+++ b/remote/remote.go
@@ -3,16 +3,16 @@
 // Written by James Shubin <james@shubin.ca> and the project contributors
 //
 // This program is free software: you can redistribute it and/or modify
-// it under the terms of the GNU Affero General Public License as published by
+// it under the terms of the GNU General Public License as published by
 // the Free Software Foundation, either version 3 of the License, or
 // (at your option) any later version.
 //
 // This program is distributed in the hope that it will be useful,
 // but WITHOUT ANY WARRANTY; without even the implied warranty of
 // MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-// GNU Affero General Public License for more details.
+// GNU General Public License for more details.
 //
-// You should have received a copy of the GNU Affero General Public License
+// You should have received a copy of the GNU General Public License
 // along with this program.  If not, see <http://www.gnu.org/licenses/>.

 // Package remote provides the remoting facilities for agentless execution.
@@ -63,6 +63,7 @@ import (

 	cv "github.com/purpleidea/mgmt/converger"
 	"github.com/purpleidea/mgmt/util"
+	"github.com/purpleidea/mgmt/util/semaphore"
 	"github.com/purpleidea/mgmt/yamlgraph"

 	multierr "github.com/hashicorp/go-multierror"
@@ -691,29 +692,30 @@ type Remotes struct {
 	fileWatch    chan string
 	cConns       uint16 // number of concurrent ssh connections, zero means unlimited
 	interactive  bool   // allow interactive prompting
-	sshPrivIdRsa string // path to ~/.ssh/id_rsa
+	sshPrivIDRsa string // path to ~/.ssh/id_rsa
 	caching      bool   // whether to try and cache the copy of the binary
 	depth        uint16 // depth of this node in the remote execution hierarchy
 	prefix       string // folder prefix to use for misc storage
 	converger    cv.Converger
 	convergerCb  func(func(map[string]bool) error) (func(), error)

-	wg                 sync.WaitGroup    // keep track of each running SSH connection
-	lock               sync.Mutex        // mutex for access to sshmap
-	sshmap             map[string]*SSH   // map to each SSH struct with the remote as the key
-	exiting            bool              // flag to let us know if we're exiting
-	exitChan           chan struct{}     // closes when we should exit
-	semaphore          Semaphore         // counting semaphore to limit concurrent connections
-	hostnames          []string          // list of hostnames we've seen so far
-	cuid               cv.UID            // convergerUID for the remote itself
-	cuids              map[string]cv.UID // map to each SSH struct with the remote as the key
-	callbackCancelFunc func()            // stored callback function cancel function
+	wg                 sync.WaitGroup       // keep track of each running SSH connection
+	lock               sync.Mutex           // mutex for access to sshmap
+	sshmap             map[string]*SSH      // map to each SSH struct with the remote as the key
+	running            chan struct{}        // closes when main loop is running
+	exiting            bool                 // flag to let us know if we're exiting
+	exitChan           chan struct{}        // closes when we should exit
+	semaphore          *semaphore.Semaphore // counting semaphore to limit concurrent connections
+	hostnames          []string             // list of hostnames we've seen so far
+	cuid               cv.UID               // convergerUID for the remote itself
+	cuids              map[string]cv.UID    // map to each SSH struct with the remote as the key
+	callbackCancelFunc func()               // stored callback function cancel function

 	flags Flags // constant runtime values
 }

 // NewRemotes builds a Remotes struct.
-func NewRemotes(clientURLs, remoteURLs []string, noop bool, remotes []string, fileWatch chan string, cConns uint16, interactive bool, sshPrivIdRsa string, caching bool, depth uint16, prefix string, converger cv.Converger, convergerCb func(func(map[string]bool) error) (func(), error), flags Flags) *Remotes {
+func NewRemotes(clientURLs, remoteURLs []string, noop bool, remotes []string, fileWatch chan string, cConns uint16, interactive bool, sshPrivIDRsa string, caching bool, depth uint16, prefix string, converger cv.Converger, convergerCb func(func(map[string]bool) error) (func(), error), flags Flags) *Remotes {
 	return &Remotes{
 		clientURLs:   clientURLs,
 		remoteURLs:   remoteURLs,
@@ -722,15 +724,16 @@ func NewRemotes(clientURLs, remoteURLs []string, noop bool, remotes []string, fi
 		fileWatch:    fileWatch,
 		cConns:       cConns,
 		interactive:  interactive,
-		sshPrivIdRsa: sshPrivIdRsa,
+		sshPrivIDRsa: sshPrivIDRsa,
 		caching:      caching,
 		depth:        depth,
 		prefix:       prefix,
 		converger:    converger,
 		convergerCb:  convergerCb,
 		sshmap:       make(map[string]*SSH),
+		running:      make(chan struct{}),
 		exitChan:     make(chan struct{}),
-		semaphore:    NewSemaphore(int(cConns)),
+		semaphore:    semaphore.NewSemaphore(int(cConns)),
 		hostnames:    make([]string, len(remotes)),
 		cuids:        make(map[string]cv.UID),
 		flags:        flags,
@@ -829,18 +832,18 @@ func (obj *Remotes) NewSSH(file string) (*SSH, error) {

 // sshKeyAuth is a helper function to get the ssh key auth struct needed
 func (obj *Remotes) sshKeyAuth() (ssh.AuthMethod, error) {
-	if obj.sshPrivIdRsa == "" {
+	if obj.sshPrivIDRsa == "" {
 		return nil, fmt.Errorf("empty path specified")
 	}
 	p := ""
 	// TODO: this doesn't match strings of the form: ~james/.ssh/id_rsa
-	if strings.HasPrefix(obj.sshPrivIdRsa, "~/") {
+	if strings.HasPrefix(obj.sshPrivIDRsa, "~/") {
 		usr, err := user.Current()
 		if err != nil {
 			log.Printf("Remote: Can't find home directory automatically.")
 			return nil, err
 		}
-		p = path.Join(usr.HomeDir, obj.sshPrivIdRsa[len("~/"):])
+		p = path.Join(usr.HomeDir, obj.sshPrivIDRsa[len("~/"):])
 	}
 	if p == "" {
 		return nil, fmt.Errorf("empty path specified")
@@ -1021,11 +1024,17 @@ func (obj *Remotes) Run() {
 		}(sshobj, f)
 		obj.lock.Unlock()
 	}
+	close(obj.running) // notify
 }

+// Ready closes its returned channel when the Run method is up and ready. It is
+// useful to know when ready, since we often execute Run in a go routine.
+func (obj *Remotes) Ready() <-chan struct{} { return obj.running }
+
 // Exit causes as much of the Remotes struct to shutdown as quickly and as
 // cleanly as possible. It only returns once everything is shutdown.
 func (obj *Remotes) Exit() error {
+	<-obj.running // wait for Run to be finished before we exit!
 	obj.lock.Lock()
 	obj.exiting = true // don't spawn new ones once this flag is set!
 	obj.lock.Unlock()
@@ -1078,29 +1087,6 @@ func cleanURL(s string) string {
 	return u.Host
 }

-// Semaphore is a counting semaphore.
-type Semaphore chan struct{}
-
-// NewSemaphore creates a new semaphore.
-func NewSemaphore(size int) Semaphore {
-	return make(Semaphore, size)
-}
-
-// P acquires n resources.
-func (s Semaphore) P(n int) {
-	e := struct{}{}
-	for i := 0; i < n; i++ {
-		s <- e // acquire one
-	}
-}
-
-// V releases n resources.
-func (s Semaphore) V(n int) {
-	for i := 0; i < n; i++ {
-		<-s // release one
-	}
-}
-
 // combinedWriter mimics what the ssh.CombinedOutput command does.
 type combinedWriter struct {
 	b  bytes.Buffer
--- a/resources/actions.go
+++ b/resources/actions.go
@@ -3,57 +3,59 @@
 // Written by James Shubin <james@shubin.ca> and the project contributors
 //
 // This program is free software: you can redistribute it and/or modify
-// it under the terms of the GNU Affero General Public License as published by
+// it under the terms of the GNU General Public License as published by
 // the Free Software Foundation, either version 3 of the License, or
 // (at your option) any later version.
 //
 // This program is distributed in the hope that it will be useful,
 // but WITHOUT ANY WARRANTY; without even the implied warranty of
 // MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-// GNU Affero General Public License for more details.
+// GNU General Public License for more details.
 //
-// You should have received a copy of the GNU Affero General Public License
+// You should have received a copy of the GNU General Public License
 // along with this program.  If not, see <http://www.gnu.org/licenses/>.

-package pgraph
+package resources

 import (
 	"fmt"
 	"log"
 	"math"
+	"strings"
 	"sync"
 	"time"

 	"github.com/purpleidea/mgmt/event"
-	"github.com/purpleidea/mgmt/resources"
+	"github.com/purpleidea/mgmt/pgraph"
+	"github.com/purpleidea/mgmt/prometheus"
+	"github.com/purpleidea/mgmt/util"

 	multierr "github.com/hashicorp/go-multierror"
 	errwrap "github.com/pkg/errors"
 	"golang.org/x/time/rate"
 )

-// GetTimestamp returns the timestamp of a vertex
-func (v *Vertex) GetTimestamp() int64 {
-	return v.timestamp
+// SentinelErr is a sentinal as an error type that wraps an arbitrary error.
+type SentinelErr struct {
+	err error
 }

-// UpdateTimestamp updates the timestamp on a vertex and returns the new value
-func (v *Vertex) UpdateTimestamp() int64 {
-	v.timestamp = time.Now().UnixNano() // update
-	return v.timestamp
+// Error is the required method to fulfill the error type.
+func (obj *SentinelErr) Error() string {
+	return obj.err.Error()
 }

 // OKTimestamp returns true if this element can run right now?
-func (g *Graph) OKTimestamp(v *Vertex) bool {
+func (obj *BaseRes) OKTimestamp() bool {
 	// these are all the vertices pointing TO v, eg: ??? -> v
-	for _, n := range g.IncomingGraphVertices(v) {
+	for _, n := range obj.Graph.IncomingGraphVertices(obj.Vertex) {
 		// if the vertex has a greater timestamp than any pre-req (n)
 		// then we can't run right now...
 		// if they're equal (eg: on init of 0) then we also can't run
 		// b/c we should let our pre-req's go first...
-		x, y := v.GetTimestamp(), n.GetTimestamp()
-		if g.Flags.Debug {
-			log.Printf("%s[%s]: OKTimestamp: (%v) >= %s[%s](%v): !%v", v.Kind(), v.GetName(), x, n.Kind(), n.GetName(), y, x >= y)
+		x, y := obj.Timestamp(), VtoR(n).Timestamp()
+		if b, ok := obj.Graph.Value("debug"); ok && util.Bool(b) {
+			log.Printf("%s: OKTimestamp: (%v) >= %s(%v): !%v", obj, x, n, y, x >= y)
 		}
 		if x >= y {
 			return false
@@ -63,28 +65,35 @@ func (g *Graph) OKTimestamp(v *Vertex) bool {
 }

 // Poke tells nodes after me in the dependency graph that they need to refresh.
-func (g *Graph) Poke(v *Vertex) error {
+func (obj *BaseRes) Poke() error {
+	// if we're pausing (or exiting) then we should suspend poke's so that
+	// the graph doesn't go on running forever until it's completely done!
+	// this is an optional feature which we can do by default on user exit
+	if obj.Graph.FastPause {
+		return nil // TODO: should this be an error instead?
+	}
+
 	var wg sync.WaitGroup
 	// these are all the vertices pointing AWAY FROM v, eg: v -> ???
-	for _, n := range g.OutgoingGraphVertices(v) {
+	for _, n := range obj.Graph.OutgoingGraphVertices(obj.Vertex) {
 		// we can skip this poke if resource hasn't done work yet... it
 		// needs to be poked if already running, or not running though!
 		// TODO: does this need an || activity flag?
-		if n.Res.GetState() != resources.ResStateProcess {
-			if g.Flags.Debug {
-				log.Printf("%s[%s]: Poke: %s[%s]", v.Kind(), v.GetName(), n.Kind(), n.GetName())
+		if VtoR(n).GetState() != ResStateProcess {
+			if b, ok := obj.Graph.Value("debug"); ok && util.Bool(b) {
+				log.Printf("%s: Poke: %s", obj, n)
 			}
 			wg.Add(1)
-			go func(nn *Vertex) error {
+			go func(nn pgraph.Vertex) error {
 				defer wg.Done()
-				//edge := g.Adjacency[v][nn] // lookup
+				//edge := obj.Graph.adjacency[v][nn] // lookup
 				//notify := edge.Notify && edge.Refresh()
-				return nn.SendEvent(event.EventPoke, nil)
+				return VtoR(nn).SendEvent(event.EventPoke, nil)
 			}(n)

 		} else {
-			if g.Flags.Debug {
-				log.Printf("%s[%s]: Poke: %s[%s]: Skipped!", v.Kind(), v.GetName(), n.Kind(), n.GetName())
+			if b, ok := obj.Graph.Value("debug"); ok && util.Bool(b) {
+				log.Printf("%s: Poke: %s: Skipped!", obj, n)
 			}
 		}
 	}
@@ -94,30 +103,30 @@ func (g *Graph) Poke(v *Vertex) error {
 }

 // BackPoke pokes the pre-requisites that are stale and need to run before I can run.
-func (g *Graph) BackPoke(v *Vertex) {
+func (obj *BaseRes) BackPoke() {
 	var wg sync.WaitGroup
 	// these are all the vertices pointing TO v, eg: ??? -> v
-	for _, n := range g.IncomingGraphVertices(v) {
-		x, y, s := v.GetTimestamp(), n.GetTimestamp(), n.Res.GetState()
+	for _, n := range obj.Graph.IncomingGraphVertices(obj.Vertex) {
+		x, y, s := obj.Timestamp(), VtoR(n).Timestamp(), VtoR(n).GetState()
 		// If the parent timestamp needs poking AND it's not running
 		// Process, then poke it. If the parent is in ResStateProcess it
 		// means that an event is pending, so we'll be expecting a poke
 		// back soon, so we can safely discard the extra parent poke...
 		// TODO: implement a stateLT (less than) to tell if something
 		// happens earlier in the state cycle and that doesn't wrap nil
-		if x >= y && (s != resources.ResStateProcess && s != resources.ResStateCheckApply) {
-			if g.Flags.Debug {
-				log.Printf("%s[%s]: BackPoke: %s[%s]", v.Kind(), v.GetName(), n.Kind(), n.GetName())
+		if x >= y && (s != ResStateProcess && s != ResStateCheckApply) {
+			if b, ok := obj.Graph.Value("debug"); ok && util.Bool(b) {
+				log.Printf("%s: BackPoke: %s", obj, n)
 			}
 			wg.Add(1)
-			go func(nn *Vertex) error {
+			go func(nn pgraph.Vertex) error {
 				defer wg.Done()
-				return nn.SendEvent(event.EventBackPoke, nil)
+				return VtoR(nn).SendEvent(event.EventBackPoke, nil)
 			}(n)

 		} else {
-			if g.Flags.Debug {
-				log.Printf("%s[%s]: BackPoke: %s[%s]: Skipped!", v.Kind(), v.GetName(), n.Kind(), n.GetName())
+			if b, ok := obj.Graph.Value("debug"); ok && util.Bool(b) {
+				log.Printf("%s: BackPoke: %s: Skipped!", obj, n)
 			}
 		}
 	}
@@ -127,10 +136,11 @@ func (g *Graph) BackPoke(v *Vertex) {

 // RefreshPending determines if any previous nodes have a refresh pending here.
 // If this is true, it means I am expected to apply a refresh when I next run.
-func (g *Graph) RefreshPending(v *Vertex) bool {
+func (obj *BaseRes) RefreshPending() bool {
 	var refresh bool
-	for _, edge := range g.IncomingGraphEdges(v) {
+	for _, edge := range obj.Graph.IncomingGraphEdges(obj.Vertex) {
 		// if we asked for a notify *and* if one is pending!
+		edge := edge.(*Edge) // panic if wrong
 		if edge.Notify && edge.Refresh() {
 			refresh = true
 			break
@@ -140,8 +150,9 @@ func (g *Graph) RefreshPending(v *Vertex) bool {
 }

 // SetUpstreamRefresh sets the refresh value to any upstream vertices.
-func (g *Graph) SetUpstreamRefresh(v *Vertex, b bool) {
-	for _, edge := range g.IncomingGraphEdges(v) {
+func (obj *BaseRes) SetUpstreamRefresh(b bool) {
+	for _, edge := range obj.Graph.IncomingGraphEdges(obj.Vertex) {
+		edge := edge.(*Edge) // panic if wrong
 		if edge.Notify {
 			edge.SetRefresh(b)
 		}
@@ -149,8 +160,9 @@ func (g *Graph) SetUpstreamRefresh(v *Vertex, b bool) {
 }

 // SetDownstreamRefresh sets the refresh value to any downstream vertices.
-func (g *Graph) SetDownstreamRefresh(v *Vertex, b bool) {
-	for _, edge := range g.OutgoingGraphEdges(v) {
+func (obj *BaseRes) SetDownstreamRefresh(b bool) {
+	for _, edge := range obj.Graph.OutgoingGraphEdges(obj.Vertex) {
+		edge := edge.(*Edge) // panic if wrong
 		// if we asked for a notify *and* if one is pending!
 		if edge.Notify {
 			edge.SetRefresh(b)
@@ -159,30 +171,51 @@ func (g *Graph) SetDownstreamRefresh(v *Vertex, b bool) {
 }

 // Process is the primary function to execute for a particular vertex in the graph.
-func (g *Graph) Process(v *Vertex) error {
-	obj := v.Res
-	if g.Flags.Debug {
-		log.Printf("%s[%s]: Process()", obj.Kind(), obj.GetName())
+func (obj *BaseRes) Process() error {
+	if obj.debug {
+		log.Printf("%s: Process()", obj)
 	}
-	defer obj.SetState(resources.ResStateNil) // reset state when finished
-	obj.SetState(resources.ResStateProcess)
-	var ok = true
-	var applied = false // did we run an apply?
+	// FIXME: should these SetState methods be here or after the sema code?
+	defer obj.SetState(ResStateNil) // reset state when finished
+	obj.SetState(ResStateProcess)
+
 	// is it okay to run dependency wise right now?
 	// if not, that's okay because when the dependency runs, it will poke
 	// us back and we will run if needed then!
-	if !g.OKTimestamp(v) {
-		go g.BackPoke(v)
+	if !obj.OKTimestamp() {
+		go obj.BackPoke()
 		return nil
 	}
 	// timestamp must be okay...
-
-	if g.Flags.Debug {
-		log.Printf("%s[%s]: OKTimestamp(%v)", obj.Kind(), obj.GetName(), v.GetTimestamp())
+	if b, ok := obj.Graph.Value("debug"); ok && util.Bool(b) {
+		log.Printf("%s: OKTimestamp(%v)", obj, obj.Timestamp())
 	}

+	// semaphores!
+	// These shouldn't ever block an exit, since the graph should eventually
+	// converge causing their them to unlock. More interestingly, since they
+	// run in a DAG alphabetically, there is no way to permanently deadlock,
+	// assuming that resources individually don't ever block from finishing!
+	// The exception is that semaphores with a zero count will always block!
+	// TODO: Add a close mechanism to close/unblock zero count semaphores...
+	semas := obj.Meta().Sema
+	if obj.debug && len(semas) > 0 {
+		log.Printf("%s: Sema: P(%s)", obj, strings.Join(semas, ", "))
+	}
+	if err := obj.Graph.SemaLock(semas); err != nil { // lock
+		// NOTE: in practice, this might not ever be truly necessary...
+		return fmt.Errorf("shutdown of semaphores")
+	}
+	defer obj.Graph.SemaUnlock(semas) // unlock
+	if obj.debug && len(semas) > 0 {
+		defer log.Printf("%s: Sema: V(%s)", obj, strings.Join(semas, ", "))
+	}
+
+	var ok = true
+	var applied = false // did we run an apply?
+
 	// connect any senders to receivers and detect if values changed
-	if updated, err := obj.SendRecv(obj); err != nil {
+	if updated, err := obj.SendRecv(obj.Res); err != nil {
 		return errwrap.Wrapf(err, "could not SendRecv in Process")
 	} else if len(updated) > 0 {
 		for _, changed := range updated {
@@ -198,16 +231,16 @@ func (g *Graph) Process(v *Vertex) error {
 	var checkOK bool
 	var err error

-	if g.Flags.Debug {
-		log.Printf("%s[%s]: CheckApply(%t)", obj.Kind(), obj.GetName(), !noop)
+	if b, ok := obj.Graph.Value("debug"); ok && util.Bool(b) {
+		log.Printf("%s: CheckApply(%t)", obj, !noop)
 	}

 	// lookup the refresh (notification) variable
-	refresh = g.RefreshPending(v) // do i need to perform a refresh?
-	obj.SetRefresh(refresh)       // tell the resource
+	refresh = obj.RefreshPending() // do i need to perform a refresh?
+	obj.SetRefresh(refresh)        // tell the resource

 	// changes can occur after this...
-	obj.SetState(resources.ResStateCheckApply)
+	obj.SetState(ResStateCheckApply)

 	// check cached state, to skip CheckApply; can't skip if refreshing
 	if !refresh && obj.IsStateOK() {
@@ -222,40 +255,37 @@ func (g *Graph) Process(v *Vertex) error {
 		// run the CheckApply!
 	} else {
 		// if this fails, don't UpdateTimestamp()
-		checkOK, err = obj.CheckApply(!noop)
+		checkOK, err = obj.Res.CheckApply(!noop)

-		if obj.Prometheus() != nil {
-			if promErr := obj.Prometheus().UpdateCheckApplyTotal(obj.Kind(), !noop, !checkOK, err != nil); promErr != nil {
-				// TODO: how to error correctly
-				log.Printf("%s[%s]: Prometheus.UpdateCheckApplyTotal() errored: %v", v.Kind(), v.GetName(), err)
-			}
+		if promErr := obj.Data().Prometheus.UpdateCheckApplyTotal(obj.GetKind(), !noop, !checkOK, err != nil); promErr != nil {
+			// TODO: how to error correctly
+			log.Printf("%s: Prometheus.UpdateCheckApplyTotal() errored: %v", obj, err)
 		}
 		// TODO: Can the `Poll` converged timeout tracking be a
 		// more general method for all converged timeouts? this
 		// would simplify the resources by removing boilerplate
-		if v.Meta().Poll > 0 {
+		if obj.Meta().Poll > 0 {
 			if !checkOK { // something changed, restart timer
-				cuid, _, _ := v.Res.ConvergerUIDs() // get the converger uid used to report status
-				cuid.ResetTimer()                   // activity!
-				if g.Flags.Debug {
-					log.Printf("%s[%s]: Converger: ResetTimer", obj.Kind(), obj.GetName())
+				obj.cuid.ResetTimer() // activity!
+				if b, ok := obj.Graph.Value("debug"); ok && util.Bool(b) {
+					log.Printf("%s: Converger: ResetTimer", obj)
 				}
 			}
 		}
 	}

 	if checkOK && err != nil { // should never return this way
-		log.Fatalf("%s[%s]: CheckApply(): %t, %+v", obj.Kind(), obj.GetName(), checkOK, err)
+		log.Fatalf("%s: CheckApply(): %t, %+v", obj, checkOK, err)
 	}
-	if g.Flags.Debug {
-		log.Printf("%s[%s]: CheckApply(): %t, %v", obj.Kind(), obj.GetName(), checkOK, err)
+	if b, ok := obj.Graph.Value("debug"); ok && util.Bool(b) {
+		log.Printf("%s: CheckApply(): %t, %v", obj, checkOK, err)
 	}

 	// if CheckApply ran without noop and without error, state should be good
 	if !noop && err == nil { // aka !noop || checkOK
 		obj.StateOK(true) // reset
 		if refresh {
-			g.SetUpstreamRefresh(v, false) // refresh happened, clear the request
+			obj.SetUpstreamRefresh(false) // refresh happened, clear the request
 			obj.SetRefresh(false)
 		}
 	}
@@ -281,14 +311,14 @@ func (g *Graph) Process(v *Vertex) error {
 		}

 		if activity { // add refresh flag to downstream edges...
-			g.SetDownstreamRefresh(v, true)
+			obj.SetDownstreamRefresh(true)
 		}

 		// update this timestamp *before* we poke or the poked
 		// nodes might fail due to having a too old timestamp!
-		v.UpdateTimestamp()                    // this was touched...
-		obj.SetState(resources.ResStatePoking) // can't cancel parent poke
-		if err := g.Poke(v); err != nil {
+		obj.UpdateTimestamp()        // this was touched...
+		obj.SetState(ResStatePoking) // can't cancel parent poke
+		if err := obj.Poke(); err != nil {
 			return errwrap.Wrapf(err, "the Poke() failed")
 		}
 	}
@@ -296,23 +326,11 @@ func (g *Graph) Process(v *Vertex) error {
 	return errwrap.Wrapf(err, "could not Process() successfully")
 }

-// SentinelErr is a sentinal as an error type that wraps an arbitrary error.
-type SentinelErr struct {
-	err error
-}
-
-// Error is the required method to fulfill the error type.
-func (obj *SentinelErr) Error() string {
-	return obj.err.Error()
-}
-
 // innerWorker is the CheckApply runner that reads from processChan.
-func (g *Graph) innerWorker(v *Vertex) {
-	obj := v.Res
+func (obj *BaseRes) innerWorker() {
 	running := false
 	done := make(chan struct{})
-	playback := false                      // do we need to run another one?
-	_, wcuid, pcuid := obj.ConvergerUIDs() // get extra cuids (worker, process)
+	playback := false // do we need to run another one?

 	waiting := false
 	var timer = time.NewTimer(time.Duration(math.MaxInt64)) // longest duration
@@ -320,9 +338,9 @@ func (g *Graph) innerWorker(v *Vertex) {
 		<-timer.C // unnecessary, shouldn't happen
 	}

-	var delay = time.Duration(v.Meta().Delay) * time.Millisecond
-	var retry = v.Meta().Retry // number of tries left, -1 for infinite
-	var limiter = rate.NewLimiter(v.Meta().Limit, v.Meta().Burst)
+	var delay = time.Duration(obj.Meta().Delay) * time.Millisecond
+	var retry = obj.Meta().Retry // number of tries left, -1 for infinite
+	var limiter = rate.NewLimiter(obj.Meta().Limit, obj.Meta().Burst)
 	limited := false

 	wg := &sync.WaitGroup{} // wait for Process routine to exit
@@ -330,46 +348,49 @@ func (g *Graph) innerWorker(v *Vertex) {
 Loop:
 	for {
 		select {
-		case ev, ok := <-obj.ProcessChan(): // must use like this
+		case ev, ok := <-obj.processChan: // must use like this
 			if !ok { // processChan closed, let's exit
 				break Loop // no event, so no ack!
 			}
-			if v.Res.Meta().Poll == 0 { // skip for polling
-				wcuid.SetConverged(false)
+			if obj.Meta().Poll == 0 { // skip for polling
+				obj.wcuid.SetConverged(false)
 			}

 			// if process started, but no action yet, skip!
-			if v.Res.GetState() == resources.ResStateProcess {
-				if g.Flags.Debug {
-					log.Printf("%s[%s]: Skipped event!", v.Kind(), v.GetName())
+			if obj.GetState() == ResStateProcess {
+				if b, ok := obj.Graph.Value("debug"); ok && util.Bool(b) {
+					log.Printf("%s: Skipped event!", obj)
 				}
 				ev.ACK() // ready for next message
+				obj.quiesceGroup.Done()
 				continue
 			}

 			// if running, we skip running a new execution!
 			// if waiting, we skip running a new execution!
 			if running || waiting {
-				if g.Flags.Debug {
-					log.Printf("%s[%s]: Playback added!", v.Kind(), v.GetName())
+				if b, ok := obj.Graph.Value("debug"); ok && util.Bool(b) {
+					log.Printf("%s: Playback added!", obj)
 				}
 				playback = true
 				ev.ACK() // ready for next message
+				obj.quiesceGroup.Done()
 				continue
 			}

 			// catch invalid rates
-			if v.Meta().Burst == 0 && !(v.Meta().Limit == rate.Inf) { // blocked
-				e := fmt.Errorf("%s[%s]: Permanently limited (rate != Inf, burst: 0)", v.Kind(), v.GetName())
-				v.SendEvent(event.EventExit, &SentinelErr{e})
+			if obj.Meta().Burst == 0 && !(obj.Meta().Limit == rate.Inf) { // blocked
+				e := fmt.Errorf("%s: Permanently limited (rate != Inf, burst: 0)", obj)
 				ev.ACK() // ready for next message
+				obj.quiesceGroup.Done()
+				obj.SendEvent(event.EventExit, &SentinelErr{e})
 				continue
 			}

 			// rate limit
 			// FIXME: consider skipping rate limit check if
 			// the event is a poke instead of a watch event
-			if !limited && !(v.Meta().Limit == rate.Inf) { // skip over the playback event...
+			if !limited && !(obj.Meta().Limit == rate.Inf) { // skip over the playback event...
 				now := time.Now()
 				r := limiter.ReserveN(now, 1) // one event
 				// r.OK() seems to always be true here!
@@ -377,11 +398,12 @@ Loop:
 				if d > 0 { // delay
 					limited = true
 					playback = true
-					log.Printf("%s[%s]: Limited (rate: %v/sec, burst: %d, next: %v)", v.Kind(), v.GetName(), v.Meta().Limit, v.Meta().Burst, d)
+					log.Printf("%s: Limited (rate: %v/sec, burst: %d, next: %v)", obj, obj.Meta().Limit, obj.Meta().Burst, d)
 					// start the timer...
 					timer.Reset(d)
 					waiting = true // waiting for retry timer
 					ev.ACK()
+					obj.quiesceGroup.Done()
 					continue
 				} // otherwise, we run directly!
 			}
@@ -390,64 +412,84 @@ Loop:
 			wg.Add(1)
 			running = true
 			go func(ev *event.Event) {
-				pcuid.SetConverged(false) // "block" Process
+				obj.pcuid.SetConverged(false) // "block" Process
 				defer wg.Done()
-				if e := g.Process(v); e != nil {
+				if e := obj.Process(); e != nil {
 					playback = true
-					log.Printf("%s[%s]: CheckApply errored: %v", v.Kind(), v.GetName(), e)
+					log.Printf("%s: CheckApply errored: %v", obj, e)
 					if retry == 0 {
+						if err := obj.Data().Prometheus.UpdateState(obj.String(), obj.GetKind(), prometheus.ResStateHardFail); err != nil {
+							// TODO: how to error this?
+							log.Printf("%s: Prometheus.UpdateState() errored: %v", obj, err)
+						}
+
 						// wrap the error in the sentinel
-						v.SendEvent(event.EventExit, &SentinelErr{e})
+						obj.quiesceGroup.Done() // before the Wait that happens in SendEvent!
+						obj.SendEvent(event.EventExit, &SentinelErr{e})
 						return
 					}
 					if retry > 0 { // don't decrement the -1
 						retry--
 					}
-					log.Printf("%s[%s]: CheckApply: Retrying after %.4f seconds (%d left)", v.Kind(), v.GetName(), delay.Seconds(), retry)
+					if err := obj.Data().Prometheus.UpdateState(obj.String(), obj.GetKind(), prometheus.ResStateSoftFail); err != nil {
+						// TODO: how to error this?
+						log.Printf("%s: Prometheus.UpdateState() errored: %v", obj, err)
+					}
+					log.Printf("%s: CheckApply: Retrying after %.4f seconds (%d left)", obj, delay.Seconds(), retry)
 					// start the timer...
 					timer.Reset(delay)
 					waiting = true // waiting for retry timer
+					// don't obj.quiesceGroup.Done() b/c
+					// the timer is running and it can exit!
 					return
 				}
-				retry = v.Meta().Retry // reset on success
-				close(done)            // trigger
+				retry = obj.Meta().Retry // reset on success
+				close(done)              // trigger
 			}(ev)
 			ev.ACK() // sync (now mostly useless)

 		case <-timer.C:
-			if v.Res.Meta().Poll == 0 { // skip for polling
-				wcuid.SetConverged(false)
+			if obj.Meta().Poll == 0 { // skip for polling
+				obj.wcuid.SetConverged(false)
 			}
 			waiting = false
 			if !timer.Stop() {
 				//<-timer.C // blocks, docs are wrong!
 			}
-			log.Printf("%s[%s]: CheckApply delay expired!", v.Kind(), v.GetName())
+			log.Printf("%s: CheckApply delay expired!", obj)
 			close(done)

 		// a CheckApply run (with possibly retry pause) finished
 		case <-done:
-			if v.Res.Meta().Poll == 0 { // skip for polling
-				wcuid.SetConverged(false)
+			if obj.Meta().Poll == 0 { // skip for polling
+				obj.wcuid.SetConverged(false)
 			}
-			if g.Flags.Debug {
-				log.Printf("%s[%s]: CheckApply finished!", v.Kind(), v.GetName())
+			if b, ok := obj.Graph.Value("debug"); ok && util.Bool(b) {
+				log.Printf("%s: CheckApply finished!", obj)
 			}
 			done = make(chan struct{}) // reset
 			// re-send this event, to trigger a CheckApply()
 			if playback {
-				playback = false
 				// this lock avoids us sending to
 				// channel after we've closed it!
 				// TODO: can this experience indefinite postponement ?
 				// see: https://github.com/golang/go/issues/11506
-				go obj.Event() // replay a new event
+				// pause or exit is in process if not quiescing!
+				if !obj.quiescing {
+					playback = false
+					obj.quiesceGroup.Add(1) // lock around it, b/c still running...
+					go func() {
+						obj.Event() // replay a new event
+						obj.quiesceGroup.Done()
+					}()
+				}
 			}
 			running = false
-			pcuid.SetConverged(true) // "unblock" Process
+			obj.pcuid.SetConverged(true) // "unblock" Process
+			obj.quiesceGroup.Done()

-		case <-wcuid.ConvergedTimer():
-			wcuid.SetConverged(true) // converged!
+		case <-obj.wcuid.ConvergedTimer():
+			obj.wcuid.SetConverged(true) // converged!
 			continue
 		}
 	}
@@ -458,22 +500,21 @@ Loop:
 // Worker is the common run frontend of the vertex. It handles all of the retry
 // and retry delay common code, and ultimately returns the final status of this
 // vertex execution.
-func (g *Graph) Worker(v *Vertex) error {
+func (obj *BaseRes) Worker() error {
 	// listen for chan events from Watch() and run
 	// the Process() function when they're received
 	// this avoids us having to pass the data into
 	// the Watch() function about which graph it is
 	// running on, which isolates things nicely...
-	obj := v.Res
-	if g.Flags.Debug {
-		log.Printf("%s[%s]: Worker: Running", v.Kind(), v.GetName())
-		defer log.Printf("%s[%s]: Worker: Stopped", v.Kind(), v.GetName())
+	if obj.debug {
+		log.Printf("%s: Worker: Running", obj)
+		defer log.Printf("%s: Worker: Stopped", obj)
 	}
 	// run the init (should match 1-1 with Close function)
-	if err := obj.Init(); err != nil {
+	if err := obj.Res.Init(); err != nil {
 		obj.ProcessExit()
 		// always exit the worker function by finishing with Close()
-		if e := obj.Close(); e != nil {
+		if e := obj.Res.Close(); e != nil {
 			err = multierr.Append(err, e) // list of errors
 		}
 		return errwrap.Wrapf(err, "could not Init() resource")
@@ -483,16 +524,15 @@ func (g *Graph) Worker(v *Vertex) error {
 	// timeout, we could inappropriately converge mid-apply!
 	// avoid this by blocking convergence with a fake report
 	// we also add a similar blocker around the worker loop!
-	_, wcuid, pcuid := obj.ConvergerUIDs() // get extra cuids (worker, process)
 	// XXX: put these in Init() ?
-	wcuid.SetConverged(true) // starts off false, and waits for loop timeout
-	pcuid.SetConverged(true) // starts off true, because it's not running...
+	// get extra cuids (worker, process)
+	obj.wcuid.SetConverged(true) // starts off false, and waits for loop timeout
+	obj.pcuid.SetConverged(true) // starts off true, because it's not running...

-	wg := obj.ProcessSync()
-	wg.Add(1)
+	obj.processSync.Add(1)
 	go func() {
-		defer wg.Done()
-		g.innerWorker(v)
+		defer obj.processSync.Done()
+		obj.innerWorker()
 	}()

 	var err error // propagate the error up (this is a permanent BAD error!)
@@ -502,7 +542,7 @@ func (g *Graph) Worker(v *Vertex) error {
 	// NOTE: we're using the same retry and delay metaparams that CheckApply
 	// uses. This is for practicality. We can separate them later if needed!
 	var watchDelay time.Duration
-	var watchRetry = v.Meta().Retry // number of tries left, -1 for infinite
+	var watchRetry = obj.Meta().Retry // number of tries left, -1 for infinite
 	// watch blocks until it ends, & errors to retry
 	for {
 		// TODO: do we have to stop the converged-timeout when in this block (perhaps we're in the delay block!)
@@ -525,7 +565,7 @@ func (g *Graph) Worker(v *Vertex) error {
 					if exit, send := obj.ReadEvent(event); exit != nil {
 						obj.ProcessExit()
 						err := *exit // exit err
-						if e := obj.Close(); err == nil {
+						if e := obj.Res.Close(); err == nil {
 							err = e
 						} else if e != nil {
 							err = multierr.Append(err, e) // list of errors
@@ -555,7 +595,7 @@ func (g *Graph) Worker(v *Vertex) error {
 				}
 			}
 			timer.Stop() // it's nice to cleanup
-			log.Printf("%s[%s]: Watch delay expired!", v.Kind(), v.GetName())
+			log.Printf("%s: Watch delay expired!", obj)
 			// NOTE: we can avoid the send if running Watch guarantees
 			// one CheckApply event on startup!
 			//if pendingSendEvent { // TODO: should this become a list in the future?
@@ -567,13 +607,12 @@ func (g *Graph) Worker(v *Vertex) error {

 		// TODO: reset the watch retry count after some amount of success
 		var e error
-		if v.Res.Meta().Poll > 0 { // poll instead of watching :(
-			cuid, _, _ := v.Res.ConvergerUIDs() // get the converger uid used to report status
-			cuid.StartTimer()
-			e = v.Res.Poll()
-			cuid.StopTimer() // clean up nicely
+		if obj.Meta().Poll > 0 { // poll instead of watching :(
+			obj.cuid.StartTimer()
+			e = obj.Poll()
+			obj.cuid.StopTimer() // clean up nicely
 		} else {
-			e = v.Res.Watch() // run the watch normally
+			e = obj.Res.Watch() // run the watch normally
 		}
 		if e == nil { // exit signal
 			err = nil // clean exit
@@ -583,7 +622,7 @@ func (g *Graph) Worker(v *Vertex) error {
 			err = sentinelErr.err
 			break // sentinel means, perma-exit
 		}
-		log.Printf("%s[%s]: Watch errored: %v", v.Kind(), v.GetName(), e)
+		log.Printf("%s: Watch errored: %v", obj, e)
 		if watchRetry == 0 {
 			err = fmt.Errorf("Permanent watch error: %v", e)
 			break
@@ -591,8 +630,8 @@ func (g *Graph) Worker(v *Vertex) error {
 		if watchRetry > 0 { // don't decrement the -1
 			watchRetry--
 		}
-		watchDelay = time.Duration(v.Meta().Delay) * time.Millisecond
-		log.Printf("%s[%s]: Watch: Retrying after %.4f seconds (%d left)", v.Kind(), v.GetName(), watchDelay.Seconds(), watchRetry)
+		watchDelay = time.Duration(obj.Meta().Delay) * time.Millisecond
+		log.Printf("%s: Watch: Retrying after %.4f seconds (%d left)", obj, watchDelay.Seconds(), watchRetry)
 		// We need to trigger a CheckApply after Watch restarts, so that
 		// we catch any lost events that happened while down. We do this
 		// by getting the Watch resource to send one event once it's up!
@@ -601,97 +640,10 @@ func (g *Graph) Worker(v *Vertex) error {

 	obj.ProcessExit()
 	// close resource and return possible errors if any
-	if e := obj.Close(); err == nil {
+	if e := obj.Res.Close(); err == nil {
 		err = e
 	} else if e != nil {
 		err = multierr.Append(err, e) // list of errors
 	}
 	return err
 }
-
-// Start is a main kick to start the graph. It goes through in reverse topological
-// sort order so that events can't hit un-started vertices.
-func (g *Graph) Start(first bool) { // start or continue
-	log.Printf("State: %v -> %v", g.setState(graphStateStarting), g.getState())
-	defer log.Printf("State: %v -> %v", g.setState(graphStateStarted), g.getState())
-	t, _ := g.TopologicalSort()
-	// TODO: only calculate indegree if `first` is true to save resources
-	indegree := g.InDegree() // compute all of the indegree's
-	for _, v := range Reverse(t) {
-		// selective poke: here we reduce the number of initial pokes
-		// to the minimum required to activate every vertex in the
-		// graph, either by direct action, or by getting poked by a
-		// vertex that was previously activated. if we poke each vertex
-		// that has no incoming edges, then we can be sure to reach the
-		// whole graph. Please note: this may mask certain optimization
-		// failures, such as any poke limiting code in Poke() or
-		// BackPoke(). You might want to disable this selective start
-		// when experimenting with and testing those elements.
-		// if we are unpausing (since it's not the first run of this
-		// function) we need to poke to *unpause* every graph vertex,
-		// and not just selectively the subset with no indegree.
-
-		// let the startup code know to poke or not
-		v.Res.Starter((!first) || indegree[v] == 0)
-
-		if !v.Res.IsWorking() { // if Worker() is not running...
-			v.Res.Setup()
-			g.wg.Add(1)
-			// must pass in value to avoid races...
-			// see: https://ttboj.wordpress.com/2015/07/27/golang-parallelism-issues-causing-too-many-open-files-error/
-			go func(vv *Vertex) {
-				defer g.wg.Done()
-				defer v.Res.Reset()
-				// TODO: if a sufficient number of workers error,
-				// should something be done? Should these restart
-				// after perma-failure if we have a graph change?
-				log.Printf("%s[%s]: Started", vv.Kind(), vv.GetName())
-				if err := g.Worker(vv); err != nil { // contains the Watch and CheckApply loops
-					log.Printf("%s[%s]: Exited with failure: %v", vv.Kind(), vv.GetName(), err)
-					return
-				}
-				log.Printf("%s[%s]: Exited", vv.Kind(), vv.GetName())
-			}(v)
-		}
-
-		select {
-		case <-v.Res.Started(): // block until started
-		case <-v.Res.Stopped(): // we failed on init
-			// if the resource Init() fails, we don't hang!
-		}
-
-		if !first { // unpause!
-			v.Res.SendEvent(event.EventStart, nil) // sync!
-		}
-	}
-	// we wait for everyone to start before exiting!
-}
-
-// Pause sends pause events to the graph in a topological sort order.
-func (g *Graph) Pause() {
-	log.Printf("State: %v -> %v", g.setState(graphStatePausing), g.getState())
-	defer log.Printf("State: %v -> %v", g.setState(graphStatePaused), g.getState())
-	t, _ := g.TopologicalSort()
-	for _, v := range t { // squeeze out the events...
-		v.SendEvent(event.EventPause, nil) // sync
-	}
-}
-
-// Exit sends exit events to the graph in a topological sort order.
-func (g *Graph) Exit() {
-	if g == nil {
-		return
-	} // empty graph that wasn't populated yet
-	t, _ := g.TopologicalSort()
-	for _, v := range t { // squeeze out the events...
-		// turn off the taps...
-		// XXX: consider instead doing this by closing the Res.events channel instead?
-		// XXX: do this by sending an exit signal, and then returning
-		// when we hit the 'default' in the select statement!
-		// XXX: we can do this to quiesce, but it's not necessary now
-
-		v.SendEvent(event.EventExit, nil)
-		v.Res.WaitGroup().Wait()
-	}
-	g.wg.Wait() // for now, this doesn't need to be a separate Wait() method
-}
--- a/resources/augeas.go
+++ b/resources/augeas.go
@@ -3,23 +3,23 @@
 // Written by James Shubin <james@shubin.ca> and the project contributors
 //
 // This program is free software: you can redistribute it and/or modify
-// it under the terms of the GNU Affero General Public License as published by
+// it under the terms of the GNU General Public License as published by
 // the Free Software Foundation, either version 3 of the License, or
 // (at your option) any later version.
 //
 // This program is distributed in the hope that it will be useful,
 // but WITHOUT ANY WARRANTY; without even the implied warranty of
 // MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-// GNU Affero General Public License for more details.
+// GNU General Public License for more details.
 //
-// You should have received a copy of the GNU Affero General Public License
+// You should have received a copy of the GNU General Public License
 // along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
 // +build !noaugeas

 package resources

 import (
-	"encoding/gob"
 	"fmt"
 	"log"
 	"os"
@@ -39,7 +39,7 @@ const (
 )

 func init() {
-	gob.Register(&AugeasRes{})
+	RegisterResource("augeas", func() Res { return &AugeasRes{} })
 }

 // AugeasRes is a resource that enables you to use the augeas resource.
@@ -47,7 +47,7 @@ func init() {
 type AugeasRes struct {
 	BaseRes `yaml:",inline"`

-	// File is the path to the file targetted by this resource.
+	// File is the path to the file targeted by this resource.
 	File string `yaml:"file"`

 	// Lens is the lens used by this resource. If specified, mgmt
@@ -93,7 +93,6 @@ func (obj *AugeasRes) Validate() error {

 // Init initiates the resource.
 func (obj *AugeasRes) Init() error {
-	obj.BaseRes.kind = "Augeas"
 	return obj.BaseRes.Init() // call base init, b/c we're overriding
 }

@@ -118,7 +117,7 @@ func (obj *AugeasRes) Watch() error {

 	for {
 		if obj.debug {
-			log.Printf("%s[%s]: Watching: %s", obj.Kind(), obj.GetName(), obj.File) // attempting to watch...
+			log.Printf("%s: Watching: %s", obj, obj.File) // attempting to watch...
 		}

 		select {
@@ -127,10 +126,10 @@ func (obj *AugeasRes) Watch() error {
 				return nil
 			}
 			if err := event.Error; err != nil {
-				return errwrap.Wrapf(err, "Unknown %s[%s] watcher error", obj.Kind(), obj.GetName())
+				return errwrap.Wrapf(err, "Unknown %s watcher error", obj)
 			}
 			if obj.debug { // don't access event.Body if event.Error isn't nil
-				log.Printf("%s[%s]: Event(%s): %v", obj.Kind(), obj.GetName(), event.Body.Name, event.Body.Op)
+				log.Printf("%s: Event(%s): %v", obj, event.Body.Name, event.Body.Op)
 			}
 			send = true
 			obj.StateOK(false) // dirty
@@ -177,7 +176,7 @@ func (obj *AugeasRes) checkApplySet(apply bool, ag *augeas.Augeas, set AugeasSet

 // CheckApply method for Augeas resource.
 func (obj *AugeasRes) CheckApply(apply bool) (bool, error) {
-	log.Printf("%s[%s]: CheckApply: %s", obj.Kind(), obj.GetName(), obj.File)
+	log.Printf("%s: CheckApply: %s", obj, obj.File)
 	// By default we do not set any option to augeas, we use the defaults.
 	opts := augeas.None
 	if obj.Lens != "" {
@@ -225,7 +224,7 @@ func (obj *AugeasRes) CheckApply(apply bool) (bool, error) {
 		return checkOK, nil
 	}

-	log.Printf("%s[%s]: changes needed, saving", obj.Kind(), obj.GetName())
+	log.Printf("%s: changes needed, saving", obj)
 	if err = ag.Save(); err != nil {
 		return false, errwrap.Wrapf(err, "augeas: error while saving augeas values")
 	}
@@ -247,15 +246,10 @@ type AugeasUID struct {
 	name string
 }

-// AutoEdges returns the AutoEdge interface. In this case no autoedges are used.
-func (obj *AugeasRes) AutoEdges() AutoEdge {
-	return nil
-}
-
 // UIDs includes all params to make a unique identification of this object.
 func (obj *AugeasRes) UIDs() []ResUID {
 	x := &AugeasUID{
-		BaseUID: BaseUID{name: obj.GetName(), kind: obj.Kind()},
+		BaseUID: BaseUID{Name: obj.GetName(), Kind: obj.GetKind()},
 		name:    obj.Name,
 	}
 	return []ResUID{x}
@@ -267,20 +261,19 @@ func (obj *AugeasRes) GroupCmp(r Res) bool {
 }

 // Compare two resources and return if they are equivalent.
-func (obj *AugeasRes) Compare(res Res) bool {
-	switch res.(type) {
-	// we can only compare AugeasRes to others of the same resource
-	case *AugeasRes:
-		res := res.(*AugeasRes)
-		if !obj.BaseRes.Compare(res) { // call base Compare
-			return false
-		}
-		if obj.Name != res.Name {
-			return false
-		}
-	default:
+func (obj *AugeasRes) Compare(r Res) bool {
+	// we can only compare AugeasRes to others of the same resource kind
+	res, ok := r.(*AugeasRes)
+	if !ok {
 		return false
 	}
+	if !obj.BaseRes.Compare(res) { // call base Compare
+		return false
+	}
+	if obj.Name != res.Name {
+		return false
+	}
+
 	return true
 }

--- a/resources/augeas_disabled.go
+++ b/resources/augeas_disabled.go
@@ -3,17 +3,18 @@
 // Written by James Shubin <james@shubin.ca> and the project contributors
 //
 // This program is free software: you can redistribute it and/or modify
-// it under the terms of the GNU Affero General Public License as published by
+// it under the terms of the GNU General Public License as published by
 // the Free Software Foundation, either version 3 of the License, or
 // (at your option) any later version.
 //
 // This program is distributed in the hope that it will be useful,
 // but WITHOUT ANY WARRANTY; without even the implied warranty of
 // MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-// GNU Affero General Public License for more details.
+// GNU General Public License for more details.
 //
-// You should have received a copy of the GNU Affero General Public License
+// You should have received a copy of the GNU General Public License
 // along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
 // +build noaugeas

 package resources
--- a/resources/autoedge.go
+++ b/resources/autoedge.go
@@ -0,0 +1,148 @@
+// Mgmt
+// Copyright (C) 2013-2017+ James Shubin and the project contributors
+// Written by James Shubin <james@shubin.ca> and the project contributors
+//
+// This program is free software: you can redistribute it and/or modify
+// it under the terms of the GNU General Public License as published by
+// the Free Software Foundation, either version 3 of the License, or
+// (at your option) any later version.
+//
+// This program is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+// GNU General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License
+// along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+package resources
+
+import (
+	"fmt"
+	"log"
+
+	"github.com/purpleidea/mgmt/pgraph"
+	"github.com/purpleidea/mgmt/util"
+
+	multierr "github.com/hashicorp/go-multierror"
+	errwrap "github.com/pkg/errors"
+)
+
+// The AutoEdge interface is used to implement the autoedges feature.
+type AutoEdge interface {
+	Next() []ResUID   // call to get list of edges to add
+	Test([]bool) bool // call until false
+}
+
+// UIDExistsInUIDs wraps the IFF method when used with a list of UID's.
+func UIDExistsInUIDs(uid ResUID, uids []ResUID) bool {
+	for _, u := range uids {
+		if uid.IFF(u) {
+			return true
+		}
+	}
+	return false
+}
+
+// addEdgesByMatchingUIDS adds edges to the vertex in a graph based on if it
+// matches a uid list.
+func addEdgesByMatchingUIDS(g *pgraph.Graph, v pgraph.Vertex, uids []ResUID) []bool {
+	// search for edges and see what matches!
+	var result []bool
+
+	// loop through each uid, and see if it matches any vertex
+	for _, uid := range uids {
+		var found = false
+		// uid is a ResUID object
+		for _, vv := range g.Vertices() { // search
+			if v == vv { // skip self
+				continue
+			}
+			if b, ok := g.Value("debug"); ok && util.Bool(b) {
+				log.Printf("Compile: AutoEdge: Match: %s with UID: %s", vv, uid)
+			}
+			// we must match to an effective UID for the resource,
+			// that is to say, the name value of a res is a helpful
+			// handle, but it is not necessarily a unique identity!
+			// remember, resources can return multiple UID's each!
+			if UIDExistsInUIDs(uid, VtoR(vv).UIDs()) {
+				// add edge from: vv -> v
+				if uid.IsReversed() {
+					txt := fmt.Sprintf("AutoEdge: %s -> %s", vv, v)
+					log.Printf("Compile: Adding %s", txt)
+					edge := &Edge{Name: txt}
+					g.AddEdge(vv, v, edge)
+				} else { // edges go the "normal" way, eg: pkg resource
+					txt := fmt.Sprintf("AutoEdge: %s -> %s", v, vv)
+					log.Printf("Compile: Adding %s", txt)
+					edge := &Edge{Name: txt}
+					g.AddEdge(v, vv, edge)
+				}
+				found = true
+				break
+			}
+		}
+		result = append(result, found)
+	}
+	return result
+}
+
+// AutoEdges adds the automatic edges to the graph.
+func AutoEdges(g *pgraph.Graph) error {
+	log.Println("Compile: Adding AutoEdges...")
+
+	// initially get all of the autoedges to seek out all possible errors
+	var err error
+	autoEdgeObjVertexMap := make(map[pgraph.Vertex]AutoEdge)
+	sorted := g.VerticesSorted()
+
+	for _, v := range sorted { // for each vertexes autoedges
+		if !VtoR(v).Meta().AutoEdge { // is the metaparam true?
+			continue
+		}
+		autoEdgeObj, e := VtoR(v).AutoEdges()
+		if e != nil {
+			err = multierr.Append(err, e) // collect all errors
+			continue
+		}
+		if autoEdgeObj == nil {
+			log.Printf("%s: No auto edges were found!", v)
+			continue // next vertex
+		}
+		autoEdgeObjVertexMap[v] = autoEdgeObj // save for next loop
+	}
+	if err != nil {
+		return errwrap.Wrapf(err, "the auto edges had errors")
+	}
+
+	// now that we're guaranteed error free, we can modify the graph safely
+	for _, v := range sorted { // stable sort order for determinism in logs
+		autoEdgeObj, exists := autoEdgeObjVertexMap[v]
+		if !exists {
+			continue
+		}
+
+		for { // while the autoEdgeObj has more uids to add...
+			uids := autoEdgeObj.Next() // get some!
+			if uids == nil {
+				log.Printf("%s: The auto edge list is empty!", v)
+				break // inner loop
+			}
+			if b, ok := g.Value("debug"); ok && util.Bool(b) {
+				log.Println("Compile: AutoEdge: UIDS:")
+				for i, u := range uids {
+					log.Printf("Compile: AutoEdge: UID%d: %v", i, u)
+				}
+			}
+
+			// match and add edges
+			result := addEdgesByMatchingUIDS(g, v, uids)
+
+			// report back, and find out if we should continue
+			if !autoEdgeObj.Test(result) {
+				break
+			}
+		}
+	}
+	return nil
+}
--- a/resources/autogroup.go
+++ b/resources/autogroup.go
@@ -3,61 +3,64 @@
 // Written by James Shubin <james@shubin.ca> and the project contributors
 //
 // This program is free software: you can redistribute it and/or modify
-// it under the terms of the GNU Affero General Public License as published by
+// it under the terms of the GNU General Public License as published by
 // the Free Software Foundation, either version 3 of the License, or
 // (at your option) any later version.
 //
 // This program is distributed in the hope that it will be useful,
 // but WITHOUT ANY WARRANTY; without even the implied warranty of
 // MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-// GNU Affero General Public License for more details.
+// GNU General Public License for more details.
 //
-// You should have received a copy of the GNU Affero General Public License
+// You should have received a copy of the GNU General Public License
 // along with this program.  If not, see <http://www.gnu.org/licenses/>.

-package pgraph
+package resources

 import (
 	"fmt"
 	"log"

+	"github.com/purpleidea/mgmt/pgraph"
+	"github.com/purpleidea/mgmt/util"
+
 	errwrap "github.com/pkg/errors"
 )

-// AutoGrouper is the required interface to implement for an autogroup algorithm
+// AutoGrouper is the required interface to implement for an autogroup algorithm.
 type AutoGrouper interface {
 	// listed in the order these are typically called in...
-	name() string                                  // friendly identifier
-	init(*Graph) error                             // only call once
-	vertexNext() (*Vertex, *Vertex, error)         // mostly algorithmic
-	vertexCmp(*Vertex, *Vertex) error              // can we merge these ?
-	vertexMerge(*Vertex, *Vertex) (*Vertex, error) // vertex merge fn to use
-	edgeMerge(*Edge, *Edge) *Edge                  // edge merge fn to use
-	vertexTest(bool) (bool, error)                 // call until false
+	name() string                                                    // friendly identifier
+	init(*pgraph.Graph) error                                        // only call once
+	vertexNext() (pgraph.Vertex, pgraph.Vertex, error)               // mostly algorithmic
+	vertexCmp(pgraph.Vertex, pgraph.Vertex) error                    // can we merge these ?
+	vertexMerge(pgraph.Vertex, pgraph.Vertex) (pgraph.Vertex, error) // vertex merge fn to use
+	edgeMerge(pgraph.Edge, pgraph.Edge) pgraph.Edge                  // edge merge fn to use
+	vertexTest(bool) (bool, error)                                   // call until false
 }

-// baseGrouper is the base type for implementing the AutoGrouper interface
+// baseGrouper is the base type for implementing the AutoGrouper interface.
 type baseGrouper struct {
-	graph    *Graph    // store a pointer to the graph
-	vertices []*Vertex // cached list of vertices
+	graph    *pgraph.Graph   // store a pointer to the graph
+	vertices []pgraph.Vertex // cached list of vertices
 	i        int
 	j        int
 	done     bool
 }

-// name provides a friendly name for the logs to see
+// name provides a friendly name for the logs to see.
 func (ag *baseGrouper) name() string {
 	return "baseGrouper"
 }

 // init is called only once and before using other AutoGrouper interface methods
 // the name method is the only exception: call it any time without side effects!
-func (ag *baseGrouper) init(g *Graph) error {
+func (ag *baseGrouper) init(g *pgraph.Graph) error {
 	if ag.graph != nil {
 		return fmt.Errorf("the init method has already been called")
 	}
-	ag.graph = g                               // pointer
-	ag.vertices = ag.graph.GetVerticesSorted() // cache in deterministic order!
+	ag.graph = g                            // pointer
+	ag.vertices = ag.graph.VerticesSorted() // cache in deterministic order!
 	ag.i = 0
 	ag.j = 0
 	if len(ag.vertices) == 0 { // empty graph
@@ -71,7 +74,7 @@ func (ag *baseGrouper) init(g *Graph) error {
 // an intelligent algorithm would selectively offer only valid pairs of vertices
 // these should satisfy logical grouping requirements for the autogroup designs!
 // the desired algorithms can override, but keep this method as a base iterator!
-func (ag *baseGrouper) vertexNext() (v1, v2 *Vertex, err error) {
+func (ag *baseGrouper) vertexNext() (v1, v2 pgraph.Vertex, err error) {
 	// this does a for v... { for w... { return v, w }} but stepwise!
 	l := len(ag.vertices)
 	if ag.i < l {
@@ -106,48 +109,49 @@ func (ag *baseGrouper) vertexNext() (v1, v2 *Vertex, err error) {
 	return
 }

-func (ag *baseGrouper) vertexCmp(v1, v2 *Vertex) error {
+func (ag *baseGrouper) vertexCmp(v1, v2 pgraph.Vertex) error {
 	if v1 == nil || v2 == nil {
 		return fmt.Errorf("the vertex is nil")
 	}
 	if v1 == v2 { // skip yourself
 		return fmt.Errorf("the vertices are the same")
 	}
-	if v1.Kind() != v2.Kind() { // we must group similar kinds
+	if VtoR(v1).GetKind() != VtoR(v2).GetKind() { // we must group similar kinds
 		// TODO: maybe future resources won't need this limitation?
 		return fmt.Errorf("the two resources aren't the same kind")
 	}
 	// someone doesn't want to group!
-	if !v1.Meta().AutoGroup || !v2.Meta().AutoGroup {
+	if !VtoR(v1).Meta().AutoGroup || !VtoR(v2).Meta().AutoGroup {
 		return fmt.Errorf("one of the autogroup flags is false")
 	}
-	if v1.Res.IsGrouped() { // already grouped!
+	if VtoR(v1).IsGrouped() { // already grouped!
 		return fmt.Errorf("already grouped")
 	}
-	if len(v2.Res.GetGroup()) > 0 { // already has children grouped!
+	if len(VtoR(v2).GetGroup()) > 0 { // already has children grouped!
 		return fmt.Errorf("already has groups")
 	}
-	if !v1.Res.GroupCmp(v2.Res) { // resource groupcmp failed!
+	if !VtoR(v1).GroupCmp(VtoR(v2)) { // resource groupcmp failed!
 		return fmt.Errorf("the GroupCmp failed")
 	}
 	return nil // success
 }

-func (ag *baseGrouper) vertexMerge(v1, v2 *Vertex) (v *Vertex, err error) {
+func (ag *baseGrouper) vertexMerge(v1, v2 pgraph.Vertex) (v pgraph.Vertex, err error) {
 	// NOTE: it's important to use w.Res instead of w, b/c
 	// the w by itself is the *Vertex obj, not the *Res obj
 	// which is contained within it! They both satisfy the
 	// Res interface, which is why both will compile! :(
-	err = v1.Res.GroupRes(v2.Res) // GroupRes skips stupid groupings
-	return                        // success or fail, and no need to merge the actual vertices!
+	err = VtoR(v1).GroupRes(VtoR(v2)) // GroupRes skips stupid groupings
+	return                            // success or fail, and no need to merge the actual vertices!
 }

-func (ag *baseGrouper) edgeMerge(e1, e2 *Edge) *Edge {
+func (ag *baseGrouper) edgeMerge(e1, e2 pgraph.Edge) pgraph.Edge {
+	// FIXME: should we merge the edge.Notify or edge.refresh values?
 	return e1 // noop
 }

 // vertexTest processes the results of the grouping for the algorithm to know
-// return an error if something went horribly wrong, and bool false to stop
+// return an error if something went horribly wrong, and bool false to stop.
 func (ag *baseGrouper) vertexTest(b bool) (bool, error) {
 	// NOTE: this particular baseGrouper version doesn't track what happens
 	// because since we iterate over every pair, we don't care which merge!
@@ -157,19 +161,20 @@ func (ag *baseGrouper) vertexTest(b bool) (bool, error) {
 	return true, nil
 }

+// NonReachabilityGrouper is the most straight-forward algorithm for grouping.
 // TODO: this algorithm may not be correct in all cases. replace if needed!
-type nonReachabilityGrouper struct {
+type NonReachabilityGrouper struct {
 	baseGrouper // "inherit" what we want, and reimplement the rest
 }

-func (ag *nonReachabilityGrouper) name() string {
-	return "nonReachabilityGrouper"
+func (ag *NonReachabilityGrouper) name() string {
+	return "NonReachabilityGrouper"
 }

-// this algorithm relies on the observation that if there's a path from a to b,
+// This algorithm relies on the observation that if there's a path from a to b,
 // then they *can't* be merged (b/c of the existing dependency) so therefore we
 // merge anything that *doesn't* satisfy this condition or that of the reverse!
-func (ag *nonReachabilityGrouper) vertexNext() (v1, v2 *Vertex, err error) {
+func (ag *NonReachabilityGrouper) vertexNext() (v1, v2 pgraph.Vertex, err error) {
 	for {
 		v1, v2, err = ag.baseGrouper.vertexNext() // get all iterable pairs
 		if err != nil {
@@ -200,15 +205,15 @@ func (ag *nonReachabilityGrouper) vertexNext() (v1, v2 *Vertex, err error) {
 // and then by deleting v2 from the graph. Since more than one edge between two
 // vertices is not allowed, duplicate edges are merged as well. an edge merge
 // function can be provided if you'd like to control how you merge the edges!
-func (g *Graph) VertexMerge(v1, v2 *Vertex, vertexMergeFn func(*Vertex, *Vertex) (*Vertex, error), edgeMergeFn func(*Edge, *Edge) *Edge) error {
+func VertexMerge(g *pgraph.Graph, v1, v2 pgraph.Vertex, vertexMergeFn func(pgraph.Vertex, pgraph.Vertex) (pgraph.Vertex, error), edgeMergeFn func(pgraph.Edge, pgraph.Edge) pgraph.Edge) error {
 	// methodology
 	// 1) edges between v1 and v2 are removed
 	//Loop:
-	for k1 := range g.Adjacency {
-		for k2 := range g.Adjacency[k1] {
+	for k1 := range g.Adjacency() {
+		for k2 := range g.Adjacency()[k1] {
 			// v1 -> v2 || v2 -> v1
 			if (k1 == v1 && k2 == v2) || (k1 == v2 && k2 == v1) {
-				delete(g.Adjacency[k1], k2) // delete map & edge
+				delete(g.Adjacency()[k1], k2) // delete map & edge
 				// NOTE: if we assume this is a DAG, then we can
 				// assume only v1 -> v2 OR v2 -> v1 exists, and
 				// we can break out of these loops immediately!
@@ -220,10 +225,10 @@ func (g *Graph) VertexMerge(v1, v2 *Vertex, vertexMergeFn func(*Vertex, *Vertex)

 	// 2) edges that point towards v2 from X now point to v1 from X (no dupes)
 	for _, x := range g.IncomingGraphVertices(v2) { // all to vertex v (??? -> v)
-		e := g.Adjacency[x][v2] // previous edge
+		e := g.Adjacency()[x][v2] // previous edge
 		r := g.Reachability(x, v1)
-		// merge e with ex := g.Adjacency[x][v1] if it exists!
-		if ex, exists := g.Adjacency[x][v1]; exists && edgeMergeFn != nil && len(r) == 0 {
+		// merge e with ex := g.Adjacency()[x][v1] if it exists!
+		if ex, exists := g.Adjacency()[x][v1]; exists && edgeMergeFn != nil && len(r) == 0 {
 			e = edgeMergeFn(e, ex)
 		}
 		if len(r) == 0 { // if not reachable, add it
@@ -236,21 +241,21 @@ func (g *Graph) VertexMerge(v1, v2 *Vertex, vertexMergeFn func(*Vertex, *Vertex)
 					continue
 				}
 				// this edge is from: prev, to: next
-				ex, _ := g.Adjacency[prev][next] // get
+				ex, _ := g.Adjacency()[prev][next] // get
 				ex = edgeMergeFn(ex, e)
-				g.Adjacency[prev][next] = ex // set
+				g.Adjacency()[prev][next] = ex // set
 				prev = next
 			}
 		}
-		delete(g.Adjacency[x], v2) // delete old edge
+		delete(g.Adjacency()[x], v2) // delete old edge
 	}

 	// 3) edges that point from v2 to X now point from v1 to X (no dupes)
 	for _, x := range g.OutgoingGraphVertices(v2) { // all from vertex v (v -> ???)
-		e := g.Adjacency[v2][x] // previous edge
+		e := g.Adjacency()[v2][x] // previous edge
 		r := g.Reachability(v1, x)
-		// merge e with ex := g.Adjacency[v1][x] if it exists!
-		if ex, exists := g.Adjacency[v1][x]; exists && edgeMergeFn != nil && len(r) == 0 {
+		// merge e with ex := g.Adjacency()[v1][x] if it exists!
+		if ex, exists := g.Adjacency()[v1][x]; exists && edgeMergeFn != nil && len(r) == 0 {
 			e = edgeMergeFn(e, ex)
 		}
 		if len(r) == 0 {
@@ -263,13 +268,13 @@ func (g *Graph) VertexMerge(v1, v2 *Vertex, vertexMergeFn func(*Vertex, *Vertex)
 					continue
 				}
 				// this edge is from: prev, to: next
-				ex, _ := g.Adjacency[prev][next]
+				ex, _ := g.Adjacency()[prev][next]
 				ex = edgeMergeFn(ex, e)
-				g.Adjacency[prev][next] = ex
+				g.Adjacency()[prev][next] = ex
 				prev = next
 			}
 		}
-		delete(g.Adjacency[v2], x)
+		delete(g.Adjacency()[v2], x)
 	}

 	// 4) merge and then remove the (now merged/grouped) vertex
@@ -277,7 +282,8 @@ func (g *Graph) VertexMerge(v1, v2 *Vertex, vertexMergeFn func(*Vertex, *Vertex)
 		if v, err := vertexMergeFn(v1, v2); err != nil {
 			return err
 		} else if v != nil { // replace v1 with the "merged" version...
-			*v1 = *v // TODO: is this safe? (replacing mutexes is undefined!)
+			//*v1 = *v // TODO: is this safe? (replacing mutexes is undefined!)
+			v1 = v
 		}
 	}
 	g.DeleteVertex(v2) // remove grouped vertex
@@ -289,8 +295,8 @@ func (g *Graph) VertexMerge(v1, v2 *Vertex, vertexMergeFn func(*Vertex, *Vertex)
 	return nil // success
 }

-// autoGroup is the mechanical auto group "runner" that runs the interface spec
-func (g *Graph) autoGroup(ag AutoGrouper) chan string {
+// autoGroup is the mechanical auto group "runner" that runs the interface spec.
+func autoGroup(g *pgraph.Graph, ag AutoGrouper) chan string {
 	strch := make(chan string) // output log messages here
 	go func(strch chan string) {
 		strch <- fmt.Sprintf("Compile: Grouping: Algorithm: %v...", ag.name())
@@ -299,7 +305,7 @@ func (g *Graph) autoGroup(ag AutoGrouper) chan string {
 		}

 		for {
-			var v, w *Vertex
+			var v, w pgraph.Vertex
 			v, w, err := ag.vertexNext() // get pair to compare
 			if err != nil {
 				log.Fatalf("error running autoGroup(vertexNext): %v", err)
@@ -310,12 +316,12 @@ func (g *Graph) autoGroup(ag AutoGrouper) chan string {
 			wStr := fmt.Sprintf("%s", w)

 			if err := ag.vertexCmp(v, w); err != nil { // cmp ?
-				if g.Flags.Debug {
+				if b, ok := g.Value("debug"); ok && util.Bool(b) {
 					strch <- fmt.Sprintf("Compile: Grouping: !GroupCmp for: %s into %s", wStr, vStr)
 				}

 				// remove grouped vertex and merge edges (res is safe)
-			} else if err := g.VertexMerge(v, w, ag.vertexMerge, ag.edgeMerge); err != nil { // merge...
+			} else if err := VertexMerge(g, v, w, ag.vertexMerge, ag.edgeMerge); err != nil { // merge...
 				strch <- fmt.Sprintf("Compile: Grouping: !VertexMerge for: %s into %s", wStr, vStr)

 			} else { // success!
@@ -337,12 +343,12 @@ func (g *Graph) autoGroup(ag AutoGrouper) chan string {
 	return strch
 }

-// AutoGroup runs the auto grouping on the graph and prints out log messages
-func (g *Graph) AutoGroup() {
+// AutoGroup runs the auto grouping on the graph and prints out log messages.
+func AutoGroup(g *pgraph.Graph, ag AutoGrouper) {
 	// receive log messages from channel...
 	// this allows test cases to avoid printing them when they're unwanted!
 	// TODO: this algorithm may not be correct in all cases. replace if needed!
-	for str := range g.autoGroup(&nonReachabilityGrouper{}) {
+	for str := range autoGroup(g, ag) {
 		log.Println(str)
 	}
 }
--- a/resources/autogroup_test.go
+++ b/resources/autogroup_test.go
@@ -0,0 +1,733 @@
+// Mgmt
+// Copyright (C) 2013-2017+ James Shubin and the project contributors
+// Written by James Shubin <james@shubin.ca> and the project contributors
+//
+// This program is free software: you can redistribute it and/or modify
+// it under the terms of the GNU General Public License as published by
+// the Free Software Foundation, either version 3 of the License, or
+// (at your option) any later version.
+//
+// This program is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+// GNU General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License
+// along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+package resources
+
+import (
+	"fmt"
+	"reflect"
+	"sort"
+	"strings"
+	"testing"
+	"time"
+
+	"github.com/purpleidea/mgmt/pgraph"
+	"github.com/purpleidea/mgmt/util"
+)
+
+// NE is a helper function to make testing easier. It creates a new noop edge.
+func NE(s string) pgraph.Edge {
+	obj := &Edge{Name: s}
+	return obj
+}
+
+type testGrouper struct {
+	// TODO: this algorithm may not be correct in all cases. replace if needed!
+	NonReachabilityGrouper // "inherit" what we want, and reimplement the rest
+}
+
+func (ag *testGrouper) name() string {
+	return "testGrouper"
+}
+
+func (ag *testGrouper) vertexMerge(v1, v2 pgraph.Vertex) (v pgraph.Vertex, err error) {
+	if err := VtoR(v1).GroupRes(VtoR(v2)); err != nil { // group them first
+		return nil, err
+	}
+	// HACK: update the name so it matches full list of self+grouped
+	obj := VtoR(v1)
+	names := strings.Split(obj.GetName(), ",") // load in stored names
+	for _, n := range obj.GetGroup() {
+		names = append(names, n.GetName()) // add my contents
+	}
+	names = util.StrRemoveDuplicatesInList(names) // remove duplicates
+	sort.Strings(names)
+	obj.SetName(strings.Join(names, ","))
+	return // success or fail, and no need to merge the actual vertices!
+}
+
+func (ag *testGrouper) edgeMerge(e1, e2 pgraph.Edge) pgraph.Edge {
+	edge1 := e1.(*Edge) // panic if wrong
+	edge2 := e2.(*Edge) // panic if wrong
+	// HACK: update the name so it makes a union of both names
+	n1 := strings.Split(edge1.Name, ",") // load
+	n2 := strings.Split(edge2.Name, ",") // load
+	names := append(n1, n2...)
+	names = util.StrRemoveDuplicatesInList(names) // remove duplicates
+	sort.Strings(names)
+	return &Edge{Name: strings.Join(names, ",")}
+}
+
+// helper function
+func runGraphCmp(t *testing.T, g1, g2 *pgraph.Graph) {
+	AutoGroup(g1, &testGrouper{}) // edits the graph
+	err := GraphCmp(g1, g2)
+	if err != nil {
+		t.Logf("  actual (g1): %v%v", g1, fullPrint(g1))
+		t.Logf("expected (g2): %v%v", g2, fullPrint(g2))
+		t.Logf("Cmp error:")
+		t.Errorf("%v", err)
+	}
+}
+
+type NoopResTest struct {
+	NoopRes
+}
+
+func (obj *NoopResTest) GroupCmp(r Res) bool {
+	res, ok := r.(*NoopResTest)
+	if !ok {
+		return false
+	}
+
+	// TODO: implement this in vertexCmp for *testGrouper instead?
+	if strings.Contains(res.Name, ",") { // HACK
+		return false // element to be grouped is already grouped!
+	}
+
+	// group if they start with the same letter! (helpful hack for testing)
+	return obj.Name[0] == res.Name[0]
+}
+
+func NewNoopResTest(name string) *NoopResTest {
+	obj := &NoopResTest{
+		NoopRes: NoopRes{
+			BaseRes: BaseRes{
+				Name: name,
+				Kind: "noop",
+				MetaParams: MetaParams{
+					AutoGroup: true, // always autogroup
+				},
+			},
+		},
+	}
+	return obj
+}
+
+// GraphCmp compares the topology of two graphs and returns nil if they're
+// equal. It also compares if grouped element groups are identical.
+// TODO: port this to use the pgraph.GraphCmp function instead.
+func GraphCmp(g1, g2 *pgraph.Graph) error {
+	if n1, n2 := g1.NumVertices(), g2.NumVertices(); n1 != n2 {
+		return fmt.Errorf("graph g1 has %d vertices, while g2 has %d", n1, n2)
+	}
+	if e1, e2 := g1.NumEdges(), g2.NumEdges(); e1 != e2 {
+		return fmt.Errorf("graph g1 has %d edges, while g2 has %d", e1, e2)
+	}
+
+	var m = make(map[pgraph.Vertex]pgraph.Vertex) // g1 to g2 vertex correspondence
+Loop:
+	// check vertices
+	for v1 := range g1.Adjacency() { // for each vertex in g1
+
+		l1 := strings.Split(VtoR(v1).GetName(), ",") // make list of everyone's names...
+		for _, x1 := range VtoR(v1).GetGroup() {
+			l1 = append(l1, x1.GetName()) // add my contents
+		}
+		l1 = util.StrRemoveDuplicatesInList(l1) // remove duplicates
+		sort.Strings(l1)
+
+		// inner loop
+		for v2 := range g2.Adjacency() { // does it match in g2 ?
+
+			l2 := strings.Split(VtoR(v2).GetName(), ",")
+			for _, x2 := range VtoR(v2).GetGroup() {
+				l2 = append(l2, x2.GetName())
+			}
+			l2 = util.StrRemoveDuplicatesInList(l2) // remove duplicates
+			sort.Strings(l2)
+
+			// does l1 match l2 ?
+			if ListStrCmp(l1, l2) { // cmp!
+				m[v1] = v2
+				continue Loop
+			}
+		}
+		return fmt.Errorf("graph g1, has no match in g2 for: %v", VtoR(v1).GetName())
+	}
+	// vertices (and groups) match :)
+
+	// check edges
+	for v1 := range g1.Adjacency() { // for each vertex in g1
+		v2 := m[v1] // lookup in map to get correspondance
+		// g1.Adjacency()[v1] corresponds to g2.Adjacency()[v2]
+		if e1, e2 := len(g1.Adjacency()[v1]), len(g2.Adjacency()[v2]); e1 != e2 {
+			return fmt.Errorf("graph g1, vertex(%v) has %d edges, while g2, vertex(%v) has %d", VtoR(v1).GetName(), e1, VtoR(v2).GetName(), e2)
+		}
+
+		for vv1, ee1 := range g1.Adjacency()[v1] {
+			vv2 := m[vv1]
+			ee1 := ee1.(*Edge)
+			ee2 := g2.Adjacency()[v2][vv2].(*Edge)
+
+			// these are edges from v1 -> vv1 via ee1 (graph 1)
+			// to cmp to edges from v2 -> vv2 via ee2 (graph 2)
+
+			// check: (1) vv1 == vv2 ? (we've already checked this!)
+			l1 := strings.Split(VtoR(vv1).GetName(), ",") // make list of everyone's names...
+			for _, x1 := range VtoR(vv1).GetGroup() {
+				l1 = append(l1, x1.GetName()) // add my contents
+			}
+			l1 = util.StrRemoveDuplicatesInList(l1) // remove duplicates
+			sort.Strings(l1)
+
+			l2 := strings.Split(VtoR(vv2).GetName(), ",")
+			for _, x2 := range VtoR(vv2).GetGroup() {
+				l2 = append(l2, x2.GetName())
+			}
+			l2 = util.StrRemoveDuplicatesInList(l2) // remove duplicates
+			sort.Strings(l2)
+
+			// does l1 match l2 ?
+			if !ListStrCmp(l1, l2) { // cmp!
+				return fmt.Errorf("graph g1 and g2 don't agree on: %v and %v", VtoR(vv1).GetName(), VtoR(vv2).GetName())
+			}
+
+			// check: (2) ee1 == ee2
+			if ee1.Name != ee2.Name {
+				return fmt.Errorf("graph g1 edge(%v) doesn't match g2 edge(%v)", ee1.Name, ee2.Name)
+			}
+		}
+	}
+
+	// check meta parameters
+	for v1 := range g1.Adjacency() { // for each vertex in g1
+		for v2 := range g2.Adjacency() { // does it match in g2 ?
+			s1, s2 := VtoR(v1).Meta().Sema, VtoR(v2).Meta().Sema
+			sort.Strings(s1)
+			sort.Strings(s2)
+			if !reflect.DeepEqual(s1, s2) {
+				return fmt.Errorf("vertex %s and vertex %s have different semaphores", VtoR(v1).GetName(), VtoR(v2).GetName())
+			}
+		}
+	}
+
+	return nil // success!
+}
+
+// ListStrCmp compares two lists of strings
+func ListStrCmp(a, b []string) bool {
+	//fmt.Printf("CMP: %v with %v\n", a, b) // debugging
+	if a == nil && b == nil {
+		return true
+	}
+	if a == nil || b == nil {
+		return false
+	}
+	if len(a) != len(b) {
+		return false
+	}
+	for i := range a {
+		if a[i] != b[i] {
+			return false
+		}
+	}
+	return true
+}
+
+func fullPrint(g *pgraph.Graph) (str string) {
+	str += "\n"
+	for v := range g.Adjacency() {
+		if semas := VtoR(v).Meta().Sema; len(semas) > 0 {
+			str += fmt.Sprintf("* v: %v; sema: %v\n", VtoR(v).GetName(), semas)
+		} else {
+			str += fmt.Sprintf("* v: %v\n", VtoR(v).GetName())
+		}
+		// TODO: add explicit grouping data?
+	}
+	for v1 := range g.Adjacency() {
+		for v2, e := range g.Adjacency()[v1] {
+			edge := e.(*Edge)
+			str += fmt.Sprintf("* e: %v -> %v # %v\n", VtoR(v1).GetName(), VtoR(v2).GetName(), edge.Name)
+		}
+	}
+	return
+}
+
+func TestDurationAssumptions(t *testing.T) {
+	var d time.Duration
+	if (d == 0) != true {
+		t.Errorf("empty time.Duration is no longer equal to zero")
+	}
+	if (d > 0) != false {
+		t.Errorf("empty time.Duration is now greater than zero")
+	}
+}
+
+// all of the following test cases are laid out with the following semantics:
+// * vertices which start with the same single letter are considered "like"
+// * "like" elements should be merged
+// * vertices can have any integer after their single letter "family" type
+// * grouped vertices should have a name with a comma separated list of names
+// * edges follow the same conventions about grouping
+
+// empty graph
+func TestPgraphGrouping1(t *testing.T) {
+	g1, _ := pgraph.NewGraph("g1") // original graph
+	g2, _ := pgraph.NewGraph("g2") // expected result
+	runGraphCmp(t, g1, g2)
+}
+
+// single vertex
+func TestPgraphGrouping2(t *testing.T) {
+	g1, _ := pgraph.NewGraph("g1") // original graph
+	{                              // grouping to limit variable scope
+		a1 := NewNoopResTest("a1")
+		g1.AddVertex(a1)
+	}
+	g2, _ := pgraph.NewGraph("g2") // expected result
+	{
+		a1 := NewNoopResTest("a1")
+		g2.AddVertex(a1)
+	}
+	runGraphCmp(t, g1, g2)
+}
+
+// two vertices
+func TestPgraphGrouping3(t *testing.T) {
+	g1, _ := pgraph.NewGraph("g1") // original graph
+	{
+		a1 := NewNoopResTest("a1")
+		b1 := NewNoopResTest("b1")
+		g1.AddVertex(a1, b1)
+	}
+	g2, _ := pgraph.NewGraph("g2") // expected result
+	{
+		a1 := NewNoopResTest("a1")
+		b1 := NewNoopResTest("b1")
+		g2.AddVertex(a1, b1)
+	}
+	runGraphCmp(t, g1, g2)
+}
+
+// two vertices merge
+func TestPgraphGrouping4(t *testing.T) {
+	g1, _ := pgraph.NewGraph("g1") // original graph
+	{
+		a1 := NewNoopResTest("a1")
+		a2 := NewNoopResTest("a2")
+		g1.AddVertex(a1, a2)
+	}
+	g2, _ := pgraph.NewGraph("g2") // expected result
+	{
+		a := NewNoopResTest("a1,a2")
+		g2.AddVertex(a)
+	}
+	runGraphCmp(t, g1, g2)
+}
+
+// three vertices merge
+func TestPgraphGrouping5(t *testing.T) {
+	g1, _ := pgraph.NewGraph("g1") // original graph
+	{
+		a1 := NewNoopResTest("a1")
+		a2 := NewNoopResTest("a2")
+		a3 := NewNoopResTest("a3")
+		g1.AddVertex(a1, a2, a3)
+	}
+	g2, _ := pgraph.NewGraph("g2") // expected result
+	{
+		a := NewNoopResTest("a1,a2,a3")
+		g2.AddVertex(a)
+	}
+	runGraphCmp(t, g1, g2)
+}
+
+// three vertices, two merge
+func TestPgraphGrouping6(t *testing.T) {
+	g1, _ := pgraph.NewGraph("g1") // original graph
+	{
+		a1 := NewNoopResTest("a1")
+		a2 := NewNoopResTest("a2")
+		b1 := NewNoopResTest("b1")
+		g1.AddVertex(a1, a2, b1)
+	}
+	g2, _ := pgraph.NewGraph("g2") // expected result
+	{
+		a := NewNoopResTest("a1,a2")
+		b1 := NewNoopResTest("b1")
+		g2.AddVertex(a, b1)
+	}
+	runGraphCmp(t, g1, g2)
+}
+
+// four vertices, three merge
+func TestPgraphGrouping7(t *testing.T) {
+	g1, _ := pgraph.NewGraph("g1") // original graph
+	{
+		a1 := NewNoopResTest("a1")
+		a2 := NewNoopResTest("a2")
+		a3 := NewNoopResTest("a3")
+		b1 := NewNoopResTest("b1")
+		g1.AddVertex(a1, a2, a3, b1)
+	}
+	g2, _ := pgraph.NewGraph("g2") // expected result
+	{
+		a := NewNoopResTest("a1,a2,a3")
+		b1 := NewNoopResTest("b1")
+		g2.AddVertex(a, b1)
+	}
+	runGraphCmp(t, g1, g2)
+}
+
+// four vertices, two&two merge
+func TestPgraphGrouping8(t *testing.T) {
+	g1, _ := pgraph.NewGraph("g1") // original graph
+	{
+		a1 := NewNoopResTest("a1")
+		a2 := NewNoopResTest("a2")
+		b1 := NewNoopResTest("b1")
+		b2 := NewNoopResTest("b2")
+		g1.AddVertex(a1, a2, b1, b2)
+	}
+	g2, _ := pgraph.NewGraph("g2") // expected result
+	{
+		a := NewNoopResTest("a1,a2")
+		b := NewNoopResTest("b1,b2")
+		g2.AddVertex(a, b)
+	}
+	runGraphCmp(t, g1, g2)
+}
+
+// five vertices, two&three merge
+func TestPgraphGrouping9(t *testing.T) {
+	g1, _ := pgraph.NewGraph("g1") // original graph
+	{
+		a1 := NewNoopResTest("a1")
+		a2 := NewNoopResTest("a2")
+		b1 := NewNoopResTest("b1")
+		b2 := NewNoopResTest("b2")
+		b3 := NewNoopResTest("b3")
+		g1.AddVertex(a1, a2, b1, b2, b3)
+	}
+	g2, _ := pgraph.NewGraph("g2") // expected result
+	{
+		a := NewNoopResTest("a1,a2")
+		b := NewNoopResTest("b1,b2,b3")
+		g2.AddVertex(a, b)
+	}
+	runGraphCmp(t, g1, g2)
+}
+
+// three unique vertices
+func TestPgraphGrouping10(t *testing.T) {
+	g1, _ := pgraph.NewGraph("g1") // original graph
+	{
+		a1 := NewNoopResTest("a1")
+		b1 := NewNoopResTest("b1")
+		c1 := NewNoopResTest("c1")
+		g1.AddVertex(a1, b1, c1)
+	}
+	g2, _ := pgraph.NewGraph("g2") // expected result
+	{
+		a1 := NewNoopResTest("a1")
+		b1 := NewNoopResTest("b1")
+		c1 := NewNoopResTest("c1")
+		g2.AddVertex(a1, b1, c1)
+	}
+	runGraphCmp(t, g1, g2)
+}
+
+// three unique vertices, two merge
+func TestPgraphGrouping11(t *testing.T) {
+	g1, _ := pgraph.NewGraph("g1") // original graph
+	{
+		a1 := NewNoopResTest("a1")
+		b1 := NewNoopResTest("b1")
+		b2 := NewNoopResTest("b2")
+		c1 := NewNoopResTest("c1")
+		g1.AddVertex(a1, b1, b2, c1)
+	}
+	g2, _ := pgraph.NewGraph("g2") // expected result
+	{
+		a1 := NewNoopResTest("a1")
+		b := NewNoopResTest("b1,b2")
+		c1 := NewNoopResTest("c1")
+		g2.AddVertex(a1, b, c1)
+	}
+	runGraphCmp(t, g1, g2)
+}
+
+// simple merge 1
+// a1   a2         a1,a2
+//   \ /     >>>     |     (arrows point downwards)
+//    b              b
+func TestPgraphGrouping12(t *testing.T) {
+	g1, _ := pgraph.NewGraph("g1") // original graph
+	{
+		a1 := NewNoopResTest("a1")
+		a2 := NewNoopResTest("a2")
+		b1 := NewNoopResTest("b1")
+		e1 := NE("e1")
+		e2 := NE("e2")
+		g1.AddEdge(a1, b1, e1)
+		g1.AddEdge(a2, b1, e2)
+	}
+	g2, _ := pgraph.NewGraph("g2") // expected result
+	{
+		a := NewNoopResTest("a1,a2")
+		b1 := NewNoopResTest("b1")
+		e := NE("e1,e2")
+		g2.AddEdge(a, b1, e)
+	}
+	runGraphCmp(t, g1, g2)
+}
+
+// simple merge 2
+//    b              b
+//   / \     >>>     |     (arrows point downwards)
+// a1   a2         a1,a2
+func TestPgraphGrouping13(t *testing.T) {
+	g1, _ := pgraph.NewGraph("g1") // original graph
+	{
+		a1 := NewNoopResTest("a1")
+		a2 := NewNoopResTest("a2")
+		b1 := NewNoopResTest("b1")
+		e1 := NE("e1")
+		e2 := NE("e2")
+		g1.AddEdge(b1, a1, e1)
+		g1.AddEdge(b1, a2, e2)
+	}
+	g2, _ := pgraph.NewGraph("g2") // expected result
+	{
+		a := NewNoopResTest("a1,a2")
+		b1 := NewNoopResTest("b1")
+		e := NE("e1,e2")
+		g2.AddEdge(b1, a, e)
+	}
+	runGraphCmp(t, g1, g2)
+}
+
+// triple merge
+// a1 a2  a3         a1,a2,a3
+//   \ | /     >>>       |      (arrows point downwards)
+//     b                 b
+func TestPgraphGrouping14(t *testing.T) {
+	g1, _ := pgraph.NewGraph("g1") // original graph
+	{
+		a1 := NewNoopResTest("a1")
+		a2 := NewNoopResTest("a2")
+		a3 := NewNoopResTest("a3")
+		b1 := NewNoopResTest("b1")
+		e1 := NE("e1")
+		e2 := NE("e2")
+		e3 := NE("e3")
+		g1.AddEdge(a1, b1, e1)
+		g1.AddEdge(a2, b1, e2)
+		g1.AddEdge(a3, b1, e3)
+	}
+	g2, _ := pgraph.NewGraph("g2") // expected result
+	{
+		a := NewNoopResTest("a1,a2,a3")
+		b1 := NewNoopResTest("b1")
+		e := NE("e1,e2,e3")
+		g2.AddEdge(a, b1, e)
+	}
+	runGraphCmp(t, g1, g2)
+}
+
+// chain merge
+//    a1             a1
+//   /  \             |
+// b1    b2   >>>   b1,b2   (arrows point downwards)
+//   \  /             |
+//    c1             c1
+func TestPgraphGrouping15(t *testing.T) {
+	g1, _ := pgraph.NewGraph("g1") // original graph
+	{
+		a1 := NewNoopResTest("a1")
+		b1 := NewNoopResTest("b1")
+		b2 := NewNoopResTest("b2")
+		c1 := NewNoopResTest("c1")
+		e1 := NE("e1")
+		e2 := NE("e2")
+		e3 := NE("e3")
+		e4 := NE("e4")
+		g1.AddEdge(a1, b1, e1)
+		g1.AddEdge(a1, b2, e2)
+		g1.AddEdge(b1, c1, e3)
+		g1.AddEdge(b2, c1, e4)
+	}
+	g2, _ := pgraph.NewGraph("g2") // expected result
+	{
+		a1 := NewNoopResTest("a1")
+		b := NewNoopResTest("b1,b2")
+		c1 := NewNoopResTest("c1")
+		e1 := NE("e1,e2")
+		e2 := NE("e3,e4")
+		g2.AddEdge(a1, b, e1)
+		g2.AddEdge(b, c1, e2)
+	}
+	runGraphCmp(t, g1, g2)
+}
+
+// re-attach 1 (outer)
+// technically the second possibility is valid too, depending on which order we
+// merge edges in, and if we don't filter out any unnecessary edges afterwards!
+// a1    a2         a1,a2        a1,a2
+//  |   /             |            |  \
+// b1  /      >>>    b1     OR    b1  /   (arrows point downwards)
+//  | /               |            | /
+// c1                c1           c1
+func TestPgraphGrouping16(t *testing.T) {
+	g1, _ := pgraph.NewGraph("g1") // original graph
+	{
+		a1 := NewNoopResTest("a1")
+		a2 := NewNoopResTest("a2")
+		b1 := NewNoopResTest("b1")
+		c1 := NewNoopResTest("c1")
+		e1 := NE("e1")
+		e2 := NE("e2")
+		e3 := NE("e3")
+		g1.AddEdge(a1, b1, e1)
+		g1.AddEdge(b1, c1, e2)
+		g1.AddEdge(a2, c1, e3)
+	}
+	g2, _ := pgraph.NewGraph("g2") // expected result
+	{
+		a := NewNoopResTest("a1,a2")
+		b1 := NewNoopResTest("b1")
+		c1 := NewNoopResTest("c1")
+		e1 := NE("e1,e3")
+		e2 := NE("e2,e3") // e3 gets "merged through" to BOTH edges!
+		g2.AddEdge(a, b1, e1)
+		g2.AddEdge(b1, c1, e2)
+	}
+	runGraphCmp(t, g1, g2)
+}
+
+// re-attach 2 (inner)
+// a1    b2          a1
+//  |   /             |
+// b1  /      >>>   b1,b2   (arrows point downwards)
+//  | /               |
+// c1                c1
+func TestPgraphGrouping17(t *testing.T) {
+	g1, _ := pgraph.NewGraph("g1") // original graph
+	{
+		a1 := NewNoopResTest("a1")
+		b1 := NewNoopResTest("b1")
+		b2 := NewNoopResTest("b2")
+		c1 := NewNoopResTest("c1")
+		e1 := NE("e1")
+		e2 := NE("e2")
+		e3 := NE("e3")
+		g1.AddEdge(a1, b1, e1)
+		g1.AddEdge(b1, c1, e2)
+		g1.AddEdge(b2, c1, e3)
+	}
+	g2, _ := pgraph.NewGraph("g2") // expected result
+	{
+		a1 := NewNoopResTest("a1")
+		b := NewNoopResTest("b1,b2")
+		c1 := NewNoopResTest("c1")
+		e1 := NE("e1")
+		e2 := NE("e2,e3")
+		g2.AddEdge(a1, b, e1)
+		g2.AddEdge(b, c1, e2)
+	}
+	runGraphCmp(t, g1, g2)
+}
+
+// re-attach 3 (double)
+// similar to "re-attach 1", technically there is a second possibility for this
+// a2   a1    b2         a1,a2
+//   \   |   /             |
+//    \ b1  /      >>>   b1,b2   (arrows point downwards)
+//     \ | /               |
+//      c1                c1
+func TestPgraphGrouping18(t *testing.T) {
+	g1, _ := pgraph.NewGraph("g1") // original graph
+	{
+		a1 := NewNoopResTest("a1")
+		a2 := NewNoopResTest("a2")
+		b1 := NewNoopResTest("b1")
+		b2 := NewNoopResTest("b2")
+		c1 := NewNoopResTest("c1")
+		e1 := NE("e1")
+		e2 := NE("e2")
+		e3 := NE("e3")
+		e4 := NE("e4")
+		g1.AddEdge(a1, b1, e1)
+		g1.AddEdge(b1, c1, e2)
+		g1.AddEdge(a2, c1, e3)
+		g1.AddEdge(b2, c1, e4)
+	}
+	g2, _ := pgraph.NewGraph("g2") // expected result
+	{
+		a := NewNoopResTest("a1,a2")
+		b := NewNoopResTest("b1,b2")
+		c1 := NewNoopResTest("c1")
+		e1 := NE("e1,e3")
+		e2 := NE("e2,e3,e4") // e3 gets "merged through" to BOTH edges!
+		g2.AddEdge(a, b, e1)
+		g2.AddEdge(b, c1, e2)
+	}
+	runGraphCmp(t, g1, g2)
+}
+
+// connected merge 0, (no change!)
+// a1            a1
+//   \     >>>     \     (arrows point downwards)
+//    a2            a2
+func TestPgraphGroupingConnected0(t *testing.T) {
+	g1, _ := pgraph.NewGraph("g1") // original graph
+	{
+		a1 := NewNoopResTest("a1")
+		a2 := NewNoopResTest("a2")
+		e1 := NE("e1")
+		g1.AddEdge(a1, a2, e1)
+	}
+	g2, _ := pgraph.NewGraph("g2") // expected result ?
+	{
+		a1 := NewNoopResTest("a1")
+		a2 := NewNoopResTest("a2")
+		e1 := NE("e1")
+		g2.AddEdge(a1, a2, e1)
+	}
+	runGraphCmp(t, g1, g2)
+}
+
+// connected merge 1, (no change!)
+// a1              a1
+//   \               \
+//    b      >>>      b      (arrows point downwards)
+//     \               \
+//      a2              a2
+func TestPgraphGroupingConnected1(t *testing.T) {
+	g1, _ := pgraph.NewGraph("g1") // original graph
+	{
+		a1 := NewNoopResTest("a1")
+		b := NewNoopResTest("b")
+		a2 := NewNoopResTest("a2")
+		e1 := NE("e1")
+		e2 := NE("e2")
+		g1.AddEdge(a1, b, e1)
+		g1.AddEdge(b, a2, e2)
+	}
+	g2, _ := pgraph.NewGraph("g2") // expected result ?
+	{
+		a1 := NewNoopResTest("a1")
+		b := NewNoopResTest("b")
+		a2 := NewNoopResTest("a2")
+		e1 := NE("e1")
+		e2 := NE("e2")
+		g2.AddEdge(a1, b, e1)
+		g2.AddEdge(b, a2, e2)
+	}
+	runGraphCmp(t, g1, g2)
+}
--- a/resources/aws_ec2.go
+++ b/resources/aws_ec2.go
@@ -0,0 +1,653 @@
+// Mgmt
+// Copyright (C) 2013-2017+ James Shubin and the project contributors
+// Written by James Shubin <james@shubin.ca> and the project contributors
+//
+// This program is free software: you can redistribute it and/or modify
+// it under the terms of the GNU General Public License as published by
+// the Free Software Foundation, either version 3 of the License, or
+// (at your option) any later version.
+//
+// This program is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+// GNU General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License
+// along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+package resources
+
+import (
+	"context"
+	"fmt"
+	"log"
+	"sync"
+	"time"
+
+	"github.com/aws/aws-sdk-go/aws"
+	"github.com/aws/aws-sdk-go/aws/awserr"
+	"github.com/aws/aws-sdk-go/aws/request"
+	"github.com/aws/aws-sdk-go/aws/session"
+	"github.com/aws/aws-sdk-go/service/ec2"
+	errwrap "github.com/pkg/errors"
+)
+
+func init() {
+	RegisterResource("aws:ec2", func() Res { return &AwsEc2Res{} })
+}
+
+// AwsPrefix is a const which gets prepended onto the instance name.
+const AwsPrefix = "mgmt:"
+
+// AwsRegions is a list of all AWS regions generated using ec2.DescribeRegions.
+// cn-north-1 and us-gov-west-1 are not returned, probably due to security.
+// List available at http://docs.aws.amazon.com/general/latest/gr/rande.html
+var AwsRegions = []string{
+	"ap-northeast-1",
+	"ap-northeast-2",
+	"ap-south-1",
+	"ap-southeast-1",
+	"ap-southeast-2",
+	"ca-central-1",
+	"cn-north-1",
+	"eu-central-1",
+	"eu-west-1",
+	"eu-west-2",
+	"sa-east-1",
+	"us-east-1",
+	"us-east-2",
+	"us-gov-west-1",
+	"us-west-1",
+	"us-west-2",
+}
+
+// AwsEc2Res is an AWS EC2 resource. In order to create a client session, your
+// AWS credentials must be present in ~/.aws - For detailed instructions see
+// http://docs.aws.amazon.com/cli/latest/userguide/cli-config-files.html
+type AwsEc2Res struct {
+	BaseRes `yaml:",inline"`
+	State   string `yaml:"state"` // state: running, stopped, terminated
+	Region  string `yaml:"region"`
+	Type    string `yaml:"type"` // the ec2 instance type, ie. t2.micro
+	ImageID string `yaml:"imageid"`
+
+	client *ec2.EC2 // client session for AWS API calls
+}
+
+// Default returns some sensible defaults for this resource.
+func (obj *AwsEc2Res) Default() Res {
+	return &AwsEc2Res{
+		BaseRes: BaseRes{
+			MetaParams: DefaultMetaParams, // force a default
+		},
+	}
+}
+
+// Validate if the params passed in are valid data.
+func (obj *AwsEc2Res) Validate() error {
+	if obj.State != "running" && obj.State != "stopped" && obj.State != "terminated" {
+		return fmt.Errorf("state must be 'running', 'stopped' or 'terminated'")
+	}
+
+	// compare obj.Region to the list of available AWS endpoints.
+	validRegion := false
+	for _, region := range AwsRegions {
+		if obj.Region == region {
+			validRegion = true
+			break
+		}
+	}
+	if !validRegion {
+		return fmt.Errorf("region must be a valid AWS endpoint")
+	}
+
+	// check the instance type
+	// there is currently no api call to enumerate available instance types
+	if obj.Type == "" {
+		return fmt.Errorf("no instance type specified")
+	}
+
+	// check imageId against a list of available images
+	sess, err := session.NewSession(&aws.Config{
+		Region: aws.String(obj.Region),
+	})
+	if err != nil {
+		return errwrap.Wrapf(err, "error creating session")
+	}
+	client := ec2.New(sess)
+
+	imagesInput := &ec2.DescribeImagesInput{}
+	images, err := client.DescribeImages(imagesInput)
+	if err != nil {
+		return errwrap.Wrapf(err, "error describing images")
+	}
+	validImage := false
+	for _, image := range images.Images {
+		if obj.ImageID == *image.ImageId {
+			validImage = true
+			break
+		}
+	}
+	if !validImage {
+		return fmt.Errorf("imageid must be a valid ami available in the specified region")
+	}
+
+	return obj.BaseRes.Validate()
+}
+
+// Init initializes the resource.
+func (obj *AwsEc2Res) Init() error {
+	// create a client session for the AWS API
+	sess, err := session.NewSession(&aws.Config{
+		Region: aws.String(obj.Region),
+	})
+	if err != nil {
+		return errwrap.Wrapf(err, "error creating session")
+	}
+	obj.client = ec2.New(sess)
+
+	return obj.BaseRes.Init() // call base init, b/c we're overriding
+}
+
+// Watch is the primary listener for this resource and it outputs events.
+func (obj *AwsEc2Res) Watch() error {
+	send := false
+	var exit *error
+	if err := obj.Running(); err != nil {
+		return err
+	}
+	type chanStruct struct {
+		str string
+		err error
+	}
+	awsChan := make(chan *chanStruct)
+	closeChan := make(chan struct{})
+	ctx, cancel := context.WithCancel(context.TODO())
+	wg := &sync.WaitGroup{}
+	wg.Add(1)
+	go func() {
+		defer wg.Done()
+		select {
+		case <-closeChan:
+			cancel()
+		}
+	}()
+	wg.Add(1)
+	defer wg.Wait()
+	go func() {
+		defer wg.Done()
+		defer close(awsChan)
+		for {
+			diInput := &ec2.DescribeInstancesInput{
+				Filters: []*ec2.Filter{
+					{
+						Name:   aws.String("tag:Name"),
+						Values: []*string{aws.String(obj.prependName())},
+					},
+					{
+						Name: aws.String("instance-state-name"),
+						Values: []*string{
+							aws.String("pending"),
+							aws.String("running"),
+							aws.String("stopping"),
+							aws.String("stopped"),
+						},
+					},
+				},
+			}
+			diOutput, err := obj.client.DescribeInstances(diInput)
+			if err != nil {
+				select {
+				case awsChan <- &chanStruct{
+					err: errwrap.Wrapf(err, "error describing instances"),
+				}:
+				case <-closeChan:
+				}
+				return
+			}
+			if obj.State == "running" {
+				stoppedInput := &ec2.DescribeInstancesInput{
+					Filters: []*ec2.Filter{
+						{
+							Name:   aws.String("tag:Name"),
+							Values: []*string{aws.String(obj.prependName())},
+						},
+						{
+							Name: aws.String("instance-state-name"),
+							Values: []*string{
+								aws.String("stopped"),
+							},
+						},
+					},
+				}
+				stoppedOutput, err := obj.client.DescribeInstances(stoppedInput)
+				if err != nil {
+					select {
+					case awsChan <- &chanStruct{
+						err: errwrap.Wrapf(err, "error describing instances"),
+					}:
+					case <-closeChan:
+					}
+					return
+				}
+				if len(diOutput.Reservations) == 1 && len(stoppedOutput.Reservations) == 0 {
+					waitInput := &ec2.DescribeInstancesInput{
+						InstanceIds: []*string{diOutput.Reservations[0].Instances[0].InstanceId},
+						Filters: []*ec2.Filter{
+							{
+								Name: aws.String("instance-state-name"),
+								Values: []*string{
+									aws.String("stopped"),
+									aws.String("terminated"),
+								},
+							},
+						},
+					}
+					log.Printf("%s: Watching: %s", obj, *diOutput.Reservations[0].Instances[0].InstanceId)
+					if err := obj.client.WaitUntilInstanceStoppedWithContext(ctx, waitInput); err != nil {
+						if aerr, ok := err.(awserr.Error); ok {
+							if aerr.Code() == request.CanceledErrorCode {
+								log.Printf("%s: Request cancelled", obj)
+							}
+						}
+						select {
+						case awsChan <- &chanStruct{
+							err: errwrap.Wrapf(err, "unknown error waiting for instance to stop"),
+						}:
+						case <-closeChan:
+						}
+						return
+					}
+					stateOutput, err := obj.client.DescribeInstances(diInput)
+					if err != nil {
+						select {
+						case awsChan <- &chanStruct{
+							err: errwrap.Wrapf(err, "error describing instances"),
+						}:
+						case <-closeChan:
+						}
+						return
+					}
+					var stateName string
+					if len(stateOutput.Reservations) == 1 {
+						stateName = *stateOutput.Reservations[0].Instances[0].State.Name
+					}
+					if len(stateOutput.Reservations) == 0 || (len(stateOutput.Reservations) == 1 && stateName != "running") {
+						select {
+						case awsChan <- &chanStruct{
+							str: "stopped",
+						}:
+						case <-closeChan:
+							return
+						}
+					}
+				}
+			}
+			if obj.State == "stopped" {
+				runningInput := &ec2.DescribeInstancesInput{
+					Filters: []*ec2.Filter{
+						{
+							Name:   aws.String("tag:Name"),
+							Values: []*string{aws.String(obj.prependName())},
+						},
+						{
+							Name: aws.String("instance-state-name"),
+							Values: []*string{
+								aws.String("running"),
+							},
+						},
+					},
+				}
+				runningOutput, err := obj.client.DescribeInstances(runningInput)
+				if err != nil {
+					select {
+					case awsChan <- &chanStruct{
+						err: errwrap.Wrapf(err, "error describing instances"),
+					}:
+					case <-closeChan:
+					}
+					return
+				}
+				if len(diOutput.Reservations) == 1 && len(runningOutput.Reservations) == 0 {
+					waitInput := &ec2.DescribeInstancesInput{
+						InstanceIds: []*string{diOutput.Reservations[0].Instances[0].InstanceId},
+						Filters: []*ec2.Filter{
+							{
+								Name:   aws.String("instance-state-name"),
+								Values: []*string{aws.String("running")},
+							},
+						},
+					}
+					log.Printf("%s: watching: %s", obj, *diOutput.Reservations[0].Instances[0].InstanceId)
+					if err := obj.client.WaitUntilInstanceRunningWithContext(ctx, waitInput); err != nil {
+						if aerr, ok := err.(awserr.Error); ok {
+							if aerr.Code() == request.CanceledErrorCode {
+								log.Printf("%s: Request cancelled", obj)
+							}
+						}
+						select {
+						case awsChan <- &chanStruct{
+							err: errwrap.Wrapf(err, "unknown error waiting for instance to start"),
+						}:
+						case <-closeChan:
+						}
+						return
+					}
+					stateOutput, err := obj.client.DescribeInstances(diInput)
+					if err != nil {
+						select {
+						case awsChan <- &chanStruct{
+							err: errwrap.Wrapf(err, "error describing instances"),
+						}:
+						case <-closeChan:
+						}
+						return
+					}
+					var stateName string
+					if len(stateOutput.Reservations) == 1 {
+						stateName = *stateOutput.Reservations[0].Instances[0].State.Name
+					}
+					if len(stateOutput.Reservations) == 0 || (len(stateOutput.Reservations) == 1 && stateName != "stopped") {
+						select {
+						case awsChan <- &chanStruct{
+							str: "running",
+						}:
+						case <-closeChan:
+							return
+						}
+					}
+				}
+			}
+			if obj.State == "terminated" {
+				obj.client.WaitUntilInstanceExistsWithContext(ctx, diInput)
+				if err := obj.client.WaitUntilInstanceExistsWithContext(ctx, diInput); err != nil {
+					if aerr, ok := err.(awserr.Error); ok {
+						if aerr.Code() == request.CanceledErrorCode {
+							log.Printf("%s: Request cancelled", obj)
+						}
+					}
+					select {
+					case awsChan <- &chanStruct{
+						err: errwrap.Wrapf(err, "unknown error waiting for instance to exist"),
+					}:
+					case <-closeChan:
+					}
+					return
+				}
+				stateOutput, err := obj.client.DescribeInstances(diInput)
+				if err != nil {
+					select {
+					case awsChan <- &chanStruct{
+						err: errwrap.Wrapf(err, "error describing instances"),
+					}:
+					case <-closeChan:
+					}
+					return
+				}
+				if len(stateOutput.Reservations) == 1 {
+					{
+						select {
+						case awsChan <- &chanStruct{
+							str: "exists",
+						}:
+						case <-closeChan:
+							return
+						}
+					}
+				}
+			}
+			select {
+			case <-closeChan:
+				return
+			default:
+			}
+		}
+	}()
+	for {
+		select {
+		case event := <-obj.Events():
+			if exit, send = obj.ReadEvent(event); exit != nil {
+				close(closeChan)
+				return *exit
+			}
+		case msg, ok := <-awsChan:
+			if !ok {
+				return *exit
+			}
+			if err := msg.err; err != nil {
+				return err
+			}
+			log.Printf("%s: State: %s", obj, msg.str)
+			obj.StateOK(false)
+			send = true
+		}
+		if send {
+			send = false
+			obj.Event()
+		}
+	}
+}
+
+// CheckApply method for AwsEc2 resource.
+func (obj *AwsEc2Res) CheckApply(apply bool) (checkOK bool, err error) {
+	log.Printf("%s: CheckApply(%t)", obj, apply)
+
+	const waitTimeout = 400
+
+	diInput := ec2.DescribeInstancesInput{
+		Filters: []*ec2.Filter{
+			{
+				Name:   aws.String("tag:Name"),
+				Values: []*string{aws.String(obj.prependName())},
+			},
+			{
+				Name: aws.String("instance-state-name"),
+				Values: []*string{
+					aws.String("running"),
+					aws.String("pending"),
+					aws.String("stopped"),
+					aws.String("stopping"),
+				},
+			},
+		},
+	}
+	diOutput, err := obj.client.DescribeInstances(&diInput)
+	if err != nil {
+		return false, errwrap.Wrapf(err, "error describing instances")
+	}
+
+	if len(diOutput.Reservations) < 1 && obj.State == "terminated" {
+		return true, nil
+	}
+	if len(diOutput.Reservations) == 1 && *diOutput.Reservations[0].Instances[0].State.Name == obj.State {
+		return true, nil
+	}
+	if !apply {
+		return false, nil
+	}
+
+	if len(diOutput.Reservations) > 1 {
+		return false, fmt.Errorf("too many reservations")
+	}
+	ctx, cancel := context.WithTimeout(context.TODO(), waitTimeout*time.Second)
+	defer cancel()
+	if len(diOutput.Reservations) == 1 {
+		instanceID := diOutput.Reservations[0].Instances[0].InstanceId
+		describeInput := &ec2.DescribeInstancesInput{
+			InstanceIds: []*string{instanceID},
+		}
+		if len(diOutput.Reservations[0].Instances) > 1 {
+			return false, fmt.Errorf("more than one instance was returned")
+		}
+		if obj.State == "running" {
+			startInput := &ec2.StartInstancesInput{
+				InstanceIds: []*string{instanceID},
+			}
+			_, err := obj.client.StartInstances(startInput)
+			if err != nil {
+				return false, errwrap.Wrapf(err, "error starting instance")
+			}
+			if err := obj.client.WaitUntilInstanceRunningWithContext(ctx, describeInput); err != nil {
+				if aerr, ok := err.(awserr.Error); ok {
+					if aerr.Code() == request.CanceledErrorCode {
+						return false, errwrap.Wrapf(err, "timeout while waiting for instance to start")
+					}
+				}
+				return false, errwrap.Wrapf(err, "unknown error waiting for instance to start")
+			}
+			log.Printf("%s: instance running", obj)
+		}
+		if obj.State == "stopped" {
+			stopInput := &ec2.StopInstancesInput{
+				InstanceIds: []*string{instanceID},
+			}
+			_, err := obj.client.StopInstances(stopInput)
+			if err != nil {
+				return false, errwrap.Wrapf(err, "error stopping instance")
+			}
+			if err := obj.client.WaitUntilInstanceStoppedWithContext(ctx, describeInput); err != nil {
+				if aerr, ok := err.(awserr.Error); ok {
+					if aerr.Code() == request.CanceledErrorCode {
+						return false, errwrap.Wrapf(err, "timeout while waiting for instance to stop")
+					}
+				}
+				return false, errwrap.Wrapf(err, "unknown error waiting for instance to stop")
+			}
+			log.Printf("%s: instance stopped", obj)
+		}
+		if obj.State == "terminated" {
+			terminateInput := &ec2.TerminateInstancesInput{
+				InstanceIds: []*string{instanceID},
+			}
+			_, err := obj.client.TerminateInstances(terminateInput)
+			if err != nil {
+				return false, errwrap.Wrapf(err, "error terminating instance")
+			}
+			if err := obj.client.WaitUntilInstanceTerminatedWithContext(ctx, describeInput); err != nil {
+				if aerr, ok := err.(awserr.Error); ok {
+					if aerr.Code() == request.CanceledErrorCode {
+						return false, errwrap.Wrapf(err, "timeout while waiting for instance to terminate")
+					}
+				}
+				return false, errwrap.Wrapf(err, "unknown error waiting for instance to terminate")
+			}
+			log.Printf("%s: instance terminated", obj)
+		}
+	}
+	if len(diOutput.Reservations) < 1 && obj.State == "running" {
+		runParams := &ec2.RunInstancesInput{
+			ImageId:      aws.String(obj.ImageID),
+			InstanceType: aws.String(obj.Type),
+		}
+		runParams.SetMinCount(1)
+		runParams.SetMaxCount(1)
+		runResult, err := obj.client.RunInstances(runParams)
+		if err != nil {
+			return false, errwrap.Wrapf(err, "could not create instance")
+		}
+		_, err = obj.client.CreateTags(&ec2.CreateTagsInput{
+			Resources: []*string{runResult.Instances[0].InstanceId},
+			Tags: []*ec2.Tag{
+				&ec2.Tag{
+					Key:   aws.String("Name"),
+					Value: aws.String(obj.prependName()),
+				},
+			},
+		})
+		if err != nil {
+			return false, errwrap.Wrapf(err, "could not create tags for instance")
+		}
+
+		describeInput := &ec2.DescribeInstancesInput{
+			InstanceIds: []*string{runResult.Instances[0].InstanceId},
+		}
+		err = obj.client.WaitUntilInstanceRunningWithContext(ctx, describeInput)
+		if err != nil {
+			if aerr, ok := err.(awserr.Error); ok {
+				if aerr.Code() == request.CanceledErrorCode {
+					return false, errwrap.Wrapf(err, "timeout while waiting for instance to start")
+				}
+			}
+			return false, errwrap.Wrapf(err, "unknown error waiting for instance to start")
+		}
+		log.Printf("%s: instance running", obj)
+	}
+	return false, nil
+}
+
+// AwsEc2UID is the UID struct for AwsEc2Res.
+type AwsEc2UID struct {
+	BaseUID
+	name string
+}
+
+// UIDs includes all params to make a unique identification of this object.
+// Most resources only return one, although some resources can return multiple.
+func (obj *AwsEc2Res) UIDs() []ResUID {
+	x := &AwsEc2UID{
+		BaseUID: BaseUID{Name: obj.GetName(), Kind: obj.GetKind()},
+		name:    obj.Name,
+	}
+	return []ResUID{x}
+}
+
+// GroupCmp returns whether two resources can be grouped together or not.
+func (obj *AwsEc2Res) GroupCmp(r Res) bool {
+	_, ok := r.(*AwsEc2Res)
+	if !ok {
+		return false
+	}
+	return false
+}
+
+// Compare two resources and return if they are equivalent.
+func (obj *AwsEc2Res) Compare(r Res) bool {
+	// we can only compare AwsEc2Res to others of the same resource kind
+	res, ok := r.(*AwsEc2Res)
+	if !ok {
+		return false
+	}
+	if !obj.BaseRes.Compare(res) { // call base Compare
+		return false
+	}
+	if obj.Name != res.Name {
+		return false
+	}
+	if obj.State != res.State {
+		return false
+	}
+	if obj.Region != res.Region {
+		return false
+	}
+	if obj.Type != res.Type {
+		return false
+	}
+	if obj.ImageID != res.ImageID {
+		return false
+	}
+	return true
+}
+
+// UnmarshalYAML is the custom unmarshal handler for this struct.
+// It is primarily useful for setting the defaults.
+func (obj *AwsEc2Res) UnmarshalYAML(unmarshal func(interface{}) error) error {
+	type rawRes AwsEc2Res // indirection to avoid infinite recursion
+
+	def := obj.Default()        // get the default
+	res, ok := def.(*AwsEc2Res) // put in the right format
+	if !ok {
+		return fmt.Errorf("could not convert to AwsEc2Res")
+	}
+	raw := rawRes(*res) // convert; the defaults go here
+
+	if err := unmarshal(&raw); err != nil {
+		return err
+	}
+
+	*obj = AwsEc2Res(raw) // restore from indirection with type conversion!
+	return nil
+}
+
+func (obj *AwsEc2Res) prependName() string {
+	return AwsPrefix + obj.GetName()
+}
--- a/resources/edge.go
+++ b/resources/edge.go
@@ -0,0 +1,56 @@
+// Mgmt
+// Copyright (C) 2013-2017+ James Shubin and the project contributors
+// Written by James Shubin <james@shubin.ca> and the project contributors
+//
+// This program is free software: you can redistribute it and/or modify
+// it under the terms of the GNU General Public License as published by
+// the Free Software Foundation, either version 3 of the License, or
+// (at your option) any later version.
+//
+// This program is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+// GNU General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License
+// along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+package resources
+
+// Edge is a struct that represents a graph's edge.
+type Edge struct {
+	Name   string
+	Notify bool // should we send a refresh notification along this edge?
+
+	refresh bool // is there a notify pending for the dest vertex ?
+}
+
+// String is a required method of the Edge interface that we must fulfill.
+func (obj *Edge) String() string {
+	return obj.Name
+}
+
+// Compare returns true if two edges are equivalent. Otherwise it returns false.
+func (obj *Edge) Compare(edge *Edge) bool {
+	if obj.Name != edge.Name {
+		return false
+	}
+	if obj.Notify != edge.Notify {
+		return false
+	}
+	// FIXME: should we compare this as well?
+	//if obj.refresh != edge.refresh {
+	//	return false
+	//}
+	return true
+}
+
+// Refresh returns the pending refresh status of this edge.
+func (obj *Edge) Refresh() bool {
+	return obj.refresh
+}
+
+// SetRefresh sets the pending refresh status of this edge.
+func (obj *Edge) SetRefresh(b bool) {
+	obj.refresh = b
+}
--- a/resources/exec.go
+++ b/resources/exec.go
@@ -3,16 +3,16 @@
 // Written by James Shubin <james@shubin.ca> and the project contributors
 //
 // This program is free software: you can redistribute it and/or modify
-// it under the terms of the GNU Affero General Public License as published by
+// it under the terms of the GNU General Public License as published by
 // the Free Software Foundation, either version 3 of the License, or
 // (at your option) any later version.
 //
 // This program is distributed in the hope that it will be useful,
 // but WITHOUT ANY WARRANTY; without even the implied warranty of
 // MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-// GNU Affero General Public License for more details.
+// GNU General Public License for more details.
 //
-// You should have received a copy of the GNU Affero General Public License
+// You should have received a copy of the GNU General Public License
 // along with this program.  If not, see <http://www.gnu.org/licenses/>.

 package resources
@@ -20,11 +20,12 @@ package resources
 import (
 	"bufio"
 	"bytes"
-	"encoding/gob"
 	"fmt"
 	"log"
 	"os/exec"
 	"strings"
+	"sync"
+	"syscall"

 	"github.com/purpleidea/mgmt/util"

@@ -32,21 +33,22 @@ import (
 )

 func init() {
-	gob.Register(&ExecRes{})
+	RegisterResource("exec", func() Res { return &ExecRes{} })
 }

 // ExecRes is an exec resource for running commands.
 type ExecRes struct {
 	BaseRes    `yaml:",inline"`
-	State      string `yaml:"state"`      // state: exists/present?, absent, (undefined?)
-	Cmd        string `yaml:"cmd"`        // the command to run
-	Shell      string `yaml:"shell"`      // the (optional) shell to use to run the cmd
-	Timeout    int    `yaml:"timeout"`    // the cmd timeout in seconds
-	WatchCmd   string `yaml:"watchcmd"`   // the watch command to run
-	WatchShell string `yaml:"watchshell"` // the (optional) shell to use to run the watch cmd
-	IfCmd      string `yaml:"ifcmd"`      // the if command to run
-	IfShell    string `yaml:"ifshell"`    // the (optional) shell to use to run the if cmd
-	PollInt    int    `yaml:"pollint"`    // the poll interval for the ifcmd
+	Cmd        string  `yaml:"cmd"`        // the command to run
+	Shell      string  `yaml:"shell"`      // the (optional) shell to use to run the cmd
+	Timeout    int     `yaml:"timeout"`    // the cmd timeout in seconds
+	WatchCmd   string  `yaml:"watchcmd"`   // the watch command to run
+	WatchShell string  `yaml:"watchshell"` // the (optional) shell to use to run the watch cmd
+	IfCmd      string  `yaml:"ifcmd"`      // the if command to run
+	IfShell    string  `yaml:"ifshell"`    // the (optional) shell to use to run the if cmd
+	Output     *string // all cmd output, read only, do not set!
+	Stdout     *string // the cmd stdout, read only, do not set!
+	Stderr     *string // the cmd stderr, read only, do not set!
 }

 // Default returns some sensible defaults for this resource.
@@ -64,17 +66,11 @@ func (obj *ExecRes) Validate() error {
 		return fmt.Errorf("command can't be empty")
 	}

-	// if we have a watch command, then we don't poll with the if command!
-	if obj.WatchCmd != "" && obj.PollInt > 0 {
-		return fmt.Errorf("don't poll when we have a watch command")
-	}
-
 	return obj.BaseRes.Validate()
 }

 // Init runs some startup code for this resource.
 func (obj *ExecRes) Init() error {
-	obj.BaseRes.kind = "Exec"
 	return obj.BaseRes.Init() // call base init, b/c we're overriding
 }

@@ -119,6 +115,11 @@ func (obj *ExecRes) Watch() error {
 		}
 		cmd := exec.Command(cmdName, cmdArgs...)
 		//cmd.Dir = "" // look for program in pwd ?
+		// ignore signals sent to parent process (we're in our own group)
+		cmd.SysProcAttr = &syscall.SysProcAttr{
+			Setpgid: true,
+			Pgid:    0,
+		}

 		cmdReader, err := cmd.StdoutPipe()
 		if err != nil {
@@ -126,11 +127,11 @@ func (obj *ExecRes) Watch() error {
 		}
 		scanner := bufio.NewScanner(cmdReader)

-		defer cmd.Wait() // XXX: is this necessary?
+		defer cmd.Wait() // wait for the command to exit before return!
 		defer func() {
 			// FIXME: without wrapping this in this func it panic's
-			// when running examples/graph8d.yaml
-			cmd.Process.Kill() // TODO: is this necessary?
+			// when running certain graphs... why?
+			cmd.Process.Kill() // shutdown the Watch command on exit
 		}()
 		if err := cmd.Start(); err != nil {
 			return errwrap.Wrapf(err, "error starting Cmd")
@@ -148,9 +149,10 @@ func (obj *ExecRes) Watch() error {
 		select {
 		case text := <-bufioch:
 			// each time we get a line of output, we loop!
-			log.Printf("%s[%s]: Watch output: %s", obj.Kind(), obj.GetName(), text)
+			log.Printf("%s: Watch output: %s", obj, text)
 			if text != "" {
 				send = true
+				obj.StateOK(false) // something made state dirty
 			}

 		case err := <-errch:
@@ -171,8 +173,6 @@ func (obj *ExecRes) Watch() error {
 		// do all our event sending all together to avoid duplicate msgs
 		if send {
 			send = false
-			// it is okay to invalidate the clean state on poke too
-			obj.StateOK(false) // something made state dirty
 			obj.Event()
 		}
 	}
@@ -181,24 +181,12 @@ func (obj *ExecRes) Watch() error {
 // CheckApply checks the resource state and applies the resource if the bool
 // input is true. It returns error info and if the state check passed or not.
 // TODO: expand the IfCmd to be a list of commands
-func (obj *ExecRes) CheckApply(apply bool) (checkOK bool, err error) {
+func (obj *ExecRes) CheckApply(apply bool) (bool, error) {
+	// If we receive a refresh signal, then the engine skips the IsStateOK()
+	// check and this will run. It is still guarded by the IfCmd, but it can
+	// have a chance to execute, and all without the check of obj.Refresh()!

-	// if there is a watch command, but no if command, run based on state
-	if obj.WatchCmd != "" && obj.IfCmd == "" {
-		if obj.IsStateOK() { // FIXME: this is done by engine now...
-			return true, nil
-		}
-
-		// if there is no watcher, but there is an onlyif check, run it to see
-	} else if obj.IfCmd != "" { // && obj.WatchCmd == ""
-		// there is a watcher, but there is also an if command
-		//} else if obj.IfCmd != "" && obj.WatchCmd != "" {
-
-		if obj.PollInt > 0 { // && obj.WatchCmd == ""
-			// XXX: have the Watch() command output onlyif poll events...
-			// XXX: we can optimize by saving those results for returning here
-			// return XXX
-		}
+	if obj.IfCmd != "" { // if there is no onlyif check, we should just run

 		var cmdName string
 		var cmdArgs []string
@@ -214,18 +202,17 @@ func (obj *ExecRes) CheckApply(apply bool) (checkOK bool, err error) {
 			cmdName = obj.IfShell // usually bash, or sh
 			cmdArgs = []string{"-c", obj.IfCmd}
 		}
-		err = exec.Command(cmdName, cmdArgs...).Run()
-		if err != nil {
+		cmd := exec.Command(cmdName, cmdArgs...)
+		// ignore signals sent to parent process (we're in our own group)
+		cmd.SysProcAttr = &syscall.SysProcAttr{
+			Setpgid: true,
+			Pgid:    0,
+		}
+		if err := cmd.Run(); err != nil {
 			// TODO: check exit value
 			return true, nil // don't run
 		}

-		// if there is no watcher and no onlyif check, assume we should run
-	} else { // if obj.WatchCmd == "" && obj.IfCmd == "" {
-		// just run if state is dirty
-		if obj.IsStateOK() { // FIXME: this is done by engine now...
-			return true, nil
-		}
 	}

 	// state is not okay, no work done, exit, but without error
@@ -234,7 +221,7 @@ func (obj *ExecRes) CheckApply(apply bool) (checkOK bool, err error) {
 	}

 	// apply portion
-	log.Printf("%s[%s]: Apply", obj.Kind(), obj.GetName())
+	log.Printf("%s: Apply", obj)
 	var cmdName string
 	var cmdArgs []string
 	if obj.Shell == "" {
@@ -252,11 +239,21 @@ func (obj *ExecRes) CheckApply(apply bool) (checkOK bool, err error) {
 	}
 	cmd := exec.Command(cmdName, cmdArgs...)
 	//cmd.Dir = "" // look for program in pwd ?
-	var out bytes.Buffer
-	cmd.Stdout = &out
+	// ignore signals sent to parent process (we're in our own group)
+	cmd.SysProcAttr = &syscall.SysProcAttr{
+		Setpgid: true,
+		Pgid:    0,
+	}
+
+	var out splitWriter
+	out.Init()
+	// from the docs: "If Stdout and Stderr are the same writer, at most one
+	// goroutine at a time will call Write." so we trick it here!
+	cmd.Stdout = out.Stdout
+	cmd.Stderr = out.Stderr

 	if err := cmd.Start(); err != nil {
-		return false, errwrap.Wrapf(err, "error starting Cmd")
+		return false, errwrap.Wrapf(err, "error starting cmd")
 	}

 	timeout := obj.Timeout
@@ -266,29 +263,57 @@ func (obj *ExecRes) CheckApply(apply bool) (checkOK bool, err error) {
 	done := make(chan error)
 	go func() { done <- cmd.Wait() }()

+	var err error // error returned by cmd
 	select {
-	case err := <-done:
-		if err != nil {
-			e := errwrap.Wrapf(err, "error waiting for Cmd")
-			return false, e
-		}
+	case e := <-done:
+		err = e // store

 	case <-util.TimeAfterOrBlock(timeout):
-		//cmd.Process.Kill() // TODO: is this necessary?
-		return false, fmt.Errorf("timeout waiting for Cmd")
+		cmd.Process.Kill() // TODO: check error?
+		return false, fmt.Errorf("timeout for cmd")
+	}
+
+	// save in memory for send/recv
+	// we use pointers to strings to indicate if used or not
+	if out.Stdout.Activity || out.Stderr.Activity {
+		str := out.String()
+		obj.Output = &str
+	}
+	if out.Stdout.Activity {
+		str := out.Stdout.String()
+		obj.Stdout = &str
+	}
+	if out.Stderr.Activity {
+		str := out.Stderr.String()
+		obj.Stderr = &str
+	}
+
+	// process the err result from cmd, we process non-zero exits here too!
+	exitErr, ok := err.(*exec.ExitError) // embeds an os.ProcessState
+	if err != nil && ok {
+		pStateSys := exitErr.Sys() // (*os.ProcessState) Sys
+		wStatus, ok := pStateSys.(syscall.WaitStatus)
+		if !ok {
+			e := errwrap.Wrapf(err, "error running cmd")
+			return false, e
+		}
+		return false, fmt.Errorf("cmd error, exit status: %d", wStatus.ExitStatus())
+
+	} else if err != nil {
+		e := errwrap.Wrapf(err, "general cmd error")
+		return false, e
 	}

 	// TODO: if we printed the stdout while the command is running, this
 	// would be nice, but it would require terminal log output that doesn't
 	// interleave all the parallel parts which would mix it all up...
 	if s := out.String(); s == "" {
-		log.Printf("%s[%s]: Command output is empty!", obj.Kind(), obj.GetName())
+		log.Printf("%s: Command output is empty!", obj)

 	} else {
-		log.Printf("%s[%s]: Command output is:", obj.Kind(), obj.GetName())
+		log.Printf("%s: Command output is:", obj)
 		log.Printf(out.String())
 	}
-	// XXX: return based on exit value!!

 	// The state tracking is for exec resources that can't "detect" their
 	// state, and assume it's invalid when the Watch() function triggers.
@@ -306,52 +331,18 @@ type ExecUID struct {
 	// TODO: add more elements here
 }

-// IFF aka if and only if they are equivalent, return true. If not, false.
-func (obj *ExecUID) IFF(uid ResUID) bool {
-	res, ok := uid.(*ExecUID)
-	if !ok {
-		return false
-	}
-	if obj.Cmd != res.Cmd {
-		return false
-	}
-	// TODO: add more checks here
-	//if obj.Shell != res.Shell {
-	//	return false
-	//}
-	//if obj.Timeout != res.Timeout {
-	//	return false
-	//}
-	//if obj.WatchCmd != res.WatchCmd {
-	//	return false
-	//}
-	//if obj.WatchShell != res.WatchShell {
-	//	return false
-	//}
-	if obj.IfCmd != res.IfCmd {
-		return false
-	}
-	//if obj.PollInt != res.PollInt {
-	//	return false
-	//}
-	//if obj.State != res.State {
-	//	return false
-	//}
-	return true
-}
-
 // AutoEdges returns the AutoEdge interface. In this case no autoedges are used.
-func (obj *ExecRes) AutoEdges() AutoEdge {
+func (obj *ExecRes) AutoEdges() (AutoEdge, error) {
 	// TODO: parse as many exec params to look for auto edges, for example
 	// the path of the binary in the Cmd variable might be from in a pkg
-	return nil
+	return nil, nil
 }

 // UIDs includes all params to make a unique identification of this object.
 // Most resources only return one, although some resources can return multiple.
 func (obj *ExecRes) UIDs() []ResUID {
 	x := &ExecUID{
-		BaseUID: BaseUID{name: obj.GetName(), kind: obj.Kind()},
+		BaseUID: BaseUID{Name: obj.GetName(), Kind: obj.GetKind()},
 		Cmd:     obj.Cmd,
 		IfCmd:   obj.IfCmd,
 		// TODO: add more params here
@@ -369,44 +360,41 @@ func (obj *ExecRes) GroupCmp(r Res) bool {
 }

 // Compare two resources and return if they are equivalent.
-func (obj *ExecRes) Compare(res Res) bool {
-	switch res.(type) {
-	case *ExecRes:
-		res := res.(*ExecRes)
-		if !obj.BaseRes.Compare(res) { // call base Compare
-			return false
-		}
-
-		if obj.Name != res.Name {
-			return false
-		}
-		if obj.Cmd != res.Cmd {
-			return false
-		}
-		if obj.Shell != res.Shell {
-			return false
-		}
-		if obj.Timeout != res.Timeout {
-			return false
-		}
-		if obj.WatchCmd != res.WatchCmd {
-			return false
-		}
-		if obj.WatchShell != res.WatchShell {
-			return false
-		}
-		if obj.IfCmd != res.IfCmd {
-			return false
-		}
-		if obj.PollInt != res.PollInt {
-			return false
-		}
-		if obj.State != res.State {
-			return false
-		}
-	default:
+func (obj *ExecRes) Compare(r Res) bool {
+	// we can only compare ExecRes to others of the same resource kind
+	res, ok := r.(*ExecRes)
+	if !ok {
 		return false
 	}
+	if !obj.BaseRes.Compare(res) { // call base Compare
+		return false
+	}
+	if obj.Name != res.Name {
+		return false
+	}
+
+	if obj.Cmd != res.Cmd {
+		return false
+	}
+	if obj.Shell != res.Shell {
+		return false
+	}
+	if obj.Timeout != res.Timeout {
+		return false
+	}
+	if obj.WatchCmd != res.WatchCmd {
+		return false
+	}
+	if obj.WatchShell != res.WatchShell {
+		return false
+	}
+	if obj.IfCmd != res.IfCmd {
+		return false
+	}
+	if obj.IfShell != res.IfShell {
+		return false
+	}
+
 	return true
 }

@@ -429,3 +417,71 @@ func (obj *ExecRes) UnmarshalYAML(unmarshal func(interface{}) error) error {
 	*obj = ExecRes(raw) // restore from indirection with type conversion!
 	return nil
 }
+
+// splitWriter mimics what the ssh.CombinedOutput command does, but stores the
+// the stdout and stderr separately. This is slightly tricky because we don't
+// want the combined output to be interleaved incorrectly. It creates sub writer
+// structs which share the same lock and a shared output buffer.
+type splitWriter struct {
+	Stdout *wrapWriter
+	Stderr *wrapWriter
+
+	stdout      bytes.Buffer // just the stdout
+	stderr      bytes.Buffer // just the stderr
+	output      bytes.Buffer // combined output
+	mutex       *sync.Mutex
+	initialized bool // is this initialized?
+}
+
+// Init initializes the splitWriter.
+func (sw *splitWriter) Init() {
+	if sw.initialized {
+		panic("splitWriter is already initialized")
+	}
+	sw.mutex = &sync.Mutex{}
+	sw.Stdout = &wrapWriter{
+		Mutex:  sw.mutex,
+		Buffer: &sw.stdout,
+		Output: &sw.output,
+	}
+	sw.Stderr = &wrapWriter{
+		Mutex:  sw.mutex,
+		Buffer: &sw.stderr,
+		Output: &sw.output,
+	}
+	sw.initialized = true
+}
+
+// String returns the contents of the combined output buffer.
+func (sw *splitWriter) String() string {
+	if !sw.initialized {
+		panic("splitWriter is not initialized")
+	}
+	return sw.output.String()
+}
+
+// wrapWriter is a simple writer which is used internally by splitWriter.
+type wrapWriter struct {
+	Mutex    *sync.Mutex
+	Buffer   *bytes.Buffer // stdout or stderr
+	Output   *bytes.Buffer // combined output
+	Activity bool          // did we get any writes?
+}
+
+// Write writes to both bytes buffers with a parent lock to mix output safely.
+func (w *wrapWriter) Write(p []byte) (int, error) {
+	// TODO: can we move the lock to only guard around the Output.Write ?
+	w.Mutex.Lock()
+	defer w.Mutex.Unlock()
+	w.Activity = true
+	i, err := w.Buffer.Write(p) // first write
+	if err != nil {
+		return i, err
+	}
+	return w.Output.Write(p) // shared write
+}
+
+// String returns the contents of the unshared buffer.
+func (w *wrapWriter) String() string {
+	return w.Buffer.String()
+}
--- a/resources/exec_test.go
+++ b/resources/exec_test.go
@@ -0,0 +1,190 @@
+// Mgmt
+// Copyright (C) 2013-2017+ James Shubin and the project contributors
+// Written by James Shubin <james@shubin.ca> and the project contributors
+//
+// This program is free software: you can redistribute it and/or modify
+// it under the terms of the GNU General Public License as published by
+// the Free Software Foundation, either version 3 of the License, or
+// (at your option) any later version.
+//
+// This program is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+// GNU General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License
+// along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+package resources
+
+import (
+	"testing"
+)
+
+func TestExecSendRecv1(t *testing.T) {
+	r1 := &ExecRes{
+		BaseRes: BaseRes{
+			Name:       "exec1",
+			Kind:       "exec",
+			MetaParams: DefaultMetaParams,
+		},
+		Cmd:   "echo hello world",
+		Shell: "/bin/bash",
+	}
+
+	r1.Setup(nil, r1, r1)
+	if err := r1.Validate(); err != nil {
+		t.Errorf("validate failed with: %v", err)
+	}
+	defer func() {
+		if err := r1.Close(); err != nil {
+			t.Errorf("close failed with: %v", err)
+		}
+	}()
+	if err := r1.Init(); err != nil {
+		t.Errorf("init failed with: %v", err)
+	}
+	// run artificially without the entire engine
+	if _, err := r1.CheckApply(true); err != nil {
+		t.Errorf("checkapply failed with: %v", err)
+	}
+
+	t.Logf("output is: %v", r1.Output)
+	if r1.Output != nil {
+		t.Logf("output is: %v", *r1.Output)
+	}
+	t.Logf("stdout is: %v", r1.Stdout)
+	if r1.Stdout != nil {
+		t.Logf("stdout is: %v", *r1.Stdout)
+	}
+	t.Logf("stderr is: %v", r1.Stderr)
+	if r1.Stderr != nil {
+		t.Logf("stderr is: %v", *r1.Stderr)
+	}
+
+	if r1.Stdout == nil {
+		t.Errorf("stdout is nil")
+	} else {
+		if out := *r1.Stdout; out != "hello world\n" {
+			t.Errorf("got wrong stdout(%d): %s", len(out), out)
+		}
+	}
+}
+
+func TestExecSendRecv2(t *testing.T) {
+	r1 := &ExecRes{
+		BaseRes: BaseRes{
+			Name:       "exec1",
+			Kind:       "exec",
+			MetaParams: DefaultMetaParams,
+		},
+		Cmd:   "echo hello world 1>&2", // to stderr
+		Shell: "/bin/bash",
+	}
+
+	r1.Setup(nil, r1, r1)
+	if err := r1.Validate(); err != nil {
+		t.Errorf("validate failed with: %v", err)
+	}
+	defer func() {
+		if err := r1.Close(); err != nil {
+			t.Errorf("close failed with: %v", err)
+		}
+	}()
+	if err := r1.Init(); err != nil {
+		t.Errorf("init failed with: %v", err)
+	}
+	// run artificially without the entire engine
+	if _, err := r1.CheckApply(true); err != nil {
+		t.Errorf("checkapply failed with: %v", err)
+	}
+
+	t.Logf("output is: %v", r1.Output)
+	if r1.Output != nil {
+		t.Logf("output is: %v", *r1.Output)
+	}
+	t.Logf("stdout is: %v", r1.Stdout)
+	if r1.Stdout != nil {
+		t.Logf("stdout is: %v", *r1.Stdout)
+	}
+	t.Logf("stderr is: %v", r1.Stderr)
+	if r1.Stderr != nil {
+		t.Logf("stderr is: %v", *r1.Stderr)
+	}
+
+	if r1.Stderr == nil {
+		t.Errorf("stderr is nil")
+	} else {
+		if out := *r1.Stderr; out != "hello world\n" {
+			t.Errorf("got wrong stderr(%d): %s", len(out), out)
+		}
+	}
+}
+
+func TestExecSendRecv3(t *testing.T) {
+	r1 := &ExecRes{
+		BaseRes: BaseRes{
+			Name:       "exec1",
+			Kind:       "exec",
+			MetaParams: DefaultMetaParams,
+		},
+		Cmd:   "echo hello world && echo goodbye world 1>&2", // to stdout && stderr
+		Shell: "/bin/bash",
+	}
+
+	r1.Setup(nil, r1, r1)
+	if err := r1.Validate(); err != nil {
+		t.Errorf("validate failed with: %v", err)
+	}
+	defer func() {
+		if err := r1.Close(); err != nil {
+			t.Errorf("close failed with: %v", err)
+		}
+	}()
+	if err := r1.Init(); err != nil {
+		t.Errorf("init failed with: %v", err)
+	}
+	// run artificially without the entire engine
+	if _, err := r1.CheckApply(true); err != nil {
+		t.Errorf("checkapply failed with: %v", err)
+	}
+
+	t.Logf("output is: %v", r1.Output)
+	if r1.Output != nil {
+		t.Logf("output is: %v", *r1.Output)
+	}
+	t.Logf("stdout is: %v", r1.Stdout)
+	if r1.Stdout != nil {
+		t.Logf("stdout is: %v", *r1.Stdout)
+	}
+	t.Logf("stderr is: %v", r1.Stderr)
+	if r1.Stderr != nil {
+		t.Logf("stderr is: %v", *r1.Stderr)
+	}
+
+	if r1.Output == nil {
+		t.Errorf("output is nil")
+	} else {
+		// it looks like bash or golang race to the write, so whichever
+		// order they come out in is ok, as long as they come out whole
+		if out := *r1.Output; out != "hello world\ngoodbye world\n" && out != "goodbye world\nhello world\n" {
+			t.Errorf("got wrong output(%d): %s", len(out), out)
+		}
+	}
+
+	if r1.Stdout == nil {
+		t.Errorf("stdout is nil")
+	} else {
+		if out := *r1.Stdout; out != "hello world\n" {
+			t.Errorf("got wrong stdout(%d): %s", len(out), out)
+		}
+	}
+
+	if r1.Stderr == nil {
+		t.Errorf("stderr is nil")
+	} else {
+		if out := *r1.Stderr; out != "goodbye world\n" {
+			t.Errorf("got wrong stderr(%d): %s", len(out), out)
+		}
+	}
+}
--- a/resources/file.go
+++ b/resources/file.go
@@ -3,16 +3,16 @@
 // Written by James Shubin <james@shubin.ca> and the project contributors
 //
 // This program is free software: you can redistribute it and/or modify
-// it under the terms of the GNU Affero General Public License as published by
+// it under the terms of the GNU General Public License as published by
 // the Free Software Foundation, either version 3 of the License, or
 // (at your option) any later version.
 //
 // This program is distributed in the hope that it will be useful,
 // but WITHOUT ANY WARRANTY; without even the implied warranty of
 // MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-// GNU Affero General Public License for more details.
+// GNU General Public License for more details.
 //
-// You should have received a copy of the GNU Affero General Public License
+// You should have received a copy of the GNU General Public License
 // along with this program.  If not, see <http://www.gnu.org/licenses/>.

 package resources
@@ -20,7 +20,6 @@ package resources
 import (
 	"bytes"
 	"crypto/sha256"
-	"encoding/gob"
 	"encoding/hex"
 	"fmt"
 	"io"
@@ -41,7 +40,7 @@ import (
 )

 func init() {
-	gob.Register(&FileRes{})
+	RegisterResource("file", func() Res { return &FileRes{} })
 }

 // FileRes is a file and directory resource.
@@ -147,7 +146,6 @@ func (obj *FileRes) Init() error {
 	obj.path = obj.GetPath()                     // compute once
 	obj.isDir = strings.HasSuffix(obj.path, "/") // dirs have trailing slashes

-	obj.BaseRes.kind = "File"
 	return obj.BaseRes.Init() // call base init, b/c we're overriding
 }

@@ -198,7 +196,7 @@ func (obj *FileRes) Watch() error {

 	for {
 		if obj.debug {
-			log.Printf("%s[%s]: Watching: %s", obj.Kind(), obj.GetName(), obj.path) // attempting to watch...
+			log.Printf("%s: Watching: %s", obj, obj.path) // attempting to watch...
 		}

 		select {
@@ -207,10 +205,10 @@ func (obj *FileRes) Watch() error {
 				return nil
 			}
 			if err := event.Error; err != nil {
-				return errwrap.Wrapf(err, "unknown %s[%s] watcher error", obj.Kind(), obj.GetName())
+				return errwrap.Wrapf(err, "unknown %s watcher error", obj)
 			}
 			if obj.debug { // don't access event.Body if event.Error isn't nil
-				log.Printf("%s[%s]: Event(%s): %v", obj.Kind(), obj.GetName(), event.Body.Name, event.Body.Op)
+				log.Printf("%s: Event(%s): %v", obj, event.Body.Name, event.Body.Op)
 			}
 			send = true
 			obj.StateOK(false) // dirty
@@ -635,7 +633,7 @@ func (obj *FileRes) syncCheckApply(apply bool, src, dst string) (bool, error) {

 // contentCheckApply performs a CheckApply for the file existence and content.
 func (obj *FileRes) contentCheckApply(apply bool) (checkOK bool, _ error) {
-	log.Printf("%s[%s]: contentCheckApply(%t)", obj.Kind(), obj.GetName(), apply)
+	log.Printf("%s: contentCheckApply(%t)", obj, apply)

 	if obj.State == "absent" {
 		if _, err := os.Stat(obj.path); os.IsNotExist(err) {
@@ -697,7 +695,7 @@ func (obj *FileRes) contentCheckApply(apply bool) (checkOK bool, _ error) {

 // chmodCheckApply performs a CheckApply for the file permissions.
 func (obj *FileRes) chmodCheckApply(apply bool) (checkOK bool, _ error) {
-	log.Printf("%s[%s]: chmodCheckApply(%t)", obj.Kind(), obj.GetName(), apply)
+	log.Printf("%s: chmodCheckApply(%t)", obj, apply)

 	if obj.State == "absent" {
 		// File is absent
@@ -743,7 +741,7 @@ func (obj *FileRes) chmodCheckApply(apply bool) (checkOK bool, _ error) {
 // chownCheckApply performs a CheckApply for the file ownership.
 func (obj *FileRes) chownCheckApply(apply bool) (checkOK bool, _ error) {
 	var expectedUID, expectedGID int
-	log.Printf("%s[%s]: chownCheckApply(%t)", obj.Kind(), obj.GetName(), apply)
+	log.Printf("%s: chownCheckApply(%t)", obj, apply)

 	if obj.State == "absent" {
 		// File is absent or no owner specified
@@ -897,17 +895,18 @@ func (obj *FileResAutoEdges) Test(input []bool) bool {

 // AutoEdges generates a simple linear sequence of each parent directory from
 // the bottom up!
-func (obj *FileRes) AutoEdges() AutoEdge {
-	var data []ResUID                              // store linear result chain here...
-	values := util.PathSplitFullReversed(obj.path) // build it
-	_, values = values[0], values[1:]              // get rid of first value which is me!
+func (obj *FileRes) AutoEdges() (AutoEdge, error) {
+	var data []ResUID // store linear result chain here...
+	// build it, but don't use obj.path because this gets called before Init
+	values := util.PathSplitFullReversed(obj.GetPath())
+	_, values = values[0], values[1:] // get rid of first value which is me!
 	for _, x := range values {
 		var reversed = true // cheat by passing a pointer
 		data = append(data, &FileUID{
 			BaseUID: BaseUID{
-				name:     obj.GetName(),
-				kind:     obj.Kind(),
-				reversed: &reversed,
+				Name:     obj.GetName(),
+				Kind:     obj.GetKind(),
+				Reversed: &reversed,
 			},
 			path: x, // what matters
 		}) // build list
@@ -916,15 +915,15 @@ func (obj *FileRes) AutoEdges() AutoEdge {
 		data:    data,
 		pointer: 0,
 		found:   false,
-	}
+	}, nil
 }

 // UIDs includes all params to make a unique identification of this object.
 // Most resources only return one, although some resources can return multiple.
 func (obj *FileRes) UIDs() []ResUID {
 	x := &FileUID{
-		BaseUID: BaseUID{name: obj.GetName(), kind: obj.Kind()},
-		path:    obj.path,
+		BaseUID: BaseUID{Name: obj.GetName(), Kind: obj.GetKind()},
+		path:    obj.GetPath(), // not obj.path b/c we didn't init yet!
 	}
 	return []ResUID{x}
 }
@@ -941,43 +940,43 @@ func (obj *FileRes) GroupCmp(r Res) bool {
 }

 // Compare two resources and return if they are equivalent.
-func (obj *FileRes) Compare(res Res) bool {
-	switch res.(type) {
-	case *FileRes:
-		res := res.(*FileRes)
-		if !obj.BaseRes.Compare(res) { // call base Compare
-			return false
-		}
-
-		if obj.Name != res.Name {
-			return false
-		}
-		if obj.path != res.path {
-			return false
-		}
-		if (obj.Content == nil) != (res.Content == nil) { // xor
-			return false
-		}
-		if obj.Content != nil && res.Content != nil {
-			if *obj.Content != *res.Content { // compare the strings
-				return false
-			}
-		}
-		if obj.Source != res.Source {
-			return false
-		}
-		if obj.State != res.State {
-			return false
-		}
-		if obj.Recurse != res.Recurse {
-			return false
-		}
-		if obj.Force != res.Force {
-			return false
-		}
-	default:
+func (obj *FileRes) Compare(r Res) bool {
+	// we can only compare FileRes to others of the same resource kind
+	res, ok := r.(*FileRes)
+	if !ok {
 		return false
 	}
+	if !obj.BaseRes.Compare(res) { // call base Compare
+		return false
+	}
+	if obj.Name != res.Name {
+		return false
+	}
+
+	if obj.path != res.path {
+		return false
+	}
+	if (obj.Content == nil) != (res.Content == nil) { // xor
+		return false
+	}
+	if obj.Content != nil && res.Content != nil {
+		if *obj.Content != *res.Content { // compare the strings
+			return false
+		}
+	}
+	if obj.Source != res.Source {
+		return false
+	}
+	if obj.State != res.State {
+		return false
+	}
+	if obj.Recurse != res.Recurse {
+		return false
+	}
+	if obj.Force != res.Force {
+		return false
+	}
+
 	return true
 }

--- a/resources/file_attrs.go
+++ b/resources/file_attrs.go
@@ -3,16 +3,16 @@
 // Written by James Shubin <james@shubin.ca> and the project contributors
 //
 // This program is free software: you can redistribute it and/or modify
-// it under the terms of the GNU Affero General Public License as published by
+// it under the terms of the GNU General Public License as published by
 // the Free Software Foundation, either version 3 of the License, or
 // (at your option) any later version.
 //
 // This program is distributed in the hope that it will be useful,
 // but WITHOUT ANY WARRANTY; without even the implied warranty of
 // MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-// GNU Affero General Public License for more details.
+// GNU General Public License for more details.
 //
-// You should have received a copy of the GNU Affero General Public License
+// You should have received a copy of the GNU General Public License
 // along with this program.  If not, see <http://www.gnu.org/licenses/>.

 // +build go1.7
--- a/resources/file_attrs_go1.6.go
+++ b/resources/file_attrs_go1.6.go
@@ -3,16 +3,16 @@
 // Written by James Shubin <james@shubin.ca> and the project contributors
 //
 // This program is free software: you can redistribute it and/or modify
-// it under the terms of the GNU Affero General Public License as published by
+// it under the terms of the GNU General Public License as published by
 // the Free Software Foundation, either version 3 of the License, or
 // (at your option) any later version.
 //
 // This program is distributed in the hope that it will be useful,
 // but WITHOUT ANY WARRANTY; without even the implied warranty of
 // MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-// GNU Affero General Public License for more details.
+// GNU General Public License for more details.
 //
-// You should have received a copy of the GNU Affero General Public License
+// You should have received a copy of the GNU General Public License
 // along with this program.  If not, see <http://www.gnu.org/licenses/>.

 // +build !go1.7
--- a/resources/file_test.go
+++ b/resources/file_test.go
@@ -0,0 +1,79 @@
+// Mgmt
+// Copyright (C) 2013-2017+ James Shubin and the project contributors
+// Written by James Shubin <james@shubin.ca> and the project contributors
+//
+// This program is free software: you can redistribute it and/or modify
+// it under the terms of the GNU General Public License as published by
+// the Free Software Foundation, either version 3 of the License, or
+// (at your option) any later version.
+//
+// This program is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+// GNU General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License
+// along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+package resources
+
+import (
+	"testing"
+
+	"github.com/purpleidea/mgmt/pgraph"
+)
+
+func TestFileAutoEdge1(t *testing.T) {
+
+	g, err := pgraph.NewGraph("TestGraph")
+	if err != nil {
+		t.Errorf("error creating graph: %v", err)
+		return
+	}
+
+	r1 := &FileRes{
+		BaseRes: BaseRes{
+			Name: "file1",
+			Kind: "file",
+			MetaParams: MetaParams{
+				AutoEdge: true,
+			},
+		},
+		Path: "/tmp/a/b/", // some dir
+	}
+	r2 := &FileRes{
+		BaseRes: BaseRes{
+			Name: "file2",
+			Kind: "file",
+			MetaParams: MetaParams{
+				AutoEdge: true,
+			},
+		},
+		Path: "/tmp/a/", // some parent dir
+	}
+	r3 := &FileRes{
+		BaseRes: BaseRes{
+			Name: "file3",
+			Kind: "file",
+			MetaParams: MetaParams{
+				AutoEdge: true,
+			},
+		},
+		Path: "/tmp/a/b/c", // some child file
+	}
+	g.AddVertex(r1, r2, r3)
+
+	if i := g.NumEdges(); i != 0 {
+		t.Errorf("should have 0 edges instead of: %d", i)
+	}
+
+	// run artificially without the entire engine
+	if err := AutoEdges(g); err != nil {
+		t.Errorf("error running autoedges: %v", err)
+	}
+
+	// two edges should have been added
+	if i := g.NumEdges(); i != 2 {
+		t.Errorf("should have 2 edges instead of: %d", i)
+	}
+}
--- a/resources/graph.go
+++ b/resources/graph.go
@@ -0,0 +1,234 @@
+// Mgmt
+// Copyright (C) 2013-2017+ James Shubin and the project contributors
+// Written by James Shubin <james@shubin.ca> and the project contributors
+//
+// This program is free software: you can redistribute it and/or modify
+// it under the terms of the GNU General Public License as published by
+// the Free Software Foundation, either version 3 of the License, or
+// (at your option) any later version.
+//
+// This program is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+// GNU General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License
+// along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+package resources
+
+import (
+	"fmt"
+
+	"github.com/purpleidea/mgmt/pgraph"
+
+	multierr "github.com/hashicorp/go-multierror"
+	errwrap "github.com/pkg/errors"
+)
+
+func init() {
+	RegisterResource("graph", func() Res { return &GraphRes{} })
+}
+
+// GraphRes is a resource that recursively runs a sub graph of resources.
+// TODO: should we name this SubGraphRes instead?
+// TODO: we could also flatten "sub graphs" into the main graph to avoid this,
+// and this could even be done with a graph transformation called flatten,
+// similar to where autogroup and autoedges run.
+// XXX: this resource is not complete, and hasn't even been tested
+type GraphRes struct {
+	BaseRes `yaml:",inline"`
+	Graph   *pgraph.Graph `yaml:"graph"` // TODO: how do we suck in a graph via yaml?
+
+	initCount int // number of successfully initialized resources
+}
+
+// GraphUID is a unique representation for a GraphRes object.
+type GraphUID struct {
+	BaseUID
+	//foo string // XXX: not implemented
+}
+
+// Default returns some sensible defaults for this resource.
+func (obj *GraphRes) Default() Res {
+	return &GraphRes{
+		BaseRes: BaseRes{
+			MetaParams: DefaultMetaParams, // force a default
+		},
+	}
+}
+
+// Validate the params and sub resources that are passed to GraphRes.
+func (obj *GraphRes) Validate() error {
+	var err error
+	for _, v := range obj.Graph.VerticesSorted() { // validate everyone
+		if e := VtoR(v).Validate(); err != nil {
+			err = multierr.Append(err, e) // list of errors
+		}
+	}
+	if err != nil {
+		return errwrap.Wrapf(err, "could not Validate() graph")
+	}
+
+	return obj.BaseRes.Validate()
+}
+
+// Init runs some startup code for this resource.
+func (obj *GraphRes) Init() error {
+	// Loop through each vertex and initialize it, but keep track of how far
+	// we've succeeded, because on failure we'll stop and prepare to reverse
+	// through from there running the Close operation on each vertex that we
+	// previously did an Init on. The engine always ensures that we run this
+	// with a 1-1 relationship between Init and Close, so we must do so too.
+	for i, v := range obj.Graph.VerticesSorted() { // deterministic order!
+		obj.initCount = i + 1 // store the number that we tried to init
+		if err := VtoR(v).Init(); err != nil {
+			return errwrap.Wrapf(err, "could not Init() graph")
+		}
+	}
+
+	return obj.BaseRes.Init() // call base init, b/c we're overrriding
+}
+
+// Close runs some cleanup code for this resource.
+func (obj *GraphRes) Close() error {
+	// The idea is to Close anything we did an Init on including the BaseRes
+	// methods which are not guaranteed to be safe if called multiple times!
+	var err error
+	vertices := obj.Graph.VerticesSorted() // deterministic order!
+	last := obj.initCount - 1              // index of last vertex we did init on
+	for i := range vertices {
+		v := vertices[last-i] // go through in reverse
+
+		// if we hit this condition, we haven't been able to get through
+		// the entire list of vertices that we'd have liked to, on init!
+		if obj.initCount == 0 {
+			// if we get here, we exit without calling BaseRes.Close
+			// because the matching BaseRes.Init did not get called!
+			return errwrap.Wrapf(err, "could not Close() partial graph")
+			//break
+		}
+
+		obj.initCount-- // count to avoid closing one that didn't init!
+		// try to close everyone that got an init, don't stop suddenly!
+		if e := VtoR(v).Close(); e != nil {
+			err = multierr.Append(err, e) // list of errors
+		}
+	}
+
+	// call base close, b/c we're overriding
+	if e := obj.BaseRes.Close(); err == nil {
+		err = e
+	} else if e != nil {
+		err = multierr.Append(err, e) // list of errors
+	}
+	// this returns nil if err is nil
+	return errwrap.Wrapf(err, "could not Close() graph")
+}
+
+// Watch is the primary listener for this resource and it outputs events.
+// XXX: should this use mgraph.Start/Pause? if so then what does CheckApply do?
+// XXX: we should probably refactor the core engine to make this work, which
+// will hopefully lead us to a more elegant core that is easier to understand
+func (obj *GraphRes) Watch() error {
+
+	return fmt.Errorf("Not implemented")
+}
+
+// CheckApply method for Graph resource.
+// XXX: not implemented
+func (obj *GraphRes) CheckApply(apply bool) (bool, error) {
+
+	return false, fmt.Errorf("Not implemented")
+
+}
+
+// UIDs includes all params to make a unique identification of this object.
+// Most resources only return one, although some resources can return multiple.
+func (obj *GraphRes) UIDs() []ResUID {
+	x := &GraphUID{
+		BaseUID: BaseUID{
+			Name: obj.GetName(),
+			Kind: obj.GetKind(),
+		},
+		//foo: obj.foo, // XXX: not implemented
+	}
+	uids := []ResUID{}
+	for _, v := range obj.Graph.VerticesSorted() {
+		uids = append(uids, VtoR(v).UIDs()...)
+	}
+	return append([]ResUID{x}, uids...)
+}
+
+// XXX: hook up the autogrouping magic!
+
+// Compare two resources and return if they are equivalent.
+func (obj *GraphRes) Compare(r Res) bool {
+	// we can only compare GraphRes to others of the same resource kind
+	res, ok := r.(*GraphRes)
+	if !ok {
+		return false
+	}
+	if !obj.BaseRes.Compare(res) {
+		return false
+	}
+	if obj.Name != res.Name {
+		return false
+	}
+
+	//if obj.Foo != res.Foo { // XXX: not implemented
+	//	return false
+	//}
+	// compare the structure of the two graphs...
+	vertexCmpFn := func(v1, v2 pgraph.Vertex) (bool, error) {
+		if v1.String() == "" || v2.String() == "" {
+			return false, fmt.Errorf("oops, empty vertex")
+		}
+		return VtoR(v1).Compare(VtoR(v2)), nil
+	}
+
+	edgeCmpFn := func(e1, e2 pgraph.Edge) (bool, error) {
+		if e1.String() == "" || e2.String() == "" {
+			return false, fmt.Errorf("oops, empty edge")
+		}
+		edge1 := e1.(*Edge) // panic if wrong
+		edge2 := e2.(*Edge) // panic if wrong
+		return edge1.Compare(edge2), nil
+	}
+	if err := obj.Graph.GraphCmp(res.Graph, vertexCmpFn, edgeCmpFn); err != nil {
+		return false
+	}
+
+	// compare individual elements in structurally equivalent graphs
+	// TODO: is this redundant with the GraphCmp?
+	g1 := obj.Graph.VerticesSorted()
+	g2 := res.Graph.VerticesSorted()
+	for i, v1 := range g1 {
+		v2 := g2[i]
+		if !VtoR(v1).Compare(VtoR(v2)) {
+			return false
+		}
+	}
+
+	return true
+}
+
+// UnmarshalYAML is the custom unmarshal handler for this struct.
+// It is primarily useful for setting the defaults.
+func (obj *GraphRes) UnmarshalYAML(unmarshal func(interface{}) error) error {
+	type rawRes GraphRes // indirection to avoid infinite recursion
+
+	def := obj.Default()       // get the default
+	res, ok := def.(*GraphRes) // put in the right format
+	if !ok {
+		return fmt.Errorf("could not convert to GraphRes")
+	}
+	raw := rawRes(*res) // convert; the defaults go here
+
+	if err := unmarshal(&raw); err != nil {
+		return err
+	}
+
+	*obj = GraphRes(raw) // restore from indirection with type conversion!
+	return nil
+}
--- a/resources/group.go
+++ b/resources/group.go
@@ -0,0 +1,314 @@
+// Mgmt
+// Copyright (C) 2013-2017+ James Shubin and the project contributors
+// Written by James Shubin <james@shubin.ca> and the project contributors
+//
+// This program is free software: you can redistribute it and/or modify
+// it under the terms of the GNU General Public License as published by
+// the Free Software Foundation, either version 3 of the License, or
+// (at your option) any later version.
+//
+// This program is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+// GNU General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License
+// along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+package resources
+
+import (
+	"fmt"
+	"io/ioutil"
+	"log"
+	"os/exec"
+	"os/user"
+	"strconv"
+	"syscall"
+
+	"github.com/purpleidea/mgmt/recwatch"
+
+	errwrap "github.com/pkg/errors"
+)
+
+func init() {
+	RegisterResource("group", func() Res { return &GroupRes{} })
+}
+
+const groupFile = "/etc/group"
+
+// GroupRes is a user group resource.
+type GroupRes struct {
+	BaseRes `yaml:",inline"`
+	State   string  `yaml:"state"` // state: exists, absent
+	GID     *uint32 `yaml:"gid"`   // the group's gid
+
+	recWatcher *recwatch.RecWatcher
+}
+
+// Default returns some sensible defaults for this resource.
+func (obj *GroupRes) Default() Res {
+	return &GroupRes{
+		BaseRes: BaseRes{
+			MetaParams: DefaultMetaParams, // force a default
+		},
+	}
+}
+
+// Validate if the params passed in are valid data.
+func (obj *GroupRes) Validate() error {
+	if obj.State != "exists" && obj.State != "absent" {
+		return fmt.Errorf("State must be 'exists' or 'absent'")
+	}
+	return obj.BaseRes.Validate()
+}
+
+// Init initializes the resource.
+func (obj *GroupRes) Init() error {
+	return obj.BaseRes.Init() // call base init, b/c we're overriding
+}
+
+// Watch is the primary listener for this resource and it outputs events.
+func (obj *GroupRes) Watch() error {
+	var err error
+	obj.recWatcher, err = recwatch.NewRecWatcher(groupFile, false)
+	if err != nil {
+		return err
+	}
+	defer obj.recWatcher.Close()
+
+	// notify engine that we're running
+	if err := obj.Running(); err != nil {
+		return err // bubble up a NACK...
+	}
+
+	var send = false // send event?
+	var exit *error
+
+	for {
+		if obj.debug {
+			log.Printf("%s: Watching: %s", obj, groupFile) // attempting to watch...
+		}
+
+		select {
+		case event, ok := <-obj.recWatcher.Events():
+			if !ok { // channel shutdown
+				return nil
+			}
+			if err := event.Error; err != nil {
+				return errwrap.Wrapf(err, "Unknown %s watcher error", obj)
+			}
+			if obj.debug { // don't access event.Body if event.Error isn't nil
+				log.Printf("%s: Event(%s): %v", obj, event.Body.Name, event.Body.Op)
+			}
+			send = true
+			obj.StateOK(false) // dirty
+
+		case event := <-obj.Events():
+			if exit, send = obj.ReadEvent(event); exit != nil {
+				return *exit // exit
+			}
+			//obj.StateOK(false) // dirty // these events don't invalidate state
+		}
+
+		// do all our event sending all together to avoid duplicate msgs
+		if send {
+			send = false
+			obj.Event()
+		}
+	}
+}
+
+// CheckApply method for Group resource.
+func (obj *GroupRes) CheckApply(apply bool) (checkOK bool, err error) {
+	log.Printf("%s: CheckApply(%t)", obj, apply)
+
+	// check if the group exists
+	exists := true
+	group, err := user.LookupGroup(obj.GetName())
+	if err != nil {
+		if _, ok := err.(user.UnknownGroupError); !ok {
+			return false, errwrap.Wrapf(err, "error looking up group")
+		}
+		exists = false
+	}
+	// if the group doesn't exist and should be absent, we are done
+	if obj.State == "absent" && !exists {
+		return true, nil
+	}
+	// if the group exists and no GID is specified, we are done
+	if obj.State == "exists" && exists && obj.GID == nil {
+		return true, nil
+	}
+	if exists && obj.GID != nil {
+		// check if GID is taken
+		lookupGID, err := user.LookupGroupId(strconv.Itoa(int(*obj.GID)))
+		if err != nil {
+			if _, ok := err.(user.UnknownGroupIdError); !ok {
+				return false, errwrap.Wrapf(err, "error looking up GID")
+			}
+		}
+		if lookupGID != nil && lookupGID.Name != obj.GetName() {
+			return false, fmt.Errorf("the requested GID belongs to another group")
+		}
+		// get the existing group's GID
+		existingGID, err := strconv.ParseUint(group.Gid, 10, 32)
+		if err != nil {
+			return false, errwrap.Wrapf(err, "error casting existing GID")
+		}
+		// check if existing group has the wrong GID
+		// if it is wrong groupmod will change it to the desired value
+		if *obj.GID != uint32(existingGID) {
+			log.Printf("%s: Inconsistent GID: %s", obj, obj.GetName())
+		}
+		// if the group exists and has the correct GID, we are done
+		if obj.State == "exists" && *obj.GID == uint32(existingGID) {
+			return true, nil
+		}
+	}
+
+	if !apply {
+		return false, nil
+	}
+
+	var cmdName string
+	args := []string{obj.GetName()}
+
+	if obj.State == "exists" {
+		if exists {
+			log.Printf("%s: Modifying group: %s", obj, obj.GetName())
+			cmdName = "groupmod"
+		} else {
+			log.Printf("%s: Adding group: %s", obj, obj.GetName())
+			cmdName = "groupadd"
+		}
+		if obj.GID != nil {
+			args = append(args, "-g", fmt.Sprintf("%d", *obj.GID))
+		}
+	}
+	if obj.State == "absent" && exists {
+		log.Printf("%s: Deleting group: %s", obj, obj.GetName())
+		cmdName = "groupdel"
+	}
+
+	cmd := exec.Command(cmdName, args...)
+	cmd.SysProcAttr = &syscall.SysProcAttr{
+		Setpgid: true,
+		Pgid:    0,
+	}
+
+	// open a pipe to get error messages from os/exec
+	stderr, err := cmd.StderrPipe()
+	if err != nil {
+		return false, errwrap.Wrapf(err, "failed to initialize stderr pipe")
+	}
+
+	// start the command
+	if err := cmd.Start(); err != nil {
+		return false, errwrap.Wrapf(err, "cmd failed to start")
+	}
+	// capture any error messages
+	slurp, err := ioutil.ReadAll(stderr)
+	if err != nil {
+		return false, errwrap.Wrapf(err, "error slurping error message")
+	}
+	// wait until cmd exits and return error message if any
+	if err := cmd.Wait(); err != nil {
+		return false, errwrap.Wrapf(err, "%s", slurp)
+	}
+
+	return false, nil
+}
+
+// GroupUID is the UID struct for GroupRes.
+type GroupUID struct {
+	BaseUID
+	name string
+	gid  *uint32
+}
+
+// IFF aka if and only if they are equivalent, return true. If not, false.
+func (obj *GroupUID) IFF(uid ResUID) bool {
+	res, ok := uid.(*GroupUID)
+	if !ok {
+		return false
+	}
+	if obj.gid != nil && res.gid != nil {
+		if *obj.gid != *res.gid {
+			return false
+		}
+	}
+	if obj.name != "" && res.name != "" {
+		if obj.name != res.name {
+			return false
+		}
+	}
+	return true
+}
+
+// UIDs includes all params to make a unique identification of this object.
+// Most resources only return one, although some resources can return multiple.
+func (obj *GroupRes) UIDs() []ResUID {
+	x := &GroupUID{
+		BaseUID: BaseUID{Name: obj.GetName(), Kind: obj.GetKind()},
+		name:    obj.Name,
+		gid:     obj.GID,
+	}
+	return []ResUID{x}
+}
+
+// GroupCmp returns whether two resources can be grouped together or not.
+func (obj *GroupRes) GroupCmp(r Res) bool {
+	_, ok := r.(*GroupRes)
+	if !ok {
+		return false
+	}
+	return false
+}
+
+// Compare two resources and return if they are equivalent.
+func (obj *GroupRes) Compare(r Res) bool {
+	// we can only compare GroupRes to others of the same resource kind
+	res, ok := r.(*GroupRes)
+	if !ok {
+		return false
+	}
+	if !obj.BaseRes.Compare(res) { // call base Compare
+		return false
+	}
+	if obj.Name != res.Name {
+		return false
+	}
+	if obj.State != res.State {
+		return false
+	}
+	if (obj.GID == nil) != (res.GID == nil) {
+		return false
+	}
+	if obj.GID != nil && res.GID != nil {
+		if *obj.GID != *res.GID {
+			return false
+		}
+	}
+	return true
+}
+
+// UnmarshalYAML is the custom unmarshal handler for this struct.
+// It is primarily useful for setting the defaults.
+func (obj *GroupRes) UnmarshalYAML(unmarshal func(interface{}) error) error {
+	type rawRes GroupRes // indirection to avoid infinite recursion
+
+	def := obj.Default()       // get the default
+	res, ok := def.(*GroupRes) // put in the right format
+	if !ok {
+		return fmt.Errorf("could not convert to GroupRes")
+	}
+	raw := rawRes(*res) // convert; the defaults go here
+
+	if err := unmarshal(&raw); err != nil {
+		return err
+	}
+
+	*obj = GroupRes(raw) // restore from indirection with type conversion!
+	return nil
+}
--- a/resources/hostname.go
+++ b/resources/hostname.go
@@ -3,22 +3,21 @@
 // Written by James Shubin <james@shubin.ca> and the project contributors
 //
 // This program is free software: you can redistribute it and/or modify
-// it under the terms of the GNU Affero General Public License as published by
+// it under the terms of the GNU General Public License as published by
 // the Free Software Foundation, either version 3 of the License, or
 // (at your option) any later version.
 //
 // This program is distributed in the hope that it will be useful,
 // but WITHOUT ANY WARRANTY; without even the implied warranty of
 // MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-// GNU Affero General Public License for more details.
+// GNU General Public License for more details.
 //
-// You should have received a copy of the GNU Affero General Public License
+// You should have received a copy of the GNU General Public License
 // along with this program.  If not, see <http://www.gnu.org/licenses/>.

 package resources

 import (
-	"encoding/gob"
 	"errors"
 	"fmt"
 	"log"
@@ -35,7 +34,7 @@ var ErrResourceInsufficientParameters = errors.New(
 	"Insufficient parameters for this resource")

 func init() {
-	gob.Register(&HostnameRes{})
+	RegisterResource("hostname", func() Res { return &HostnameRes{} })
 }

 const (
@@ -87,7 +86,6 @@ func (obj *HostnameRes) Validate() error {

 // Init runs some startup code for this resource.
 func (obj *HostnameRes) Init() error {
-	obj.BaseRes.kind = "Hostname"
 	if obj.PrettyHostname == "" {
 		obj.PrettyHostname = obj.Hostname
 	}
@@ -227,16 +225,11 @@ type HostnameUID struct {
 	transientHostname string
 }

-// AutoEdges returns the AutoEdge interface. In this case no autoedges are used.
-func (obj *HostnameRes) AutoEdges() AutoEdge {
-	return nil
-}
-
 // UIDs includes all params to make a unique identification of this object.
 // Most resources only return one, although some resources can return multiple.
 func (obj *HostnameRes) UIDs() []ResUID {
 	x := &HostnameUID{
-		BaseUID:           BaseUID{name: obj.GetName(), kind: obj.Kind()},
+		BaseUID:           BaseUID{Name: obj.GetName(), Kind: obj.GetKind()},
 		name:              obj.Name,
 		prettyHostname:    obj.PrettyHostname,
 		staticHostname:    obj.StaticHostname,
@@ -251,28 +244,29 @@ func (obj *HostnameRes) GroupCmp(r Res) bool {
 }

 // Compare two resources and return if they are equivalent.
-func (obj *HostnameRes) Compare(res Res) bool {
-	switch res := res.(type) {
-	// we can only compare HostnameRes to others of the same resource
-	case *HostnameRes:
-		if !obj.BaseRes.Compare(res) { // call base Compare
-			return false
-		}
-		if obj.Name != res.Name {
-			return false
-		}
-		if obj.PrettyHostname != res.PrettyHostname {
-			return false
-		}
-		if obj.StaticHostname != res.StaticHostname {
-			return false
-		}
-		if obj.TransientHostname != res.TransientHostname {
-			return false
-		}
-	default:
+func (obj *HostnameRes) Compare(r Res) bool {
+	// we can only compare HostnameRes to others of the same resource kind
+	res, ok := r.(*HostnameRes)
+	if !ok {
 		return false
 	}
+	if !obj.BaseRes.Compare(res) { // call base Compare
+		return false
+	}
+	if obj.Name != res.Name {
+		return false
+	}
+
+	if obj.PrettyHostname != res.PrettyHostname {
+		return false
+	}
+	if obj.StaticHostname != res.StaticHostname {
+		return false
+	}
+	if obj.TransientHostname != res.TransientHostname {
+		return false
+	}
+
 	return true
 }

--- a/resources/kv.go
+++ b/resources/kv.go
@@ -0,0 +1,296 @@
+// Mgmt
+// Copyright (C) 2013-2017+ James Shubin and the project contributors
+// Written by James Shubin <james@shubin.ca> and the project contributors
+//
+// This program is free software: you can redistribute it and/or modify
+// it under the terms of the GNU General Public License as published by
+// the Free Software Foundation, either version 3 of the License, or
+// (at your option) any later version.
+//
+// This program is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+// GNU General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License
+// along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+package resources
+
+import (
+	"fmt"
+	"log"
+	"strconv"
+
+	errwrap "github.com/pkg/errors"
+)
+
+func init() {
+	RegisterResource("kv", func() Res { return &KVRes{} })
+}
+
+// KVResSkipCmpStyle represents the different styles of comparison when using SkipLessThan.
+type KVResSkipCmpStyle int
+
+// These are the different allowed comparison styles. Most folks will want SkipCmpStyleInt.
+const (
+	SkipCmpStyleInt KVResSkipCmpStyle = iota
+	SkipCmpStyleString
+)
+
+// KVRes is a resource which writes a key/value pair into cluster wide storage.
+// It will ensure that the key is set to the requested value. The one exception
+// is that if you use the SkipLessThan parameter, then it will only replace the
+// stored value with the requested value if it is greater than that stored one.
+// This allows the KV resource to be used in fast acting, finite state machines
+// which have monotonically increasing state values that represent progression.
+// The one exception is that when this resource receives a refresh signal, then
+// it will set the value to be the exact one if they are not identical already.
+type KVRes struct {
+	BaseRes      `yaml:",inline"`
+	Key          string            `yaml:"key"`          // key to set
+	Value        *string           `yaml:"value"`        // value to set (nil to delete)
+	SkipLessThan bool              `yaml:"skiplessthan"` // skip updates as long as stored value is greater
+	SkipCmpStyle KVResSkipCmpStyle `yaml:"skipcmpstyle"` // how to do the less than cmp
+	// TODO: does it make sense to have different backends here? (eg: local)
+}
+
+// Default returns some sensible defaults for this resource.
+func (obj *KVRes) Default() Res {
+	return &KVRes{
+		BaseRes: BaseRes{
+			MetaParams: DefaultMetaParams, // force a default
+		},
+	}
+}
+
+// Validate if the params passed in are valid data.
+// FIXME: This will catch most issues unless data is passed in after Init with
+// the Send/Recv mechanism. Should the engine re-call Validate after Send/Recv?
+func (obj *KVRes) Validate() error {
+	if obj.Key == "" {
+		return fmt.Errorf("key must not be empty")
+	}
+	if obj.SkipLessThan {
+		if obj.SkipCmpStyle != SkipCmpStyleInt && obj.SkipCmpStyle != SkipCmpStyleString {
+			return fmt.Errorf("the SkipCmpStyle of %v is invalid", obj.SkipCmpStyle)
+		}
+
+		if v := obj.Value; obj.SkipCmpStyle == SkipCmpStyleInt && v != nil {
+			if _, err := strconv.Atoi(*v); err != nil {
+				return fmt.Errorf("the set value of %v can't convert to int", v)
+			}
+		}
+	}
+	return obj.BaseRes.Validate()
+}
+
+// Init initializes the resource.
+func (obj *KVRes) Init() error {
+	return obj.BaseRes.Init() // call base init, b/c we're overriding
+}
+
+// Watch is the primary listener for this resource and it outputs events.
+func (obj *KVRes) Watch() error {
+
+	// notify engine that we're running
+	if err := obj.Running(); err != nil {
+		return err // bubble up a NACK...
+	}
+
+	ch := obj.Data().World.StrMapWatch(obj.Key) // get possible events!
+
+	var send = false // send event?
+	var exit *error
+	for {
+		select {
+		// NOTE: this part is very similar to the file resource code
+		case err, ok := <-ch:
+			if !ok { // channel shutdown
+				return nil
+			}
+			if err != nil {
+				return errwrap.Wrapf(err, "unknown %s watcher error", obj)
+			}
+			if obj.Data().Debug {
+				log.Printf("%s: Event!", obj)
+			}
+			send = true
+			obj.StateOK(false) // dirty
+
+		case event := <-obj.Events():
+			// we avoid sending events on unpause
+			if exit, send = obj.ReadEvent(event); exit != nil {
+				return *exit // exit
+			}
+		}
+
+		// do all our event sending all together to avoid duplicate msgs
+		if send {
+			send = false
+			obj.Event()
+		}
+	}
+}
+
+// lessThanCheck checks for less than validity.
+func (obj *KVRes) lessThanCheck(value string) (checkOK bool, err error) {
+
+	v := *obj.Value
+	if value == v { // redundant check for safety
+		return true, nil
+	}
+
+	var refresh = obj.Refresh()       // do we have a pending reload to apply?
+	if !obj.SkipLessThan || refresh { // update lessthan on refresh
+		return false, nil
+	}
+
+	switch obj.SkipCmpStyle {
+	case SkipCmpStyleInt:
+		intValue, err := strconv.Atoi(value)
+		if err != nil {
+			// NOTE: We don't error here since we're going to write
+			// over the value anyways. It could be from an old run!
+			return false, nil // value is bad (old/corrupt), fix it
+		}
+		if vint, err := strconv.Atoi(v); err != nil {
+			return false, errwrap.Wrapf(err, "can't convert %v to int", v)
+		} else if vint < intValue {
+			return true, nil
+		}
+
+	case SkipCmpStyleString:
+		if v < value { // weird way to cmp, but valid
+			return true, nil
+		}
+
+	default:
+		return false, fmt.Errorf("unmatches SkipCmpStyle style %v", obj.SkipCmpStyle)
+	}
+
+	return false, nil
+}
+
+// CheckApply method for Password resource. Does nothing, returns happy!
+func (obj *KVRes) CheckApply(apply bool) (checkOK bool, err error) {
+	log.Printf("%s: CheckApply(%t)", obj, apply)
+
+	if val, exists := obj.Recv["Value"]; exists && val.Changed {
+		// if we received on Value, and it changed, wooo, nothing to do.
+		log.Printf("CheckApply: `Value` was updated!")
+	}
+
+	hostname := obj.Data().Hostname // me
+	keyMap, err := obj.Data().World.StrMapGet(obj.Key)
+	if err != nil {
+		return false, errwrap.Wrapf(err, "check error during StrGet")
+	}
+
+	if value, ok := keyMap[hostname]; ok && obj.Value != nil {
+		if value == *obj.Value {
+			return true, nil
+		}
+
+		if c, err := obj.lessThanCheck(value); err != nil {
+			return false, err
+		} else if c {
+			return true, nil
+		}
+
+	} else if !ok && obj.Value == nil {
+		return true, nil // nothing to delete, we're good!
+
+	} else if ok && obj.Value == nil { // delete
+		err := obj.Data().World.StrMapDel(obj.Key)
+		return false, errwrap.Wrapf(err, "apply error during StrDel")
+	}
+
+	if !apply {
+		return false, nil
+	}
+
+	if err := obj.Data().World.StrMapSet(obj.Key, *obj.Value); err != nil {
+		return false, errwrap.Wrapf(err, "apply error during StrSet")
+	}
+
+	return false, nil
+}
+
+// KVUID is the UID struct for KVRes.
+type KVUID struct {
+	BaseUID
+	name string
+}
+
+// UIDs includes all params to make a unique identification of this object.
+// Most resources only return one, although some resources can return multiple.
+func (obj *KVRes) UIDs() []ResUID {
+	x := &KVUID{
+		BaseUID: BaseUID{Name: obj.GetName(), Kind: obj.GetKind()},
+		name:    obj.Name,
+	}
+	return []ResUID{x}
+}
+
+// GroupCmp returns whether two resources can be grouped together or not.
+func (obj *KVRes) GroupCmp(r Res) bool {
+	_, ok := r.(*KVRes)
+	if !ok {
+		return false
+	}
+	return false // TODO: this is doable!
+	// TODO: it could be useful to group our writes and watches!
+}
+
+// Compare two resources and return if they are equivalent.
+func (obj *KVRes) Compare(r Res) bool {
+	// we can only compare KVRes to others of the same resource kind
+	res, ok := r.(*KVRes)
+	if !ok {
+		return false
+	}
+	if !obj.BaseRes.Compare(res) { // call base Compare
+		return false
+	}
+
+	if obj.Key != res.Key {
+		return false
+	}
+	if (obj.Value == nil) != (res.Value == nil) { // xor
+		return false
+	}
+	if obj.Value != nil && res.Value != nil {
+		if *obj.Value != *res.Value { // compare the strings
+			return false
+		}
+	}
+	if obj.SkipLessThan != res.SkipLessThan {
+		return false
+	}
+	if obj.SkipCmpStyle != res.SkipCmpStyle {
+		return false
+	}
+
+	return true
+}
+
+// UnmarshalYAML is the custom unmarshal handler for this struct.
+// It is primarily useful for setting the defaults.
+func (obj *KVRes) UnmarshalYAML(unmarshal func(interface{}) error) error {
+	type rawRes KVRes // indirection to avoid infinite recursion
+
+	def := obj.Default()    // get the default
+	res, ok := def.(*KVRes) // put in the right format
+	if !ok {
+		return fmt.Errorf("could not convert to KVRes")
+	}
+	raw := rawRes(*res) // convert; the defaults go here
+
+	if err := unmarshal(&raw); err != nil {
+		return err
+	}
+
+	*obj = KVRes(raw) // restore from indirection with type conversion!
+	return nil
+}
--- a/resources/metaparams.go
+++ b/resources/metaparams.go
@@ -0,0 +1,65 @@
+// Mgmt
+// Copyright (C) 2013-2017+ James Shubin and the project contributors
+// Written by James Shubin <james@shubin.ca> and the project contributors
+//
+// This program is free software: you can redistribute it and/or modify
+// it under the terms of the GNU General Public License as published by
+// the Free Software Foundation, either version 3 of the License, or
+// (at your option) any later version.
+//
+// This program is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+// GNU General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License
+// along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+package resources
+
+import (
+	"golang.org/x/time/rate"
+)
+
+// MetaParams is a struct will all params that apply to every resource.
+type MetaParams struct {
+	AutoEdge  bool `yaml:"autoedge"`  // metaparam, should we generate auto edges?
+	AutoGroup bool `yaml:"autogroup"` // metaparam, should we auto group?
+	Noop      bool `yaml:"noop"`
+	// NOTE: there are separate Watch and CheckApply retry and delay values,
+	// but I've decided to use the same ones for both until there's a proper
+	// reason to want to do something differently for the Watch errors.
+	Retry int16      `yaml:"retry"` // metaparam, number of times to retry on error. -1 for infinite
+	Delay uint64     `yaml:"delay"` // metaparam, number of milliseconds to wait between retries
+	Poll  uint32     `yaml:"poll"`  // metaparam, number of seconds between poll intervals, 0 to watch
+	Limit rate.Limit `yaml:"limit"` // metaparam, number of events per second to allow through
+	Burst int        `yaml:"burst"` // metaparam, number of events to allow in a burst
+	Sema  []string   `yaml:"sema"`  // metaparam, list of semaphore ids (id | id:count)
+}
+
+// UnmarshalYAML is the custom unmarshal handler for the MetaParams struct. It
+// is primarily useful for setting the defaults.
+func (obj *MetaParams) UnmarshalYAML(unmarshal func(interface{}) error) error {
+	type rawMetaParams MetaParams           // indirection to avoid infinite recursion
+	raw := rawMetaParams(DefaultMetaParams) // convert; the defaults go here
+
+	if err := unmarshal(&raw); err != nil {
+		return err
+	}
+
+	*obj = MetaParams(raw) // restore from indirection with type conversion!
+	return nil
+}
+
+// DefaultMetaParams are the defaults to be used for undefined metaparams.
+var DefaultMetaParams = MetaParams{
+	AutoEdge:  true,
+	AutoGroup: true,
+	Noop:      false,
+	Retry:     0,        // TODO: is this a good default?
+	Delay:     0,        // TODO: is this a good default?
+	Poll:      0,        // defaults to watching for events
+	Limit:     rate.Inf, // defaults to no limit
+	Burst:     0,        // no burst needed on an infinite rate // TODO: is this a good default?
+	//Sema:      []string{},
+}
--- a/resources/mgraph.go
+++ b/resources/mgraph.go
@@ -0,0 +1,215 @@
+// Mgmt
+// Copyright (C) 2013-2017+ James Shubin and the project contributors
+// Written by James Shubin <james@shubin.ca> and the project contributors
+//
+// This program is free software: you can redistribute it and/or modify
+// it under the terms of the GNU General Public License as published by
+// the Free Software Foundation, either version 3 of the License, or
+// (at your option) any later version.
+//
+// This program is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+// GNU General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License
+// along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+package resources // TODO: can this be a separate package or will it break the dag?
+
+import (
+	"log"
+	"sync"
+
+	"github.com/purpleidea/mgmt/event"
+	"github.com/purpleidea/mgmt/pgraph"
+	"github.com/purpleidea/mgmt/util/semaphore"
+)
+
+//go:generate stringer -type=graphState -output=graphstate_stringer.go
+type graphState uint
+
+const (
+	graphStateNil graphState = iota
+	graphStateStarting
+	graphStateStarted
+	graphStatePausing
+	graphStatePaused
+)
+
+// MGraph is a meta graph structure used to encapsulate a generic graph
+// structure alongside some non-generic elements.
+type MGraph struct {
+	//Graph *pgraph.Graph
+	*pgraph.Graph // wrap a graph, and use its methods directly
+
+	Data      *ResData
+	FastPause bool
+	Debug     bool
+
+	state graphState
+	// ptr b/c: Mutex/WaitGroup must not be copied after first use
+	mutex *sync.Mutex
+	wg    *sync.WaitGroup
+	slock *sync.Mutex
+	semas map[string]*semaphore.Semaphore
+}
+
+// Init initializes the internal structures.
+func (obj *MGraph) Init() {
+	obj.mutex = &sync.Mutex{}
+	obj.wg = &sync.WaitGroup{}
+	obj.slock = &sync.Mutex{} // semaphore lock
+	obj.semas = make(map[string]*semaphore.Semaphore)
+}
+
+// getState returns the state of the graph. This state is used for optimizing
+// certain algorithms by knowing what part of processing the graph is currently
+// undergoing.
+func (obj *MGraph) getState() graphState {
+	//obj.mutex.Lock()
+	//defer obj.mutex.Unlock()
+	return obj.state
+}
+
+// setState sets the graph state and returns the previous state.
+func (obj *MGraph) setState(state graphState) graphState {
+	obj.mutex.Lock()
+	defer obj.mutex.Unlock()
+	prev := obj.getState()
+	obj.state = state
+	return prev
+}
+
+// Update switches our graph structure to the new graph that we pass to it. This
+// also updates any references to the old graph so that they're now correct. It
+// also updates references to the Data structure that should be passed around.
+func (obj *MGraph) Update(newGraph *pgraph.Graph) {
+	obj.Graph = newGraph.Copy() // store as new active graph
+	// update stored reference to graph and other values that need storing!
+	for _, v := range obj.Graph.Vertices() {
+		res := VtoR(v)          // resource
+		*res.Data() = *obj.Data // push the data around
+	}
+}
+
+// Start is a main kick to start the graph. It goes through in reverse
+// topological sort order so that events can't hit un-started vertices.
+func (obj *MGraph) Start(first bool) { // start or continue
+	log.Printf("State: %v -> %v", obj.setState(graphStateStarting), obj.getState())
+	defer log.Printf("State: %v -> %v", obj.setState(graphStateStarted), obj.getState())
+	t, _ := obj.Graph.TopologicalSort()
+	indegree := obj.Graph.InDegree() // compute all of the indegree's
+	reversed := pgraph.Reverse(t)
+	wg := &sync.WaitGroup{}
+	for _, v := range reversed { // run the Setup() for everyone first
+		// run these in parallel, as long as we wait before continuing
+		wg.Add(1)
+		go func(vertex pgraph.Vertex, res Res) {
+			defer wg.Done()
+			// TODO: can't we do this check outside of the goroutine?
+			if !*res.Working() { // if Worker() is not running...
+				// NOTE: vertex == res here, but pass in both in
+				// case we ever wrap the res in something before
+				// we store it as the vertex in the graph struct
+				res.Setup(obj, vertex, res) // initialize some vars in the resource
+			}
+		}(v, VtoR(v))
+	}
+	wg.Wait()
+
+	// run through the topological reverse, and start or unpause each vertex
+	for _, v := range reversed {
+		res := VtoR(v)
+		// selective poke: here we reduce the number of initial pokes
+		// to the minimum required to activate every vertex in the
+		// graph, either by direct action, or by getting poked by a
+		// vertex that was previously activated. if we poke each vertex
+		// that has no incoming edges, then we can be sure to reach the
+		// whole graph. Please note: this may mask certain optimization
+		// failures, such as any poke limiting code in Poke() or
+		// BackPoke(). You might want to disable this selective start
+		// when experimenting with and testing those elements.
+		// if we are unpausing (since it's not the first run of this
+		// function) we need to poke to *unpause* every graph vertex,
+		// and not just selectively the subset with no indegree.
+
+		// let the startup code know to poke or not
+		// this triggers a CheckApply AFTER Watch is Running()
+		// We *don't* need to also do this to new nodes or nodes that
+		// are about to get unpaused, because they'll get poked by one
+		// of the indegree == 0 vertices, and an important aspect of the
+		// Process() function is that even if the state is correct, it
+		// will pass through the Poke so that it flows through the DAG.
+		res.Starter(indegree[v] == 0)
+
+		var unpause = true
+		if !*res.Working() { // if Worker() is not running...
+			*res.Working() = true // set Worker() running flag
+
+			unpause = false // doesn't need unpausing on first start
+			obj.wg.Add(1)
+			// must pass in value to avoid races...
+			// see: https://ttboj.wordpress.com/2015/07/27/golang-parallelism-issues-causing-too-many-open-files-error/
+			go func(vv pgraph.Vertex) {
+				defer obj.wg.Done()
+				// unset Worker() running flag just before exit
+				defer func() { *VtoR(vv).Working() = false }()
+				defer VtoR(vv).Reset()
+				// TODO: if a sufficient number of workers error,
+				// should something be done? Should these restart
+				// after perma-failure if we have a graph change?
+				log.Printf("%s: Started", vv)
+				if err := VtoR(vv).Worker(); err != nil { // contains the Watch and CheckApply loops
+					log.Printf("%s: Exited with failure: %v", vv, err)
+					return
+				}
+				log.Printf("%s: Exited", vv)
+			}(v)
+		}
+
+		select {
+		case <-res.Started(): // block until started
+		case <-res.Stopped(): // we failed on init
+			// if the resource Init() fails, we don't hang!
+		}
+
+		if unpause { // unpause (if needed)
+			res.SendEvent(event.EventStart, nil) // sync!
+		}
+	}
+	// we wait for everyone to start before exiting!
+}
+
+// Pause sends pause events to the graph in a topological sort order. If you set
+// the fastPause argument to true, then it will ask future propagation waves to
+// not run through the graph before exiting, and instead will exit much quicker.
+func (obj *MGraph) Pause(fastPause bool) {
+	log.Printf("State: %v -> %v", obj.setState(graphStatePausing), obj.getState())
+	defer log.Printf("State: %v -> %v", obj.setState(graphStatePaused), obj.getState())
+	if fastPause {
+		obj.FastPause = true // set flag
+	}
+	t, _ := obj.Graph.TopologicalSort()
+	for _, v := range t { // squeeze out the events...
+		VtoR(v).SendEvent(event.EventPause, nil) // sync
+	}
+	obj.FastPause = false // reset flag
+}
+
+// Exit sends exit events to the graph in a topological sort order.
+func (obj *MGraph) Exit() {
+	if obj.Graph == nil { // empty graph that wasn't populated yet
+		return
+	}
+
+	// FIXME: a second ^C could put this into fast pause, but do it for now!
+	obj.Pause(true) // implement this with pause to avoid duplicating the code
+
+	t, _ := obj.Graph.TopologicalSort()
+	for _, v := range t { // squeeze out the events...
+		// turn off the taps...
+		VtoR(v).Exit() // sync
+	}
+	obj.wg.Wait() // for now, this doesn't need to be a separate Wait() method
+}
--- a/resources/msg.go
+++ b/resources/msg.go
@@ -3,22 +3,21 @@
 // Written by James Shubin <james@shubin.ca> and the project contributors
 //
 // This program is free software: you can redistribute it and/or modify
-// it under the terms of the GNU Affero General Public License as published by
+// it under the terms of the GNU General Public License as published by
 // the Free Software Foundation, either version 3 of the License, or
 // (at your option) any later version.
 //
 // This program is distributed in the hope that it will be useful,
 // but WITHOUT ANY WARRANTY; without even the implied warranty of
 // MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-// GNU Affero General Public License for more details.
+// GNU General Public License for more details.
 //
-// You should have received a copy of the GNU Affero General Public License
+// You should have received a copy of the GNU General Public License
 // along with this program.  If not, see <http://www.gnu.org/licenses/>.

 package resources

 import (
-	"encoding/gob"
 	"fmt"
 	"log"
 	"regexp"
@@ -28,7 +27,7 @@ import (
 )

 func init() {
-	gob.Register(&MsgRes{})
+	RegisterResource("msg", func() Res { return &MsgRes{} })
 }

 // MsgRes is a resource that writes messages to logs.
@@ -70,12 +69,23 @@ func (obj *MsgRes) Validate() error {
 			return fmt.Errorf("fields cannot begin with _")
 		}
 	}
+	switch obj.Priority {
+	case "Emerg":
+	case "Alert":
+	case "Crit":
+	case "Err":
+	case "Warning":
+	case "Notice":
+	case "Info":
+	case "Debug":
+	default:
+		return fmt.Errorf("invalid Priority '%s'", obj.Priority)
+	}
 	return obj.BaseRes.Validate()
 }

 // Init runs some startup code for this resource.
 func (obj *MsgRes) Init() error {
-	obj.BaseRes.kind = "Msg"
 	return obj.BaseRes.Init() // call base init, b/c we're overrriding
 }

@@ -96,7 +106,6 @@ func (obj *MsgRes) updateStateOK() {
 }

 // JournalPriority converts a string description to a numeric priority.
-// XXX: Have Validate() make sure it actually is one of these.
 func (obj *MsgRes) journalPriority() journal.Priority {
 	switch obj.Priority {
 	case "Emerg":
@@ -167,7 +176,7 @@ func (obj *MsgRes) CheckApply(apply bool) (bool, error) {
 	}

 	if !obj.logStateOK {
-		log.Printf("%s[%s]: Body: %s", obj.Kind(), obj.GetName(), obj.Body)
+		log.Printf("%s: Body: %s", obj, obj.Body)
 		obj.logStateOK = true
 		obj.updateStateOK()
 	}
@@ -195,44 +204,40 @@ func (obj *MsgRes) CheckApply(apply bool) (bool, error) {
 func (obj *MsgRes) UIDs() []ResUID {
 	x := &MsgUID{
 		BaseUID: BaseUID{
-			name: obj.GetName(),
-			kind: obj.Kind(),
+			Name: obj.GetName(),
+			Kind: obj.GetKind(),
 		},
 		body: obj.Body,
 	}
 	return []ResUID{x}
 }

-// AutoEdges returns the AutoEdges. In this case none are used.
-func (obj *MsgRes) AutoEdges() AutoEdge {
-	return nil
-}
-
 // Compare two resources and return if they are equivalent.
-func (obj *MsgRes) Compare(res Res) bool {
-	switch res.(type) {
-	case *MsgRes:
-		res := res.(*MsgRes)
-		if !obj.BaseRes.Compare(res) {
-			return false
-		}
-		if obj.Body != res.Body {
-			return false
-		}
-		if obj.Priority != res.Priority {
-			return false
-		}
-		if len(obj.Fields) != len(res.Fields) {
-			return false
-		}
-		for field, value := range obj.Fields {
-			if res.Fields[field] != value {
-				return false
-			}
-		}
-	default:
+func (obj *MsgRes) Compare(r Res) bool {
+	// we can only compare MsgRes to others of the same resource kind
+	res, ok := r.(*MsgRes)
+	if !ok {
 		return false
 	}
+	if !obj.BaseRes.Compare(res) {
+		return false
+	}
+
+	if obj.Body != res.Body {
+		return false
+	}
+	if obj.Priority != res.Priority {
+		return false
+	}
+	if len(obj.Fields) != len(res.Fields) {
+		return false
+	}
+	for field, value := range obj.Fields {
+		if res.Fields[field] != value {
+			return false
+		}
+	}
+
 	return true
 }

--- a/resources/msg_test.go
+++ b/resources/msg_test.go
@@ -0,0 +1,54 @@
+// Mgmt
+// Copyright (C) 2013-2017+ James Shubin and the project contributors
+// Written by James Shubin <james@shubin.ca> and the project contributors
+//
+// This program is free software: you can redistribute it and/or modify
+// it under the terms of the GNU General Public License as published by
+// the Free Software Foundation, either version 3 of the License, or
+// (at your option) any later version.
+//
+// This program is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+// GNU General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License
+// along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+package resources
+
+import (
+	"testing"
+)
+
+func TestMsgValidate1(t *testing.T) {
+	r1 := &MsgRes{
+		BaseRes: BaseRes{
+			Name:       "msg1",
+			Kind:       "msg",
+			MetaParams: DefaultMetaParams,
+		},
+		Priority: "Debug",
+	}
+
+	r1.Setup(nil, r1, r1)
+	if err := r1.Validate(); err != nil {
+		t.Errorf("validate failed with: %v", err)
+	}
+}
+
+func TestMsgValidate2(t *testing.T) {
+	r1 := &MsgRes{
+		BaseRes: BaseRes{
+			Name:       "msg1",
+			Kind:       "msg",
+			MetaParams: DefaultMetaParams,
+		},
+		Priority: "UnrealPriority",
+	}
+
+	r1.Setup(nil, r1, r1)
+	if err := r1.Validate(); err == nil {
+		t.Errorf("validation error is nil")
+	}
+}
--- a/resources/noop.go
+++ b/resources/noop.go
@@ -3,28 +3,27 @@
 // Written by James Shubin <james@shubin.ca> and the project contributors
 //
 // This program is free software: you can redistribute it and/or modify
-// it under the terms of the GNU Affero General Public License as published by
+// it under the terms of the GNU General Public License as published by
 // the Free Software Foundation, either version 3 of the License, or
 // (at your option) any later version.
 //
 // This program is distributed in the hope that it will be useful,
 // but WITHOUT ANY WARRANTY; without even the implied warranty of
 // MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-// GNU Affero General Public License for more details.
+// GNU General Public License for more details.
 //
-// You should have received a copy of the GNU Affero General Public License
+// You should have received a copy of the GNU General Public License
 // along with this program.  If not, see <http://www.gnu.org/licenses/>.

 package resources

 import (
-	"encoding/gob"
 	"fmt"
 	"log"
 )

 func init() {
-	gob.Register(&NoopRes{})
+	RegisterResource("noop", func() Res { return &NoopRes{} })
 }

 // NoopRes is a no-op resource that does nothing.
@@ -49,7 +48,6 @@ func (obj *NoopRes) Validate() error {

 // Init runs some startup code for this resource.
 func (obj *NoopRes) Init() error {
-	obj.BaseRes.kind = "Noop"
 	return obj.BaseRes.Init() // call base init, b/c we're overriding
 }

@@ -82,7 +80,7 @@ func (obj *NoopRes) Watch() error {
 // CheckApply method for Noop resource. Does nothing, returns happy!
 func (obj *NoopRes) CheckApply(apply bool) (checkOK bool, err error) {
 	if obj.Refresh() {
-		log.Printf("%s[%s]: Received a notification!", obj.Kind(), obj.GetName())
+		log.Printf("%s: Received a notification!", obj)
 	}
 	return true, nil // state is always okay
 }
@@ -93,16 +91,11 @@ type NoopUID struct {
 	name string
 }

-// AutoEdges returns the AutoEdge interface. In this case no autoedges are used.
-func (obj *NoopRes) AutoEdges() AutoEdge {
-	return nil
-}
-
 // UIDs includes all params to make a unique identification of this object.
 // Most resources only return one, although some resources can return multiple.
 func (obj *NoopRes) UIDs() []ResUID {
 	x := &NoopUID{
-		BaseUID: BaseUID{name: obj.GetName(), kind: obj.Kind()},
+		BaseUID: BaseUID{Name: obj.GetName(), Kind: obj.GetKind()},
 		name:    obj.Name,
 	}
 	return []ResUID{x}
@@ -122,21 +115,20 @@ func (obj *NoopRes) GroupCmp(r Res) bool {
 }

 // Compare two resources and return if they are equivalent.
-func (obj *NoopRes) Compare(res Res) bool {
-	switch res.(type) {
-	// we can only compare NoopRes to others of the same resource
-	case *NoopRes:
-		res := res.(*NoopRes)
-		// calling base Compare is unneeded for the noop res
-		//if !obj.BaseRes.Compare(res) { // call base Compare
-		//	return false
-		//}
-		if obj.Name != res.Name {
-			return false
-		}
-	default:
+func (obj *NoopRes) Compare(r Res) bool {
+	// we can only compare NoopRes to others of the same resource kind
+	res, ok := r.(*NoopRes)
+	if !ok {
 		return false
 	}
+	// calling base Compare is probably unneeded for the noop res, but do it
+	if !obj.BaseRes.Compare(res) { // call base Compare
+		return false
+	}
+	if obj.Name != res.Name {
+		return false
+	}
+
 	return true
 }

--- a/resources/nspawn.go
+++ b/resources/nspawn.go
@@ -3,32 +3,34 @@
 // Written by James Shubin <james@shubin.ca> and the project contributors
 //
 // This program is free software: you can redistribute it and/or modify
-// it under the terms of the GNU Affero General Public License as published by
+// it under the terms of the GNU General Public License as published by
 // the Free Software Foundation, either version 3 of the License, or
 // (at your option) any later version.
 //
 // This program is distributed in the hope that it will be useful,
 // but WITHOUT ANY WARRANTY; without even the implied warranty of
 // MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-// GNU Affero General Public License for more details.
+// GNU General Public License for more details.
 //
-// You should have received a copy of the GNU Affero General Public License
+// You should have received a copy of the GNU General Public License
 // along with this program.  If not, see <http://www.gnu.org/licenses/>.

 package resources

 import (
-	"encoding/gob"
 	"errors"
 	"fmt"
 	"log"
+	"strconv"
+	"unicode"

 	"github.com/purpleidea/mgmt/util"

+	systemdDbus "github.com/coreos/go-systemd/dbus"
+	machined "github.com/coreos/go-systemd/machine1"
 	systemdUtil "github.com/coreos/go-systemd/util"
 	"github.com/godbus/dbus"
 	errwrap "github.com/pkg/errors"
-	machined "github.com/purpleidea/go-systemd/machine1"
 )

 const (
@@ -41,18 +43,18 @@ const (
 )

 func init() {
-	gob.Register(&NspawnRes{})
+	RegisterResource("nspawn", func() Res { return &NspawnRes{} })
 }

-// NspawnRes is an nspawn container resource
+// NspawnRes is an nspawn container resource.
 type NspawnRes struct {
 	BaseRes `yaml:",inline"`
 	State   string `yaml:"state"`
-	// we're using the svc resource to start the machine because that's
-	// what machinectl does. We're not using svc.Watch because then we
+	// We're using the svc resource to start and stop the machine because
+	// that's what machinectl does. We're not using svc.Watch because then we
 	// would have two watches potentially racing each other and producing
-	// potentially unexpected results. We get everything we need to
-	// monitor the machine state changes from the org.freedesktop.machine1 object.
+	// potentially unexpected results. We get everything we need to monitor
+	// the machine state changes from the org.freedesktop.machine1 object.
 	svc *SvcRes
 }

@@ -66,37 +68,94 @@ func (obj *NspawnRes) Default() Res {
 	}
 }

+// makeComposite creates a pointer to a SvcRes. The pointer is used to
+// validate and initialize the nested svc.
+func (obj *NspawnRes) makeComposite() (*SvcRes, error) {
+	res, err := NewNamedResource("svc", fmt.Sprintf(nspawnServiceTmpl, obj.GetName()))
+	if err != nil {
+		return nil, err
+	}
+	svc := res.(*SvcRes)
+	svc.State = obj.State
+	return svc, nil
+}
+
+// systemdVersion uses dbus to check which version of systemd is installed.
+func systemdVersion() (uint16, error) {
+	// check if systemd is running
+	if !systemdUtil.IsRunningSystemd() {
+		return 0, fmt.Errorf("systemd is not running")
+	}
+	bus, err := systemdDbus.NewSystemdConnection()
+	if err != nil {
+		return 0, errwrap.Wrapf(err, "failed to connect to bus")
+	}
+	defer bus.Close()
+	// get the systemd version
+	verString, err := bus.GetManagerProperty("Version")
+	if err != nil {
+		return 0, errwrap.Wrapf(err, "could not get version property")
+	}
+	// lose the surrounding quotes
+	verNum, err := strconv.Unquote(verString)
+	if err != nil {
+		return 0, errwrap.Wrapf(err, "error unquoting version number")
+	}
+	// cast to uint16
+	ver, err := strconv.ParseUint(verNum, 10, 16)
+	if err != nil {
+		return 0, errwrap.Wrapf(err, "error casting systemd version number")
+	}
+	return uint16(ver), nil
+}
+
 // Validate if the params passed in are valid data.
 func (obj *NspawnRes) Validate() error {
-	// TODO: validStates should be an enum!
-	validStates := map[string]struct{}{
-		stopped: {},
-		running: {},
+	if len(obj.GetName()) > 64 {
+		return fmt.Errorf("name must be 64 characters or less")
 	}
-	if _, exists := validStates[obj.State]; !exists {
-		return fmt.Errorf("Invalid State: %s", obj.State)
+	// check if systemd version is higher than 231 to allow non-alphanumeric
+	// machine names, as previous versions would error in such cases
+	ver, err := systemdVersion()
+	if err != nil {
+		return err
+	}
+	if ver < 231 {
+		for _, char := range obj.GetName() {
+			if !unicode.IsLetter(char) && !unicode.IsNumber(char) {
+				return fmt.Errorf("name must only contain alphanumeric characters for systemd versions < 231")
+			}
+		}
 	}

-	if err := obj.svc.Validate(); err != nil { // composite resource
-		return errwrap.Wrapf(err, "validate failed for embedded svc")
+	if obj.State != running && obj.State != stopped {
+		return fmt.Errorf("invalid state: %s", obj.State)
+	}
+
+	svc, err := obj.makeComposite()
+	if err != nil {
+		return errwrap.Wrapf(err, "makeComposite failed in validate")
+	}
+	if err := svc.Validate(); err != nil { // composite resource
+		return errwrap.Wrapf(err, "validate failed for embedded svc: %s", obj.svc)
 	}
 	return obj.BaseRes.Validate()
 }

 // Init runs some startup code for this resource.
 func (obj *NspawnRes) Init() error {
-	var serviceName = fmt.Sprintf(nspawnServiceTmpl, obj.GetName())
-	obj.svc = &SvcRes{}
-	obj.svc.Name = serviceName
-	obj.svc.State = obj.State
+	svc, err := obj.makeComposite()
+	if err != nil {
+		return errwrap.Wrapf(err, "makeComposite failed in init")
+	}
+	obj.svc = svc
 	if err := obj.svc.Init(); err != nil {
 		return err
 	}
-	obj.BaseRes.kind = "Nspawn"
 	return obj.BaseRes.Init()
 }

-// Watch for state changes and sends a message to the bus if there is a change
+// Watch for state changes and sends a message to the bus if there is a change.
 func (obj *NspawnRes) Watch() error {
 	// this resource depends on systemd ensure that it's running
 	if !systemdUtil.IsRunningSystemd() {
@@ -117,8 +176,10 @@ func (obj *NspawnRes) Watch() error {
 	if err := call.Err; err != nil {
 		return err
 	}
-	buschan := make(chan *dbus.Signal, 10)
-	bus.Signal(buschan)
+	// TODO: verify that implementation doesn't deadlock if there are unread
+	// messages left in the channel
+	busChan := make(chan *dbus.Signal, 10)
+	bus.Signal(busChan)

 	// notify engine that we're running
 	if err := obj.Running(); err != nil {
@@ -128,16 +189,19 @@ func (obj *NspawnRes) Watch() error {
 	var send = false
 	var exit *error

+	defer close(busChan)
+	defer bus.Close()
+	defer bus.RemoveSignal(busChan)
 	for {
 		select {
-		case event := <-buschan:
+		case event := <-busChan:
 			// process org.freedesktop.machine1 events for this resource's name
 			if event.Body[0] == obj.GetName() {
-				log.Printf("%s[%s]: Event received: %v", obj.Kind(), obj.GetName(), event.Name)
+				log.Printf("%s: Event received: %v", obj, event.Name)
 				if event.Name == machineNew {
-					log.Printf("%s[%s]: Machine started", obj.Kind(), obj.GetName())
+					log.Printf("%s: Machine started", obj)
 				} else if event.Name == machineRemoved {
-					log.Printf("%s[%s]: Machine stopped", obj.Kind(), obj.GetName())
+					log.Printf("%s: Machine stopped", obj)
 				} else {
 					return fmt.Errorf("unknown event: %s", event.Name)
 				}
@@ -160,8 +224,8 @@ func (obj *NspawnRes) Watch() error {
 }

 // CheckApply is run to check the state and, if apply is true, to apply the
-// necessary changes to reach the desired state. this is run before Watch and
-// again if watch finds a change occurring to the state
+// necessary changes to reach the desired state. This is run before Watch and
+// again if Watch finds a change occurring to the state.
 func (obj *NspawnRes) CheckApply(apply bool) (checkOK bool, err error) {
 	// this resource depends on systemd ensure that it's running
 	if !systemdUtil.IsRunningSystemd() {
@@ -177,7 +241,7 @@ func (obj *NspawnRes) CheckApply(apply bool) (checkOK bool, err error) {
 	// compare the current state with the desired state and perform the
 	// appropriate action
 	var exists = true
-	properties, err := conn.GetProperties(obj.GetName())
+	properties, err := conn.DescribeMachine(obj.GetName())
 	if err != nil {
 		if err, ok := err.(dbus.Error); ok && err.Name !=
 			"org.freedesktop.machine1.NoSuchMachine" {
@@ -194,13 +258,13 @@ func (obj *NspawnRes) CheckApply(apply bool) (checkOK bool, err error) {
 		}
 	}
 	if obj.debug {
-		log.Printf("%s[%s]: properties: %v", obj.Kind(), obj.GetName(), properties)
+		log.Printf("%s: properties: %v", obj, properties)
 	}
 	// if the machine doesn't exist and is supposed to
 	// be stopped or the state matches we're done
 	if !exists && obj.State == stopped || properties["State"] == obj.State {
 		if obj.debug {
-			log.Printf("%s[%s]: CheckApply() in valid state", obj.Kind(), obj.GetName())
+			log.Printf("%s: CheckApply() in valid state", obj)
 		}
 		return true, nil
 	}
@@ -210,41 +274,26 @@ func (obj *NspawnRes) CheckApply(apply bool) (checkOK bool, err error) {
 		return false, nil
 	}

-	if obj.debug {
-		log.Printf("%s[%s]: CheckApply() applying '%s' state", obj.Kind(), obj.GetName(), obj.State)
-	}
-
-	if obj.State == running {
-		// start the machine using svc resource
-		log.Printf("%s[%s]: Starting machine", obj.Kind(), obj.GetName())
-		// assume state had to be changed at this point, ignore checkOK
-		if _, err := obj.svc.CheckApply(apply); err != nil {
-			return false, errwrap.Wrapf(err, "nested svc failed")
-		}
-	}
-	if obj.State == stopped {
-		// terminate the machine with
-		// org.freedesktop.machine1.Manager.KillMachine
-		log.Printf("%s[%s]: Stopping machine", obj.Kind(), obj.GetName())
-		if err := conn.TerminateMachine(obj.GetName()); err != nil {
-			return false, errwrap.Wrapf(err, "failed to stop machine")
-		}
+	log.Printf("%s: CheckApply() applying '%s' state", obj, obj.State)
+	// use the embedded svc to apply the correct state
+	if _, err := obj.svc.CheckApply(apply); err != nil {
+		return false, errwrap.Wrapf(err, "nested svc failed")
 	}

 	return false, nil
 }

-// NspawnUID is a unique resource identifier
+// NspawnUID is a unique resource identifier.
 type NspawnUID struct {
-	// NOTE: there is also a name variable in the BaseUID struct, this is
+	// NOTE: There is also a name variable in the BaseUID struct, this is
 	// information about where this UID came from, and is unrelated to the
 	// information about the resource we're matching. That data which is
-	// used in the IFF function, is what you see in the struct fields here
+	// used in the IFF function, is what you see in the struct fields here.
 	BaseUID
 	name string // the machine name
 }

-// IFF aka if and only if they are equivalent, return true. If not, false
+// IFF aka if and only if they are equivalent, return true. If not, false.
 func (obj *NspawnUID) IFF(uid ResUID) bool {
 	res, ok := uid.(*NspawnUID)
 	if !ok {
@@ -253,17 +302,17 @@ func (obj *NspawnUID) IFF(uid ResUID) bool {
 	return obj.name == res.name
 }

-// UIDs includes all params to make a unique identification of this object
-// most resources only return one although some resources can return multiple
+// UIDs includes all params to make a unique identification of this object.
+// Most resources only return one although some resources can return multiple.
 func (obj *NspawnRes) UIDs() []ResUID {
 	x := &NspawnUID{
-		BaseUID: BaseUID{name: obj.GetName(), kind: obj.Kind()},
+		BaseUID: BaseUID{Name: obj.GetName(), Kind: obj.GetKind()},
 		name:    obj.Name, // svc name
 	}
 	return append([]ResUID{x}, obj.svc.UIDs()...)
 }

-// GroupCmp returns whether two resources can be grouped together or not
+// GroupCmp returns whether two resources can be grouped together or not.
 func (obj *NspawnRes) GroupCmp(r Res) bool {
 	_, ok := r.(*NspawnRes)
 	if !ok {
@@ -273,29 +322,28 @@ func (obj *NspawnRes) GroupCmp(r Res) bool {
 	return false
 }

-// Compare two resources and return if they are equivalent
-func (obj *NspawnRes) Compare(res Res) bool {
-	switch res.(type) {
-	case *NspawnRes:
-		res := res.(*NspawnRes)
-		if !obj.BaseRes.Compare(res) {
-			return false
-		}
-		if obj.Name != res.Name {
-			return false
-		}
-		if !obj.svc.Compare(res.svc) {
-			return false
-		}
-	default:
+// Compare two resources and return if they are equivalent.
+func (obj *NspawnRes) Compare(r Res) bool {
+	// we can only compare NspawnRes to others of the same resource kind
+	res, ok := r.(*NspawnRes)
+	if !ok {
+		return false
+	}
+	if !obj.BaseRes.Compare(res) {
+		return false
+	}
+	if obj.Name != res.Name {
+		return false
+	}
+	if obj.State != res.State {
 		return false
 	}
-	return true
-}

-// AutoEdges returns the AutoEdge interface in this case no autoedges are used
-func (obj *NspawnRes) AutoEdges() AutoEdge {
-	return nil
+	if !obj.svc.Compare(res.svc) {
+		return false
+	}
+
+	return true
 }

 // UnmarshalYAML is the custom unmarshal handler for this struct.
--- a/resources/packagekit/packagekit.go
+++ b/resources/packagekit/packagekit.go
@@ -3,16 +3,16 @@
 // Written by James Shubin <james@shubin.ca> and the project contributors
 //
 // This program is free software: you can redistribute it and/or modify
-// it under the terms of the GNU Affero General Public License as published by
+// it under the terms of the GNU General Public License as published by
 // the Free Software Foundation, either version 3 of the License, or
 // (at your option) any later version.
 //
 // This program is distributed in the hope that it will be useful,
 // but WITHOUT ANY WARRANTY; without even the implied warranty of
 // MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-// GNU Affero General Public License for more details.
+// GNU General Public License for more details.
 //
-// You should have received a copy of the GNU Affero General Public License
+// You should have received a copy of the GNU General Public License
 // along with this program.  If not, see <http://www.gnu.org/licenses/>.

 // Package packagekit provides an interface to interact with packagekit.
@@ -33,8 +33,8 @@ import (

 // global tweaks of verbosity and code path
 const (
-	PK_DEBUG = false
-	PARANOID = false // enable if you see any ghosts
+	Debug    = false
+	Paranoid = false // enable if you see any ghosts
 )

 // constants which might need to be tweaked or which contain special dbus strings.
@@ -74,76 +74,76 @@ var (
 //type enum_filter uint64
 // https://github.com/hughsie/PackageKit/blob/master/lib/packagekit-glib2/pk-enum.c
 const ( //static const PkEnumMatch enum_filter[]
-	PK_FILTER_ENUM_UNKNOWN         uint64 = 1 << iota // "unknown"
-	PK_FILTER_ENUM_NONE                               // "none"
-	PK_FILTER_ENUM_INSTALLED                          // "installed"
-	PK_FILTER_ENUM_NOT_INSTALLED                      // "~installed"
-	PK_FILTER_ENUM_DEVELOPMENT                        // "devel"
-	PK_FILTER_ENUM_NOT_DEVELOPMENT                    // "~devel"
-	PK_FILTER_ENUM_GUI                                // "gui"
-	PK_FILTER_ENUM_NOT_GUI                            // "~gui"
-	PK_FILTER_ENUM_FREE                               // "free"
-	PK_FILTER_ENUM_NOT_FREE                           // "~free"
-	PK_FILTER_ENUM_VISIBLE                            // "visible"
-	PK_FILTER_ENUM_NOT_VISIBLE                        // "~visible"
-	PK_FILTER_ENUM_SUPPORTED                          // "supported"
-	PK_FILTER_ENUM_NOT_SUPPORTED                      // "~supported"
-	PK_FILTER_ENUM_BASENAME                           // "basename"
-	PK_FILTER_ENUM_NOT_BASENAME                       // "~basename"
-	PK_FILTER_ENUM_NEWEST                             // "newest"
-	PK_FILTER_ENUM_NOT_NEWEST                         // "~newest"
-	PK_FILTER_ENUM_ARCH                               // "arch"
-	PK_FILTER_ENUM_NOT_ARCH                           // "~arch"
-	PK_FILTER_ENUM_SOURCE                             // "source"
-	PK_FILTER_ENUM_NOT_SOURCE                         // "~source"
-	PK_FILTER_ENUM_COLLECTIONS                        // "collections"
-	PK_FILTER_ENUM_NOT_COLLECTIONS                    // "~collections"
-	PK_FILTER_ENUM_APPLICATION                        // "application"
-	PK_FILTER_ENUM_NOT_APPLICATION                    // "~application"
-	PK_FILTER_ENUM_DOWNLOADED                         // "downloaded"
-	PK_FILTER_ENUM_NOT_DOWNLOADED                     // "~downloaded"
+	PkFilterEnumUnknown        uint64 = 1 << iota // "unknown"
+	PkFilterEnumNone                              // "none"
+	PkFilterEnumInstalled                         // "installed"
+	PkFilterEnumNotInstalled                      // "~installed"
+	PkFilterEnumDevelopment                       // "devel"
+	PkFilterEnumNotDevelopment                    // "~devel"
+	PkFilterEnumGui                               // "gui"
+	PkFilterEnumNotGui                            // "~gui"
+	PkFilterEnumFree                              // "free"
+	PkFilterEnumNotFree                           // "~free"
+	PkFilterEnumVisible                           // "visible"
+	PkFilterEnumNotVisible                        // "~visible"
+	PkFilterEnumSupported                         // "supported"
+	PkFilterEnumNotSupported                      // "~supported"
+	PkFilterEnumBasename                          // "basename"
+	PkFilterEnumNotBasename                       // "~basename"
+	PkFilterEnumNewest                            // "newest"
+	PkFilterEnumNotNewest                         // "~newest"
+	PkFilterEnumArch                              // "arch"
+	PkFilterEnumNotArch                           // "~arch"
+	PkFilterEnumSource                            // "source"
+	PkFilterEnumNotSource                         // "~source"
+	PkFilterEnumCollections                       // "collections"
+	PkFilterEnumNotCollections                    // "~collections"
+	PkFilterEnumApplication                       // "application"
+	PkFilterEnumNotApplication                    // "~application"
+	PkFilterEnumDownloaded                        // "downloaded"
+	PkFilterEnumNotDownloaded                     // "~downloaded"
 )

 // constants from packagekit c library.
 const ( //static const PkEnumMatch enum_transaction_flag[]
-	PK_TRANSACTION_FLAG_ENUM_NONE            uint64 = 1 << iota // "none"
-	PK_TRANSACTION_FLAG_ENUM_ONLY_TRUSTED                       // "only-trusted"
-	PK_TRANSACTION_FLAG_ENUM_SIMULATE                           // "simulate"
-	PK_TRANSACTION_FLAG_ENUM_ONLY_DOWNLOAD                      // "only-download"
-	PK_TRANSACTION_FLAG_ENUM_ALLOW_REINSTALL                    // "allow-reinstall"
-	PK_TRANSACTION_FLAG_ENUM_JUST_REINSTALL                     // "just-reinstall"
-	PK_TRANSACTION_FLAG_ENUM_ALLOW_DOWNGRADE                    // "allow-downgrade"
+	PkTransactionFlagEnumNone           uint64 = 1 << iota // "none"
+	PkTransactionFlagEnumOnlyTrusted                       // "only-trusted"
+	PkTransactionFlagEnumSimulate                          // "simulate"
+	PkTransactionFlagEnumOnlyDownload                      // "only-download"
+	PkTransactionFlagEnumAllowReinstall                    // "allow-reinstall"
+	PkTransactionFlagEnumJustReinstall                     // "just-reinstall"
+	PkTransactionFlagEnumAllowDowngrade                    // "allow-downgrade"
 )

 // constants from packagekit c library.
 const ( //typedef enum
-	PK_INFO_ENUM_UNKNOWN uint64 = 1 << iota
-	PK_INFO_ENUM_INSTALLED
-	PK_INFO_ENUM_AVAILABLE
-	PK_INFO_ENUM_LOW
-	PK_INFO_ENUM_ENHANCEMENT
-	PK_INFO_ENUM_NORMAL
-	PK_INFO_ENUM_BUGFIX
-	PK_INFO_ENUM_IMPORTANT
-	PK_INFO_ENUM_SECURITY
-	PK_INFO_ENUM_BLOCKED
-	PK_INFO_ENUM_DOWNLOADING
-	PK_INFO_ENUM_UPDATING
-	PK_INFO_ENUM_INSTALLING
-	PK_INFO_ENUM_REMOVING
-	PK_INFO_ENUM_CLEANUP
-	PK_INFO_ENUM_OBSOLETING
-	PK_INFO_ENUM_COLLECTION_INSTALLED
-	PK_INFO_ENUM_COLLECTION_AVAILABLE
-	PK_INFO_ENUM_FINISHED
-	PK_INFO_ENUM_REINSTALLING
-	PK_INFO_ENUM_DOWNGRADING
-	PK_INFO_ENUM_PREPARING
-	PK_INFO_ENUM_DECOMPRESSING
-	PK_INFO_ENUM_UNTRUSTED
-	PK_INFO_ENUM_TRUSTED
-	PK_INFO_ENUM_UNAVAILABLE
-	PK_INFO_ENUM_LAST
+	PkInfoEnumUnknown uint64 = 1 << iota
+	PkInfoEnumInstalled
+	PkInfoEnumAvailable
+	PkInfoEnumLow
+	PkInfoEnumEnhancement
+	PkInfoEnumNormal
+	PkInfoEnumBugfix
+	PkInfoEnumImportant
+	PkInfoEnumSecurity
+	PkInfoEnumBlocked
+	PkInfoEnumDownloading
+	PkInfoEnumUpdating
+	PkInfoEnumInstalling
+	PkInfoEnumRemoving
+	PkInfoEnumCleanup
+	PkInfoEnumObsoleting
+	PkInfoEnumCollectionInstalled
+	PkInfoEnumCollectionAvailable
+	PkInfoEnumFinished
+	PkInfoEnumReinstalling
+	PkInfoEnumDowngrading
+	PkInfoEnumPreparing
+	PkInfoEnumDecompressing
+	PkInfoEnumUntrusted
+	PkInfoEnumTrusted
+	PkInfoEnumUnavailable
+	PkInfoEnumLast
 )

 // Conn is a wrapper struct so we can pass bus connection around in the struct.
@@ -184,7 +184,7 @@ func (bus *Conn) Close() error {

 // internal helper to add signal matches to the bus, should only be called once
 func (bus *Conn) matchSignal(ch chan *dbus.Signal, path dbus.ObjectPath, iface string, signals []string) error {
-	if PK_DEBUG {
+	if Debug {
 		log.Printf("PackageKit: matchSignal(%v, %v, %v, %v)", ch, path, iface, signals)
 	}
 	// eg: gdbus monitor --system --dest org.freedesktop.PackageKit --object-path /org/freedesktop/PackageKit | grep <signal>
@@ -224,7 +224,7 @@ func (bus *Conn) WatchChanges() (chan *dbus.Signal, error) {
 	if err != nil {
 		return nil, err
 	}
-	if PARANOID { // TODO: this filtering might not be necessary anymore...
+	if Paranoid { // TODO: this filtering might not be necessary anymore...
 		// try to handle the filtering inside this function!
 		rch := make(chan *dbus.Signal)
 		go func() {
@@ -257,7 +257,7 @@ func (bus *Conn) WatchChanges() (chan *dbus.Signal, error) {

 // CreateTransaction creates and returns a transaction path.
 func (bus *Conn) CreateTransaction() (dbus.ObjectPath, error) {
-	if PK_DEBUG {
+	if Debug {
 		log.Println("PackageKit: CreateTransaction()")
 	}
 	var interfacePath dbus.ObjectPath
@@ -266,7 +266,7 @@ func (bus *Conn) CreateTransaction() (dbus.ObjectPath, error) {
 	if call != nil {
 		return "", call
 	}
-	if PK_DEBUG {
+	if Debug {
 		log.Printf("PackageKit: CreateTransaction(): %v", interfacePath)
 	}
 	return interfacePath, nil
@@ -284,12 +284,12 @@ func (bus *Conn) ResolvePackages(packages []string, filter uint64) ([]string, er
 	// add signal matches for Package and Finished which will always be last
 	var signals = []string{"Package", "Finished", "Error", "Destroy"}
 	bus.matchSignal(ch, interfacePath, PkIfaceTransaction, signals)
-	if PK_DEBUG {
+	if Debug {
 		log.Printf("PackageKit: ResolvePackages(): Object(%v, %v)", PkIface, interfacePath)
 	}
 	obj := bus.GetBus().Object(PkIface, interfacePath) // pass in found transaction path
 	call := obj.Call(FmtTransactionMethod("Resolve"), 0, filter, packages)
-	if PK_DEBUG {
+	if Debug {
 		log.Println("PackageKit: ResolvePackages(): Call: Success!")
 	}
 	if call.Err != nil {
@@ -300,7 +300,7 @@ loop:
 		// FIXME: add a timeout option to error in case signals are dropped!
 		select {
 		case signal := <-ch:
-			if PK_DEBUG {
+			if Debug {
 				log.Printf("PackageKit: ResolvePackages(): Signal: %+v", signal)
 			}
 			if signal.Path != interfacePath {
@@ -338,8 +338,8 @@ loop:

 // IsInstalledList queries a list of packages to see if they are installed.
 func (bus *Conn) IsInstalledList(packages []string) ([]bool, error) {
-	var filter uint64             // initializes at the "zero" value of 0
-	filter += PK_FILTER_ENUM_ARCH // always search in our arch
+	var filter uint64          // initializes at the "zero" value of 0
+	filter += PkFilterEnumArch // always search in our arch
 	packageIDs, e := bus.ResolvePackages(packages, filter)
 	if e != nil {
 		return nil, fmt.Errorf("ResolvePackages error: %v", e)
@@ -396,7 +396,11 @@ func (bus *Conn) InstallPackages(packageIDs []string, transactionFlags uint64) e
 	bus.matchSignal(ch, interfacePath, PkIfaceTransaction, signals)

 	obj := bus.GetBus().Object(PkIface, interfacePath) // pass in found transaction path
-	call := obj.Call(FmtTransactionMethod("InstallPackages"), 0, transactionFlags, packageIDs)
+	call := obj.Call(FmtTransactionMethod("RefreshCache"), 0, false)
+	if call.Err != nil {
+		return call.Err
+	}
+	call = obj.Call(FmtTransactionMethod("InstallPackages"), 0, transactionFlags, packageIDs)
 	if call.Err != nil {
 		return call.Err
 	}
@@ -593,7 +597,7 @@ loop:

 // GetUpdates gets a list of packages that are installed and which can be updated, mod filter.
 func (bus *Conn) GetUpdates(filter uint64) ([]string, error) {
-	if PK_DEBUG {
+	if Debug {
 		log.Println("PackageKit: GetUpdates()")
 	}
 	packageIDs := []string{}
@@ -664,11 +668,11 @@ func (bus *Conn) PackagesToPackageIDs(packageMap map[string]string, filter uint6
 		count++
 	}

-	if !(filter&PK_FILTER_ENUM_ARCH == PK_FILTER_ENUM_ARCH) {
-		filter += PK_FILTER_ENUM_ARCH // always search in our arch
+	if !(filter&PkFilterEnumArch == PkFilterEnumArch) {
+		filter += PkFilterEnumArch // always search in our arch
 	}

-	if PK_DEBUG {
+	if Debug {
 		log.Printf("PackageKit: PackagesToPackageIDs(): %v", strings.Join(packages, ", "))
 	}
 	resolved, e := bus.ResolvePackages(packages, filter)
@@ -771,7 +775,7 @@ func (bus *Conn) PackagesToPackageIDs(packageMap map[string]string, filter uint6
 	// this check is for packages that need to verify their "newest" status
 	// we need to know this so we can install the correct newest packageID!
 	recursion := make(map[string]*PkPackageIDActionData)
-	if !(filter&PK_FILTER_ENUM_NEWEST == PK_FILTER_ENUM_NEWEST) {
+	if !(filter&PkFilterEnumNewest == PkFilterEnumNewest) {
 		checkPackages := []string{}
 		filteredPackageMap := make(map[string]string)
 		for index, pkg := range packages {
@@ -788,13 +792,13 @@ func (bus *Conn) PackagesToPackageIDs(packageMap map[string]string, filter uint6
 		}

 		// we _could_ do a second resolve and then parse like this...
-		//resolved, e := bus.ResolvePackages(..., filter+PK_FILTER_ENUM_NEWEST)
+		//resolved, e := bus.ResolvePackages(..., filter+PkFilterEnumNewest)
 		// but that's basically what recursion here could do too!
 		if len(checkPackages) > 0 {
-			if PK_DEBUG {
+			if Debug {
 				log.Printf("PackageKit: PackagesToPackageIDs(): Recurse: %v", strings.Join(checkPackages, ", "))
 			}
-			recursion, e = bus.PackagesToPackageIDs(filteredPackageMap, filter+PK_FILTER_ENUM_NEWEST)
+			recursion, e = bus.PackagesToPackageIDs(filteredPackageMap, filter+PkFilterEnumNewest)
 			if e != nil {
 				return nil, fmt.Errorf("Recursion error: %v", e)
 			}
--- a/resources/password.go
+++ b/resources/password.go
@@ -3,23 +3,22 @@
 // Written by James Shubin <james@shubin.ca> and the project contributors
 //
 // This program is free software: you can redistribute it and/or modify
-// it under the terms of the GNU Affero General Public License as published by
+// it under the terms of the GNU General Public License as published by
 // the Free Software Foundation, either version 3 of the License, or
 // (at your option) any later version.
 //
 // This program is distributed in the hope that it will be useful,
 // but WITHOUT ANY WARRANTY; without even the implied warranty of
 // MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-// GNU Affero General Public License for more details.
+// GNU General Public License for more details.
 //
-// You should have received a copy of the GNU Affero General Public License
+// You should have received a copy of the GNU General Public License
 // along with this program.  If not, see <http://www.gnu.org/licenses/>.

 package resources

 import (
 	"crypto/rand"
-	"encoding/gob"
 	"fmt"
 	"io/ioutil"
 	"log"
@@ -34,7 +33,7 @@ import (
 )

 func init() {
-	gob.Register(&PasswordRes{})
+	RegisterResource("password", func() Res { return &PasswordRes{} })
 }

 const (
@@ -73,7 +72,6 @@ func (obj *PasswordRes) Validate() error {
 // Init generates a new password for this resource if one was not provided. It
 // will save this into a local file. It will load it back in from previous runs.
 func (obj *PasswordRes) Init() error {
-	obj.BaseRes.kind = "Password" // must be set before using VarDir

 	dir, err := obj.VarDir("")
 	if err != nil {
@@ -187,7 +185,7 @@ func (obj *PasswordRes) Watch() error {
 				return nil
 			}
 			if err := event.Error; err != nil {
-				return errwrap.Wrapf(err, "unknown %s[%s] watcher error", obj.Kind(), obj.GetName())
+				return errwrap.Wrapf(err, "unknown %s watcher error", obj)
 			}
 			send = true
 			obj.StateOK(false) // dirty
@@ -228,7 +226,7 @@ func (obj *PasswordRes) CheckApply(apply bool) (checkOK bool, err error) {
 			if !obj.CheckRecovery {
 				return false, errwrap.Wrapf(err, "check failed")
 			}
-			log.Printf("%s[%s]: Integrity check failed", obj.Kind(), obj.GetName())
+			log.Printf("%s: Integrity check failed", obj)
 			generate = true // okay to build a new one
 			write = true    // make sure to write over the old one
 		}
@@ -262,7 +260,7 @@ func (obj *PasswordRes) CheckApply(apply bool) (checkOK bool, err error) {
 		}
 		// generate the actual password
 		var err error
-		log.Printf("%s[%s]: Generating new password...", obj.Kind(), obj.GetName())
+		log.Printf("%s: Generating new password...", obj)
 		if password, err = obj.generate(); err != nil { // generate one!
 			return false, errwrap.Wrapf(err, "could not generate password")
 		}
@@ -279,7 +277,7 @@ func (obj *PasswordRes) CheckApply(apply bool) (checkOK bool, err error) {
 			output = password
 		}
 		// write either an empty token, or the password
-		log.Printf("%s[%s]: Writing password token...", obj.Kind(), obj.GetName())
+		log.Printf("%s: Writing password token...", obj)
 		if _, err := obj.write(output); err != nil {
 			return false, errwrap.Wrapf(err, "can't write to file")
 		}
@@ -294,16 +292,11 @@ type PasswordUID struct {
 	name string
 }

-// AutoEdges returns the AutoEdge interface. In this case no autoedges are used.
-func (obj *PasswordRes) AutoEdges() AutoEdge {
-	return nil
-}
-
 // UIDs includes all params to make a unique identification of this object.
 // Most resources only return one, although some resources can return multiple.
 func (obj *PasswordRes) UIDs() []ResUID {
 	x := &PasswordUID{
-		BaseUID: BaseUID{name: obj.GetName(), kind: obj.Kind()},
+		BaseUID: BaseUID{Name: obj.GetName(), Kind: obj.GetKind()},
 		name:    obj.Name,
 	}
 	return []ResUID{x}
@@ -321,32 +314,31 @@ func (obj *PasswordRes) GroupCmp(r Res) bool {
 }

 // Compare two resources and return if they are equivalent.
-func (obj *PasswordRes) Compare(res Res) bool {
-	switch res.(type) {
-	// we can only compare PasswordRes to others of the same resource
-	case *PasswordRes:
-		res := res.(*PasswordRes)
-		if !obj.BaseRes.Compare(res) { // call base Compare
-			return false
-		}
-
-		if obj.Name != res.Name {
-			return false
-		}
-		if obj.Length != res.Length {
-			return false
-		}
-		// TODO: we *could* optimize by allowing CheckApply to move from
-		// saved->!saved, by removing the file, but not likely worth it!
-		if obj.Saved != res.Saved {
-			return false
-		}
-		if obj.CheckRecovery != res.CheckRecovery {
-			return false
-		}
-	default:
+func (obj *PasswordRes) Compare(r Res) bool {
+	// we can only compare PasswordRes to others of the same resource kind
+	res, ok := r.(*PasswordRes)
+	if !ok {
 		return false
 	}
+	if !obj.BaseRes.Compare(res) { // call base Compare
+		return false
+	}
+	if obj.Name != res.Name {
+		return false
+	}
+
+	if obj.Length != res.Length {
+		return false
+	}
+	// TODO: we *could* optimize by allowing CheckApply to move from
+	// saved->!saved, by removing the file, but not likely worth it!
+	if obj.Saved != res.Saved {
+		return false
+	}
+	if obj.CheckRecovery != res.CheckRecovery {
+		return false
+	}
+
 	return true
 }

--- a/resources/pkg.go
+++ b/resources/pkg.go
@@ -3,22 +3,21 @@
 // Written by James Shubin <james@shubin.ca> and the project contributors
 //
 // This program is free software: you can redistribute it and/or modify
-// it under the terms of the GNU Affero General Public License as published by
+// it under the terms of the GNU General Public License as published by
 // the Free Software Foundation, either version 3 of the License, or
 // (at your option) any later version.
 //
 // This program is distributed in the hope that it will be useful,
 // but WITHOUT ANY WARRANTY; without even the implied warranty of
 // MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-// GNU Affero General Public License for more details.
+// GNU General Public License for more details.
 //
-// You should have received a copy of the GNU Affero General Public License
+// You should have received a copy of the GNU General Public License
 // along with this program.  If not, see <http://www.gnu.org/licenses/>.

 package resources

 import (
-	"encoding/gob"
 	"fmt"
 	"log"
 	"path"
@@ -31,7 +30,7 @@ import (
 )

 func init() {
-	gob.Register(&PkgRes{})
+	RegisterResource("pkg", func() Res { return &PkgRes{} })
 }

 // PkgRes is a package resource for packagekit.
@@ -66,36 +65,16 @@ func (obj *PkgRes) Validate() error {

 // Init runs some startup code for this resource.
 func (obj *PkgRes) Init() error {
-	obj.BaseRes.kind = "Pkg"
 	if err := obj.BaseRes.Init(); err != nil { // call base init, b/c we're overriding
 		return err
 	}

-	bus := packagekit.NewBus()
-	if bus == nil {
-		return fmt.Errorf("can't connect to PackageKit bus")
-	}
-	defer bus.Close()
-
-	result, err := obj.pkgMappingHelper(bus)
-	if err != nil {
-		return errwrap.Wrapf(err, "the pkgMappingHelper failed")
+	if obj.fileList == nil {
+		if err := obj.populateFileList(); err != nil {
+			return errwrap.Wrapf(err, "error populating file list in init")
+		}
 	}

-	data, ok := result[obj.Name] // lookup single package (init does just one)
-	// package doesn't exist, this is an error!
-	if !ok || !data.Found {
-		return fmt.Errorf("can't find package named '%s'", obj.Name)
-	}
-
-	packageIDs := []string{data.PackageID} // just one for now
-	filesMap, err := bus.GetFilesByPackageID(packageIDs)
-	if err != nil {
-		return errwrap.Wrapf(err, "can't run GetFilesByPackageID")
-	}
-	if files, ok := filesMap[data.PackageID]; ok {
-		obj.fileList = util.DirifyFileList(files, false)
-	}
 	return nil
 }

@@ -178,9 +157,9 @@ func (obj *PkgRes) getNames() []string {
 // pretty print for header values
 func (obj *PkgRes) fmtNames(names []string) string {
 	if len(obj.GetGroup()) > 0 { // grouped elements
-		return fmt.Sprintf("%s[autogroup:(%v)]", obj.Kind(), strings.Join(names, ","))
+		return fmt.Sprintf("%s[autogroup:(%s)]", obj.GetKind(), strings.Join(names, ","))
 	}
-	return fmt.Sprintf("%s[%s]", obj.Kind(), obj.GetName())
+	return obj.String()
 }

 func (obj *PkgRes) groupMappingHelper() map[string]string {
@@ -189,7 +168,7 @@ func (obj *PkgRes) groupMappingHelper() map[string]string {
 		for _, x := range g {
 			pkg, ok := x.(*PkgRes) // convert from Res
 			if !ok {
-				log.Fatalf("grouped member %v is not a %s", x, obj.Kind())
+				log.Fatalf("grouped member %v is not a %s", x, obj.GetKind())
 			}
 			result[pkg.Name] = pkg.State
 		}
@@ -198,22 +177,22 @@ func (obj *PkgRes) groupMappingHelper() map[string]string {
 }

 func (obj *PkgRes) pkgMappingHelper(bus *packagekit.Conn) (map[string]*packagekit.PkPackageIDActionData, error) {
-	packageMap := obj.groupMappingHelper()   // get the grouped values
-	packageMap[obj.Name] = obj.State         // key is pkg name, value is pkg state
-	var filter uint64                        // initializes at the "zero" value of 0
-	filter += packagekit.PK_FILTER_ENUM_ARCH // always search in our arch (optional!)
+	packageMap := obj.groupMappingHelper() // get the grouped values
+	packageMap[obj.Name] = obj.State       // key is pkg name, value is pkg state
+	var filter uint64                      // initializes at the "zero" value of 0
+	filter += packagekit.PkFilterEnumArch  // always search in our arch (optional!)
 	// we're requesting latest version, or to narrow down install choices!
 	if obj.State == "newest" || obj.State == "installed" {
 		// if we add this, we'll still see older packages if installed
 		// this is an optimization, and is *optional*, this logic is
 		// handled inside of PackagesToPackageIDs now automatically!
-		filter += packagekit.PK_FILTER_ENUM_NEWEST // only search for newest packages
+		filter += packagekit.PkFilterEnumNewest // only search for newest packages
 	}
 	if !obj.AllowNonFree {
-		filter += packagekit.PK_FILTER_ENUM_FREE
+		filter += packagekit.PkFilterEnumFree
 	}
 	if !obj.AllowUnsupported {
-		filter += packagekit.PK_FILTER_ENUM_SUPPORTED
+		filter += packagekit.PkFilterEnumSupported
 	}
 	result, err := bus.PackagesToPackageIDs(packageMap, filter)
 	if err != nil {
@@ -222,6 +201,39 @@ func (obj *PkgRes) pkgMappingHelper(bus *packagekit.Conn) (map[string]*packageki
 	return result, nil
 }

+// populateFileList fills in the fileList structure with what is in the package.
+// TODO: should this work properly if pkg has been autogrouped ?
+func (obj *PkgRes) populateFileList() error {
+
+	bus := packagekit.NewBus()
+	if bus == nil {
+		return fmt.Errorf("can't connect to PackageKit bus")
+	}
+	defer bus.Close()
+
+	result, err := obj.pkgMappingHelper(bus)
+	if err != nil {
+		return errwrap.Wrapf(err, "the pkgMappingHelper failed")
+	}
+
+	data, ok := result[obj.Name] // lookup single package (init does just one)
+	// package doesn't exist, this is an error!
+	if !ok || !data.Found {
+		return fmt.Errorf("can't find package named '%s'", obj.Name)
+	}
+
+	packageIDs := []string{data.PackageID} // just one for now
+	filesMap, err := bus.GetFilesByPackageID(packageIDs)
+	if err != nil {
+		return errwrap.Wrapf(err, "can't run GetFilesByPackageID")
+	}
+	if files, ok := filesMap[data.PackageID]; ok {
+		obj.fileList = util.DirifyFileList(files, false)
+	}
+
+	return nil
+}
+
 // CheckApply checks the resource state and applies the resource if the bool
 // input is true. It returns error info and if the state check passed or not.
 func (obj *PkgRes) CheckApply(apply bool) (checkOK bool, err error) {
@@ -286,7 +298,7 @@ func (obj *PkgRes) CheckApply(apply bool) (checkOK bool, err error) {

 	var transactionFlags uint64 // initializes at the "zero" value of 0
 	if !obj.AllowUntrusted {    // allow
-		transactionFlags += packagekit.PK_TRANSACTION_FLAG_ENUM_ONLY_TRUSTED
+		transactionFlags += packagekit.PkTransactionFlagEnumOnlyTrusted
 	}
 	// apply correct state!
 	log.Printf("%s: Set: %v...", obj.fmtNames(util.StrListIntersection(applyPackages, obj.getNames())), obj.State)
@@ -355,9 +367,9 @@ func (obj *PkgResAutoEdges) Next() []ResUID {
 		var reversed = false // cheat by passing a pointer
 		result = append(result, &FileUID{
 			BaseUID: BaseUID{
-				name:     obj.name,
-				kind:     obj.kind,
-				reversed: &reversed,
+				Name:     obj.name,
+				Kind:     obj.kind,
+				Reversed: &reversed,
 			},
 			path: x, // what matters
 		}) // build list
@@ -418,10 +430,16 @@ func (obj *PkgResAutoEdges) Test(input []bool) bool {

 // AutoEdges produces an object which generates a minimal pkg file optimization
 // sequence of edges.
-func (obj *PkgRes) AutoEdges() AutoEdge {
+func (obj *PkgRes) AutoEdges() (AutoEdge, error) {
 	// in contrast with the FileRes AutoEdges() function which contains
 	// more of the mechanics, most of the AutoEdge mechanics for the PkgRes
-	// is contained in the Test() method! This design is completely okay!
+	// are contained in the Test() method! This design is completely okay!
+
+	if obj.fileList == nil {
+		if err := obj.populateFileList(); err != nil {
+			return nil, errwrap.Wrapf(err, "error populating file list for automatic edges")
+		}
+	}

 	// add matches for any svc resources found in pkg definition!
 	var svcUIDs []ResUID
@@ -429,9 +447,9 @@ func (obj *PkgRes) AutoEdges() AutoEdge {
 		var reversed = false
 		svcUIDs = append(svcUIDs, &SvcUID{
 			BaseUID: BaseUID{
-				name:     obj.GetName(),
-				kind:     obj.Kind(),
-				reversed: &reversed,
+				Name:     obj.GetName(),
+				Kind:     obj.GetKind(),
+				Reversed: &reversed,
 			},
 			name: x, // the svc name itself in the SvcUID object!
 		}) // build list
@@ -442,15 +460,15 @@ func (obj *PkgRes) AutoEdges() AutoEdge {
 		svcUIDs:    svcUIDs,
 		testIsNext: false,         // start with Next() call
 		name:       obj.GetName(), // save data for PkgResAutoEdges obj
-		kind:       obj.Kind(),
-	}
+		kind:       obj.GetKind(),
+	}, nil
 }

 // UIDs includes all params to make a unique identification of this object.
 // Most resources only return one, although some resources can return multiple.
 func (obj *PkgRes) UIDs() []ResUID {
 	x := &PkgUID{
-		BaseUID: BaseUID{name: obj.GetName(), kind: obj.Kind()},
+		BaseUID: BaseUID{Name: obj.GetName(), Kind: obj.GetKind()},
 		name:    obj.Name,
 		state:   obj.State,
 	}
@@ -481,52 +499,33 @@ func (obj *PkgRes) GroupCmp(r Res) bool {
 }

 // Compare two resources and return if they are equivalent.
-func (obj *PkgRes) Compare(res Res) bool {
-	switch res.(type) {
-	case *PkgRes:
-		res := res.(*PkgRes)
-		if !obj.BaseRes.Compare(res) { // call base Compare
-			return false
-		}
-
-		if obj.Name != res.Name {
-			return false
-		}
-		if obj.State != res.State {
-			return false
-		}
-		if obj.AllowUntrusted != res.AllowUntrusted {
-			return false
-		}
-		if obj.AllowNonFree != res.AllowNonFree {
-			return false
-		}
-		if obj.AllowUnsupported != res.AllowUnsupported {
-			return false
-		}
-	default:
+func (obj *PkgRes) Compare(r Res) bool {
+	// we can only compare PkgRes to others of the same resource kind
+	res, ok := r.(*PkgRes)
+	if !ok {
 		return false
 	}
-	return true
-}
-
-// ReturnSvcInFileList returns a list of svc names for matches like: `/usr/lib/systemd/system/*.service`.
-func ReturnSvcInFileList(fileList []string) []string {
-	result := []string{}
-	for _, x := range fileList {
-		dirname, basename := path.Split(path.Clean(x))
-		// TODO: do we also want to look for /etc/systemd/system/ ?
-		if dirname != "/usr/lib/systemd/system/" {
-			continue
-		}
-		if !strings.HasSuffix(basename, ".service") {
-			continue
-		}
-		if s := strings.TrimSuffix(basename, ".service"); !util.StrInList(s, result) {
-			result = append(result, s)
-		}
+	if !obj.BaseRes.Compare(res) { // call base Compare
+		return false
 	}
-	return result
+	if obj.Name != res.Name {
+		return false
+	}
+
+	if obj.State != res.State {
+		return false
+	}
+	if obj.AllowUntrusted != res.AllowUntrusted {
+		return false
+	}
+	if obj.AllowNonFree != res.AllowNonFree {
+		return false
+	}
+	if obj.AllowUnsupported != res.AllowUnsupported {
+		return false
+	}
+
+	return true
 }

 // UnmarshalYAML is the custom unmarshal handler for this struct.
@@ -548,3 +547,22 @@ func (obj *PkgRes) UnmarshalYAML(unmarshal func(interface{}) error) error {
 	*obj = PkgRes(raw) // restore from indirection with type conversion!
 	return nil
 }
+
+// ReturnSvcInFileList returns a list of svc names for matches like: `/usr/lib/systemd/system/*.service`.
+func ReturnSvcInFileList(fileList []string) []string {
+	result := []string{}
+	for _, x := range fileList {
+		dirname, basename := path.Split(path.Clean(x))
+		// TODO: do we also want to look for /etc/systemd/system/ ?
+		if dirname != "/usr/lib/systemd/system/" {
+			continue
+		}
+		if !strings.HasSuffix(basename, ".service") {
+			continue
+		}
+		if s := strings.TrimSuffix(basename, ".service"); !util.StrInList(s, result) {
+			result = append(result, s)
+		}
+	}
+	return result
+}
--- a/resources/pkg_test.go
+++ b/resources/pkg_test.go
@@ -0,0 +1,33 @@
+// Mgmt
+// Copyright (C) 2013-2017+ James Shubin and the project contributors
+// Written by James Shubin <james@shubin.ca> and the project contributors
+//
+// This program is free software: you can redistribute it and/or modify
+// it under the terms of the GNU General Public License as published by
+// the Free Software Foundation, either version 3 of the License, or
+// (at your option) any later version.
+//
+// This program is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+// GNU General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License
+// along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+package resources
+
+import (
+	"testing"
+)
+
+func TestNilList1(t *testing.T) {
+	var x []string
+	if x != nil { // we have this expectation for obj.fileList in pkg
+		t.Errorf("list should have been nil, was: %+v", x)
+	}
+	x = []string{} // empty list
+	if x == nil {
+		t.Errorf("list should have been empty, was: %+v", x)
+	}
+}
--- a/Show More
+++ b/Show More