This is a monster patch that finally gets the iterative pkg auto edges
working the way they should. For each file, as soon as one matches, we
don't want to keep add dependencies on other file objects under that
tree structure. This reduces the number of necessary edges considerably,
and allows the graph to run more concurrently.
This allows for resources to automatically add necessary edges to the
graph so that the event system doesn't have to work overtime due to
sub-optimal execution order.
This is based on PackageKit, which means events, *and* we automatically
get support for any of the backends that PackageKit supports. This means
dpkg, and rpm are both first class citizens! Many other backends will
surely work, although thorough testing is left as an exercise to the
reader, or to someone who would like to write more test cases!
Unfortunately at the moment, there are a few upstream PackageKit bugs
which cause us issues, but those have been apparently resolved upstream.
If you experience issues with an old version of PackageKit, test if it
is working correctly before blaming mgmt :)
In parallel, mgmt might increase the testing surface for PackageKit, so
hopefully this makes it more robust for everyone involved!
Lastly, I'd like to point out that many great things that are typically
used for servers do start in the GNOME desktop world. Help support your
GNOME GNU/Linux desktop today!
Naming the resources "type" was a stupid mistake, and is a huge source
of confusion when also talking about real types. Fix this before it gets
out of hand.
* Fixup graph state readability
* Rename original SetState() to SetConvergedState() and friends...
* Add type state management for proper BackPoke() commands...
* Add better DEBUG logging
This is an important optimization that prevents running a BackPoke on a
parent which is in the process of running and will most certainly poke
the caller back in a moment. This avoids unnecessary roundtrips.
Unfortunately, there are still other algorithms required so that races
can't cause the graph to run for longer than necessary.
* Fix Process() object calling
* Add PokeParent() to poke upwards
* Break linear exec chains :(
This was the issue where in a graph f1 -> f2, if you were to rm f2 &&
cat f2, then f2 would not come back because we didn't poke upwards to
refresh the timestamp. Unfortunately this adds another bug which we
solve in a later patch.
* Add exec type
* Switch erroneous use of fmt to log instead
* Check for edge existence for safety before using
* Avoid recalling etcd channel maker
* Clean up logging output
This is the third main feature of this system. The code needs a bunch of
polish, but it actually all works :)
I've tested this briefly with N <= 3.
Currently you have to build your own etcd cluster. It's quite easy, just
run `etcd` and it will be ready. I usually run it in a throw away /tmp/
dir so that I can blow away the stored data easily.
This is still a dirty prototype, so please excuse the mess. Please
excuse the fact that this is a mega patch. Once things settle down this
won't happen any more.
Some of the changes squashed into here include:
* Merge vertex loop with type loop
(The file watcher seems to cache events anyways)
* Improve pgraph library
* Add indegree, outdegree, and topological sort with tests
* Add reverse function for vertex list
* Tons of additional cleanup!
Amazingly, on my first successful compile, this seemed to run!
A special thanks to Ira Cooper who helped me talk through some of the
algorithmic decisions and for his help in finding better ones!
This is a prototype that i'm attempting to "release early". Expect a lot
of changes! It is intended to be a config management tool that will:
* be event based
* execute actions in parallel
* function as a distributed system
There are a bunch more design ideas going into this, please stay tuned!
