Commit Graph

285 Commits

Author SHA1 Message Date
Felix Frank
e0107b1dda have 'make clean' get rid of generated code files again 2016-09-27 05:20:56 -04:00
James Shubin
ccc00f913d Resources: Update interface to support errors
This was an early interface mistake I made and is now corrected.
We'll plumb in checking the error state of Init() and running Validate()
later on.
2016-09-27 05:16:37 -04:00
James Shubin
ad3c6bdc88 TODO: add TODO item about DHT idea 2016-09-27 05:07:34 -04:00
James Shubin
8fe3891ea9 Makefile: Limit depth for yamlfmt 2016-09-27 05:06:43 -04:00
James Shubin
63f21952f4 golang: Split things into packages
This makes this logically more separate! :) As an aside...

I really hate the way golang does dependencies and packages. Yes, some
people insist on nesting their code deep into a $GOPATH, which is fine
if you're a google dev and are forced to work this way, but annoying for
the rest of the world. Your code shouldn't need a git commit to switch
to a a different vcs host! Gah I hate this so much.
2016-09-26 12:30:28 -04:00
James Shubin
361d643ce7 resources: Compare Delay and Retry metaparams on graph change
We forgot to add a number of important properties to the Compare. This
means that we'll rebuild elements for these changes too now.
2016-09-20 05:35:04 -04:00
Antoine Racine
abe1ffaab6 Add pcap.h dependency on Debian/Ubuntu to fix make deps error 2016-09-19 22:51:10 -04:00
James Shubin
fc24c91dde Resources: Add retry and retry delay meta parameters
All resources can now set a retry limit (-1 for infinite) and a delay
between retries. This applies to both the CheckApply methods, and the
Watch methods as well. They each have their own separate counts, but use
the same input meta param, since I decided it wouldn't be useful to have
a separate watchRetry and watchDelay set of meta parameters.

In the process, we got rid of about 15 error cases which would normally
panic.

This patch required a slight overhaul of the Event system.

The previous commit is an earlier version of this patch which I decided
to leave in to "show my work" as I used to have to do in math class.
It's slightly more correct with the current event system, and this
version is less correct and has a few bugs, but that is because the
event system needs a massive overhaul, and once that's done this should
all work properly for the corner cases.
2016-09-19 06:32:21 -04:00
James Shubin
53cabd5ee4 Resources: Prototype retry and retry delay meta parameters
This was the initial cut of the retry and delay meta parameters.
Instead, I decided to move the delay action into the common space
outside of the Watch resource. This is more complicated in the short
term, but will be more beneficial in the long run as each resource won't
have to implement this part itself (even if it uses boiler plate).

This is the first version of this patch without this fix. I decided to
include it because I think it has more correct event processing.
2016-09-19 06:32:21 -04:00
James Shubin
2b1e8cdbee travis: bump to newer golang version (fedora 23) 2016-09-19 03:39:09 -04:00
Michaël Faille
9715146495 travis : use go1.7 to be future proof.
And, to reflect my own config `;-)`
2016-09-19 03:37:55 -04:00
Michaël Faille
22b0b89949 Let env find bash in shebang.
On Nixos and GNUIX-SD, bash is chroot in package store path.
In my case : /nix/store/qvccmr6fsis4kqlvlk8pb1c8c0r0cwai-system-path/bin/bash

In any case, using `/usr/bin/env bash` is the recommended way to get bash
portable across UNIX-like systems.
2016-09-19 03:36:38 -04:00
Michaël Faille
2ebc23a777 cli : Prefer github.com/urfave/cli as cli
Over  github.com/codegangsta/cli
Note : I just adapt the README.md and dependency manager
2016-09-19 02:26:28 -04:00
James Shubin
0199285319 readme: Add information for new users 2016-09-18 05:59:46 -04:00
James Shubin
277ab2fe44 readme: We now have an arch aur build.
Thanks Joe Julian!
2016-09-18 05:52:48 -04:00
James Shubin
8a96dfdc8a art: Add a logo
Sarah Jane Cox made us a logo. Thanks SJ!
2016-09-18 05:47:33 -04:00
James Shubin
66fbbb940a test: temporarily disable test 2016-09-18 03:46:42 -04:00
James Shubin
716ea1bb3c etcd: Update examples to be more useful for single machine tests 2016-09-14 23:00:59 -04:00
James Shubin
3d701d3daa todo: Update TODO file 2016-09-12 02:01:44 -04:00
James Shubin
598c74657c file: Overhaul file resource and add recursion
The file resource contained some of the early golang code that I wrote
for this project. Needless to say, some of it was quite yucky, and it
was also lacking a number of important features. This patch builds upon
it so that it starts being usable for directories of files too.

Many thanks to Sam Gélineau for helping with the recursive watching. My
brain officially didn't want to look at that code anymore.
2016-09-12 01:55:31 -04:00
Felix Frank
4bd53d5ab0 add puppet support documentation 2016-09-12 02:40:09 +02:00
James Shubin
70f8d54a31 thanks: More people really needed to be on the list 0.0.4 2016-09-07 19:41:39 -04:00
James Shubin
4ef25a33fc docs: Add an FAQ entry about difference between two similar methods
Felix wins the points for first asking the question I knew would
eventually come but didn't document earlier.
2016-09-07 19:31:13 -04:00
Felix Frank
f5dd90a8dd add unit tests for UUID comparison and resource event passing 2016-09-08 01:12:56 +02:00
James Shubin
a84defd689 readme: Add new blog posts by Felix
The last post here was one of my favourites! Super impressive stuff!
2016-09-07 01:08:21 -04:00
James Shubin
1cf88d9540 test: Increase timeouts of t8
Increase the timeouts in the rare chance that this is slow performing
travis, and not just an etcd regression.
2016-09-02 02:27:39 -04:00
James Shubin
644a0ee8c8 puppet: golint fixes 2016-09-02 02:25:41 -04:00
James Shubin
e9d5dc8fee packagekit: golint fixes 2016-09-02 02:23:13 -04:00
James Shubin
8003202beb resources: golint fixes 2016-09-02 02:03:26 -04:00
James Shubin
b46432b5b6 misc: Golint fixes 2016-09-02 01:46:45 -04:00
James Shubin
5e3f03df06 etcd: Catch possible raft grpc error 2016-09-02 00:35:05 -04:00
James Shubin
8ab8e6679a test: provider usage text for shell test runner 2016-09-01 22:52:32 -04:00
James Shubin
786b896018 remote: small cleanups to update misc notes 2016-08-31 22:56:00 -04:00
James Shubin
40723f8705 docs: Add additional documentation about remote execution 2016-08-31 22:42:09 -04:00
James Shubin
2a0721bddf remote: allow converge during corner cases
This allows the system to converge during corner cases where there is an
error, or when there are no remotes being used, but we are using the
--no-watch variable.

I deliberately left this in as a separate commit instead of rebasing
into the remote execution development branch because the placement of
the Unregister() and semaphore.V(1) were quite subtle and easy to forget
about.
2016-08-31 22:42:09 -04:00
James Shubin
ff01e4a5e7 remote: Add distributed converged timeout
This patch extends the --converged-timeout argument so that when used
with --remote it waits for the entire set of remote mgmt agents to
converge simultaneously before exiting.

purpleidea says: This particular part of the patch probably took as much
work as all of the work required for the initial remote patches alone!
2016-08-31 21:55:19 -04:00
James Shubin
6794aff77c miscellaneous cleanups and fixes 2016-08-31 21:55:19 -04:00
James Shubin
636f2a36b1 etcd: Use new converged timers and allow skipping them
This implements the new extensions to the converged UUID API so that we
can keep a consistent timer running and reset it when needed. This is
useful because we now allow certain Watcher callbacks and other
operations to explicitly _not_ cause a convergerUUID to un-converge.
This is a necessary dependency for the distributed remote
converged-timeout work.
2016-08-31 21:55:19 -04:00
James Shubin
eee652cefe converger: Add new timer system for determining convergence
This adds a new method of marking whether a particular UUID has
converged or not. You can now Start, Stop, or Reset a convergence timer
on the individual UUID's. This wraps the existing SetConverged calls
with a hidden go routine. It is not recommended to use the SetConverged
calls and the Timer calls on the same UUID.
2016-08-31 21:55:19 -04:00
James Shubin
6d45cd45d1 converger: Add new methods to the API
This adds new helper methods to the API such as the ability to query the
set timeout, and to set the state change function after initialization.
The first makes it easier to pass in the timeout value to functions and
structs, because you don't need to pass it separately from the converger
object. The second makes it easy to specify the state change functon
when you don't know what it is at creation time. In addition, it is more
powerful now, and tells you when we converge or un-converge in case we
want to take different actions on each.
2016-08-31 21:55:19 -04:00
James Shubin
f5fb135793 converger: Update the API for errors and naming
We updated the API and behaviour to make two changes:

1) We remove the log.* stuff, and replace the Fatal calls with straight
calls to panic(). These are meant to be like an assert, and shouldn't
happen unless there is a user error or a bug.

2) We made the !uuid.IsValid() checks return an error instead of causing
a panic. These returns errors instead, and makes the process safer for
users who are okay calling a function that won't have an effect.
2016-08-31 21:55:19 -04:00
James Shubin
6bf32c978a etcd: Rename loop to be more consistent in messages
Small nitpick fixups
2016-08-31 21:55:19 -04:00
James Shubin
8d3011fb9c etcd: Add a timeout for etcd server to start correctly
This also updates etcd to a newer version with a fix that allows this
detection and timeout operation to be possible.
2016-08-31 21:55:19 -04:00
James Shubin
9260066fa3 tests: Workaround regression in two host etcd clusters
If you don't give your two host cluster enough time to "feel healthy",
it will generate an error if you do operations within five seconds. This
is a regression and the five seconds is also quite arbitrary. This is
detailed at: https://github.com/coreos/etcd/issues/6305

This seems to be a bit of a race condition, even with a 10s timer, so
this also disables the StrictReconfigCheck. Re-enable this as soon as
possible.
2016-08-31 21:55:19 -04:00
James Shubin
5e45c5805b Improve internal etcd error handling 2016-08-31 21:55:19 -04:00
James Shubin
db4de12767 Add more flexibility around the prefixes available
This allows you to specify a custom prefix, or a tmp prefix which is
chosen automatically.
2016-08-31 21:55:19 -04:00
James Shubin
d429795737 Improve the configWatcher array to allow N files
This simplifies the code in main and hides it in the watcher!
2016-08-31 21:55:19 -04:00
James Shubin
276219a691 Run tests with a tmp prefix
This will avoid failures if /var/lib/mgmt/ isn't writable.
2016-08-31 21:55:19 -04:00
James Shubin
03c1df98f4 Improve prefix creation and feedback
This makes the default /var/lib/mgmt/ directory, but also allows you to
use a prefix in /tmp/ automatically if you can't write anywhere else.
2016-08-31 21:55:19 -04:00
James Shubin
79ba750dd5 Automatically update remote files on change
This extends the automatic watching of graph definition files across the
remote SSH boundary.
2016-08-31 21:55:19 -04:00