Commit Graph

702 Commits

Author SHA1 Message Date
Jonathan Gold
032d0992d6 resources: aws: ec2: Refactor CheckApply
CheckApply was rewritten, using the new describe methods to improve
readability and maintainability.
2018-01-03 21:34:28 -05:00
jonathangold
67837a47ac resources: aws: ec2: Refactor longpollWatch
Complete rewrite of longpollWatch() for correctness and maintanability.
2018-01-03 21:34:28 -05:00
Jonathan Gold
32e3c4e029 resources: aws: ec2: Refactor longpollWatch
This patch simplifies longpollwatch by getting rid of some unnecessary
api calls and breaking the waiters out into their own functions.
2018-01-03 21:34:28 -05:00
Jonathan Gold
76fcb7a06e resources: aws: ec2: Wait for stop and terminate concurrently
In longpollWatch it was no longer sufficient to use only
WaitUntilInstanceStopped as it would block if the instance was
terminated. This patch launches two goroutines in its place, one
waits until the instance stops and the other waits until it
terminates. When either one returns, it cancels their context,
and execution continues.
2018-01-03 21:34:28 -05:00
Jonathan Gold
149a2188e2 resources: aws: ec2: Retry on exceeded wait attempts error
The waiters now return the AwsErr error "ResourceNotReady: exceeded wait
attempts" when the instance state does not converge after 40 retries.
During longpollWatch() we need to detect this error and continue to
the top of the loop so we can restart the waiters and keep watching for
events.
2018-01-03 21:34:28 -05:00
Jonathan Gold
08e7caea6b resources: aws: ec2: CheckApply fix pending and stopping cases
If CheckApply was called when the instance was pending or stopping, it
would return an error. This patch supresses these errors and tells the
engine that the state can't yet be changed.
2018-01-03 21:34:28 -05:00
Jonathan Gold
e330ebc8c9 resources: aws: ec2: Verify SNS message signatures 2018-01-03 21:34:28 -05:00
Jonathan Gold
388a08e13a resources: aws: ec2: Check that policy.Statement != nil 2018-01-03 21:34:28 -05:00
Jonathan Gold
9ba9ef1cbf resources: aws: ec2: Close closeChan before server shutdown
This patch makes sure that closeChan is closed as soon as the main loop
returns, so any channel operations are unblocked before we run shutdown.
This ensures that the server's goroutine can return before shutdown
completes and we don't panic by trying to serve the client after
shutdown returns.
2018-01-03 21:34:27 -05:00
Jonathan Gold
fac004b774 resources: aws: ec2: Update postHandler to process messages 2018-01-03 21:34:27 -05:00
Jonathan Gold
8cd3f28734 resources: aws: ec2: Authorize CloudWatch to publish to sns 2018-01-03 21:34:27 -05:00
Jonathan Gold
dcd23fcf75 resources: aws: ec2: Add CloudWatch rule and target SNS
This patch creates the cloudwatch rule that detects ec2 instance
state changes, and targets the rule to publish on our sns topic
which, in turn, pushes those event notifications to our endpoint.
2018-01-03 21:34:27 -05:00
Jonathan Gold
1162485c2c resources: aws: ec2: Subscribe SNS endpoint to topic
This patch adds methods to subscribe and confirm the subscription
to the sns topic.
2018-01-03 21:34:27 -05:00
Jonathan Gold
966172eac6 resources: aws: ec2: Use custom listener for snsServer
This patch replaces the call to Server.ListenAndServe() with
Server.Serve(listener) in order to make sure the listener is up
and running before we subscribe to the topic in a future patch.
2018-01-03 21:34:27 -05:00
James Shubin
12fce52cd7 legal: Happy 2018 everyone...
Done with:

ack '2017+' -l | xargs sed -i -e 's/2017+/2018+/g'

Checked manually with:

git add -p

Hello to future James from 2019, and Happy Hacking!
2018-01-03 21:22:07 -05:00
Felix Frank
5ca1e2a23f puppet: Avoid empty parameters to puppet mgmtgraph
This solves an issue first observed with golang 1.8.

Creating an exec.Command with an empty string parameter (when no puppet.conf
file is specified) would lead to an error from Puppet, stating that an
unexpected argument was passed to "puppet mgmtgraph print".

The workaround is to not include *any* positional argument (not even the
empty string) when --puppet-conf is not used.
2017-12-26 00:18:46 +01:00
Paul Morgan
98f8a61e83 git: Configure editorconfig to indent with tabs in bash scripts
This follows `test/test-bashfmt.sh` style check(s).
2017-12-20 21:09:15 +00:00
Paul Morgan
2e86d7c5ab git: Ensure the tagging script is idempotent 2017-12-20 21:04:57 +00:00
Jonathan Gold
62ca12608d cli: Add license flag
This patch adds the option to print the license with a cli flag. It
uses go-bindata to store the license file. The file is generated by
running `make bindata` and the result is stored in the bindata
directory.
2017-12-08 00:57:58 -05:00
Jonathan Gold
406aa55667 resources: virt: Update libvirt-xml target
Builds started failing due to go-libvirt-xml 6d97448. In that patch,
the DomainChannelTarget struct was changed from having a single type
field, to having an individual field for each virtualization type.

This patch updates the connection check in Init to reflect the changes
to go-libvirt-xml, so that builds no longer fail.
2017-11-29 19:03:56 -05:00
James Shubin
a76dce8b15 docs: Add missing blog post about augeas resource 2017-11-26 17:15:49 -05:00
James Shubin
b01d453ae3 docs: Refresh documentation to provide a better new user experience
This does some cleanups and moves some things around for a better
experience. If you're an expert in this area, or are a new user who has
some feedback about their first impressions and experiences, please let
us know!
2017-11-25 20:45:57 -05:00
Guillaume Herail
ac629404f4 test: Switch to goimports instead of gofmt
see https://github.com/purpleidea/mgmt/pull/256#issuecomment-346360414
2017-11-25 06:49:00 -05:00
Guillaume Herail
3575d597f7 resources: Add User/Group to ExecRes 2017-11-24 10:38:16 -05:00
Toshaan Bharvani
2affcba3b4 build: Added build option to strip binary
This is a build option in Golang that will strip the binary.
The binary becomes about 50% smaller.

Signed-off-by: Toshaan Bharvani <toshaan@vantosh.com>
2017-11-24 10:26:48 -05:00
James Shubin
846c5f8762 test: Add another check for off-by-one-error commit tags 2017-11-24 09:46:32 -05:00
Julien Pivotto
086af712d2 example: Remove content out of directory definition
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2017-11-24 14:26:20 +01:00
Julien Pivotto
2b6e39f283 build: Remove go 1.3 and 1.4 support
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2017-11-24 05:35:09 -05:00
Julien Pivotto
472663193a prometheus: Initialize all metrics
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2017-11-24 11:02:36 +01:00
James Shubin
879ff838ae resources: Replace golang 1.6 specific code with newer 1.7 version
We now require at least 1.8 so we might as well fix this up.
2017-11-23 10:57:11 -05:00
Julien Pivotto
5e9a085e39 exec: Add autoEdges between ExecRes and PkgRes
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2017-11-23 16:30:22 +01:00
Julien Pivotto
c2b5729ebd build: Build mgmt on any go file change
Prior to this commit, running make would only rebuild mgmt when
main.go was changed. It means that make clean build was needed.

With this commit, any go file change in this directory will
trigger a new compilation.

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2017-11-23 09:32:02 -05:00
Julien Pivotto
fdce9d6a6a prometheus: Initialize mgmt_checkapply_total metrics
It is recommended by Prometheus to initialize metrics:

https://prometheus.io/docs/practices/instrumentation/#avoid-missing-metrics

This commits initialize the mgmt_checkapply_total metric
for each registered resource.

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2017-11-23 15:23:41 +01:00
Guillaume Herail
bfc2549289 resources: Move FileRes.uid()/.gid() to util.go 2017-11-23 08:34:38 -05:00
James Shubin
52fd1ae73e test: Add check for common doc vs docs ambiguity 2017-11-23 08:20:44 -05:00
Julien Pivotto
23e167616f doc: Fix link to the prometheus wiki
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2017-11-23 09:52:28 +01:00
James Shubin
51ce83f20b test: Add extra commit message tests for some common mistakes
Feel free to add more if we identify them.
2017-11-21 11:05:20 -05:00
Jonathan Gold
5e5bbf4b39 travis: Allow travis builds to access target branches
Because travis builds only fetch a single branch (master) by default,
test-commit-message.sh only had access to commits in the master branch.
In order to fetch the correct branch for our build, we need to run
'git config remote.origin.fetch..' with the target branch's information
before executing git fetch on the repo in before_install.

Now git will always fetch the appropriate branch.
2017-11-18 21:04:12 -05:00
Guillaume Herail
cbc3a691b9 docker: Bump to golang 1.8 2017-11-16 17:36:35 +01:00
Jonathan Gold
a5247d6e69 resources: aws: ec2: Change event messages to iota consts 2017-11-14 16:48:51 -05:00
Jonathan Gold
d698b82a83 resources: aws: ec2: Start and stop SNS endpoint in snsWatch
This patch adds snsWatch which launches the HTTP server and listens
for messages on awsChan to forward as events to the mgmt engine.
2017-11-11 23:07:12 -05:00
Jonathan Gold
91eff75288 resources: aws: ec2: Add method to make sns topic 2017-11-10 17:31:19 -05:00
James Shubin
91a9edb322 resources: aws: ec2: Fix deadlock on rare error scenarios
If we get an error in the Watch loop, it will send this on awsChan,
which will cause Watch to loop. However, in this scenario it will never
cause closeChan to close, and we will deadlock because we have a
waitGroup in a helper goroutine which is waiting on this channel to
close the context.

Normally this wouldn't be an issue, but since we have more than one
goroutine (with associated waitGroup) it is. It's also good practice to
close all the channels to help avoid this kind of bug.

This patch also moves the waitGroup Wait into a more logical place for
visibility.
2017-11-10 14:17:54 -05:00
Jonathan Gold
c8ddbeaa5c resource: aws: ec2: Add http server 2017-11-09 13:13:42 -05:00
Jonathan Gold
3634b3450d resource: aws: ec2: Move waitgroup to resource struct 2017-11-08 16:57:41 -05:00
Jonathan Gold
c2a5e3f5d8 resources: aws: ec2: Move watch channels into struct 2017-11-08 16:16:01 -05:00
Jonathan Gold
db49fe85e4 resources: aws: ec2: Move chanStruct type out of longpollWatch 2017-11-08 16:08:25 -05:00
Jonathan Gold
567a2e9fd8 resources: aws: ec2: Reorganized consts 2017-11-08 16:02:29 -05:00
Jonathan Gold
987de00e17 resources: aws: ec2: Remove extra wait from Watch
There were two calls to WaitUntilInstanceTerminatedWithContext in a row.
There's no reason to make the call twice.
2017-11-08 16:02:24 -05:00
Jonathan Gold
baeafec74a resources: aws: ec2: Move Watch to longpollWatch 2017-11-08 16:02:12 -05:00