Commit Graph

203 Commits

Author SHA1 Message Date
Johan Bloemberg
8fb0373f82 resources: Do not return GID for UID lookup
On linux it is convention for users to have a group with the same GID as the users UID. On macOS this is not the case. This broke the test which lead to discovering this bug.
2018-02-10 05:01:12 -05:00
Carsten Thiel
eb33a5a5df docs: Improve file resource documentation
Info on how to create a directory.
Explain more parameter options.
2018-02-07 14:19:14 +01:00
James Shubin
b19583e7d3 lang: Initial implementation of the mgmt language
This is an initial implementation of the mgmt language. It is a
declarative (immutable) functional, reactive, domain specific
programming language. It is intended to be a language that is:

* safe
* powerful
* easy to reason about

With these properties, we hope this language, and the mgmt engine will
allow you to model the real-time systems that you'd like to automate.

This also includes a number of other associated changes. Sorry for the
large size of this patch.
2018-01-20 08:09:29 -05:00
jonathangold
3c677543e0 resources: aws: ec2: Fix closed channel handling
If awschan closes, longpollWatch and snsWatch return nil
instead of an error. This will prevent the engine from
shutting down in case we choose to close the channel
early or from other struct methods.
2018-01-06 15:15:30 -05:00
jonathangold
c455ef2c62 resources: aws: ec2: Send IP addresses and InstanceID 2018-01-03 21:34:28 -05:00
Jonathan Gold
032d0992d6 resources: aws: ec2: Refactor CheckApply
CheckApply was rewritten, using the new describe methods to improve
readability and maintainability.
2018-01-03 21:34:28 -05:00
jonathangold
67837a47ac resources: aws: ec2: Refactor longpollWatch
Complete rewrite of longpollWatch() for correctness and maintanability.
2018-01-03 21:34:28 -05:00
Jonathan Gold
32e3c4e029 resources: aws: ec2: Refactor longpollWatch
This patch simplifies longpollwatch by getting rid of some unnecessary
api calls and breaking the waiters out into their own functions.
2018-01-03 21:34:28 -05:00
Jonathan Gold
76fcb7a06e resources: aws: ec2: Wait for stop and terminate concurrently
In longpollWatch it was no longer sufficient to use only
WaitUntilInstanceStopped as it would block if the instance was
terminated. This patch launches two goroutines in its place, one
waits until the instance stops and the other waits until it
terminates. When either one returns, it cancels their context,
and execution continues.
2018-01-03 21:34:28 -05:00
Jonathan Gold
149a2188e2 resources: aws: ec2: Retry on exceeded wait attempts error
The waiters now return the AwsErr error "ResourceNotReady: exceeded wait
attempts" when the instance state does not converge after 40 retries.
During longpollWatch() we need to detect this error and continue to
the top of the loop so we can restart the waiters and keep watching for
events.
2018-01-03 21:34:28 -05:00
Jonathan Gold
08e7caea6b resources: aws: ec2: CheckApply fix pending and stopping cases
If CheckApply was called when the instance was pending or stopping, it
would return an error. This patch supresses these errors and tells the
engine that the state can't yet be changed.
2018-01-03 21:34:28 -05:00
Jonathan Gold
e330ebc8c9 resources: aws: ec2: Verify SNS message signatures 2018-01-03 21:34:28 -05:00
Jonathan Gold
388a08e13a resources: aws: ec2: Check that policy.Statement != nil 2018-01-03 21:34:28 -05:00
Jonathan Gold
9ba9ef1cbf resources: aws: ec2: Close closeChan before server shutdown
This patch makes sure that closeChan is closed as soon as the main loop
returns, so any channel operations are unblocked before we run shutdown.
This ensures that the server's goroutine can return before shutdown
completes and we don't panic by trying to serve the client after
shutdown returns.
2018-01-03 21:34:27 -05:00
Jonathan Gold
fac004b774 resources: aws: ec2: Update postHandler to process messages 2018-01-03 21:34:27 -05:00
Jonathan Gold
8cd3f28734 resources: aws: ec2: Authorize CloudWatch to publish to sns 2018-01-03 21:34:27 -05:00
Jonathan Gold
dcd23fcf75 resources: aws: ec2: Add CloudWatch rule and target SNS
This patch creates the cloudwatch rule that detects ec2 instance
state changes, and targets the rule to publish on our sns topic
which, in turn, pushes those event notifications to our endpoint.
2018-01-03 21:34:27 -05:00
Jonathan Gold
1162485c2c resources: aws: ec2: Subscribe SNS endpoint to topic
This patch adds methods to subscribe and confirm the subscription
to the sns topic.
2018-01-03 21:34:27 -05:00
Jonathan Gold
966172eac6 resources: aws: ec2: Use custom listener for snsServer
This patch replaces the call to Server.ListenAndServe() with
Server.Serve(listener) in order to make sure the listener is up
and running before we subscribe to the topic in a future patch.
2018-01-03 21:34:27 -05:00
James Shubin
12fce52cd7 legal: Happy 2018 everyone...
Done with:

ack '2017+' -l | xargs sed -i -e 's/2017+/2018+/g'

Checked manually with:

git add -p

Hello to future James from 2019, and Happy Hacking!
2018-01-03 21:22:07 -05:00
Jonathan Gold
406aa55667 resources: virt: Update libvirt-xml target
Builds started failing due to go-libvirt-xml 6d97448. In that patch,
the DomainChannelTarget struct was changed from having a single type
field, to having an individual field for each virtualization type.

This patch updates the connection check in Init to reflect the changes
to go-libvirt-xml, so that builds no longer fail.
2017-11-29 19:03:56 -05:00
Guillaume Herail
ac629404f4 test: Switch to goimports instead of gofmt
see https://github.com/purpleidea/mgmt/pull/256#issuecomment-346360414
2017-11-25 06:49:00 -05:00
Guillaume Herail
3575d597f7 resources: Add User/Group to ExecRes 2017-11-24 10:38:16 -05:00
James Shubin
879ff838ae resources: Replace golang 1.6 specific code with newer 1.7 version
We now require at least 1.8 so we might as well fix this up.
2017-11-23 10:57:11 -05:00
Julien Pivotto
5e9a085e39 exec: Add autoEdges between ExecRes and PkgRes
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2017-11-23 16:30:22 +01:00
Julien Pivotto
fdce9d6a6a prometheus: Initialize mgmt_checkapply_total metrics
It is recommended by Prometheus to initialize metrics:

https://prometheus.io/docs/practices/instrumentation/#avoid-missing-metrics

This commits initialize the mgmt_checkapply_total metric
for each registered resource.

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2017-11-23 15:23:41 +01:00
Guillaume Herail
bfc2549289 resources: Move FileRes.uid()/.gid() to util.go 2017-11-23 08:34:38 -05:00
Jonathan Gold
a5247d6e69 resources: aws: ec2: Change event messages to iota consts 2017-11-14 16:48:51 -05:00
Jonathan Gold
d698b82a83 resources: aws: ec2: Start and stop SNS endpoint in snsWatch
This patch adds snsWatch which launches the HTTP server and listens
for messages on awsChan to forward as events to the mgmt engine.
2017-11-11 23:07:12 -05:00
Jonathan Gold
91eff75288 resources: aws: ec2: Add method to make sns topic 2017-11-10 17:31:19 -05:00
James Shubin
91a9edb322 resources: aws: ec2: Fix deadlock on rare error scenarios
If we get an error in the Watch loop, it will send this on awsChan,
which will cause Watch to loop. However, in this scenario it will never
cause closeChan to close, and we will deadlock because we have a
waitGroup in a helper goroutine which is waiting on this channel to
close the context.

Normally this wouldn't be an issue, but since we have more than one
goroutine (with associated waitGroup) it is. It's also good practice to
close all the channels to help avoid this kind of bug.

This patch also moves the waitGroup Wait into a more logical place for
visibility.
2017-11-10 14:17:54 -05:00
Jonathan Gold
c8ddbeaa5c resource: aws: ec2: Add http server 2017-11-09 13:13:42 -05:00
Jonathan Gold
3634b3450d resource: aws: ec2: Move waitgroup to resource struct 2017-11-08 16:57:41 -05:00
Jonathan Gold
c2a5e3f5d8 resources: aws: ec2: Move watch channels into struct 2017-11-08 16:16:01 -05:00
Jonathan Gold
db49fe85e4 resources: aws: ec2: Move chanStruct type out of longpollWatch 2017-11-08 16:08:25 -05:00
Jonathan Gold
567a2e9fd8 resources: aws: ec2: Reorganized consts 2017-11-08 16:02:29 -05:00
Jonathan Gold
987de00e17 resources: aws: ec2: Remove extra wait from Watch
There were two calls to WaitUntilInstanceTerminatedWithContext in a row.
There's no reason to make the call twice.
2017-11-08 16:02:24 -05:00
Jonathan Gold
baeafec74a resources: aws: ec2: Move Watch to longpollWatch 2017-11-08 16:02:12 -05:00
Jonathan Gold
e7c4bc7f47 resources: Add UserData field to AwsEc2
UserData specifies first-launch bash and cloud-init commands. See
https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/user-data.html
for documentation and examples.
2017-10-30 00:22:30 -04:00
Jonathan Gold
9907c12eda resources: Enhancements to user and group
This patch adds autoedges between users and groups, and extends
users with additional fields for supplementary groups and a named
primary group. Also, some small fixes to log and error messages.
2017-10-23 19:18:52 -04:00
Jonathan Gold
19533a32b5 resources: Add a group resource 2017-10-21 01:28:22 -04:00
Jonathan Gold
c5a5004f9e resources: Fix user gid compare 2017-10-19 06:58:31 -04:00
Jonathan Gold
677cdea99d resources: Improve nspawn resource 2017-10-17 19:23:04 -04:00
Jonathan Gold
4d7c0ddbce resources: Add an Aws resource 2017-10-09 04:05:13 -04:00
Juan Luis de Sousa-Valadas Castaño
21f7f87716 resources: Refresh packagekit cache before install
Fixes #80
2017-09-17 22:29:15 +02:00
Jonathan Gold
831c7e2c32 resources: Add user resource 2017-09-17 01:04:36 -04:00
James Shubin
46be83f8f7 legal: Re-license to GPLv3 2017-09-11 18:07:47 -04:00
James Shubin
28560e2045 resources: Fix formatting 2017-09-11 18:06:34 -04:00
Jonathan Gold
69f479b67e virt: Allow more than 26 disks 2017-09-08 02:15:40 +00:00
Arthur Mello
80b8f8740f virt: Added support for ~user into expandHome
- Enabled expandHome to expand both ~/ and ~username/ paths
- Added some unit tests for expandHome
2017-09-06 14:59:08 -04:00