Resources: Add retry and retry delay meta parameters

All resources can now set a retry limit (-1 for infinite) and a delay
between retries. This applies to both the CheckApply methods, and the
Watch methods as well. They each have their own separate counts, but use
the same input meta param, since I decided it wouldn't be useful to have
a separate watchRetry and watchDelay set of meta parameters.

In the process, we got rid of about 15 error cases which would normally
panic.

This patch required a slight overhaul of the Event system.

The previous commit is an earlier version of this patch which I decided
to leave in to "show my work" as I used to have to do in math class.
It's slightly more correct with the current event system, and this
version is less correct and has a few bugs, but that is because the
event system needs a massive overhaul, and once that's done this should
all work properly for the corner cases.
This commit is contained in:
James Shubin
2016-09-19 05:47:11 -04:00
parent 53cabd5ee4
commit fc24c91dde
11 changed files with 237 additions and 356 deletions

10
main.go
View File

@@ -231,7 +231,7 @@ func run(c *cli.Context) error {
converger.SetStateFn(convergerStateFn)
}
exitchan := make(chan Event) // exit event
exitchan := make(chan struct{}) // exit on close
go func() {
startchan := make(chan struct{}) // start signal
go func() { startchan <- struct{}{} }()
@@ -268,8 +268,7 @@ func run(c *cli.Context) error {
}
// XXX: case compile_event: ...
// ...
case msg := <-exitchan:
msg.ACK()
case <-exitchan:
return
}
@@ -384,16 +383,13 @@ func run(c *cli.Context) error {
G.Exit() // tell all the children to exit
// tell inner main loop to exit
resp := NewResp()
go func() { exitchan <- Event{eventExit, resp, "", false} }()
close(exitchan)
// cleanup etcd main loop last so it can process everything first
if err := EmbdEtcd.Destroy(); err != nil { // shutdown and cleanup etcd
log.Printf("Etcd exited poorly with: %v", err)
}
resp.ACKWait() // let inner main loop finish cleanly just in case
if DEBUG {
log.Printf("Graph: %v", G)
}