Commit Graph

158 Commits

Author SHA1 Message Date
James Shubin
1b68845b00 test: Fix up token vet test
I forgot some of the cases to catch earlier.
2018-12-19 22:24:20 -05:00
James Shubin
27ac7481f9 test: Increase the vet testing for irregular strings
Catch some inconsistent comments to keep things neat. Hey, anything we
can automate, we do :)
2018-12-19 06:52:23 -05:00
James Shubin
d990d2ad86 travis: Bump to golang 1.10
This requires breaking changes in gofmt. It is hilarious that this was
changed. Oh well. This also moves to the latest stable etcd. Lastly,
this changes the `go vet` testing to test by package, since the new go
vet changed how it works and now fails without this change.
2018-12-11 01:46:17 -05:00
Jonathan Gold
56db31ca43 engine: resources: file: Add shell test for source field 2018-12-10 22:08:59 -05:00
Jonathan Gold
8e25667f87 engine: resources: net: test: Add shell test for net resource
This patch adds a shell test for net, which creates a dummy interface
and runs mgmt to bring it up and assign it with an address. It then
checks if the state was applied correctly. Finally, it runs mgmt again
to bring the interface down, and tests that it comes down and stays
down.
2018-12-04 17:12:57 -05:00
James Shubin
4c93bc3599 test: Add doc note about skipping docker tests
This is useful if you don't have docker running, since otherwise it
causes all the tests to fail.
2018-12-03 23:55:20 -05:00
James Shubin
5431bfdc29 test: Improve commit message tests 2018-11-24 04:42:50 -05:00
James Shubin
81bb87f4cd test: Add a test to ensure the parser doesn't have any conflicts
Our grammar shouldn't be ambiguous, and it makes sense to test this.
2018-06-18 16:06:23 -04:00
James Shubin
c4b97fadcc lang: Update map type definition to include a prefix
It turns out that some planned additions to the parser make it so that
the map type definition can be ambiguous. As a result, this patch
updates the definition so that the map definition is not confused with
an open curly bracket anywhere.

Thanks to pestle and stbenjamin for their help understanding yacc!
2018-06-18 16:06:23 -04:00
James Shubin
5f12ff6178 lang: Add indentation test to parser
This adds a test case to catch some common typos.
2018-06-12 17:40:18 -04:00
James Shubin
22c0ff3cf5 test: Improve golang tests with root and disabling cache
This allows golang tests to be marked as root or !root using build tags.
The matching tests are then run as expected using our test runner.

This also disables test caching which is unfriendly to repeated test
running and is an absurd golang default to add.

Lastly this hooks up the testing verbose flag to tests that accept a
debug variable.

These tests aren't enabled on travis yet because of how it installs
golang.
2018-05-09 13:44:01 -04:00
James Shubin
a407771eaf test: Catch naked returns and check for canonically named imports
This catches scenarios where we forgot to prefix the error with return.
One of our contributors occasionally made this typo, and since core go
vet didn't (surprisingly) catch it, we should add a test!

It also adds a simple check for import naming aliases. Expanding this
test to add other cases and check for differently named values might
make sense.
2018-05-06 15:18:46 -04:00
James Shubin
a589e2ecf3 docs, test: Remove old reference to resources package
Forgot to change this previously. Also updated the resources list in the
documentation.
2018-05-02 15:28:15 -04:00
James Shubin
412a25462e test: Improve commit message test
We can classify better now that we have the new engine.
2018-04-21 19:29:26 -04:00
James Shubin
9969286224 engine: Resources package rewrite
This giant patch makes some much needed improvements to the code base.

* The engine has been rewritten and lives within engine/graph/
* All of the common interfaces and code now live in engine/
* All of the resources are in one package called engine/resources/
* The Res API can use different "traits" from engine/traits/
* The Res API has been simplified to hide many of the old internals
* The Watch & Process loops were previously inverted, but is now fixed
* The likelihood of package cycles has been reduced drastically
* And much, much more...

Unfortunately, some code had to be temporarily removed. The remote code
had to be taken out, as did the prometheus code. We hope to have these
back in new forms as soon as possible.
2018-04-19 01:10:58 -04:00
James Shubin
f3b99b3940 test, integration: Add an integration test framework
This adds an initial implementation of an integration test framework for
writing more complicated tests. In particular this also makes some small
additions to the mgmt core so that testing is easier.
2018-03-13 06:38:21 -04:00
James Shubin
df1e50e599 lang: funcs: Add math pow function and a few examples
Just a few small things I think should be committed.
2018-02-25 19:48:25 -05:00
James Shubin
40dcd6ec99 all: Misc fixes and test fixes 2018-02-25 02:13:51 -05:00
Johan Bloemberg
f33f84d2f2 lang: Add getenv function
$x = getenv("NAME")
    $y = defaultenv("NOTEXIST", "defaultvalue")
    $z = hasenv("NAME")
    $a = env()
    $b = maplookup($a, "NAME", "defaultvalue")
2018-02-23 20:02:13 +01:00
Johan Bloemberg
9544ab2e02 recwatch: Fix watching newly created files on macOS
Fixes: https://github.com/purpleidea/mgmt/issues/33
2018-02-22 16:52:26 -05:00
James Shubin
05c60d9a59 test, docs: Restrict long lines in markdown linter
It's getting out of hand...
2018-02-22 16:19:23 -05:00
James Shubin
cffdb06181 test, docs: Add a linter for testing markdown, and fix up our docs
While writing docs, I couldn't remember what the correct style was
supposed to be, and I remember someone complaining about this
previously, so I decided to add a linter! I excluded a bunch of annoying
style rules, but if we find more we can add those to the list too.

Hopefully this gives us a more consistent feel throughout.
2018-02-21 22:52:41 -05:00
James Shubin
915b022901 test: Show test output as it happens 2018-02-18 17:31:45 -05:00
Johan Bloemberg
7508161c39 test: Exclude generated files from golint 2018-02-18 15:07:54 -05:00
Johan Bloemberg
d33861ccb4 test: Fix augeas test for macOS, improve test debuggability
- resolve a discrepancy in augeas behaviour on macOS
- on macOS `sed` requires an argument for `-i`.
- made the test fail as early as it can
- provide information about why the test is failing
2018-02-18 14:36:59 -05:00
Johan Bloemberg
572b2575c5 test: Export the mgmt command to be used during test 2018-02-18 14:19:05 -05:00
James Shubin
e3a26483e8 test: Improve gometalinter test so that it skips generated files
This should improve things significantly, and avoid the failures now
that we're testing after the files have already been built.
2018-02-15 16:26:06 -05:00
Johan Bloemberg
b34b359860 test: Streamline test suite a little
This change aims to streamline the integrationtest suite and reduce friction when running (parts of) test suites.

Changes:
- add `test-testname` to makefile to easily run one suite
- made skipping tests first class citizen in test.sh (all available testsuites and the reasons they are skipped are now better exposed and discovered)
- suppress some output of gotest unless there is an error
- no longer build binary for examples and gotest suites
- removed .SILENT from makefile as it being applied to only some targets makes it feel weird (I just learned about this option btw, feel free to comment on this change)
- move individual tests out of `test.sh` and into `test-misc.sh`
- introduced the concept of testsuites to `test.sh`
2018-02-15 17:21:49 +01:00
James Shubin
714afe35a1 test: Fix broken gometalinter test
The test for gometalinter got silently broken in an earlier commit.
Look for the missing space that was added back in this commit to see
why! In any case, this now fixes some of the things that weren't
previously caught by this change.

If anyone knows how to run these sorts of tests properly so that entire
packages are tested and so that we can enable additional tests, please
let me know!

It's also unclear why goreportcard catches a few additional problems
which aren't found by running this ourselves.

See:
https://goreportcard.com/report/github.com/purpleidea/mgmt
for more information.
2018-02-14 14:34:36 -05:00
Johan Bloemberg
22e4dfa534 build: Unify build/crossbuild
Changes:

- allows explicit crossbuild targets (eg: `make mgmt-darwin-amd64`)
- adds darwin/amd64 to default crossbuild targets
- gitignore only build artifacts (eg: not all files starting with `mgmt-`)
- `build` and `crossbuild` target now utilize the same build function (`build` still generates only a `mgmt` binary for the current os/arch)
- test crossbuilding
- allow specifying custom GOOSARCHES envvar to override defaults
- crossbuild artifacts go into `build/` now
- add `build-debug` which includes symbol tables and debug info
- the build function now has `-s -w` linker arguments which discards some debug info afaict, to build a debug release use `make build-debug`

On my mac crossbuilding won't work unless I disable augeas and libvirt:

```
~/.g/s/g/p/mgmt (build|●1✚8…3) $ make build
Generating: bindata...
Generating: lang...
/Applications/Xcode.app/Contents/Developer/usr/bin/make --quiet -C lang
Building: mgmt, os/arch: darwin-amd64, version: 0.0.14-12-g94c8bc1-dirty...
env GOOS=darwin GOARCH=amd64 time go build -ldflags "-X main.program=mgmt -X main.version=0.0.14-12-g94c8bc1-dirty -s -w" -o mgmt-darwin-amd64 ;
        7.14 real        10.36 user         1.73 sys
mv mgmt-darwin-amd64 mgmt
```

```
~/.g/s/g/p/mgmt (build|●1✚8…3) $ time env GOTAGS='noaugeas novirt' make crossbuild
Generating: bindata...
Generating: lang...
/Applications/Xcode.app/Contents/Developer/usr/bin/make --quiet -C lang
Building: mgmt, os/arch: linux-amd64, version: 0.0.14-12-g94c8bc1-dirty...
env GOOS=linux GOARCH=amd64 time go build -ldflags "-X main.program=mgmt -X main.version=0.0.14-12-g94c8bc1-dirty -s -w" -o mgmt-linux-amd64 -tags 'noaugeas novirt';
       18.48 real        50.02 user         5.83 sys
Building: mgmt, os/arch: linux-ppc64, version: 0.0.14-12-g94c8bc1-dirty...
env GOOS=linux GOARCH=ppc64 time go build -ldflags "-X main.program=mgmt -X main.version=0.0.14-12-g94c8bc1-dirty -s -w" -o mgmt-linux-ppc64 -tags 'noaugeas novirt';
       29.83 real        85.09 user        11.54 sys
Building: mgmt, os/arch: linux-ppc64le, version: 0.0.14-12-g94c8bc1-dirty...
env GOOS=linux GOARCH=ppc64le time go build -ldflags "-X main.program=mgmt -X main.version=0.0.14-12-g94c8bc1-dirty -s -w" -o mgmt-linux-ppc64le -tags 'noaugeas novirt';
       29.74 real        85.84 user        11.76 sys
Building: mgmt, os/arch: linux-arm64, version: 0.0.14-12-g94c8bc1-dirty...
env GOOS=linux GOARCH=arm64 time go build -ldflags "-X main.program=mgmt -X main.version=0.0.14-12-g94c8bc1-dirty -s -w" -o mgmt-linux-arm64 -tags 'noaugeas novirt';
       28.33 real        83.24 user        11.40 sys
Building: mgmt, os/arch: darwin-amd64, version: 0.0.14-12-g94c8bc1-dirty...
env GOOS=darwin GOARCH=amd64 time go build -ldflags "-X main.program=mgmt -X main.version=0.0.14-12-g94c8bc1-dirty -s -w" -o mgmt-darwin-amd64 -tags 'noaugeas novirt';
        7.16 real        10.15 user         1.74 sys
      114.71 real       315.26 user        42.44 sys
```
2018-02-14 12:37:49 -05:00
James Shubin
644e5164b1 test: Increase timeouts for when travis is slow
Hopefully this cuts down on spurious failures.
2018-02-12 15:08:47 -05:00
Johan Bloemberg
d567dc3769 lang: Use universal way to retrieve load
Sysinfo is not supported on macOS and results in a build error.
2018-02-10 05:01:12 -05:00
Johan Bloemberg
ba21554c5f build, docs: Improve macOS building
- New docker command for quickly running tasks in a Linux environment.
- Updated docs with macOS specific details.
- Fixed some test issues.
- Add (fallible) macOS test target for Travis.
2018-02-10 05:01:12 -05:00
James Shubin
e37bb3ac8a test: Add new test for language prefix 2018-02-07 21:52:06 -05:00
James Shubin
b19583e7d3 lang: Initial implementation of the mgmt language
This is an initial implementation of the mgmt language. It is a
declarative (immutable) functional, reactive, domain specific
programming language. It is intended to be a language that is:

* safe
* powerful
* easy to reason about

With these properties, we hope this language, and the mgmt engine will
allow you to model the real-time systems that you'd like to automate.

This also includes a number of other associated changes. Sorry for the
large size of this patch.
2018-01-20 08:09:29 -05:00
Joe Julian
bee1aa00f1 misc: Use bash's command instead of which
Bash has a built-in command, `command`, that will search the path and
return the full path to a command if it exists (or an exit code of 1 if
it does not), preventing the requirement of the `which` package.
2018-02-03 11:23:52 -05:00
Jonathan Gold
62ca12608d cli: Add license flag
This patch adds the option to print the license with a cli flag. It
uses go-bindata to store the license file. The file is generated by
running `make bindata` and the result is stored in the bindata
directory.
2017-12-08 00:57:58 -05:00
Guillaume Herail
ac629404f4 test: Switch to goimports instead of gofmt
see https://github.com/purpleidea/mgmt/pull/256#issuecomment-346360414
2017-11-25 06:49:00 -05:00
Guillaume Herail
3575d597f7 resources: Add User/Group to ExecRes 2017-11-24 10:38:16 -05:00
James Shubin
846c5f8762 test: Add another check for off-by-one-error commit tags 2017-11-24 09:46:32 -05:00
James Shubin
52fd1ae73e test: Add check for common doc vs docs ambiguity 2017-11-23 08:20:44 -05:00
James Shubin
51ce83f20b test: Add extra commit message tests for some common mistakes
Feel free to add more if we identify them.
2017-11-21 11:05:20 -05:00
James Shubin
81daf10157 test: Fix linter issues
These are some linter issues that were found in a new version of the
linter. Let's fix them now before that linter hits our test suite.
2017-09-26 19:38:53 -04:00
James Shubin
0df4824a56 test: Increase timeouts for slow travis
Should prevent more intermittent failures.
2017-09-09 15:31:06 -04:00
James Shubin
653c76709a test: Fix another intermittent failure
Some of the tests had very precise timeouts, which weren't very
important. Here's another one that timed out early.
2017-09-04 16:39:01 -04:00
James Shubin
6c8588c019 test: Increase timeouts because travis is slow
Should hopefully prevent some intermittent failures.
2017-09-04 13:02:05 -04:00
James Shubin
0edba74091 etcd: Bump to version 3.2.6 and update all the grpc deps
Note: When go-grpc-prometheus was in the main $gopath (even at this
version) and everyone else was where they always were in vendor/ this
didn't build! It gave errors like:

	have SendHeader("github.com/purpleidea/mgmt/vendor/google.golang.org/grpc/metadata".MD) error
	want SendHeader("google.golang.org/grpc/metadata".MD) error

and I got frustrated. Putting it "next" to the other vendored deps seems
to have fixed this. Where are the golang docs that explain this
phenomenon?

This also requires golang 1.8+ as that is a requirement for etcd. It's
probably a reasonable thing for us too.

Note the older versions of etcd had some bugs with the concurrency
package and other things, so this is a necessary bump.
2017-08-30 14:16:02 -04:00
James Shubin
0dadf3d78a resources: Add NewNamedResource helper
This makes the common pattern of NewResource, SetName, easier. It also
makes it less likely for you to forget to use SetName.
2017-06-17 18:09:49 -04:00
James Shubin
a8bbb22fe8 resources: Fix golint issues
Including a trick to get the golinter to allow our compact code!
2017-06-08 04:38:25 -04:00
James Shubin
f1db088af4 test: Don't be noisy when running cd during testing 2017-06-08 01:05:58 -04:00