Commit Graph

77 Commits

Author SHA1 Message Date
James Shubin
d30ff6cfae legal: Remove year
Instead of constantly making these updates, let's just remove the year
since things are stored in git anyways, and this is not an actual modern
legal risk anymore.
2025-01-26 16:24:51 -05:00
James Shubin
1d3f2dbe3c util: Add a pprof helper package
As it turns out, there are other helper packages out there. I wrote this
before I realized that, and it didn't take me that long anyways.
2025-01-24 03:06:31 -05:00
James Shubin
68ee163eb1 entry, lang: core: embedded: provisioner: Allow more than one entry
This changes the entry API slightly to allow for more than one entry
registered, which makes building, testing and user tooling easier.
2024-04-16 14:11:34 -04:00
Felix Frank
29ec867ac7 gapi: Bring back puppet and langpuppet
This reverts commit e767655ede.

In addition, it applies required changes to function with the new CLI backend.
2024-04-02 21:07:02 -04:00
James Shubin
e0d6d35b56 entry: Execute embedded commands if you use their correct name
I think this is the more intuitive way to do things. Force the user the
explicitly choose the embedded command which prevents accidental running
of a different command, and allows us to potentially have multiple
commands in the future.
2024-03-22 01:36:04 -04:00
James Shubin
8cf7719476 cli: Refactor the logger setup 2024-03-22 01:36:04 -04:00
James Shubin
3e180eafb4 cli: Refactor out log to the top level
Remove some cruft at the same time.
2024-03-22 00:55:03 -04:00
James Shubin
90f6d4e563 legal: Update http to https 2024-03-05 01:05:50 -05:00
James Shubin
3e31ee9455 legal: Additional permission under GNU GPL version 3 section 7
With the recent merging of embedded package imports and the entry CLI
package, it is now possible for users to build in mcl code into a single
binary. This additional permission makes it explicitly clear that this
is permitted to make it easier for those users. The condition is phrased
so that the terms can be "patched" by the original author if it's
necessary for the project. For example, if the name of the language
(mcl) changes, has a differently named new version, someone finds a
phrasing improvement or a legal loophole, or for some other
reasonable circumstance. Now go write some beautiful embedded tools!
2024-03-05 01:04:09 -05:00
James Shubin
d52c90ede4 cli, entry, lang: Add an entry package for embeddable CLI's
This adds a new entry package that allows embedded programs to exist
inside of mgmt. This took a lot of refactoring to get the API right, but
I think it's incredibly elegant now. There is a chance we tweak things a
bit, but it's a good first start. All-in-one programs are coming soon!
2024-03-05 01:03:54 -05:00
James Shubin
601fcf40c4 cli, gapi: empty, lang, yaml: Refactor the args structs
Put these datastructures into an external package so they can be re-used
for parsing elsewhere.

Since we remove these dependencies, we need to manually import the
GAPI's so that they register. Despite efforts to embed them deeper into
the import tree without cycles, this failed. Logically what this told me
is that it actually makes sense to allow a different binary with only
one of the multiple GAPI's contained within.
2024-03-03 17:17:21 -05:00
James Shubin
589a5f9aeb cli, lib, lang: Port to new cli library
The new version of the urfave/cli library is moving to generics, and
it's completely unclear to me why this is an improvement. Their new API
is very complicated to understand, which for me, defeats the purpose of
golang.

In parallel, I needed to do some upcoming cli API refactoring, so this
was a good time to look into new libraries. After a review of the
landscape, I found the alexflint/go-arg library which has a delightfully
elegant API. It does have a few rough edges, but it's otherwise very
usable, and I think it would be straightforward to add features and fix
issues.

Thanks Alex!
2024-03-01 21:02:55 -05:00
James Shubin
80af171a35 cli: Further refactor our input cli data 2024-02-28 16:36:49 -05:00
James Shubin
71c54ab212 cli: Refactor even more code out of cli package
Quite honestly, I'm trying to clean things up, and removing anything
non-consequential will help!
2024-02-28 16:36:49 -05:00
James Shubin
abe3e0e7a5 lib: Split off the CLI portions into a separate package
This cleans things up a bit more and forces the lib package to not
contain any accidental CLI parsing code.
2024-02-28 16:36:49 -05:00
James Shubin
a8f194259b legal: Happy 2024 everyone...
Done with:

ack '2023+' -l | xargs sed -i -e 's/2023+/2024+/g'

Checked manually with:

git add -p

Hello to future James from 2025, and Happy Hacking!
2024-01-22 15:52:49 -05:00
James Shubin
fb57fb714a main, lib: Build in standalone etcd
Standalone etcd is useful for when we don't want to use the embedded
version to make it easier to deploy somewhere or for testing.

This pulls in about the same amount of code since we already embedded
etcd previously. Since the embedded etcd feature of mgmt is not very
stable, we'll add this for now.
2023-08-30 03:44:12 -04:00
James Shubin
a7624a2bf9 legal: Happy 2023 everyone...
Done with:

ack '2022+' -l | xargs sed -i -e 's/2022+/2023+/g'

Checked manually with:

git add -p

Hello to future James from 2024, and Happy Hacking!
2023-03-05 18:31:52 -05:00
James Shubin
3cea422365 legal: Happy 2022 everyone...
Done with:

ack '2021+' -l | xargs sed -i -e 's/2021+/2022+/g'

Checked manually with:

git add -p

Hello to future James from 2023, and Happy Hacking!
2022-08-05 23:06:27 -04:00
James Shubin
88516546fa lib: Move to go modules and shed a few tears
The old system with vendor/ and git submodules worked great,
unfortunately FUD around git submodules seemed to scare people away and
golang moved to a go.mod system that adds a new lock file format instead
of using the built-in git version. It's now almost impossible to use
modern golang without this, so we've switched.

So much for the golang compatibility promise-- turns out it doesn't
apply to the useful parts that I actually care about like this.

Thanks to frebib for his incredibly valuable contributions to this
patch. This snide commit message is mine alone.

This patch also mixes in some changes due to legacy golang as we've also
bumped the minimum version to 1.16 in the docs and tests.

Lastly, we had to disable some tests and fix up a few other misc things
to get this passing. We've definitely hot bugs in the go.mod system, and
our Makefile tries to workaround those.
2021-10-05 08:34:51 -04:00
James Shubin
336a38081a legal: Happy 2021 everyone...
Done with:

ack '2020+' -l | xargs sed -i -e 's/2020+/2021+/g'

Checked manually with:

git add -p

Hello to future James from 2022, and Happy Hacking!
2021-01-31 16:52:46 -05:00
James Shubin
2648fb1bb1 legal: Happy 2020 everyone...
Done with:

ack '2019+' -l | xargs sed -i -e 's/2019+/2020+/g'

Checked manually with:

git add -p

Hello to future James from 2021, and Happy Hacking!
2020-01-03 20:08:37 -05:00
James Shubin
a5842a41b2 etcd: Rewrite embed etcd implementation
This is a giant cleanup of the etcd code. The earlier version was
written when I was less experienced with golang.

This is still not perfect, and does contain some races, but at least
it's a decent base to start from. The automatic elastic clustering
should be considered an experimental feature. If you need a more
battle-tested cluster, then you should manage etcd manually and point
mgmt at your existing cluster.
2019-04-11 21:43:48 -04:00
James Shubin
07f542b4d7 legal: Happy 2019 everyone...
Done with:

ack '2018+' -l | xargs sed -i -e 's/2018+/2019+/g'

Checked manually with:

git add -p

Hello to future James from 2020, and Happy Hacking!
2019-03-24 15:08:50 -04:00
James Shubin
9969286224 engine: Resources package rewrite
This giant patch makes some much needed improvements to the code base.

* The engine has been rewritten and lives within engine/graph/
* All of the common interfaces and code now live in engine/
* All of the resources are in one package called engine/resources/
* The Res API can use different "traits" from engine/traits/
* The Res API has been simplified to hide many of the old internals
* The Watch & Process loops were previously inverted, but is now fixed
* The likelihood of package cycles has been reduced drastically
* And much, much more...

Unfortunately, some code had to be temporarily removed. The remote code
had to be taken out, as did the prometheus code. We hope to have these
back in new forms as soon as possible.
2018-04-19 01:10:58 -04:00
James Shubin
12fce52cd7 legal: Happy 2018 everyone...
Done with:

ack '2017+' -l | xargs sed -i -e 's/2017+/2018+/g'

Checked manually with:

git add -p

Hello to future James from 2019, and Happy Hacking!
2018-01-03 21:22:07 -05:00
James Shubin
46be83f8f7 legal: Re-license to GPLv3 2017-09-11 18:07:47 -04:00
James Shubin
a981cfa053 legal: Oh yeah, it is 2017 2017-02-16 01:34:32 -05:00
James Shubin
4803be1987 misc: Rename mgmtmain to lib and remove global package
This refactor should make it cleaner to use mgmt.
2016-12-08 23:31:45 -05:00
James Shubin
0dbd2004ad main: Split apart logic in main
This splits most of the main logic from the cli logic so that they can
be used independently, in particular for if we ever libify mgmt.
2016-10-23 20:23:04 -04:00
James Shubin
cfde54261b golint: Outdent else statement 2016-10-03 15:21:08 -04:00
James Shubin
46893e84c3 recwatch: Split recursive watching into sub-package
This splits the recursive watching bit of the file file resource into
it's own package. This also de-duplicates the configwatch code and puts
it into the same package. With these bits refactored, it was also easier
to clean up the error code in main.
2016-10-03 14:48:57 -04:00
James Shubin
63f21952f4 golang: Split things into packages
This makes this logically more separate! :) As an aside...

I really hate the way golang does dependencies and packages. Yes, some
people insist on nesting their code deep into a $GOPATH, which is fine
if you're a google dev and are forced to work this way, but annoying for
the rest of the world. Your code shouldn't need a git commit to switch
to a a different vcs host! Gah I hate this so much.
2016-09-26 12:30:28 -04:00
James Shubin
fc24c91dde Resources: Add retry and retry delay meta parameters
All resources can now set a retry limit (-1 for infinite) and a delay
between retries. This applies to both the CheckApply methods, and the
Watch methods as well. They each have their own separate counts, but use
the same input meta param, since I decided it wouldn't be useful to have
a separate watchRetry and watchDelay set of meta parameters.

In the process, we got rid of about 15 error cases which would normally
panic.

This patch required a slight overhaul of the Event system.

The previous commit is an earlier version of this patch which I decided
to leave in to "show my work" as I used to have to do in math class.
It's slightly more correct with the current event system, and this
version is less correct and has a few bugs, but that is because the
event system needs a massive overhaul, and once that's done this should
all work properly for the corner cases.
2016-09-19 06:32:21 -04:00
James Shubin
b46432b5b6 misc: Golint fixes 2016-09-02 01:46:45 -04:00
James Shubin
ff01e4a5e7 remote: Add distributed converged timeout
This patch extends the --converged-timeout argument so that when used
with --remote it waits for the entire set of remote mgmt agents to
converge simultaneously before exiting.

purpleidea says: This particular part of the patch probably took as much
work as all of the work required for the initial remote patches alone!
2016-08-31 21:55:19 -04:00
James Shubin
6794aff77c miscellaneous cleanups and fixes 2016-08-31 21:55:19 -04:00
James Shubin
eee652cefe converger: Add new timer system for determining convergence
This adds a new method of marking whether a particular UUID has
converged or not. You can now Start, Stop, or Reset a convergence timer
on the individual UUID's. This wraps the existing SetConverged calls
with a hidden go routine. It is not recommended to use the SetConverged
calls and the Timer calls on the same UUID.
2016-08-31 21:55:19 -04:00
James Shubin
db4de12767 Add more flexibility around the prefixes available
This allows you to specify a custom prefix, or a tmp prefix which is
chosen automatically.
2016-08-31 21:55:19 -04:00
James Shubin
d429795737 Improve the configWatcher array to allow N files
This simplifies the code in main and hides it in the watcher!
2016-08-31 21:55:19 -04:00
James Shubin
03c1df98f4 Improve prefix creation and feedback
This makes the default /var/lib/mgmt/ directory, but also allows you to
use a prefix in /tmp/ automatically if you can't write anywhere else.
2016-08-31 21:55:19 -04:00
James Shubin
79ba750dd5 Automatically update remote files on change
This extends the automatic watching of graph definition files across the
remote SSH boundary.
2016-08-31 21:55:19 -04:00
James Shubin
1d0e187838 Etcd: switch to using a directory prefix 2016-08-31 21:55:19 -04:00
James Shubin
ad1e48aa2d Add caching for remote execution
This speeds up copying of the binary for slow connections. It also
finally adds a universal directory prefix for mgmt!
2016-08-31 21:55:19 -04:00
James Shubin
7032eea045 Remote "agent-less" mode
This is a new mode to be used for bootstrapping mgmt clusters or in
situations with tight operational restrictions.

This includes the basics, additional functionality will follow!
2016-08-31 21:55:19 -04:00
James Shubin
bdb970203c Start converger even if graph is empty 2016-08-30 17:47:25 -04:00
James Shubin
a5e9f6a6fc Fix stupid gofmt issue 2016-06-19 02:47:36 -04:00
James Shubin
f821afdf3e Update cli library path 2016-06-19 02:35:45 -04:00
James Shubin
5363839ac8 Embedded etcd
This monster patch embeds the etcd server. It took a good deal of
iterative work to tweak small details, and survived a rewrite from the
initial etcd v2 API implementation to the beta version of v3.

It has a notable race, and is missing some features, but it is ready for
git master and external developer consumption.
2016-06-18 04:43:19 -04:00
Felix Frank
8f83ecee65 add puppet integration code
Puppet can be used on the basis of the ffrank-mgmtgraph module.
There are three modes available:
* fetching catalogs from the master (--puppet agent)
* compiling a manifest from a local file (--puppet /path/to/file.pp)
* compiling a manifest from the cli (--puppet "<manifest>")

Catalogs from the master are currently never refreshed. We should
add some more code to re-run the parsing function at an interval
equal to Puppet's local 'runinterval' setting.

There is also still a distinct lack of tests.

Still, this fixes #8
2016-06-01 00:34:40 +02:00