Commit Graph

1784 Commits

Author SHA1 Message Date
James Shubin
59e133d3bc make: Find on all mcl files
We forgot to port this line when we moved the core package up one
directory.
2024-03-08 17:41:57 -05:00
James Shubin
90f6d4e563 legal: Update http to https 2024-03-05 01:05:50 -05:00
James Shubin
3e31ee9455 legal: Additional permission under GNU GPL version 3 section 7
With the recent merging of embedded package imports and the entry CLI
package, it is now possible for users to build in mcl code into a single
binary. This additional permission makes it explicitly clear that this
is permitted to make it easier for those users. The condition is phrased
so that the terms can be "patched" by the original author if it's
necessary for the project. For example, if the name of the language
(mcl) changes, has a differently named new version, someone finds a
phrasing improvement or a legal loophole, or for some other
reasonable circumstance. Now go write some beautiful embedded tools!
2024-03-05 01:04:09 -05:00
James Shubin
d52c90ede4 cli, entry, lang: Add an entry package for embeddable CLI's
This adds a new entry package that allows embedded programs to exist
inside of mgmt. This took a lot of refactoring to get the API right, but
I think it's incredibly elegant now. There is a chance we tweak things a
bit, but it's a good first start. All-in-one programs are coming soon!
2024-03-05 01:03:54 -05:00
James Shubin
9527d0dcbd lib, cli: Move cli struct tags and embed this datastructure
This moves over the cli `arg` struct tags which are used to generate and
parse things on the command line. Furthermore, we then embed this data
directly in our more general parser struct so that we avoid duplication.
Finally, since the data shares a common struct type, we don't need to do
the manual field-by-field copying to pull things in!
2024-03-03 20:07:04 -05:00
James Shubin
51d21b8dab lib: Split out all of our config data as a separate struct
Hopefully this will make things more reusable.
2024-03-03 19:18:32 -05:00
James Shubin
601fcf40c4 cli, gapi: empty, lang, yaml: Refactor the args structs
Put these datastructures into an external package so they can be re-used
for parsing elsewhere.

Since we remove these dependencies, we need to manually import the
GAPI's so that they register. Despite efforts to embed them deeper into
the import tree without cycles, this failed. Logically what this told me
is that it actually makes sense to allow a different binary with only
one of the multiple GAPI's contained within.
2024-03-03 17:17:21 -05:00
James Shubin
d537c3d523 cli: Lookup subcommand names dynamically
This lets us get the chosen subcommands dynamically and removes the
eventual need for three package imports.
2024-03-03 15:50:54 -05:00
James Shubin
a65c87b584 gapi: Add a new Names API
This gets the list more directly. We could also just lookup a key in the
big map to get the same effect, but this is more logical for now. This
is useful for removing the importing of three packages.
2024-03-03 15:50:54 -05:00
James Shubin
589a5f9aeb cli, lib, lang: Port to new cli library
The new version of the urfave/cli library is moving to generics, and
it's completely unclear to me why this is an improvement. Their new API
is very complicated to understand, which for me, defeats the purpose of
golang.

In parallel, I needed to do some upcoming cli API refactoring, so this
was a good time to look into new libraries. After a review of the
landscape, I found the alexflint/go-arg library which has a delightfully
elegant API. It does have a few rough edges, but it's otherwise very
usable, and I think it would be straightforward to add features and fix
issues.

Thanks Alex!
2024-03-01 21:02:55 -05:00
James Shubin
e767655ede cli, puppet, langpuppet: Remove puppet GAPI's
I'm currently refactoring the CLI code. Unfortunately this means a
pretty big churn in the various GAPI frontends. Since nobody is actively
using the puppet frontend code, I'm removing it for now. If someone is
actively using it, and wants to either port it to the new API, or
sponsor the porting of it to the new API, I'm happy to allow it back in.

Sorry Felix, it was a fun idea, and I loved seeing it work, but I can't
personally afford the maintenance cost of having this in right now.
2024-03-01 21:02:55 -05:00
James Shubin
62295e370c test: shell: Disable flaky test
Not sure what's up, but we should investigate eventually.
2024-03-01 21:02:55 -05:00
James Shubin
f818f5ccf5 util: errwrap: Add more tests 2024-02-28 16:37:28 -05:00
James Shubin
f28d22d20f cli: Typo fix 2024-02-28 16:36:49 -05:00
James Shubin
80af171a35 cli: Further refactor our input cli data 2024-02-28 16:36:49 -05:00
James Shubin
71c54ab212 cli: Refactor even more code out of cli package
Quite honestly, I'm trying to clean things up, and removing anything
non-consequential will help!
2024-02-28 16:36:49 -05:00
James Shubin
c37ff3efce lang, test: Fix capitalization for consistency 2024-02-28 16:36:49 -05:00
James Shubin
dd0e67540f all: Remove deprecated io/ioutil package
Porting everything to the newer imports was trivial except for one
instance which required a very small refactor.
2024-02-28 16:36:49 -05:00
James Shubin
8db41e7701 cli: Rename confusing obj variable name 2024-02-28 16:36:49 -05:00
James Shubin
37569aae17 lang: core: Add package documentation string 2024-02-28 16:36:49 -05:00
James Shubin
ffd6385dd5 lang: inputs: Add visualization of parsed input struct
Occasionally helps with debugging.
2024-02-28 16:36:49 -05:00
James Shubin
abe3e0e7a5 lib: Split off the CLI portions into a separate package
This cleans things up a bit more and forces the lib package to not
contain any accidental CLI parsing code.
2024-02-28 16:36:49 -05:00
James Shubin
70b5ed7067 lang: Add an embedded package for embedded imports
This adds a new "embedded" package which can be used to import
system-like packages that are embedded into the binary.
2024-02-28 16:36:49 -05:00
James Shubin
9d8beb85d7 lang: ast: Refactor import scope for sanity
This makes it easier to integrate future additions to the import code.
2024-02-28 16:36:49 -05:00
James Shubin
296fc484ba test: shell: Disable flaky test
Legacy yaml anyways...
2024-02-22 17:41:13 -05:00
James Shubin
ad900fc8f1 etcd: Store the scheme as a constant 2024-02-22 17:41:13 -05:00
James Shubin
f60c25aacf util: Add afero relpath implementation
This new filesystem implements a relative filesystem modifier which can
be useful for converting between absolute filesystems rooted at / and
relative ones. This is particularly useful when interfacing with the
golang embed package.

The upstream requires a CLA, so we'll just store this here instead.
https://github.com/spf13/afero/pull/417
2024-02-22 17:41:13 -05:00
James Shubin
d6cf595899 lang: Unnested the core package from the functions dir
The core package could contain non-functions, so we might as well move
it upwards.
2024-02-22 17:19:02 -05:00
James Shubin
9329ed1e37 lang: ast: Add a new flavour and signature for scope importing
We'll need this more flexible version for the future.
2024-02-22 13:46:27 -05:00
James Shubin
90628dc5c1 engine: Improve docs
Things were messy here, give this a small cleanup.
2024-02-22 13:46:27 -05:00
James Shubin
73ae197d20 util: Add scheme and path support to our fs util struct
This lets us have a custom URI when wrapping an Afero.Fs interface.
2024-02-22 13:46:27 -05:00
James Shubin
15fa6b82a5 lang: funcs: core: Formatting cleanup 2024-02-22 13:36:46 -05:00
James Shubin
d887e7fea5 util: More error handling 2024-02-22 13:36:46 -05:00
James Shubin
871f0e73c0 lang, lib, util: Rename fs to be more unique
Trying to do a big refactor and this will help.
2024-02-22 13:36:46 -05:00
James Shubin
d117cb8ed5 util: Improve tree printing function
This makes it behave more like the core GNU tree util.
2024-02-22 13:36:46 -05:00
James Shubin
733d7fb55f lang: ast: Clean up code slightly 2024-02-22 13:15:39 -05:00
James Shubin
6ae3481ae9 engine, lang, gapi: Split out some functions to a writeable API
Start breaking down the filesystem interface to make things more
flexible.
2024-02-22 13:15:39 -05:00
James Shubin
b7efd94147 lang: Pass through the fs and be consistent in usage
This simplifies the API by passing through the filesystem so that
function signatures don't need to be as complicated, and furthermore use
that consistently throughout.
2024-02-22 13:15:39 -05:00
James Shubin
a05b6a927e lang: funcs: core: os: Add an args function to read argv 2024-02-22 13:15:39 -05:00
James Shubin
30648a7858 lib: Print out timing from core graph generation
This should help us determine what steps need improving. As it turns
out, autoedges are approximately the slowest. (Highly dependent on the
specific mcl code being used.)

The trend is clear though; note the units:

main: new graph took: 913.653µs
main: auto edges took: 9.273807153s
main: auto grouping took: 28.690819ms
main: send/recv building took: 566ns

main: new graph took: 779.255µs
main: auto edges took: 4.03670168s
main: auto grouping took: 37.682101ms
main: send/recv building took: 121.017µs

main: new graph took: 1.157479ms
main: auto edges took: 3.794132165s
main: auto grouping took: 49.732836ms
main: send/recv building took: 95.921µs

main: new graph took: 900.937µs
main: auto edges took: 7.206085s
main: auto grouping took: 25.508671ms
main: send/recv building took: 489ns

main: new graph took: 794.224µs
main: auto edges took: 4.313729756s
main: auto grouping took: 47.970533ms
main: send/recv building took: 207.62µs

main: new graph took: 884.49µs
main: auto edges took: 7.585529786s
main: auto grouping took: 24.327938ms
main: send/recv building took: 72.741µs

main: new graph took: 774.157µs
main: auto edges took: 2.827380129s
main: auto grouping took: 28.303023ms
main: send/recv building took: 85.246µs

main: new graph took: 746.841µs
main: auto edges took: 2.775868117s
main: auto grouping took: 33.11291ms
main: send/recv building took: 104.875µs

main: new graph took: 796.445µs
main: auto edges took: 2.71556122s
main: auto grouping took: 24.03827ms
main: send/recv building took: 106.414µs

main: new graph took: 1.217452ms
main: auto edges took: 2.908416104s
main: auto grouping took: 61.175916ms
main: send/recv building took: 92.328µs

main: new graph took: 807.894µs
main: auto edges took: 3.222089261s
main: auto grouping took: 40.032629ms
main: send/recv building took: 106.49µs

main: new graph took: 986.963µs
main: auto edges took: 3.538425263s
main: auto grouping took: 30.660849ms
main: send/recv building took: 99.74µs
2024-02-22 13:15:39 -05:00
James Shubin
26bce5ddc0 engine: resources: Add a new http:proxy resource
This adds a new caching http proxy resource that can be autogrouped into
the core http:server resource, and which caches and serves files that it
received from a different http server. It first pulls them down when
they are initially requested, which makes it possible for us to use this
for provisioning a Linux installation without having to pre-rsync the
entire package repository.
2024-02-22 13:15:39 -05:00
James Shubin
bd708159a1 engine: resources: Close files after use
Don't need to wait for gc.
2024-02-22 13:15:39 -05:00
James Shubin
87ce637bd2 engine: resources: Add a helper function for creating errors 2024-02-22 13:15:39 -05:00
James Shubin
7698b1b5fd engine: resources: Add an http error wrapper
This makes it easier to return errors more succinctly.
2024-02-16 08:06:20 -05:00
James Shubin
e256d886e0 engine: resources: Add a fancy http error wrapper
This lets use use a regular error signature but also specify which http
error code we want to return.
2024-02-16 08:06:20 -05:00
James Shubin
6f268e3a40 engine: resources: Fix up docstring
This was a copy+pasta mistake.
2024-02-16 08:06:20 -05:00
James Shubin
57910470a9 docs: Add new talks from Belgium 2024 2024-02-09 10:49:35 -05:00
James Shubin
890b6e9a28 docs: Add empty list declarations to style guide
This seems to have come up lately, so add my reasoning. Debate is
welcome-- to a point.
2024-02-09 04:48:16 -05:00
Eng Zer Jun
b09b21e939 lang: ast: Remove redundant nil check
From the Go specification [1]:

  "1. ... For a nil slice, the number of iterations is 0."

Therefore, an additional nil check for around the loop is unnecessary.

[1]: https://go.dev/ref/spec#For_range

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2024-02-09 04:31:00 +08:00
James Shubin
edf47a1737 lang: gapi, docs: Add a --skip-unify option to speed up run
This is useful for run and dangerous for deploy. If we make type
unification blazing fast, we should probably get rid of this option.
2024-02-08 10:49:22 -05:00