710 Commits

Author SHA1 Message Date
James Shubin
6de7d8b254 lang: funcs: Catch non-specific type build error
If you had ambiguous code, and specified an invalid type, this could
sneak through and become a runtime error, instead of a compile-time
error. We fix this and add a test.
2024-07-31 17:29:42 -04:00
James Shubin
bfb5d983c1 lang: types, unification: Don't recurse into private fields
We forgot to omit looking deeper into private struct fields. I don't
know why we didn't catch this earlier, I can only assume some subtlety
changed, since we've previously used many of the resources this would
fail on. Maybe golang broke some API that they didn't consider stable?

This also adds a new test for this, and ensures each resource can be
inspected too!
2024-07-31 17:29:42 -04:00
James Shubin
0a183dfff9 lang: funcs: txn, util: Fix typos 2024-07-25 12:44:58 -04:00
James Shubin
aa03b5ce2f lang: core: iter: Add filter iterator function
This was fun to write and adds a new core iterator function.
2024-07-03 21:25:19 -04:00
James Shubin
d1753c592a lang: core: iter: Misc formatting fixes
Also fix up the examples.
2024-07-02 23:49:56 -04:00
James Shubin
e10e92596f lang: types: Add stringer information manually
This lets us get the more correct lowercase versions of type kinds in
error messages. (These match what the user would type.)
2024-07-01 18:35:20 -04:00
James Shubin
28253c4bd2 lang: Move stateful test objects into a per-test mode
Was this causing failures? Does this make things much slower?
2024-07-01 18:34:42 -04:00
James Shubin
f2976deb02 pgraph, lang: ast: Fix failing tests due to non-deterministic topo sort
This causes inconsistent type unification when running our tests. It's a
bad user experience too.
2024-07-01 18:34:24 -04:00
James Shubin
14577a0c46 lang: Add modern type unification implementation
This adds a modern type unification algorithm, which drastically
improves performance, particularly for bigger programs.

This required a change to the AST to add TypeCheck methods (for Stmt)
and Infer/Check methods (for Expr). This also changed how the functions
express their invariants, and as a result this was changed as well.

This greatly improves the way we express these invariants, and as a
result it makes adding new polymorphic functions significantly easier.

This also makes error output for the user a lot better in pretty much
all scenarios.

The one downside of this patch is that a good chunk of it is merged in
this giant single commit since it was hard to do it step-wise. That's
not the end of the world.

This couldn't be done without the guidance of Sam who helped me in
explaining, debugging, and writing all the sneaky algorithmic parts and
much more. Thanks again Sam!

Co-authored-by: Samuel Gélineau <gelisam@gmail.com>
2024-07-01 18:33:47 -04:00
James Shubin
4e18c9c67a lang: Plumb through the unified state facility 2024-07-01 16:07:14 -04:00
James Shubin
d326917432 lang: interfaces: Add some unification basics
This includes the GenericCheck helper which we'll use everywhere, and
the standard single invariant which we use throughout.
2024-07-01 16:07:14 -04:00
James Shubin
ad4eb86262 lang: unification: util: Add the core unification helpers
This adds the core unification helper functions that do the core work of
solving the invariants. This includes the actual Unify, OccursCheck, and
Extract which is sometimes known as "zonk".

A few other small functions are also included.

Co-authored-by: Samuel Gélineau <gelisam@gmail.com>
2024-07-01 16:06:58 -04:00
James Shubin
5c73e7c582 lang: types: Add a facility for printing consistent unification vars
When we look at unification variables from two different places, the
default printer will always start numbering them from ?1 and therefore
if we look at two unrelated systems, they might both print as ?1 when
they are in fact different pointers.

We don't collect them all by default since it's usually not necessary
except for debugging, but in those situations, we want a consistent
unification store which we can pass around to get sensible debug output.
2024-07-01 14:16:11 -04:00
James Shubin
dc33d9aab7 lang: types: Add a comparable helper to our types library 2024-07-01 14:14:20 -04:00
James Shubin
cdc2439f89 lang: types: Add an iterator helper to our types lib 2024-07-01 14:13:41 -04:00
James Shubin
318ee0d002 lang: types: Fixup small log messages 2024-07-01 14:12:22 -04:00
James Shubin
653299a88f lang: types: Plumb in a unification variable into our type
This is used for representing a unification variable in our type during
type unification. For example, this allows us to have a [?1] or a
map{?1:[?2]} and so on...
2024-07-01 14:11:03 -04:00
James Shubin
3c661ab674 lang: core: embedded: provisioner: Version flag needs a unique name
Conflicts with the stock --version.
2024-05-23 07:57:30 -04:00
James Shubin
415e22abe2 lang: core, funcs, types: Add ctx to simple func
Plumb through the standard context.Context so that a function can be
cancelled if someone requests this. It makes it less awkward to write
simple functions that might depend on io or network access.
2024-05-09 19:25:46 -04:00
James Shubin
453cd4409e lang: ast: Remove unnecessary metaparam exclusive
Originally, I considered having more than one way to express the meta
param. After thinking about it for longer, it probably makes sense to
have a second meta param if necessary, and to avoid the exclusive.
2024-04-18 00:44:34 -04:00
James Shubin
51cf1e2921 lang: ast: The res and edge names should not use exclusives
This removes the exclusive from the res names and edge names. We now
require that the names should be lists of strings, however they can
still be single strings if that can be determined statically.
Programmers should explicitly wrap their variables in a string by
interpolation to force this, or in square brackets to force a list. The
former is generally preferable because it generates a small function
graph since it doesn't need to build a list.
2024-04-18 00:07:53 -04:00
James Shubin
dc45c90ccd lang: Add common type to global variables
We use the list of strings so often, we might as well give it a global
variable.
2024-04-16 14:31:03 -04:00
James Shubin
6782d65577 test: shell, lang: core: embedded: provisioner: Check it compiles
Add a test to guarantee we continue to keep compiling, in case something
in the language changes.
2024-04-16 14:30:56 -04:00
James Shubin
68ee163eb1 entry, lang: core: embedded: provisioner: Allow more than one entry
This changes the entry API slightly to allow for more than one entry
registered, which makes building, testing and user tooling easier.
2024-04-16 14:11:34 -04:00
James Shubin
bc4b5d96b0 lang: core: embedded: provisioner: Better name for firewall entries 2024-04-15 15:03:27 -04:00
James Shubin
909dbb531d lang: ast: Fix small typos 2024-04-06 15:32:31 -04:00
James Shubin
1b00af6926 lang: Add unification optimizations
This adds a unification optimizations API, and uses it to optimize the
embedded provisioner. With these turned on, type unification drops from
around 1m45s to 2.5s which is a 40x speedup.
2024-03-30 18:05:16 -04:00
James Shubin
ddf1be653e lang: Plumb data and unification strategy through the lang struct
This adds some plumbing to pass values into the lang struct.
2024-03-30 17:36:49 -04:00
James Shubin
cede7e5ac0 lang: Structurally refactor type unification
This will make it easier to add new solvers and also cleans up some
pending issues.
2024-03-30 16:55:20 -04:00
James Shubin
4d9c78003a lang: core: embedded: Add standalone provisioning tool
This commit adds the ability to build a standalone provisioning tool.
This is the first useful public mcl code base as well. It is not
perfect, but does serve as a rough starting point to show what is
possible. In the future as the language and the engine evolve, this will
likely get more elegant, and also grow new features.

To build this, run `make clean && GOTAGS='embedded_provisioner' make`.
To run this, run `mgmt provisioner`.
2024-03-26 20:33:43 -04:00
James Shubin
89784e86bd lang: gapi, unification: Shutdown unification quickly when asked
This is a bit of a hack until we improve the GAPI a bit, but will let us
shut down type unification a bit faster if we want to interrupt a long
running operation. Hopefully our future algorthmic performance
improvements will obliviate the need for this to be a common issue.
2024-03-22 03:23:46 -04:00
James Shubin
c4b14ac40d lang: download, cli: Move to new git version
Apparently there were some security issues with the old one.
2024-03-22 02:56:59 -04:00
James Shubin
12b519a543 lang: gapi: Improve logging to be cleaner 2024-03-22 02:56:59 -04:00
James Shubin
43b5a0ea6a util: recwatch: Remove last use of core logger
We've cleaned up quite a bit, yay!
2024-03-22 02:56:59 -04:00
James Shubin
946468dc99 lang: ast: Ensure a list doesn't sneak through type interpolation
If we had a single list wrapped in an interpolated string, it could
sneak through type unification, which is not correct. Wrapping a
variable by interpolation in a string, must force it to be a string.
2024-03-20 18:36:47 -04:00
James Shubin
340a832884 lang: ast: Optimize unification invariants for common case
This optimizes the list of type unification invariants that we generate
for the common case where a resource or edge name is known statically.

For one code base this halved the type unification time in half. More
work can be done though!
2024-03-20 18:07:59 -04:00
James Shubin
ba6d816186 lang: unification: Clean up the solver plumbing
This refactors the solver into a separate struct that can be extended as
needed.
2024-03-16 01:38:33 -04:00
James Shubin
4d18044851 util: Move recwatch package to util directory
This makes more sense and this is also a reminded that it needs a bit of
a cleanup.
2024-03-16 01:21:32 -04:00
James Shubin
d6a6734b65 lang: Plumb the ctx deeper into the lang API 2024-03-16 01:21:32 -04:00
James Shubin
10319dd641 lang: Plumb through a context into unification
If we have a long type unification, we might want to cancel it early.
This also helps us visualize where we want context to be seen.
2024-03-16 01:21:32 -04:00
James Shubin
a8b945e36e lang: types, core: Skip over invalid functions
It's not clear how best to support complex functions with struct fields
in templates at the moment. Skip these for now.
2024-03-16 01:21:32 -04:00
James Shubin
719c56e754 lang: core: convert: Add a function to convert an int to a str
Faster than using printf.
2024-03-16 01:21:32 -04:00
James Shubin
cbf10bdb44 lang: core: os: Add a simple distro uid parsing function
I had previous good success using this kind of pattern for discussing
distro UID's. Let's put this in as a reminded to see if it's worth
pursuing longer term.
2024-03-16 01:21:32 -04:00
James Shubin
a410b4981f lang: Add timing information to lang execution
This makes performance optimizing a little easier.
2024-03-16 01:21:32 -04:00
James Shubin
f973009b83 lang: Rephrase the scope building log messages 2024-03-16 00:36:06 -04:00
James Shubin
1f37ac83e4 lang: types: Fix capitalization 2024-03-08 17:57:13 -05:00
James Shubin
9351eee3f1 lang: funcs: simple: Add a function from struct generator
This is a helper function that can generate a bunch of functions from a
struct type. This is most useful when using a CLI args struct for
command line parsing and then storing the values as functions.

An alternative version of this might choose to return all of the values
as a single giant struct.
2024-03-08 17:57:13 -05:00
James Shubin
d412d6502d lang: gapi: Fix a very rare nil pointer issue
This only showed up while debugging a bug I caused, but technically if
we fail in Init() then we could hit this nil pointer condition.
2024-03-08 17:57:13 -05:00
James Shubin
b85f81d529 lang: types: Add simple parsing of net.HardwareAddr
If we want to use special struct types from our CLI parser, we also need
to be able to both identify, and convert them to our language type and
value representations.

For as long as we don't have fancier types in our language, these should
both be strings. Tests and extensions to these additions are welcome!
2024-03-08 17:57:13 -05:00
James Shubin
4140492d56 lang: ast: Embedded imports have function scopes too
If we're importing an embedded module, we need to also include any
possible system scope imports (like functions) that might be using the
same namespace. These might be used by a custom CLI frontend to extend
the code with values from the CLI parsing, for example.
2024-03-08 17:41:57 -05:00