Commit Graph

435 Commits

Author SHA1 Message Date
James Shubin
9dd5dfdde2 engine: graph: autogroup: Improve the autogrouping algorithm
This improves the autogrouping algorithm to support hierarchical
autogrouping. It's not guaranteed to work if we replace the reachability
grouper with something more efficient, but it's good enough for now.
2024-01-22 20:37:53 -05:00
James Shubin
024aa60209 lang: parser: Allow edges of resources with colons
Due to a small copy-pasta bug in the parser, resource edges with kinds
containing colons didn't work. This fixes the mistake and adds a test.
2024-01-22 19:40:15 -05:00
James Shubin
32916f9a6f engine: resources: Receive keys should match mcl, not golang
The capitalization of these keys was wrong and they weren't getting
seen. Add a test as well.
2024-01-22 19:40:15 -05:00
James Shubin
6f1f69683d lang: funcs: core: world: Add a function to get world values
Useful for following single values that we set.
2024-01-22 17:43:24 -05:00
James Shubin
f59f3c3c83 lang: funcs: core: convert: Add format and parse for bool 2024-01-22 17:43:24 -05:00
James Shubin
18c66ae7ac lang: funcs: core: Remove unused struct fields and bool
Old copy-pasta misc!
2024-01-22 17:43:24 -05:00
James Shubin
d01c168450 lang: Add per-test config with count maximums
Some of our special tests can only be run once per `go test` invocation.
That is, using the test -count flag will cause a guaranteed failure
since we depend on a global being initialized only once as part of that
test.

This adds a per-test config option so that a user can specify to never
run a particular test more than once. This lets us continue to use the
-count flag with the test suite, without it causing some tests to fail.
2024-01-22 16:55:45 -05:00
James Shubin
f0a4a9e3c4 lang: Don't print test comments if empty 2024-01-22 16:53:47 -05:00
James Shubin
a8f194259b legal: Happy 2024 everyone...
Done with:

ack '2023+' -l | xargs sed -i -e 's/2023+/2024+/g'

Checked manually with:

git add -p

Hello to future James from 2025, and Happy Hacking!
2024-01-22 15:52:49 -05:00
James Shubin
c39fdcb8ac lang: funcs: dage: Have the compiler verify expectations
This engine should fulfill the GraphAPI interface. Specify that here in
case it's not obvious.
2024-01-22 15:52:49 -05:00
James Shubin
26e46a6e18 lang: funcs: txn: Add a simple graph implementation
This fulfills the GraphAPI that we use.
2024-01-22 15:52:49 -05:00
James Shubin
4ad7edc35e lang: funcs: txn: Move transaction code to a new package
This also makes it public, although it is designed for internal use
only.
2024-01-22 15:38:18 -05:00
James Shubin
28eacdb2bb lang: funcs: ref: Move reference counting code to a new package
This also makes it public, although it is designed for internal use
only.
2024-01-22 15:38:18 -05:00
James Shubin
66b826a8e1 pgraph, lang: funcs: dage: Add an awkward graphviz interface
This refactors the code to make it not depend on the specific engine.
2024-01-22 15:35:21 -05:00
James Shubin
20c8a856a2 lang: funcs: dage: Add a Graph method to improve the API
This also adds it to the GraphAPI so that users of it can pull out a
graph when needed. It's not likely to be used by the dage engine
consumers.
2024-01-22 15:09:32 -05:00
James Shubin
dd20bd5486 lang: ast: Improve ordering to eliminate false positives
The Ordering and DAG detection code is challenging because we need
Ordering to do SetScope, but Ordering itself needs to know about scopes.
This improved variant should hopefully catch all the scenarios of
identically named variables causing invalid loops.

Co-authored-by: Samuel Gélineau <gelisam@gmail.com>
2024-01-22 13:19:10 -05:00
James Shubin
f8077d9fc4 lang: ast: Add a util function for node map copying 2024-01-22 13:07:17 -05:00
Samuel Gélineau
16dfb7b5f5 lang: Add more test cases related to future graph optimization
We are planning to implement an optimization in which some function
calls are compiled to a single static graph rather than to a CallFunc
which dynamically creates a sub-graph at runtime. These test cases
exercise corner cases for which it would be theoretically possible to
use a static graph, but which we might not be able to optimize.
2024-01-20 15:01:56 -05:00
James Shubin
aae0e16350 lang: ast, parser, interfaces: Implementation of nested class sugar
This implements a new type of syntactic sugar for the common pattern of
a base class which returns a child class, and so on. Instead of needing
to repeatedly indent the child classes, we can instead prefix them at
the definition site (where created with the class keyword) with the name
of the parent class, followed by a colon, to get the desired embedded
sugar.

For example, instead of writing:

class base() {
	class inner() {
		class deepest() {
		}
	}
}

You can instead write:

class base() {
}
class base:inner() {
}
class base:inner:deepest() {
}

Of course, you can only access any of the inner classes by first
including (with the include keyword) a parent class, and then
subsequently including the inner one.
2024-01-14 17:10:31 -05:00
James Shubin
741a71b490 lang: parser: Clean up the parser to avoid parsing res separately
This cleans up some cruft in the parser, and makes it easier for us to
extend this to future colon separated identifiers.
2024-01-14 17:10:31 -05:00
James Shubin
44ee578a3a lang: parser, ast, interfaces: Implement include as
This adds support for `include as <identifier>` type statements which in
addition to pulling in any defined resources, it also makes the contents
of the scope of the class available to the scope of the include
statement, but prefixed by the identifier specified.

This makes passing data between scopes much more powerful, and it also
allows classes to return useful classes for subsequent use.

This also improves the SetScope procedure and adds to the Ordering
stage. It's unclear if the current Ordering stage can handle all code,
or if there exist corner-cases which are valid code, but which would
produce a wrong or imprecise topological sort.

Some extraneous scoping bugs still exist, which expose certain variables
that we should not depend on in future code.

Co-authored-by: Samuel Gélineau <gelisam@gmail.com>
2024-01-14 17:08:51 -05:00
James Shubin
f92f34dc54 lang: The in keyword can't be a variable name
At least for now until we figure out some fancier parser magic. (If
at all possible!)
2024-01-12 18:22:00 -05:00
James Shubin
dff9c9ad22 lang: ast: Add more validation on struct fields
Catch more errors early, just in case.
2024-01-12 18:11:39 -05:00
James Shubin
acae7eccc4 lang: interfaces: Expand docs on compiler constants
Improve the docs to make things clearer.
2024-01-07 18:44:58 -05:00
Samuel Gélineau
3553eb1f2a lang: ast: Fix data race in ExprSingleton
Init the mutex everywhere, but consider calling Init instead and
plumbing though the data input field in the future.
2024-01-05 16:27:41 -05:00
James Shubin
c2f508e261 engine: graph: Use an rwmutex around read/write of vertex state
This fixes two small races we had while simultaneously reading from and
writing to the vertex timestamp, and simultaneously writing to the
vertex isStateOK (dirty) flag.

They were actually "safe" races, in that it doesn't matter if the
read/write race got the old or new value, or that the double write
happened. The time sequencing was correct (I believe) in both cases, but
this triggers the race detector now that we have tests for it.
2024-01-02 18:18:34 -05:00
James Shubin
a07dc0a511 lang: interpolate, funcs: Add a concat function for faster interpolation
This adds a concat function which can be used directly by string
interpolation to avoid having to constantly unify the plus operator
which is much slower at this time.

The new monomorphisms changes caused type unification of a notable
example to go from ~25s to ~5m30s which was obviously not bearable. With
this fix, things are now down to ~6s.

This is an important optimization, but it's also a good reminder that
type unification of polymorphic functions needs to be improved in
general too.
2024-01-02 15:23:40 -05:00
James Shubin
d8db320722 lang: gapi: Prettier printing of type unification timing
This makes things a little nicer to visualize.
2024-01-02 13:26:51 -05:00
James Shubin
24054f905f lib, lang, docs: Add --only-unify option
This adds a new run flag for the lang frontend to exit immediately
following type unification. This makes it easier to use this as a step
in CI, and also to type the execution for performance comparison
reasons.
2024-01-02 13:06:19 -05:00
James Shubin
fa5949e191 lang: funcs: core: test: Make a new instance for each test
Since this special one_instance function uses global state, if it's
re-used in more than one test, this won't work since they still all use
the whole global state. Make new ones for each test.

This also breaks the count=2 feature (any number other than 1) when
running these, which is not ideal. Create a cleanup API that we can run
between tests to reset the global state.
2023-12-27 18:25:53 -05:00
Samuel Gélineau
1c0a98a0cc lang: ast: ExprBind is now monomorphic
This adds ExprTopLevel and ExprSingleton and ensures that ExprBind is
now monomorphic.

This corrects a previous design bug where it was not monomorphic and
would thus cause spawning of many more copies than necessary. In most
cases this was only harmful to memory and performance, and not
behaviour, since these functions were pure, and we didn't have a test
for this.

This also adds a bunch more tests. Most notably, the graph shape tests
generally produce smaller graphs now.

Lastly, a lambda cannot have two different types when used at two
different call sites. It is rare that this would be used, and when it
would make sense, there are easy workarounds to accomplish equivalent
goals.

This was mostly authored by Sam, James helped with some cleanup and
debugging.

Co-authored-by: James Shubin <james@shubin.ca>
2023-12-27 16:33:38 -05:00
James Shubin
9d208e8795 lang: Test more iter funcs without polymorphic id function
This tests the lambdas in more ways so that we are sure non-polymorphic
$id functions work the way we want.
2023-12-27 16:33:38 -05:00
James Shubin
72fe0cd6db lang: Polymorphic lambda functions aren't allowed for now
We don't allow a lambda variable to be polymorphic anymore. Of course
this isn't bad, but it makes things too difficult, at least for now.
It's also likely that we don't need this specific feature very often I
think.
2023-12-27 16:33:38 -05:00
James Shubin
734590b6bd lang: Add a test for duplicate functions called
We should not call either of these functions more than once for their
values. If we do, it means we have made a mistake with a compiler
optimization.

This is important, because otherwise if you had code like:

$x = random_password()

Then this would obviously be a problem. Thankfully, the situations where
functions generate unique data is rare, but it's probably something we
should take care of.
2023-12-27 16:33:38 -05:00
James Shubin
7cc231e8b9 lang: unification, interfaces: Add a skip invariant
This is a cleaner way of telling the type unifier that we don't want
this particular expression in the solution set at the end.
2023-12-27 16:33:38 -05:00
James Shubin
c2bf4ef7d4 lang: Test a top-level var explicitly
This makes sure the top-level scope is really seen.
2023-12-27 16:33:38 -05:00
James Shubin
439179e37f lang: Remove duplicate test
It's contents were identical to the stmtfunc-recursive-double.txtar
test.
2023-12-27 16:33:38 -05:00
James Shubin
c333cb542c lang: ast: Expr Param and Poly should not have values
Sometimes a recursive call through ExprVar's Value method would hit one
of these and return (nil, nil) which would throw off things.
2023-12-17 21:39:06 -05:00
James Shubin
6a6546db8d lang: ast: Save the non-polyfunc args
We probably want to save these args too in case.
2023-12-16 14:26:29 -05:00
James Shubin
a6b6aa570e test: Fix small test warnings
These were issues we should be catching.
2023-12-12 00:01:29 -05:00
James Shubin
8fcaa4abf2 lang: Output of tests should also show autogrouped resources
This makes our test cases slightly more complete.
2023-12-08 18:25:54 -05:00
James Shubin
18e1f08156 engine: graph: Allow send/recv to work with autogrouped resources
We've previously not received a value from within an autogrouped
resource. It turns out this would be quite useful, and so this patch
implements the additional plumbing and testing so that this works!

Testing that an autogrouped resource can still send values has not been
done at this time.
2023-12-08 18:18:17 -05:00
James Shubin
b7d8a769db lang: funcs: core: Add value functions
This adds a new series of "get*" functions which can read values from
the associated "value" resources. The key name of the function must
match the name value of the resource for things to work.

Type unification isn't yet perfect in these scenarios, so you should use
casually and with caution.
2023-12-03 18:24:20 -05:00
James Shubin
c05af3b9b3 lib, lang, engine: local: Add Value mechanisms to local API
This expands the Local API with the first (and in theory, only ever) API
for reading and writing simple values. This is a coordination point for
resources and functions to share things directly.
2023-12-03 18:24:20 -05:00
James Shubin
3e37a60635 lang: funcs: core: test: Fix small formatting issue 2023-12-03 18:24:20 -05:00
James Shubin
9d47b6843f engine, gapi, lang, lib: Plumb through new local API
This is a new API that is similar in spirit and plumbing to the World
API, but it intended for all local machine operations and will likely
only ever have one implementation.
2023-12-03 18:24:20 -05:00
James Shubin
12ffac1f06 lang: Add placeholders for future prefix passing 2023-12-03 18:02:00 -05:00
James Shubin
7991b4ab25 engine, lang, lib: Re-order for consistency
Some small cleanups to make room for future patches.
2023-12-03 18:02:00 -05:00
Samuel Gélineau
55932cf3c7 lang: ast: Clear env on ExprCall.Graph()
Fix the same bug in ExprCall.Graph().
2023-12-02 01:19:31 -05:00
Samuel Gélineau
1ff2c9bbd9 lang: ast: Clear env in ExprVar.Graph()
Fix the bug in ExprVar.Graph() described in the previous commit.
2023-12-02 01:19:31 -05:00