Commit Graph

1709 Commits

Author SHA1 Message Date
James Shubin
3739fa401e engine: resources: Refactor HTTP constants 2024-01-22 17:43:24 -05:00
James Shubin
377d62999f engine: resources: The kv resource can set un-mapped values
Previously the resource could only set values in a per-hostname
namespace, but for single, user-managed values, we'd like to be able to
control things entirely. Now this resource can do that.
2024-01-22 17:43:24 -05:00
James Shubin
6f1f69683d lang: funcs: core: world: Add a function to get world values
Useful for following single values that we set.
2024-01-22 17:43:24 -05:00
James Shubin
b4bb4ca397 engine: resources: Update kv error messages 2024-01-22 17:43:24 -05:00
James Shubin
6b4fb434da engine: resources: Rename var
More accurate.
2024-01-22 17:43:24 -05:00
James Shubin
9e70f53afa engine: resources: Add missing apply guard 2024-01-22 17:43:24 -05:00
James Shubin
81885dec63 engine: resources: Add some spacing 2024-01-22 17:43:24 -05:00
James Shubin
f59f3c3c83 lang: funcs: core: convert: Add format and parse for bool 2024-01-22 17:43:24 -05:00
James Shubin
18c66ae7ac lang: funcs: core: Remove unused struct fields and bool
Old copy-pasta misc!
2024-01-22 17:43:24 -05:00
James Shubin
d01c168450 lang: Add per-test config with count maximums
Some of our special tests can only be run once per `go test` invocation.
That is, using the test -count flag will cause a guaranteed failure
since we depend on a global being initialized only once as part of that
test.

This adds a per-test config option so that a user can specify to never
run a particular test more than once. This lets us continue to use the
-count flag with the test suite, without it causing some tests to fail.
2024-01-22 16:55:45 -05:00
James Shubin
f0a4a9e3c4 lang: Don't print test comments if empty 2024-01-22 16:53:47 -05:00
James Shubin
78e59a9400 github: New versions of github actions 2024-01-22 15:52:49 -05:00
James Shubin
a8f194259b legal: Happy 2024 everyone...
Done with:

ack '2023+' -l | xargs sed -i -e 's/2023+/2024+/g'

Checked manually with:

git add -p

Hello to future James from 2025, and Happy Hacking!
2024-01-22 15:52:49 -05:00
James Shubin
c39fdcb8ac lang: funcs: dage: Have the compiler verify expectations
This engine should fulfill the GraphAPI interface. Specify that here in
case it's not obvious.
2024-01-22 15:52:49 -05:00
James Shubin
26e46a6e18 lang: funcs: txn: Add a simple graph implementation
This fulfills the GraphAPI that we use.
2024-01-22 15:52:49 -05:00
James Shubin
4ad7edc35e lang: funcs: txn: Move transaction code to a new package
This also makes it public, although it is designed for internal use
only.
2024-01-22 15:38:18 -05:00
James Shubin
28eacdb2bb lang: funcs: ref: Move reference counting code to a new package
This also makes it public, although it is designed for internal use
only.
2024-01-22 15:38:18 -05:00
James Shubin
66b826a8e1 pgraph, lang: funcs: dage: Add an awkward graphviz interface
This refactors the code to make it not depend on the specific engine.
2024-01-22 15:35:21 -05:00
James Shubin
20c8a856a2 lang: funcs: dage: Add a Graph method to improve the API
This also adds it to the GraphAPI so that users of it can pull out a
graph when needed. It's not likely to be used by the dage engine
consumers.
2024-01-22 15:09:32 -05:00
James Shubin
dd20bd5486 lang: ast: Improve ordering to eliminate false positives
The Ordering and DAG detection code is challenging because we need
Ordering to do SetScope, but Ordering itself needs to know about scopes.
This improved variant should hopefully catch all the scenarios of
identically named variables causing invalid loops.

Co-authored-by: Samuel Gélineau <gelisam@gmail.com>
2024-01-22 13:19:10 -05:00
James Shubin
f8077d9fc4 lang: ast: Add a util function for node map copying 2024-01-22 13:07:17 -05:00
Samuel Gélineau
16dfb7b5f5 lang: Add more test cases related to future graph optimization
We are planning to implement an optimization in which some function
calls are compiled to a single static graph rather than to a CallFunc
which dynamically creates a sub-graph at runtime. These test cases
exercise corner cases for which it would be theoretically possible to
use a static graph, but which we might not be able to optimize.
2024-01-20 15:01:56 -05:00
James Shubin
aae0e16350 lang: ast, parser, interfaces: Implementation of nested class sugar
This implements a new type of syntactic sugar for the common pattern of
a base class which returns a child class, and so on. Instead of needing
to repeatedly indent the child classes, we can instead prefix them at
the definition site (where created with the class keyword) with the name
of the parent class, followed by a colon, to get the desired embedded
sugar.

For example, instead of writing:

class base() {
	class inner() {
		class deepest() {
		}
	}
}

You can instead write:

class base() {
}
class base:inner() {
}
class base:inner:deepest() {
}

Of course, you can only access any of the inner classes by first
including (with the include keyword) a parent class, and then
subsequently including the inner one.
2024-01-14 17:10:31 -05:00
James Shubin
741a71b490 lang: parser: Clean up the parser to avoid parsing res separately
This cleans up some cruft in the parser, and makes it easier for us to
extend this to future colon separated identifiers.
2024-01-14 17:10:31 -05:00
James Shubin
44ee578a3a lang: parser, ast, interfaces: Implement include as
This adds support for `include as <identifier>` type statements which in
addition to pulling in any defined resources, it also makes the contents
of the scope of the class available to the scope of the include
statement, but prefixed by the identifier specified.

This makes passing data between scopes much more powerful, and it also
allows classes to return useful classes for subsequent use.

This also improves the SetScope procedure and adds to the Ordering
stage. It's unclear if the current Ordering stage can handle all code,
or if there exist corner-cases which are valid code, but which would
produce a wrong or imprecise topological sort.

Some extraneous scoping bugs still exist, which expose certain variables
that we should not depend on in future code.

Co-authored-by: Samuel Gélineau <gelisam@gmail.com>
2024-01-14 17:08:51 -05:00
James Shubin
f92f34dc54 lang: The in keyword can't be a variable name
At least for now until we figure out some fancier parser magic. (If
at all possible!)
2024-01-12 18:22:00 -05:00
James Shubin
dff9c9ad22 lang: ast: Add more validation on struct fields
Catch more errors early, just in case.
2024-01-12 18:11:39 -05:00
James Shubin
acae7eccc4 lang: interfaces: Expand docs on compiler constants
Improve the docs to make things clearer.
2024-01-07 18:44:58 -05:00
James Shubin
837739d7dd pgraph: Implement FilterGraph in terms of FilterGraphWithFn
Simplify the code base by having one core implementation.
2024-01-07 18:44:58 -05:00
James Shubin
2567303fd7 pgraph: Add FilterGraphWithFn implementation
This lets us filter more programmatically.
2024-01-07 18:44:58 -05:00
James Shubin
4939ae1a2f pgraph: FilterGraph doesn't need a name arg
It's not being consumed anywhere, so remove it. If you really want to
rename the graph, this can be done as a second step.
2024-01-07 18:25:50 -05:00
James Shubin
58607e2ca0 pgraph: Filter graph should not include unfiltered vertices
This adds a test for another sneaky aspect of filter graph: It should
not pull in a vertex because another across the edge was included. If we
want this behaviour, then we need a different function or a config
option for this filter function.
2024-01-07 18:24:56 -05:00
James Shubin
cb021e3a25 pgraph: Filter graph should preserve vertices without an edge
There was a bug in filter graph that caused it to not preserve any
vertices that weren't bound to another with an edge. This fixes the bug
and adds a test too.
2024-01-07 17:56:02 -05:00
James Shubin
d61936c06d test: Add some simple commit body tests
I never remember the capitalization rules, so hopefully this helps me!
2024-01-05 16:57:42 -05:00
Samuel Gélineau
3553eb1f2a lang: ast: Fix data race in ExprSingleton
Init the mutex everywhere, but consider calling Init instead and
plumbing though the data input field in the future.
2024-01-05 16:27:41 -05:00
James Shubin
c4a9560d53 engine: local: Fix benign race
Use the mutex in a safer manner to eliminate the benign data race.
2024-01-05 15:50:43 -05:00
James Shubin
bc63b7608e engine: graph: Use an atomic bool instead of a mutex
The isStateOK variable can be accessed concurrently as these are
supposed to be "benign" races. As such, they need to be labelled as such
so that we don't hit some undefined compiler behaviour.

Here are five good references relating to "benign" data races in golang.

1) https://web.archive.org/web/20181022150257/https://software.intel.com/en-us/blogs/2013/01/06/benign-data-races-what-could-possibly-go-wrong

2) https://go.dev/ref/mem - "Informal Overview" section.

3) https://docs.oracle.com/cd/E19205-01/820-0619/gecqt/index.html

4) https://www.usenix.org/legacy/event/hotpar11/tech/final_files/Boehm.pdf

5) https://go.dev/doc/articles/race_detector

TL;DR: wrap your benign races with sync/atomic or eliminate them.
2024-01-05 15:50:42 -05:00
James Shubin
c2f508e261 engine: graph: Use an rwmutex around read/write of vertex state
This fixes two small races we had while simultaneously reading from and
writing to the vertex timestamp, and simultaneously writing to the
vertex isStateOK (dirty) flag.

They were actually "safe" races, in that it doesn't matter if the
read/write race got the old or new value, or that the double write
happened. The time sequencing was correct (I believe) in both cases, but
this triggers the race detector now that we have tests for it.
2024-01-02 18:18:34 -05:00
James Shubin
a07dc0a511 lang: interpolate, funcs: Add a concat function for faster interpolation
This adds a concat function which can be used directly by string
interpolation to avoid having to constantly unify the plus operator
which is much slower at this time.

The new monomorphisms changes caused type unification of a notable
example to go from ~25s to ~5m30s which was obviously not bearable. With
this fix, things are now down to ~6s.

This is an important optimization, but it's also a good reminder that
type unification of polymorphic functions needs to be improved in
general too.
2024-01-02 15:23:40 -05:00
James Shubin
d8db320722 lang: gapi: Prettier printing of type unification timing
This makes things a little nicer to visualize.
2024-01-02 13:26:51 -05:00
James Shubin
24054f905f lib, lang, docs: Add --only-unify option
This adds a new run flag for the lang frontend to exit immediately
following type unification. This makes it easier to use this as a step
in CI, and also to type the execution for performance comparison
reasons.
2024-01-02 13:06:19 -05:00
James Shubin
32ca815896 lib, docs: Small cleanups
Formatting and removing of old docs.
2024-01-02 13:04:49 -05:00
James Shubin
fa5949e191 lang: funcs: core: test: Make a new instance for each test
Since this special one_instance function uses global state, if it's
re-used in more than one test, this won't work since they still all use
the whole global state. Make new ones for each test.

This also breaks the count=2 feature (any number other than 1) when
running these, which is not ideal. Create a cleanup API that we can run
between tests to reset the global state.
2023-12-27 18:25:53 -05:00
Samuel Gélineau
1c0a98a0cc lang: ast: ExprBind is now monomorphic
This adds ExprTopLevel and ExprSingleton and ensures that ExprBind is
now monomorphic.

This corrects a previous design bug where it was not monomorphic and
would thus cause spawning of many more copies than necessary. In most
cases this was only harmful to memory and performance, and not
behaviour, since these functions were pure, and we didn't have a test
for this.

This also adds a bunch more tests. Most notably, the graph shape tests
generally produce smaller graphs now.

Lastly, a lambda cannot have two different types when used at two
different call sites. It is rare that this would be used, and when it
would make sense, there are easy workarounds to accomplish equivalent
goals.

This was mostly authored by Sam, James helped with some cleanup and
debugging.

Co-authored-by: James Shubin <james@shubin.ca>
2023-12-27 16:33:38 -05:00
James Shubin
9d208e8795 lang: Test more iter funcs without polymorphic id function
This tests the lambdas in more ways so that we are sure non-polymorphic
$id functions work the way we want.
2023-12-27 16:33:38 -05:00
James Shubin
72fe0cd6db lang: Polymorphic lambda functions aren't allowed for now
We don't allow a lambda variable to be polymorphic anymore. Of course
this isn't bad, but it makes things too difficult, at least for now.
It's also likely that we don't need this specific feature very often I
think.
2023-12-27 16:33:38 -05:00
James Shubin
734590b6bd lang: Add a test for duplicate functions called
We should not call either of these functions more than once for their
values. If we do, it means we have made a mistake with a compiler
optimization.

This is important, because otherwise if you had code like:

$x = random_password()

Then this would obviously be a problem. Thankfully, the situations where
functions generate unique data is rare, but it's probably something we
should take care of.
2023-12-27 16:33:38 -05:00
James Shubin
7cc231e8b9 lang: unification, interfaces: Add a skip invariant
This is a cleaner way of telling the type unifier that we don't want
this particular expression in the solution set at the end.
2023-12-27 16:33:38 -05:00
James Shubin
c2bf4ef7d4 lang: Test a top-level var explicitly
This makes sure the top-level scope is really seen.
2023-12-27 16:33:38 -05:00
James Shubin
439179e37f lang: Remove duplicate test
It's contents were identical to the stmtfunc-recursive-double.txtar
test.
2023-12-27 16:33:38 -05:00