Commit Graph

101 Commits

Author SHA1 Message Date
James Shubin
55eeb50fb4 lang: Refactor all the highlight helping together
Keep this cleaner and add a bit more.
2025-06-07 17:52:15 -04:00
James Shubin
58461323b9 lang: parser: Try to add the end values in parser
Not sure if this is right, but it's a start.
2025-06-06 03:11:06 -04:00
James Shubin
75bafa4fd3 mcl, docs: Use the less ambiguous form of the import
Update the style guide as well!
2025-06-05 22:47:38 -04:00
James Shubin
654e958d3f engine: resources: Add the proper prefix to grouped http resources
Resources that can be grouped into the http:server resource must have
that prefix. Grouping is basically hierarchical, and without that common
prefix, it means we'd have to special-case our grouping algorithm.
2025-05-25 01:40:25 -04:00
James Shubin
876834ff29 lang: core: fmt: Catch printf edge case 2025-05-25 01:17:28 -04:00
James Shubin
ad0dd44130 engine: Don't force validation for hidden resources
I think this is what I want in most scenarios, is there a reason to do
otherwise? This is because we may wish to export incomplete resources,
where the remaining necessary fields for validation happens on collect.
2025-05-06 03:36:01 -04:00
James Shubin
807c4b3430 engine: resources: Add an http ui resource
Many years ago I built and demoed a prototype of a simple web ui with a
slider, and as you moved it left and right, it started up or shutdown
some number of virtual machines.

The webui was standalone code, but the rough idea of having events from
a high-level overview flow into mgmt, was what I wanted to test out. At
this stage, I didn't even have the language built yet. This prototype
helped convince me of the way a web ui would fit into everything.

Years later, I build an autogrouping prototype which looks quite similar
to what we have today. I recently picked it back up to polish it a bit
more. It's certainly not perfect, and might even be buggy, but it's
useful enough that it's worth sharing.

If I had more cycles, I'd probably consider removing the "store" mode,
and replace it with the normal "value" system, but we would need the
resource "mutate" API if we wanted this. This would allow us to directly
change the "value" field, without triggering a graph swap, which would
be a lot less clunky than the "store" situation.

Of course I'd love to see a GTK version of this concept, but I figured
it would be more practical to have a web ui over HTTP.

One notable missing feature, is that if the "web ui" changes (rather
than just a value changing) we need to offer to the user to reload it.
It currently doesn't get an event for that, and so don't confuse your
users. We also need to be better at validating "untrusted" input here.

There's also no major reason to use the "gin" framework, we should
probably redo this with the standard library alone, but it was easier
for me to push out something quick this way. We can optimize that later.

Lastly, this is all quite ugly since I'm not a very good web dev, so if
you want to make this polished, please do! The wasm code is also quite
terrible due to limitations in the compiler, and maybe one day when that
works better and doesn't constantly deadlock, we can improve it.
2025-05-02 02:14:14 -04:00
Lourenço Vales
ae68dd79cb lang: core: iter: Add a range function
This commit implements a range function that mimicks python's range
built-in by having a start, stop, and range argument. There's also
a few examples and tests to mimick Python's examples to guarantee
we're consistent with their behaviour.
2025-04-22 02:37:35 -04:00
James Shubin
9c1c587f7b lang: parse, core: world: Add a collect package
Some checks failed
/ Test (basic) on ubuntu-latest with golang 1.23 (push) Has been cancelled
/ Test (race) on ubuntu-latest with golang 1.23 (push) Has been cancelled
/ Test (shell) on ubuntu-latest with golang 1.23 (push) Has been cancelled
This lets us look at the available resource data for collection, and to
filter it so we can decide what we want to collect on our machine.

Other types of collect functions could be added in the future.
2025-04-05 17:00:53 -04:00
James Shubin
a6057319a9 lang: Make scope error messages be more consistent 2025-03-12 03:33:08 -04:00
James Shubin
8544a66257 lang: Allow more than one possible error in tests
There are some rare situations with completely symmetrical graphs which
mean that there isn't a "more correct" error. This is due to the
annoying map iteration non-determinism, and so instead of fighting to
remove every bit of that, let's just accept more than one error here.
2025-03-09 03:03:37 -04:00
James Shubin
2899bc234a lang: Add a forkv loop statement for iterating over a map
This adds a forkv statement which is used to iterate over a map with a
body of statements. This is an important data transformation tool which
should be used sparingly, but is important to have.

An import statement inside of a forkv loop is not currently supported.
We have a simple hack to detect the obvious cases, but more deeply
nested scenarios probably won't be caught, and you'll get an obscure
error message if you try to do this.

This was incredibly challenging to get right, and it's all thanks to Sam
for his brilliance.

Note, I couldn't think of a better keyword that "forkv" but suggestions
are welcome if you think you have a better idea. Other ideas were formap
and foreach, but neither got me very excited.
2025-03-08 17:45:29 -05:00
James Shubin
cf7e73bbf6 lang: Add a for loop statement for iterating over a list
This adds a for statement which is used to iterate over a list with a
body of statements. This is an important data transformation tool which
should be used sparingly, but is important to have.

An import statement inside of a for loop is not currently supported. We
have a simple hack to detect the obvious cases, but more deeply nested
scenarios probably won't be caught, and you'll get an obscure error
message if you try to do this.

This was incredibly challenging to get right, and it's all thanks to Sam
for his brilliance.

Co-authored-by: Samuel Gélineau <gelisam@gmail.com>
2025-03-08 17:45:29 -05:00
James Shubin
d7ecc72b41 lang: ast, gapi, interfaces, parser: Print line numbers on error
This adds an initial implementation of printing line numbers on type
unification errors. It also attempts to print a visual position
indicator for most scenarios.

This patch was started by Felix Frank and finished by James Shubin.

Co-authored-by: Felix Frank <Felix.Frank.de@gmail.com>
2025-02-25 20:15:02 -05:00
Lourenço
1f90de31e7 lang: core: net: Add a new func for URL parsing
This is a first attempt to add a new function for URL parsing, using
go's net/url package and the simple API. This is still a barebones
implementation, there's possibility to expose more information. It also
includes simple tests.
2025-02-19 13:35:20 +01:00
James Shubin
a064a87ecd lang: Add a weird test case
Mark Smith was concerned we might not handle this case correctly. It
seems we do in fact catch this scenario, so it's not an issue. Yay!
2025-02-07 17:57:36 -05:00
James Shubin
315a493565 lang: Add a few more tests 2025-01-26 19:48:17 -05:00
James Shubin
3f202c6a7a lang: core: Fix struct lookup corner case
We forgot to reject this corner case which could lead to a runtime error
since the expected type from the incoming struct would not match what
we're handling.
2025-01-26 19:12:13 -05:00
James Shubin
d46c43df5a lang: core: Let lookup function specialize earlier
If we happen to know some information, we can specialize early and help
type unification solve things.
2025-01-26 18:21:54 -05:00
James Shubin
1538befc93 lang: ast, parser: Allow calling anonymous functions
I forgot to plumb this in through the parser. Pretty easy to add,
hopefully I didn't forget any weird corner scope cases here.
2025-01-26 17:21:11 -05:00
James Shubin
81b102ed7f lang: ast: Allow multiple star imports
If more than one star import is present in the same scope, allow it. If
one star import could overwrite something, ordering is not guaranteed.
We allow this for now, but we might create a compiler fix to stop it.
This adds a test to notice both of these behaviours.
2025-01-17 14:03:48 -05:00
James Shubin
097efdd66a engine: graph: autoedge: Clean up redundant logs
They repeat themselves, this is cleaner.
2024-12-03 00:56:22 -05:00
James Shubin
29eebd0d07 lang: core: Move template to golang namespace
I don't think this template function should be in any way authoritative,
so let's namespace it.
2024-09-13 15:51:24 -04:00
James Shubin
a0972c0752 lang, engine: Add a metaparam for catching accidental dollar signs
Let's make our life easier for users!
2024-08-22 20:41:48 -04:00
James Shubin
8594b6e2a9 lang: funcs: Hint the struct_lookup functions better
If we have static information, use it to help unification.
2024-08-21 19:00:51 -04:00
James Shubin
6de7d8b254 lang: funcs: Catch non-specific type build error
If you had ambiguous code, and specified an invalid type, this could
sneak through and become a runtime error, instead of a compile-time
error. We fix this and add a test.
2024-07-31 17:29:42 -04:00
James Shubin
e10e92596f lang: types: Add stringer information manually
This lets us get the more correct lowercase versions of type kinds in
error messages. (These match what the user would type.)
2024-07-01 18:35:20 -04:00
James Shubin
f2976deb02 pgraph, lang: ast: Fix failing tests due to non-deterministic topo sort
This causes inconsistent type unification when running our tests. It's a
bad user experience too.
2024-07-01 18:34:24 -04:00
James Shubin
14577a0c46 lang: Add modern type unification implementation
This adds a modern type unification algorithm, which drastically
improves performance, particularly for bigger programs.

This required a change to the AST to add TypeCheck methods (for Stmt)
and Infer/Check methods (for Expr). This also changed how the functions
express their invariants, and as a result this was changed as well.

This greatly improves the way we express these invariants, and as a
result it makes adding new polymorphic functions significantly easier.

This also makes error output for the user a lot better in pretty much
all scenarios.

The one downside of this patch is that a good chunk of it is merged in
this giant single commit since it was hard to do it step-wise. That's
not the end of the world.

This couldn't be done without the guidance of Sam who helped me in
explaining, debugging, and writing all the sneaky algorithmic parts and
much more. Thanks again Sam!

Co-authored-by: Samuel Gélineau <gelisam@gmail.com>
2024-07-01 18:33:47 -04:00
James Shubin
51cf1e2921 lang: ast: The res and edge names should not use exclusives
This removes the exclusive from the res names and edge names. We now
require that the names should be lists of strings, however they can
still be single strings if that can be determined statically.
Programmers should explicitly wrap their variables in a string by
interpolation to force this, or in square brackets to force a list. The
former is generally preferable because it generates a small function
graph since it doesn't need to build a list.
2024-04-18 00:07:53 -04:00
James Shubin
946468dc99 lang: ast: Ensure a list doesn't sneak through type interpolation
If we had a single list wrapped in an interpolated string, it could
sneak through type unification, which is not correct. Wrapping a
variable by interpolation in a string, must force it to be a string.
2024-03-20 18:36:47 -04:00
James Shubin
d01c168450 lang: Add per-test config with count maximums
Some of our special tests can only be run once per `go test` invocation.
That is, using the test -count flag will cause a guaranteed failure
since we depend on a global being initialized only once as part of that
test.

This adds a per-test config option so that a user can specify to never
run a particular test more than once. This lets us continue to use the
-count flag with the test suite, without it causing some tests to fail.
2024-01-22 16:55:45 -05:00
James Shubin
dd20bd5486 lang: ast: Improve ordering to eliminate false positives
The Ordering and DAG detection code is challenging because we need
Ordering to do SetScope, but Ordering itself needs to know about scopes.
This improved variant should hopefully catch all the scenarios of
identically named variables causing invalid loops.

Co-authored-by: Samuel Gélineau <gelisam@gmail.com>
2024-01-22 13:19:10 -05:00
Samuel Gélineau
16dfb7b5f5 lang: Add more test cases related to future graph optimization
We are planning to implement an optimization in which some function
calls are compiled to a single static graph rather than to a CallFunc
which dynamically creates a sub-graph at runtime. These test cases
exercise corner cases for which it would be theoretically possible to
use a static graph, but which we might not be able to optimize.
2024-01-20 15:01:56 -05:00
James Shubin
aae0e16350 lang: ast, parser, interfaces: Implementation of nested class sugar
This implements a new type of syntactic sugar for the common pattern of
a base class which returns a child class, and so on. Instead of needing
to repeatedly indent the child classes, we can instead prefix them at
the definition site (where created with the class keyword) with the name
of the parent class, followed by a colon, to get the desired embedded
sugar.

For example, instead of writing:

class base() {
	class inner() {
		class deepest() {
		}
	}
}

You can instead write:

class base() {
}
class base:inner() {
}
class base:inner:deepest() {
}

Of course, you can only access any of the inner classes by first
including (with the include keyword) a parent class, and then
subsequently including the inner one.
2024-01-14 17:10:31 -05:00
James Shubin
44ee578a3a lang: parser, ast, interfaces: Implement include as
This adds support for `include as <identifier>` type statements which in
addition to pulling in any defined resources, it also makes the contents
of the scope of the class available to the scope of the include
statement, but prefixed by the identifier specified.

This makes passing data between scopes much more powerful, and it also
allows classes to return useful classes for subsequent use.

This also improves the SetScope procedure and adds to the Ordering
stage. It's unclear if the current Ordering stage can handle all code,
or if there exist corner-cases which are valid code, but which would
produce a wrong or imprecise topological sort.

Some extraneous scoping bugs still exist, which expose certain variables
that we should not depend on in future code.

Co-authored-by: Samuel Gélineau <gelisam@gmail.com>
2024-01-14 17:08:51 -05:00
James Shubin
f92f34dc54 lang: The in keyword can't be a variable name
At least for now until we figure out some fancier parser magic. (If
at all possible!)
2024-01-12 18:22:00 -05:00
James Shubin
fa5949e191 lang: funcs: core: test: Make a new instance for each test
Since this special one_instance function uses global state, if it's
re-used in more than one test, this won't work since they still all use
the whole global state. Make new ones for each test.

This also breaks the count=2 feature (any number other than 1) when
running these, which is not ideal. Create a cleanup API that we can run
between tests to reset the global state.
2023-12-27 18:25:53 -05:00
Samuel Gélineau
1c0a98a0cc lang: ast: ExprBind is now monomorphic
This adds ExprTopLevel and ExprSingleton and ensures that ExprBind is
now monomorphic.

This corrects a previous design bug where it was not monomorphic and
would thus cause spawning of many more copies than necessary. In most
cases this was only harmful to memory and performance, and not
behaviour, since these functions were pure, and we didn't have a test
for this.

This also adds a bunch more tests. Most notably, the graph shape tests
generally produce smaller graphs now.

Lastly, a lambda cannot have two different types when used at two
different call sites. It is rare that this would be used, and when it
would make sense, there are easy workarounds to accomplish equivalent
goals.

This was mostly authored by Sam, James helped with some cleanup and
debugging.

Co-authored-by: James Shubin <james@shubin.ca>
2023-12-27 16:33:38 -05:00
James Shubin
9d208e8795 lang: Test more iter funcs without polymorphic id function
This tests the lambdas in more ways so that we are sure non-polymorphic
$id functions work the way we want.
2023-12-27 16:33:38 -05:00
James Shubin
72fe0cd6db lang: Polymorphic lambda functions aren't allowed for now
We don't allow a lambda variable to be polymorphic anymore. Of course
this isn't bad, but it makes things too difficult, at least for now.
It's also likely that we don't need this specific feature very often I
think.
2023-12-27 16:33:38 -05:00
James Shubin
734590b6bd lang: Add a test for duplicate functions called
We should not call either of these functions more than once for their
values. If we do, it means we have made a mistake with a compiler
optimization.

This is important, because otherwise if you had code like:

$x = random_password()

Then this would obviously be a problem. Thankfully, the situations where
functions generate unique data is rare, but it's probably something we
should take care of.
2023-12-27 16:33:38 -05:00
James Shubin
c2bf4ef7d4 lang: Test a top-level var explicitly
This makes sure the top-level scope is really seen.
2023-12-27 16:33:38 -05:00
James Shubin
439179e37f lang: Remove duplicate test
It's contents were identical to the stmtfunc-recursive-double.txtar
test.
2023-12-27 16:33:38 -05:00
James Shubin
c333cb542c lang: ast: Expr Param and Poly should not have values
Sometimes a recursive call through ExprVar's Value method would hit one
of these and return (nil, nil) which would throw off things.
2023-12-17 21:39:06 -05:00
James Shubin
a6b6aa570e test: Fix small test warnings
These were issues we should be catching.
2023-12-12 00:01:29 -05:00
Samuel Gélineau
72235b0fd4 lang: ast: New test "clear-env-on-var"
This test detects a mistake which is easy to make: when making a
recursive call to the target of an ExprVar, it would be easy to
accidentally pass the environment, like we usually do with every other
recursive call. For variables, this is a mistake, because the lambda
parameters which are in scope where the variable is used must not be in
scope where the variable is defined.

In fact, ExprVar.Graph() currently makes this mistake. The test passes
anyway, because an earlier phase (SetScope) correctly clears the
environment and detects the problem before the Graph phase. Thus, this
test does not guarantee that all the phases correctly clear their
environment, it merely detects the unlikely case in which all the phases
make the same mistake.
2023-12-02 01:19:21 -05:00
James Shubin
8251c8f259 lang: funcs: core: Add a second panic signature
This modifies the panic feature to accept a boolean or a string. If true
or not empty, then it will cause the panic. This makes some of the error
code a little less ugly.
2023-11-28 14:43:39 -05:00
James Shubin
9c0bde0b29 lang: funcs, parser: Add improved panic magic
This is a newer implementation of the panic magic. I kept the old commit
in for posterity and to show the difference. The two versions are
identical to the end-user with one exception: the newer version doesn't
include a useless panic resource in the graph when there is no panic. In
this version, the panic function returns false and the if statement it's
the condition of, doesn't produce the resource within. On error, we
still consume the function in the if expression, and doing so causes
everything to shutdown.

The other benefit is that the implementation is much cleaner and doesn't
need the interpolate hack.
2023-11-28 14:40:21 -05:00
James Shubin
2cbce963b7 engine: resources, lang: funcs, parser: Add panic magic
It's valuable to check your runtime values and to shut down the entire
engine in case something doesn't match. This patch adds some magic
plumbing to support a "panic" mechanism.

A new "panic" statement gets transparently converted into a panic
function and panic resource. The former errors if the input is not
empty. The latter must be present to consume the value, but doesn't
actually do anything.
2023-11-28 13:49:31 -05:00