710 Commits

Author SHA1 Message Date
James Shubin
2948644536 lang: ast: Remove the double dash
Not sure why this is here?
2025-05-25 01:17:28 -04:00
James Shubin
876834ff29 lang: core: fmt: Catch printf edge case 2025-05-25 01:17:28 -04:00
James Shubin
7c5adb1fec lang: core: net: Add a helper to return the network ip 2025-05-09 02:49:02 -04:00
James Shubin
20e1c461b8 lang: core: embedded: provisioner: Update import style 2025-05-09 02:48:51 -04:00
James Shubin
e9d485b7f6 lang: ast, core: Add some safety checks
I don't think I'm hitting these, but good for debugging.
2025-05-09 01:08:12 -04:00
James Shubin
fddebb2474 engine, lang: core: Match exported resources properly
I inverted the logic for complex setups and forgot to handle the zero
cases. I also didn't notice my loop continue error. This cleans all this
up so that we can have proper exported resource matching.
2025-05-08 22:29:03 -04:00
James Shubin
ad0dd44130 engine: Don't force validation for hidden resources
I think this is what I want in most scenarios, is there a reason to do
otherwise? This is because we may wish to export incomplete resources,
where the remaining necessary fields for validation happens on collect.
2025-05-06 03:36:01 -04:00
James Shubin
2ee403bab9 git: Update gitignore files
We were overly matching in some cases by not starting with a slash. This
updates a few other cases too.
2025-05-06 02:52:26 -04:00
James Shubin
4903995052 lang: core: os: Add readfilewait which won't error easily
Just a useful helper function which we may want for a while.
2025-05-05 23:54:28 -04:00
James Shubin
774d408e13 engine: Fix up some send/recv corner cases
Initially I wasn't 100% clear or decided on the send/recv semantics.
After some experimenting, I think this is much closer to what we want.
Nothing should break or regress here, this only enables more
possibilities.
2025-05-05 23:53:37 -04:00
James Shubin
807c4b3430 engine: resources: Add an http ui resource
Many years ago I built and demoed a prototype of a simple web ui with a
slider, and as you moved it left and right, it started up or shutdown
some number of virtual machines.

The webui was standalone code, but the rough idea of having events from
a high-level overview flow into mgmt, was what I wanted to test out. At
this stage, I didn't even have the language built yet. This prototype
helped convince me of the way a web ui would fit into everything.

Years later, I build an autogrouping prototype which looks quite similar
to what we have today. I recently picked it back up to polish it a bit
more. It's certainly not perfect, and might even be buggy, but it's
useful enough that it's worth sharing.

If I had more cycles, I'd probably consider removing the "store" mode,
and replace it with the normal "value" system, but we would need the
resource "mutate" API if we wanted this. This would allow us to directly
change the "value" field, without triggering a graph swap, which would
be a lot less clunky than the "store" situation.

Of course I'd love to see a GTK version of this concept, but I figured
it would be more practical to have a web ui over HTTP.

One notable missing feature, is that if the "web ui" changes (rather
than just a value changing) we need to offer to the user to reload it.
It currently doesn't get an event for that, and so don't confuse your
users. We also need to be better at validating "untrusted" input here.

There's also no major reason to use the "gin" framework, we should
probably redo this with the standard library alone, but it was easier
for me to push out something quick this way. We can optimize that later.

Lastly, this is all quite ugly since I'm not a very good web dev, so if
you want to make this polished, please do! The wasm code is also quite
terrible due to limitations in the compiler, and maybe one day when that
works better and doesn't constantly deadlock, we can improve it.
2025-05-02 02:14:14 -04:00
James Shubin
6b10477ebc lang: core: convert: Add a simple str to int function 2025-05-02 00:24:21 -04:00
James Shubin
47aecd25c3 lang: funcs: structs: Pass through the type
Not sure why this wasn't done or if it should be, but seems plausible
for now.
2025-04-27 22:23:42 -04:00
James Shubin
fb6eae184a lang: ast: Refactor to unindent slightly 2025-04-27 22:19:14 -04:00
James Shubin
16d3e3063c lang: funcs: facts: Do not reuse fact pointers
In my carelessness, I was re-using pointers when a fact was used twice!
This could cause disastrous consequences like a double close panic on a
datetime.now() fact for example.

In other news, we should consider if it's possible to get more clever
about graph shape optimization so that we don't need more than once
instance of certain functions like datetime.now() in our graph.
2025-04-27 22:14:51 -04:00
James Shubin
37bb67dffd lang: Improve graph shape with speculative execution
Most of the time, we don't need to have a dynamic call sub graph, since
the actual function call could be represented statically as it
originally was before lambda functions were implemented. Simplifying the
graph shape has important performance benefits in terms of both keep the
graph smaller (memory, etc) and in avoiding the need to run transactions
at runtime (speed) to reshape the graph.

Co-authored-by: Samuel Gélineau <gelisam@gmail.com>
2025-04-27 22:14:51 -04:00
James Shubin
9c9f2f558a lang: Move out this legacy execution function
Hasn't been used in a while, but it's fine if we want to use it for
tests.
2025-04-22 03:24:23 -04:00
James Shubin
1a81e57410 lang: interfaces: Update stale comments 2025-04-22 03:24:23 -04:00
James Shubin
7096293885 lang: funcs: dage: Return better errors
Helps a lot with debugging.
2025-04-22 03:24:23 -04:00
James Shubin
1536a94026 lang: Functions that build should be copyable
It's not entirely clear if this is required, but it's probably a good
idea. We should consider making it a requirement of the BuildableFunc
interface.
2025-04-22 03:24:23 -04:00
James Shubin
1bb1e056c4 lang: funcs: structs: Add some extra safety checks
Not sure if these are even needed.
2025-04-22 03:24:23 -04:00
James Shubin
e71b11f843 lang: funcs: facts: Check if a fact is callable 2025-04-22 03:24:23 -04:00
James Shubin
b4769eefd9 lang: funcs: facts: Add a separate callable interface
Add some symmetry to our interfaces for now, even though I'd love to
drop the idea of "facts" altogether.
2025-04-22 03:24:23 -04:00
James Shubin
d4a24d4c9d lang: funcs: wrapped: Simplify the implementation 2025-04-22 03:24:23 -04:00
Lourenço Vales
ae68dd79cb lang: core: iter: Add a range function
This commit implements a range function that mimicks python's range
built-in by having a start, stop, and range argument. There's also
a few examples and tests to mimick Python's examples to guarantee
we're consistent with their behaviour.
2025-04-22 02:37:35 -04:00
James Shubin
60a3d7c65e lang: interfaces: Add more information about graph semantics
Sam thoughts.
2025-04-19 13:02:51 -04:00
James Shubin
9c1c587f7b lang: parse, core: world: Add a collect package
Some checks failed
/ Test (basic) on ubuntu-latest with golang 1.23 (push) Has been cancelled
/ Test (race) on ubuntu-latest with golang 1.23 (push) Has been cancelled
/ Test (shell) on ubuntu-latest with golang 1.23 (push) Has been cancelled
This lets us look at the available resource data for collection, and to
filter it so we can decide what we want to collect on our machine.

Other types of collect functions could be added in the future.
2025-04-05 17:00:53 -04:00
James Shubin
af04d364d0 lang: core: fmt: Make printf handle more cases
Until we make a clean determination about what this should print, this
should handle things for now.
2025-04-05 16:14:11 -04:00
James Shubin
045b29291e engine, lang: Modern exported resources
I've been waiting to write this patch for a long time. I firmly believe
that the idea of "exported resources" was truly a brilliant one, but
which was never even properly understood by its original inventors! This
patch set aims to show how it should have been done.

The main differences are:

* Real-time modelling, since "once per run" makes no sense.
* Filter with code/functions not language syntax.
* Directed exporting to limit the intended recipients.

The next step is to add more "World" reading and filtering functions to
make it easy and expressive to make your selection of resources to
collect!
2025-04-05 12:45:23 -04:00
James Shubin
8ff187b4e9 lang: core, funcs: Rename things for consistency
Seems we had different patterns going on. This makes those all
consistent now.
2025-04-05 12:45:23 -04:00
Karpfen
7e8ced534f misc: Use /usr/bin/env for a more generic shebang
Use path based SHELL in Makefiles. It was suggested that this is a
better solution for make for cases when there is no /usr/bin/env.

See: https://github.com/purpleidea/mgmt/pull/694#discussion_r1015596204
2025-03-22 14:53:21 -04:00
Lourenço Vales
f2d9219218 lang: core: os: Add is_virtual function
This is a basic implementation of a detection method for whether mgmt is
running in a virtualized environment. We achieve this by doing two types
of checks: on one hand, we check if the CPU flags can confirm the
presence of a virtualized env; on the other, we check if the presence
of known files related with DMI (and their contents) can confirm whether
 we're inside a virt env. Either of these situations will cause the
function to return true, with the default case being false. All of these
checks are relatively naive and can be improved by looking at the main
inspiration for this implementation, which was systemd's own check for
the presence of virtualization.
2025-03-21 14:18:55 -04:00
James Shubin
a7a5237b07 cli, engine, etcd, lib: Pass in init args
Improve the API and make it more general.
2025-03-18 04:54:13 -04:00
James Shubin
7ad54fe3e8 cli, engine, etcd, lib: Split out the deployer into world
This should hopefully make the refactor into a clean world API a bit
better. Still more to do though!
2025-03-18 04:54:13 -04:00
James Shubin
59c33a354c engine, lang: core: world: Split out the scheduler interface 2025-03-18 03:32:42 -04:00
James Shubin
642c6b952f lang: core, funcs: Port some functions to CallableFunc API
Some modern features of our function engine and language might require
this new API, so port what we can and figure out the rest later.
2025-03-16 23:23:57 -04:00
James Shubin
37308b950b cli, gapi: Add more information that deploy is running
There can be a non-obvious pause, so give some hint here...
2025-03-12 05:45:26 -04:00
James Shubin
a6057319a9 lang: Make scope error messages be more consistent 2025-03-12 03:33:08 -04:00
James Shubin
17b859d0d7 cli, gapi, lang, lib: Add a flag to skip autoedges
The GAPI API is a bit of a mess, but I think this seems to work for
standalone run and also deploys. Hopefully I didn't add any unnecessary
extra dead code here, but that's archaeology for another day.
2025-03-11 04:18:03 -04:00
James Shubin
8544a66257 lang: Allow more than one possible error in tests
There are some rare situations with completely symmetrical graphs which
mean that there isn't a "more correct" error. This is due to the
annoying map iteration non-determinism, and so instead of fighting to
remove every bit of that, let's just accept more than one error here.
2025-03-09 03:03:37 -04:00
James Shubin
a50765393d lang: ast: Catch ordering errors 2025-03-09 01:50:28 -05:00
James Shubin
f87c550be1 lang: ast, interfaces: Improve speculation safety checks
We want to speculate in more cases, so make sure that speculation is
safe!
2025-03-08 17:45:29 -05:00
James Shubin
aea894a706 lang: ast: Add more context to pointer errors
This makes debugging easier. We don't expect these errors to occur with
normal usage.
2025-03-08 17:45:29 -05:00
James Shubin
a549a30f71 lang: ast: Add more context to table errors
This makes debugging easier. We don't expect these errors to occur with
normal usage.
2025-03-08 17:45:29 -05:00
James Shubin
2899bc234a lang: Add a forkv loop statement for iterating over a map
This adds a forkv statement which is used to iterate over a map with a
body of statements. This is an important data transformation tool which
should be used sparingly, but is important to have.

An import statement inside of a forkv loop is not currently supported.
We have a simple hack to detect the obvious cases, but more deeply
nested scenarios probably won't be caught, and you'll get an obscure
error message if you try to do this.

This was incredibly challenging to get right, and it's all thanks to Sam
for his brilliance.

Note, I couldn't think of a better keyword that "forkv" but suggestions
are welcome if you think you have a better idea. Other ideas were formap
and foreach, but neither got me very excited.
2025-03-08 17:45:29 -05:00
James Shubin
cf7e73bbf6 lang: Add a for loop statement for iterating over a list
This adds a for statement which is used to iterate over a list with a
body of statements. This is an important data transformation tool which
should be used sparingly, but is important to have.

An import statement inside of a for loop is not currently supported. We
have a simple hack to detect the obvious cases, but more deeply nested
scenarios probably won't be caught, and you'll get an obscure error
message if you try to do this.

This was incredibly challenging to get right, and it's all thanks to Sam
for his brilliance.

Co-authored-by: Samuel Gélineau <gelisam@gmail.com>
2025-03-08 17:45:29 -05:00
James Shubin
c456a5ab97 lang: types: Add some length methods for list and map 2025-03-06 16:55:55 -05:00
James Shubin
b5ae96e0d4 lang: types: Add some helpful true and false values
In case we need one, we don't need to build it.
2025-03-06 16:55:55 -05:00
James Shubin
f792facde9 lang: gapi: Debug stalled graph errors
This makes debugging these scenarios a lot easier.
2025-03-05 17:24:26 -05:00
James Shubin
c5257dd64b lang: parser: Simplify code and format it
This would get done by gofmt -s anyways.
2025-02-27 17:13:31 -05:00