Commit Graph

648 Commits

Author SHA1 Message Date
James Shubin
47aecd25c3 lang: funcs: structs: Pass through the type
Not sure why this wasn't done or if it should be, but seems plausible
for now.
2025-04-27 22:23:42 -04:00
James Shubin
fb6eae184a lang: ast: Refactor to unindent slightly 2025-04-27 22:19:14 -04:00
James Shubin
16d3e3063c lang: funcs: facts: Do not reuse fact pointers
In my carelessness, I was re-using pointers when a fact was used twice!
This could cause disastrous consequences like a double close panic on a
datetime.now() fact for example.

In other news, we should consider if it's possible to get more clever
about graph shape optimization so that we don't need more than once
instance of certain functions like datetime.now() in our graph.
2025-04-27 22:14:51 -04:00
James Shubin
37bb67dffd lang: Improve graph shape with speculative execution
Most of the time, we don't need to have a dynamic call sub graph, since
the actual function call could be represented statically as it
originally was before lambda functions were implemented. Simplifying the
graph shape has important performance benefits in terms of both keep the
graph smaller (memory, etc) and in avoiding the need to run transactions
at runtime (speed) to reshape the graph.

Co-authored-by: Samuel Gélineau <gelisam@gmail.com>
2025-04-27 22:14:51 -04:00
James Shubin
9c9f2f558a lang: Move out this legacy execution function
Hasn't been used in a while, but it's fine if we want to use it for
tests.
2025-04-22 03:24:23 -04:00
James Shubin
1a81e57410 lang: interfaces: Update stale comments 2025-04-22 03:24:23 -04:00
James Shubin
7096293885 lang: funcs: dage: Return better errors
Helps a lot with debugging.
2025-04-22 03:24:23 -04:00
James Shubin
1536a94026 lang: Functions that build should be copyable
It's not entirely clear if this is required, but it's probably a good
idea. We should consider making it a requirement of the BuildableFunc
interface.
2025-04-22 03:24:23 -04:00
James Shubin
1bb1e056c4 lang: funcs: structs: Add some extra safety checks
Not sure if these are even needed.
2025-04-22 03:24:23 -04:00
James Shubin
e71b11f843 lang: funcs: facts: Check if a fact is callable 2025-04-22 03:24:23 -04:00
James Shubin
b4769eefd9 lang: funcs: facts: Add a separate callable interface
Add some symmetry to our interfaces for now, even though I'd love to
drop the idea of "facts" altogether.
2025-04-22 03:24:23 -04:00
James Shubin
d4a24d4c9d lang: funcs: wrapped: Simplify the implementation 2025-04-22 03:24:23 -04:00
Lourenço Vales
ae68dd79cb lang: core: iter: Add a range function
This commit implements a range function that mimicks python's range
built-in by having a start, stop, and range argument. There's also
a few examples and tests to mimick Python's examples to guarantee
we're consistent with their behaviour.
2025-04-22 02:37:35 -04:00
James Shubin
60a3d7c65e lang: interfaces: Add more information about graph semantics
Sam thoughts.
2025-04-19 13:02:51 -04:00
James Shubin
9c1c587f7b lang: parse, core: world: Add a collect package
Some checks failed
/ Test (basic) on ubuntu-latest with golang 1.23 (push) Has been cancelled
/ Test (race) on ubuntu-latest with golang 1.23 (push) Has been cancelled
/ Test (shell) on ubuntu-latest with golang 1.23 (push) Has been cancelled
This lets us look at the available resource data for collection, and to
filter it so we can decide what we want to collect on our machine.

Other types of collect functions could be added in the future.
2025-04-05 17:00:53 -04:00
James Shubin
af04d364d0 lang: core: fmt: Make printf handle more cases
Until we make a clean determination about what this should print, this
should handle things for now.
2025-04-05 16:14:11 -04:00
James Shubin
045b29291e engine, lang: Modern exported resources
I've been waiting to write this patch for a long time. I firmly believe
that the idea of "exported resources" was truly a brilliant one, but
which was never even properly understood by its original inventors! This
patch set aims to show how it should have been done.

The main differences are:

* Real-time modelling, since "once per run" makes no sense.
* Filter with code/functions not language syntax.
* Directed exporting to limit the intended recipients.

The next step is to add more "World" reading and filtering functions to
make it easy and expressive to make your selection of resources to
collect!
2025-04-05 12:45:23 -04:00
James Shubin
8ff187b4e9 lang: core, funcs: Rename things for consistency
Seems we had different patterns going on. This makes those all
consistent now.
2025-04-05 12:45:23 -04:00
Karpfen
7e8ced534f misc: Use /usr/bin/env for a more generic shebang
Use path based SHELL in Makefiles. It was suggested that this is a
better solution for make for cases when there is no /usr/bin/env.

See: https://github.com/purpleidea/mgmt/pull/694#discussion_r1015596204
2025-03-22 14:53:21 -04:00
Lourenço Vales
f2d9219218 lang: core: os: Add is_virtual function
This is a basic implementation of a detection method for whether mgmt is
running in a virtualized environment. We achieve this by doing two types
of checks: on one hand, we check if the CPU flags can confirm the
presence of a virtualized env; on the other, we check if the presence
of known files related with DMI (and their contents) can confirm whether
 we're inside a virt env. Either of these situations will cause the
function to return true, with the default case being false. All of these
checks are relatively naive and can be improved by looking at the main
inspiration for this implementation, which was systemd's own check for
the presence of virtualization.
2025-03-21 14:18:55 -04:00
James Shubin
a7a5237b07 cli, engine, etcd, lib: Pass in init args
Improve the API and make it more general.
2025-03-18 04:54:13 -04:00
James Shubin
7ad54fe3e8 cli, engine, etcd, lib: Split out the deployer into world
This should hopefully make the refactor into a clean world API a bit
better. Still more to do though!
2025-03-18 04:54:13 -04:00
James Shubin
59c33a354c engine, lang: core: world: Split out the scheduler interface 2025-03-18 03:32:42 -04:00
James Shubin
642c6b952f lang: core, funcs: Port some functions to CallableFunc API
Some modern features of our function engine and language might require
this new API, so port what we can and figure out the rest later.
2025-03-16 23:23:57 -04:00
James Shubin
37308b950b cli, gapi: Add more information that deploy is running
There can be a non-obvious pause, so give some hint here...
2025-03-12 05:45:26 -04:00
James Shubin
a6057319a9 lang: Make scope error messages be more consistent 2025-03-12 03:33:08 -04:00
James Shubin
17b859d0d7 cli, gapi, lang, lib: Add a flag to skip autoedges
The GAPI API is a bit of a mess, but I think this seems to work for
standalone run and also deploys. Hopefully I didn't add any unnecessary
extra dead code here, but that's archaeology for another day.
2025-03-11 04:18:03 -04:00
James Shubin
8544a66257 lang: Allow more than one possible error in tests
There are some rare situations with completely symmetrical graphs which
mean that there isn't a "more correct" error. This is due to the
annoying map iteration non-determinism, and so instead of fighting to
remove every bit of that, let's just accept more than one error here.
2025-03-09 03:03:37 -04:00
James Shubin
a50765393d lang: ast: Catch ordering errors 2025-03-09 01:50:28 -05:00
James Shubin
f87c550be1 lang: ast, interfaces: Improve speculation safety checks
We want to speculate in more cases, so make sure that speculation is
safe!
2025-03-08 17:45:29 -05:00
James Shubin
aea894a706 lang: ast: Add more context to pointer errors
This makes debugging easier. We don't expect these errors to occur with
normal usage.
2025-03-08 17:45:29 -05:00
James Shubin
a549a30f71 lang: ast: Add more context to table errors
This makes debugging easier. We don't expect these errors to occur with
normal usage.
2025-03-08 17:45:29 -05:00
James Shubin
2899bc234a lang: Add a forkv loop statement for iterating over a map
This adds a forkv statement which is used to iterate over a map with a
body of statements. This is an important data transformation tool which
should be used sparingly, but is important to have.

An import statement inside of a forkv loop is not currently supported.
We have a simple hack to detect the obvious cases, but more deeply
nested scenarios probably won't be caught, and you'll get an obscure
error message if you try to do this.

This was incredibly challenging to get right, and it's all thanks to Sam
for his brilliance.

Note, I couldn't think of a better keyword that "forkv" but suggestions
are welcome if you think you have a better idea. Other ideas were formap
and foreach, but neither got me very excited.
2025-03-08 17:45:29 -05:00
James Shubin
cf7e73bbf6 lang: Add a for loop statement for iterating over a list
This adds a for statement which is used to iterate over a list with a
body of statements. This is an important data transformation tool which
should be used sparingly, but is important to have.

An import statement inside of a for loop is not currently supported. We
have a simple hack to detect the obvious cases, but more deeply nested
scenarios probably won't be caught, and you'll get an obscure error
message if you try to do this.

This was incredibly challenging to get right, and it's all thanks to Sam
for his brilliance.

Co-authored-by: Samuel Gélineau <gelisam@gmail.com>
2025-03-08 17:45:29 -05:00
James Shubin
c456a5ab97 lang: types: Add some length methods for list and map 2025-03-06 16:55:55 -05:00
James Shubin
b5ae96e0d4 lang: types: Add some helpful true and false values
In case we need one, we don't need to build it.
2025-03-06 16:55:55 -05:00
James Shubin
f792facde9 lang: gapi: Debug stalled graph errors
This makes debugging these scenarios a lot easier.
2025-03-05 17:24:26 -05:00
James Shubin
c5257dd64b lang: parser: Simplify code and format it
This would get done by gofmt -s anyways.
2025-02-27 17:13:31 -05:00
James Shubin
d56896cb0d lang: core: Simplify list and map lookup default functions 2025-02-26 19:59:47 -05:00
James Shubin
d579787bcd lang: core: Simplify list and map lookup functions 2025-02-26 19:59:47 -05:00
James Shubin
37fffce9f5 lang: core: Simplify implementation of the "contains" function 2025-02-26 18:12:38 -05:00
James Shubin
d7ecc72b41 lang: ast, gapi, interfaces, parser: Print line numbers on error
This adds an initial implementation of printing line numbers on type
unification errors. It also attempts to print a visual position
indicator for most scenarios.

This patch was started by Felix Frank and finished by James Shubin.

Co-authored-by: Felix Frank <Felix.Frank.de@gmail.com>
2025-02-25 20:15:02 -05:00
Edward Toroshchyn
c25a2a257b misc: Fix typos and spelling errors 2025-02-24 16:01:46 +01:00
Lourenço
1f90de31e7 lang: core: net: Add a new func for URL parsing
This is a first attempt to add a new function for URL parsing, using
go's net/url package and the simple API. This is still a barebones
implementation, there's possibility to expose more information. It also
includes simple tests.
2025-02-19 13:35:20 +01:00
James Shubin
d80ec4aaa7 engine: resources: Detect simple self-referential frags
It would be a likely mistake to create a self-referential frag, and mgmt
would spin forever updating the file... We probably don't want this, so
let's just catch this case in Validate.

Of course you could get around this with multiple files, and a fancier
search could statically check the graph, but the goal isn't to prevent
any bad code, since that's not likely to be possible.
2025-02-15 06:58:15 -05:00
James Shubin
a064a87ecd lang: Add a weird test case
Mark Smith was concerned we might not handle this case correctly. It
seems we do in fact catch this scenario, so it's not an issue. Yay!
2025-02-07 17:57:36 -05:00
James Shubin
ecd5a0f304 util, lang, etcd: Move the error type to our util package
We use this error in a lot of places, let's centralize it a bit.
2025-02-07 17:57:36 -05:00
James Shubin
315a493565 lang: Add a few more tests 2025-01-26 19:48:17 -05:00
James Shubin
6268b61a7d lang: core: Lookup function (with default) can be more precise 2025-01-26 19:12:13 -05:00
James Shubin
3f202c6a7a lang: core: Fix struct lookup corner case
We forgot to reject this corner case which could lead to a runtime error
since the expected type from the incoming struct would not match what
we're handling.
2025-01-26 19:12:13 -05:00