lourenco/mgmt
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

lang: types, funcs: Add simple function API

Browse Source 
This patch adds a simple function API for writing simple, pure
functions. This should reduce the amount of boilerplate required for
most functions, and make growing a stdlib significantly easier. If you
need to build more complex, event-generating functions, or statically
polymorphic functions, then you'll still need to use the normal API for
now.

This also makes all of these pure functions available automatically
within templates. It might make sense to group these functions into
packages to make their logical organization easier, but this is a good
enough start for now.

Lastly, this added some missing pieces to our types library. You can now
use `ValueOf` to convert from a `reflect.Value` to the corresponding
`Value` in our type system, if an equivalent exists.

Unfortunately, we're severely lacking in tests for these new types
library additions, but look forward to growing some in the future!
This commit is contained in:
James Shubin
2018-02-20 20:43:32 -05:00
parent cbd2bdd4c5
commit 837388ae4e
15 changed files with 864 additions and 22 deletions
Download Patch File Download Diff File Expand all files Collapse all files

318
docs/function-guide.md Normal file
View File

@@ -0,0 +1,318 @@
# Function guide
## Overview
The `mgmt` tool has built-in functions which add useful, reactive functionality
to the language. This guide describes the different function API's that are
available. It is meant to instruct developers on how to write new functions.
Since `mgmt` and the core functions are written in golang, some prior golang
knowledge is assumed.
## Theory
Functions in `mgmt` are similar to functions in other languages, however they
also have a [reactive](https://en.wikipedia.org/wiki/Functional_reactive_programming)
component. Our functions can produce events over time, and there are different
ways to write functions. For some background on this design, please read the
[original article](https://purpleidea.com/blog/2018/02/05/mgmt-configuration-language/)
on the subject.
## Native Functions
Native functions are functions which are implemented in the mgmt language
itself. These are currently not available yet, but are coming soon. Stay tuned!
## Simple Function API
Most functions should be implemented using the simple function API. This API
allows you to implement simple, static, [pure](https://en.wikipedia.org/wiki/Pure_function)
functions that don't require you to write much boilerplate code. They will be
automatically re-evaluated as needed when their input values change. These will
all be automatically made available as helper functions within mgmt templates,
and are also available for use anywhere inside mgmt programs.
You'll need some basic knowledge of using the [`types`](https://github.com/purpleidea/mgmt/tree/master/lang/types)
library which is included with mgmt. This library lets you interact with the
available types and values in the mgmt language. It is very easy to use, and
should be fairly intuitive. Most of what you'll need to know can be inferred
from looking at example code.
To implement a function, you'll need to create a file in
[`lang/funcs/simple/`](https://github.com/purpleidea/mgmt/tree/master/lang/funcs/simple/).
The function should be implemented as a `FuncValue` in our type system. It is
then registered with the engine during `init()`. An example explains it best:
### Example
```golang
package simple
import (
"fmt"
"github.com/purpleidea/mgmt/lang/types"
)
// you must register your functions in init when the program starts up
func init() {
// Example function that squares an int and prints out answer as an str.
Register("talkingsquare", &types.FuncValue{
T: types.NewType("func(a int) str"), // declare the signature
V: func(input []types.Value) (types.Value, error) {
i := input[0].Int() // get first arg as an int64
// must return the above specified value
return &types.StrValue{
V: fmt.Sprintf("%d^2 is %d", i, i * i),
}, nil // no serious errors occurred
},
})
}
```
This simple function accepts one `int` as input, and returns one `str`.
Functions can have zero or more inputs, and must have exactly one output. You
must be sure to use the `types` library correctly, since if you try and access
an input which should not exist (eg: `input[2]`, when there are only two
that are expected), then you will cause a panic. If you have declared that a
particular argument is an `int` but you try to read it with `.Bool()` you will
also cause a panic. Lastly, make sure that you return a value in the correct
type or you will also cause a panic!
If anything goes wrong, you can return an error, however this will cause the
mgmt engine to shutdown. It should be seen as the equivalent to calling a
`panic()`, however it is safer because it brings the engine down cleanly.
Ideally, your functions should never need to error. You should never cause a
real `panic()`, since this could have negative consequences to the system.
## Function API
To implement a reactive function in `mgmt` it must satisfy the
[`Func`](https://github.com/purpleidea/mgmt/blob/master/lang/interfaces/func.go)
interface. Using the [Simple Function API](#simple-function-api) is preferable
if it meets your needs. Most functions will be able to use that API. If you
really need something more powerful, then you can use the regular function API.
What follows are each of the method signatures and a description of each.
### Default
```golang
Info() *interfaces.Info
```
This returns some information about the function. It is necessary so that the
compiler can type check the code correctly, and know what optimizations can be
performed. This is usually the first method which is called by the engine.
#### Example
```golang
func (obj *FooFunc) Info() *interfaces.Info {
return &interfaces.Info{
Pure: true,
Sig: types.NewType("func(a int) str"),
}
}
```
### Init
```golang
Init(init *interfaces.Init) error
```
This is called to initialize the function. If something goes wrong, it should
return an error. It is passed a struct that contains all the important
information and poiinters that it might need to work with throughout its
lifetime. As a result, it will need to save a copy to that pointer for future
use in the other methods.
#### Example
```golang
// Init runs some startup code for this function.
func (obj *FooFunc) Init(init *interfaces.Init) error {
obj.init = init
obj.closeChan = make(chan struct{}) // shutdown signal
return nil
}
```
### Close
```golang
Close() error
```
This is called to cleanup the function. It usually causes the stream to
shutdown. Even if `Stream()` decided to shutdown early, it might still get
called. It is usually called by the engine to tell the function to shutdown.
#### Example
```golang
// Close runs some shutdown code for this function and turns off the stream.
func (obj *FooFunc) Close() error {
close(obj.closeChan) // send a signal to tell the stream to close
return nil
}
```
### Stream
```golang
Stream() error
```
`Stream` is where the real _work_ is done. This method is started by the
language function engine. It will run this function while simultaneously sending
it values on the `input` channel. It will only send a complete set of input
values. You should send a value to the output channel when you have decided that
one should be produced. Make sure to only use input values of the expected type
as declared in the `Info` struct, and send values of the similarly declared
appropriate return type. Failure to do so will may result in a panic and
sadness.
#### Example
```golang
// Stream returns the single value that was generated and then closes.
func (obj *FooFunc) Stream() error {
defer close(obj.init.Output) // the sender closes
var result string
for {
select {
case input, ok := <-obj.init.Input:
if !ok {
return nil // can't output any more
}
ix := input.Struct()["a"].Int()
if ix < 0 {
return fmt.Errorf("we can't deal with negatives")
}
result = fmt.Sprintf("the input is: %d", ix)
case <-obj.closeChan:
return nil
}
select {
case obj.init.Output <- &types.StrValue{
V: result,
}:
case <-obj.closeChan:
return nil
}
}
}
```
As you can see, we read our inputs from the `input` channel, and write to the
`output` channel. Our code is careful to never block or deadlock, and can always
exit if a close signal is requested. It also cleans up after itself by closing
the `output` channel when it is done using it. This is done easily with `defer`.
If it notices that the `input` channel closes, then it knows that no more input
values are coming and it can consider shutting down early.
## Further considerations
There is some additional information that any function author will need to know.
Each issue is listed separately below!
### Function struct
Each function will implement methods as pointer receivers on a function struct.
The naming convention for resources is that they end with a `Func` suffix.
#### Example
```golang
type FooFunc struct {
init *interfaces.Init
// this space can be used if needed
closeChan chan struct{} // shutdown signal
}
```
### Function registration
All functions must be registered with the engine so that they can be found. This
also ensures they can be encoded and decoded. Make sure to include the following
code snippet for this to work.
```golang
func init() { // special golang method that runs once
funcs.Register("foo", func() interfaces.Func { return &FooFunc{} })
}
```
### Composite functions
Composite functions are functions which import one or more existing functions.
This is useful to prevent code duplication in higher level function scenarios.
Unfortunately no further documentation about this subject has been written. To
expand this section, please send a patch! Please contact us if you'd like to
work on a function that uses this feature, or to add it to an existing one!
We don't expect this functionality to be particularly useful or common, as it's
probably easier and preferable to simply import common golang library code into
multiple different functions instead.
## Polymorphic Function API
The polymorphic function API is an API that lets you implement functions which
do not necessarily have a single static function signature. After compile time,
all functions must have a static function signature. We also know that there
might be different ways you would want to call `printf`, such as:
`printf("the %s is %d", "answer", 42)` or `printf("3 * 2 = %d", 3 * 2)`. Since
you couldn't implement the infinite number of possible signatures, this API lets
you write code which can be coerced into different forms. This makes
implementing what would appear to be generic or polymorphic, instead something
that is actually static and that still has the static type safety properties
that were guaranteed by the mgmt language.
Since this is an advanced topic, it is not described in full at this time. For
more information please have a look at the source code comments, some of the
existing implementations, and ask around in the community.
## Frequently asked questions
(Send your questions as a patch to this FAQ! I'll review it, merge it, and
respond by commit with the answer.)
### Can I use global variables?
Probably not. You must assume that multiple copies of your function may be used
at the same time. If they require a global variable, it's likely this won't
work. Instead it's probably better to use a struct local variable if you need to
store some state.
There might be some rare instances where a global would be acceptable, but if
you need one of these, you're probably already an internals expert. If you think
they need to lock or synchronize so as to not overwhelm an external resource,
then you have to be especially careful not to cause deadlocking the mgmt engine.
### Can I write functions in a different language?
Currently `golang` is the only supported language for built-in functions. We
might consider allowing external functions to be imported in the future. This
will likely require a language that can expose a C-like API, such as `python` or
`ruby`. Custom `golang` functions are already possible when using mgmt as a lib.
### What new functions need writing?
There are still many ideas for new functions that haven't been written yet. If
you'd like to contribute one, please contact us and tell us about your idea!
### Where can I find more information about mgmt?
Additional blog posts, videos and other material [is available!](https://github.com/purpleidea/mgmt/blob/master/docs/on-the-web.md).
## Suggestions
If you have any ideas for API changes or other improvements to function writing,
please let us know! We're still pre 1.0 and pre 0.1 and happy to break API in
order to get it right!

8
docs/language-guide.md
View File

@@ -212,6 +212,8 @@ Your function must have a specific type. For example, a simple math function
might have a signature of `func(x int, x int) int`. As you can see, all the
types are known _before_ compile time.
A separate discussion on this matter can be found in the [function guide](function-guide.md).
What follows are each of the method signatures and a description of each.
Failure to implement the API correctly can cause the function graph engine to
block, or the program to panic.
@@ -279,7 +281,7 @@ one value must be produced.
#### Example
```golang
Please see the example functions in
[lang/funcs/public/](https://github.com/purpleidea/mgmt/tree/master/lang/funcs/public/).
[lang/funcs/core/](https://github.com/purpleidea/mgmt/tree/master/lang/funcs/core/).
```
### Stream
@@ -298,7 +300,7 @@ whether or not you close the `Output` channel.
#### Example
```golang
Please see the example functions in
[lang/funcs/public/](https://github.com/purpleidea/mgmt/tree/master/lang/funcs/public/).
[lang/funcs/core/](https://github.com/purpleidea/mgmt/tree/master/lang/funcs/core/).
```
### Close
@@ -312,7 +314,7 @@ return.
#### Example
```golang
Please see the example functions in
[lang/funcs/public/](https://github.com/purpleidea/mgmt/tree/master/lang/funcs/public/).
[lang/funcs/core/](https://github.com/purpleidea/mgmt/tree/master/lang/funcs/core/).
```
### Polymorphic Function API

2
examples/lang/datetime1.mcl
View File

@@ -1,4 +1,4 @@
$d = datetime()
file "/tmp/mgmt/datetime" {
content => template("Hello! It is now: {{ datetimeprint . }}\n", $d),
content => template("Hello! It is now: {{ datetime_print . }}\n", $d),
}

2
examples/lang/datetime2.mcl
View File

@@ -9,6 +9,6 @@ $theload = structlookup(load(), "x1")
if 5 > 3 {
file "/tmp/mgmt/datetime" {
content => template("Now + 1 year is: {{ .year }} seconds, aka: {{ datetimeprint .year }}\n\nload average: {{ .load }}\n", $tmplvalues),
content => template("Now + 1 year is: {{ .year }} seconds, aka: {{ datetime_print .year }}\n\nload average: {{ .load }}\n", $tmplvalues),
}
}

2
examples/lang/datetime3.mcl
View File

@@ -10,5 +10,5 @@ $theload = structlookup(load(), "x1")
$vumeter = vumeter("====", 10, 0.9)
file "/tmp/mgmt/datetime" {
content => template("Now + 1 year is: {{ .year }} seconds, aka: {{ datetimeprint .year }}\n\nload average: {{ .load }}\n\nvu: {{ .vumeter }}\n", $tmplvalues),
content => template("Now + 1 year is: {{ .year }} seconds, aka: {{ datetime_print .year }}\n\nload average: {{ .load }}\n\nvu: {{ .vumeter }}\n", $tmplvalues),
}

2
examples/lang/history1.mcl
View File

@@ -3,5 +3,5 @@ $dt = datetime()
$hystvalues = {"ix0" => $dt, "ix1" => $dt{1}, "ix2" => $dt{2}, "ix3" => $dt{3},}
file "/tmp/mgmt/history" {
content => template("Index(0) {{.ix0}}: {{ datetimeprint .ix0 }}\nIndex(1) {{.ix1}}: {{ datetimeprint .ix1 }}\nIndex(2) {{.ix2}}: {{ datetimeprint .ix2 }}\nIndex(3) {{.ix3}}: {{ datetimeprint .ix3 }}\n", $hystvalues),
content => template("Index(0) {{.ix0}}: {{ datetime_print .ix0 }}\nIndex(1) {{.ix1}}: {{ datetime_print .ix1 }}\nIndex(2) {{.ix2}}: {{ datetime_print .ix2 }}\nIndex(3) {{.ix3}}: {{ datetime_print .ix3 }}\n", $hystvalues),
}

10
examples/lang/template0.mcl Normal file
View File

@@ -0,0 +1,10 @@
$answer = 42
$s = int2str($answer)
print "print1" {
msg => printf("an str is: %s", $s),
}
print "print2" {
msg => template("an str is: {{ int2str . }}", $answer),
}

118
lang/funcs/core/template_polyfunc.go
View File

@@ -20,16 +20,23 @@ package core // TODO: should this be in its own individual package?
import (
"bytes"
"fmt"
"reflect"
"text/template"
"time"
"github.com/purpleidea/mgmt/lang/funcs"
"github.com/purpleidea/mgmt/lang/funcs/simple"
"github.com/purpleidea/mgmt/lang/interfaces"
"github.com/purpleidea/mgmt/lang/types"
errwrap "github.com/pkg/errors"
)
var (
// errorType represents a reflection type of error as seen in:
// https://github.com/golang/go/blob/ec62ee7f6d3839fe69aeae538dadc1c9dc3bf020/src/text/template/exec.go#L612
errorType = reflect.TypeOf((*error)(nil)).Elem()
)
func init() {
funcs.Register("template", func() interfaces.Func { return &TemplateFunc{} })
}
@@ -42,7 +49,8 @@ const TemplateName = "template"
// to it. It examines the type of the second argument (the input data vars) at
// compile time and then determines the static functions signature by including
// that in the overall signature.
// XXX: do we need to add events if any of the internal functions change over time?
// TODO: We *might* need to add events for internal function changes over time,
// but only if they are not pure. We currently only use simple, pure functions.
type TemplateFunc struct {
Type *types.Type // type of vars
@@ -159,13 +167,40 @@ func (obj *TemplateFunc) Init(init *interfaces.Init) error {
// run runs a template and returns the result.
func (obj *TemplateFunc) run(templateText string, vars types.Value) (string, error) {
// see: https://golang.org/pkg/text/template/#FuncMap for more info
// note: we can override any other functions by adding them here...
funcMap := map[string]interface{}{
// XXX: can these functions come from normal funcValue things
// that we build for the interfaces.Func part?
// TODO: add a bunch of stdlib-like stuff here...
"datetimeprint": func(epochDelta int64) string { // TODO: rename
return time.Unix(epochDelta, 0).String()
},
//"test1": func(in interface{}) (interface{}, error) { // ok
// return fmt.Sprintf("got(%T): %+v", in, in), nil
//},
//"test2": func(in interface{}) interface{} { // NOT ok
// panic("panic") // a panic here brings down everything!
//},
//"test3": func(foo int64) (string, error) { // ok, but errors
// return "", fmt.Errorf("i am an error")
//},
//"test4": func(in1, in2 reflect.Value) (reflect.Value, error) { // ok
// s := fmt.Sprintf("got: %+v and: %+v", in1, in2)
// return reflect.ValueOf(s), nil
//},
}
// FIXME: should we do this once in init() instead, or in the Register
// function in the simple package?
// TODO: loop through this map in a sorted, deterministic order
for name, fn := range simple.RegisteredFuncs {
if _, exists := funcMap[name]; exists {
obj.init.Logf("warning, existing function named: `%s` exists", name)
continue
}
// When template execution invokes a function with an argument
// list, that list must be assignable to the function's
// parameter types. Functions meant to apply to arguments of
// arbitrary type can use parameters of type interface{} or of
// type reflect.Value.
f := wrap(name, fn) // wrap it so that it meets API expectations
funcMap[name] = f // add it
}
var err error
@@ -288,3 +323,70 @@ func (obj *TemplateFunc) Close() error {
close(obj.closeChan)
return nil
}
// wrap builds a function in the format expected by the template engine, and
// returns it as an interface{}. It does so by wrapping our type system and
// function API with what is expected from the reflection API. It returns a
// version that includes the optional second error return value so that our
// functions can return errors without causing a panic.
func wrap(name string, fn *types.FuncValue) interface{} {
if fn.T.Map == nil {
panic("malformed func type")
}
if len(fn.T.Map) != len(fn.T.Ord) {
panic("malformed func length")
}
in := []reflect.Type{}
for _, k := range fn.T.Ord {
t, ok := fn.T.Map[k]
if !ok {
panic("malformed func order")
}
if t == nil {
panic("malformed func arg")
}
in = append(in, t.Reflect())
}
out := []reflect.Type{fn.T.Out.Reflect(), errorType}
var variadic = false // currently not supported in our function value
typ := reflect.FuncOf(in, out, variadic)
// wrap our function with the translation that is necessary
f := func(args []reflect.Value) (results []reflect.Value) { // build
innerArgs := []types.Value{}
zeroValue := reflect.Zero(fn.T.Out.Reflect()) // zero value of return type
for _, x := range args {
v, err := types.ValueOf(x) // reflect.Value -> Value
if err != nil {
r := reflect.ValueOf(errwrap.Wrapf(err, "function `%s` errored", name))
if !r.Type().ConvertibleTo(errorType) { // for fun!
r = reflect.ValueOf(fmt.Errorf("function `%s` errored: %+v", name, err))
}
e := r.Convert(errorType) // must be seen as an `error`
return []reflect.Value{zeroValue, e}
}
innerArgs = append(innerArgs, v)
}
result, err := fn.Call(innerArgs) // call it
if err != nil { // function errored :(
// errwrap is a better way to report errors, if allowed!
r := reflect.ValueOf(errwrap.Wrapf(err, "function `%s` errored", name))
if !r.Type().ConvertibleTo(errorType) { // for fun!
r = reflect.ValueOf(fmt.Errorf("function `%s` errored: %+v", name, err))
}
e := r.Convert(errorType) // must be seen as an `error`
return []reflect.Value{zeroValue, e}
} else if result == nil { // someone wrote a bad function
r := reflect.ValueOf(fmt.Errorf("function `%s` returned nil", name))
e := r.Convert(errorType) // must be seen as an `error`
return []reflect.Value{zeroValue, e}
}
nilError := reflect.Zero(errorType)
return []reflect.Value{reflect.ValueOf(result.Value()), nilError}
}
val := reflect.MakeFunc(typ, f)
return val.Interface()
}

42
lang/funcs/simple/datetime_print_func.go Normal file
View File

@@ -0,0 +1,42 @@
// Mgmt
// Copyright (C) 2013-2018+ James Shubin and the project contributors
// Written by James Shubin <james@shubin.ca> and the project contributors
//
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// This program is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
//
// You should have received a copy of the GNU General Public License
// along with this program. If not, see <http://www.gnu.org/licenses/>.
package simple // TODO: should this be in its own individual package?
import (
"fmt"
"time"
"github.com/purpleidea/mgmt/lang/types"
)
func init() {
// TODO: should we support namespacing these, eg: datetime.print ?
// FIXME: consider renaming this to printf, and add in a format string?
Register("datetime_print", &types.FuncValue{
T: types.NewType("func(a int) str"),
V: func(input []types.Value) (types.Value, error) {
epochDelta := input[0].Int()
if epochDelta < 0 {
return nil, fmt.Errorf("epoch delta must be positive")
}
return &types.StrValue{
V: time.Unix(epochDelta, 0).String(),
}, nil
},
})
}

39
lang/funcs/simple/example_errorbool_func.go Normal file
View File

@@ -0,0 +1,39 @@
// Mgmt
// Copyright (C) 2013-2018+ James Shubin and the project contributors
// Written by James Shubin <james@shubin.ca> and the project contributors
//
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// This program is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
//
// You should have received a copy of the GNU General Public License
// along with this program. If not, see <http://www.gnu.org/licenses/>.
package simple // TODO: should this be in its own individual package?
import (
"fmt"
"github.com/purpleidea/mgmt/lang/types"
)
func init() {
// TODO: should we support namespacing these, eg: example.errorbool ?
Register("example_errorbool", &types.FuncValue{
T: types.NewType("func(a bool) str"),
V: func(input []types.Value) (types.Value, error) {
if input[0].Bool() {
return nil, fmt.Errorf("we errored on request")
}
return &types.StrValue{
V: "set input to true to generate an error",
}, nil
},
})
}

35
lang/funcs/simple/int2str_func.go Normal file
View File

@@ -0,0 +1,35 @@
// Mgmt
// Copyright (C) 2013-2018+ James Shubin and the project contributors
// Written by James Shubin <james@shubin.ca> and the project contributors
//
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// This program is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
//
// You should have received a copy of the GNU General Public License
// along with this program. If not, see <http://www.gnu.org/licenses/>.
package simple // TODO: should this be in its own individual package?
import (
"fmt"
"github.com/purpleidea/mgmt/lang/types"
)
func init() {
Register("int2str", &types.FuncValue{
T: types.NewType("func(a int) str"),
V: func(input []types.Value) (types.Value, error) {
return &types.StrValue{
V: fmt.Sprintf("%d", input[0].Int()),
}, nil
},
})
}

135
lang/funcs/simple/simple.go Normal file
View File

@@ -0,0 +1,135 @@
// Mgmt
// Copyright (C) 2013-2018+ James Shubin and the project contributors
// Written by James Shubin <james@shubin.ca> and the project contributors
//
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// This program is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
//
// You should have received a copy of the GNU General Public License
// along with this program. If not, see <http://www.gnu.org/licenses/>.
package simple // TODO: should this be in its own individual package?
import (
"fmt"
"github.com/purpleidea/mgmt/lang/funcs"
"github.com/purpleidea/mgmt/lang/interfaces"
"github.com/purpleidea/mgmt/lang/types"
errwrap "github.com/pkg/errors"
)
// RegisteredFuncs maps a function name to the corresponding static, pure func.
var RegisteredFuncs = make(map[string]*types.FuncValue) // must initialize
// Register registers a simple, static, pure function. It is easier to use that
// the raw function API, but also limits you to simple, static, pure functions.
func Register(name string, fn *types.FuncValue) {
if _, exists := RegisteredFuncs[name]; exists {
panic(fmt.Sprintf("a simple func named %s is already registered", name))
}
RegisteredFuncs[name] = fn // store a copy for ourselves
// register a copy in the main function database
funcs.Register(name, func() interfaces.Func { return &simpleFunc{Fn: fn} })
}
// simpleFunc is a scaffolding function struct which fulfills the boiler-plate
// for the function API, but that can run a very simple, static, pure function.
type simpleFunc struct {
Fn *types.FuncValue
init *interfaces.Init
last types.Value // last value received to use for diff
result types.Value // last calculated output
closeChan chan struct{}
}
// Validate makes sure we've built our struct properly. It is usually unused for
// normal functions that users can use directly.
func (obj *simpleFunc) Validate() error {
if obj.Fn == nil { // build must be run first
return fmt.Errorf("type is still unspecified")
}
return nil
}
// Info returns some static info about itself.
func (obj *simpleFunc) Info() *interfaces.Info {
return &interfaces.Info{
Pure: true,
Memo: false, // TODO: should this be something we specify here?
Sig: obj.Fn.Type(),
Err: obj.Validate(),
}
}
// Init runs some startup code for this function.
func (obj *simpleFunc) Init(init *interfaces.Init) error {
obj.init = init
obj.closeChan = make(chan struct{})
return nil
}
// Stream returns the changing values that this func has over time.
func (obj *simpleFunc) Stream() error {
defer close(obj.init.Output) // the sender closes
for {
select {
case input, ok := <-obj.init.Input:
if !ok {
return nil // can't output any more
}
//if err := input.Type().Cmp(obj.Info().Sig.Input); err != nil {
// return errwrap.Wrapf(err, "wrong function input")
//}
if obj.last != nil && input.Cmp(obj.last) == nil {
continue // value didn't change, skip it
}
obj.last = input // store for next
values := []types.Value{}
for _, name := range obj.Fn.Type().Ord {
x := input.Struct()[name]
values = append(values, x)
}
result, err := obj.Fn.Call(values) // (Value, error)
if err != nil {
return errwrap.Wrapf(err, "simple function errored")
}
if obj.result == result {
continue // result didn't change
}
obj.result = result // store new result
case <-obj.closeChan:
return nil
}
select {
case obj.init.Output <- obj.result: // send
// pass
case <-obj.closeChan:
return nil
}
}
}
// Close runs some shutdown code for this function and turns off the stream.
func (obj *simpleFunc) Close() error {
close(obj.closeChan)
return nil
}

1
lang/lang.go
View File

@@ -26,6 +26,7 @@ import (
"github.com/purpleidea/mgmt/lang/funcs"
_ "github.com/purpleidea/mgmt/lang/funcs/core" // import so the funcs register
_ "github.com/purpleidea/mgmt/lang/funcs/facts/core" // import so the facts register
_ "github.com/purpleidea/mgmt/lang/funcs/simple" // import so the funcs register
"github.com/purpleidea/mgmt/lang/interfaces"
"github.com/purpleidea/mgmt/lang/unification"
"github.com/purpleidea/mgmt/pgraph"

168
lang/types/value.go
View File

@@ -47,8 +47,154 @@ type Value interface {
}
// ValueOf takes a reflect.Value and returns an equivalent Value.
func ValueOf(value reflect.Value) Value {
panic("not implemented") // XXX: not implemented
func ValueOf(v reflect.Value) (Value, error) {
value := v
typ := value.Type()
kind := typ.Kind()
for kind == reflect.Ptr {
typ = typ.Elem() // un-nest one pointer
kind = typ.Kind()
// un-nest value from pointer
value = value.Elem() // XXX: is this correct?
}
switch kind { // match on destination field kind
case reflect.Bool:
return &BoolValue{V: value.Bool()}, nil
case reflect.String:
return &StrValue{V: value.String()}, nil
case reflect.Int, reflect.Int64, reflect.Int32, reflect.Int16, reflect.Int8:
return &IntValue{V: value.Int()}, nil
case reflect.Uint, reflect.Uint64, reflect.Uint32, reflect.Uint16, reflect.Uint8:
return &IntValue{V: int64(value.Uint())}, nil
case reflect.Float64, reflect.Float32:
return &FloatValue{V: value.Float()}, nil
case reflect.Array, reflect.Slice:
values := []Value{}
for i := 0; i < value.Len(); i++ {
x := value.Index(i)
v, err := ValueOf(x) // recurse
if err != nil {
return nil, err
}
values = append(values, v)
}
t, err := TypeOf(value.Type().Elem()) // type of contents
if err != nil {
return nil, errwrap.Wrapf(err, "can't determine type of %+v", value)
}
return &ListValue{
T: NewType(fmt.Sprintf("[]%s", t.String())),
V: values,
}, nil
case reflect.Map:
m := make(map[Value]Value)
// loop through the list of map keys in undefined order
for _, mk := range value.MapKeys() {
mv := value.MapIndex(mk)
k, err := ValueOf(mk) // recurse
if err != nil {
return nil, err
}
v, err := ValueOf(mv) // recurse
if err != nil {
return nil, err
}
m[k] = v
}
kt, err := TypeOf(value.Type().Key()) // type of key
if err != nil {
return nil, errwrap.Wrapf(err, "can't determine key type of %+v", value)
}
vt, err := TypeOf(value.Type().Elem()) // type of value
if err != nil {
return nil, errwrap.Wrapf(err, "can't determine value type of %+v", value)
}
return &MapValue{
T: NewType(fmt.Sprintf("{%s: %s}", kt.String(), vt.String())),
V: m,
}, nil
case reflect.Struct:
// TODO: we could take this simpler "get the full type" approach
// for all the values, but I think that building them up when
// possible for the other cases is a more robust approach!
t, err := TypeOf(value.Type())
if err != nil {
return nil, errwrap.Wrapf(err, "can't determine type of %+v", value)
}
l := value.Len() // number of struct fields according to value
if l != len(t.Ord) {
// programming error?
return nil, fmt.Errorf("incompatible number of fields")
}
values := make(map[string]Value)
for i := 0; i < l; i++ {
x := value.Field(i)
v, err := ValueOf(x) // recurse
if err != nil {
return nil, err
}
name := t.Ord[i] // how else can we get the field name?
values[name] = v
}
return &StructValue{
T: t,
V: values,
}, nil
case reflect.Func:
t, err := TypeOf(value.Type())
if err != nil {
return nil, errwrap.Wrapf(err, "can't determine type of %+v", value)
}
if t.Out == nil {
return nil, fmt.Errorf("cannot only represent functions with one output value")
}
f := func(args []Value) (Value, error) {
in := []reflect.Value{}
for _, x := range args {
// TODO: should we build this method instead?
//v := x.Reflect() // types.Value -> reflect.Value
v := reflect.ValueOf(x.Value())
in = append(in, v)
}
// FIXME: can we trap panic's ?
out := value.Call(in) // []reflect.Value
if len(out) != 1 { // TODO: panic, b/c already checked in TypeOf?
return nil, fmt.Errorf("cannot only represent functions with one output value")
}
return ValueOf(out[0]) // recurse
}
return &FuncValue{
T: t,
V: f,
}, nil
default:
return nil, fmt.Errorf("unable to represent value of %+v", v)
}
}
// ValueSlice is a linear list of values. It is used for sorting purposes.
@@ -800,7 +946,10 @@ func (obj *FuncValue) Value() interface{} {
fn := func(args []reflect.Value) (results []reflect.Value) { // build
innerArgs := []Value{}
for _, x := range args {
v := ValueOf(x) // reflect.Value -> Value
v, err := ValueOf(x) // reflect.Value -> Value
if err != nil {
panic(fmt.Sprintf("can't determine value of %+v", x))
}
innerArgs = append(innerArgs, v)
}
result, err := obj.V(innerArgs) // call it
@@ -836,18 +985,23 @@ func (obj *FuncValue) Call(args []Value) (Value, error) {
// cmp input args type to obj.T
length := len(obj.T.Ord)
if length != len(args) {
panic(fmt.Sprintf("arg length of %d does not match expected of %d", len(args), length))
return nil, fmt.Errorf("arg length of %d does not match expected of %d", len(args), length)
}
for i := 0; i < length; i++ {
if err := args[i].Type().Cmp(obj.T.Map[obj.T.Ord[i]]); err != nil {
panic(fmt.Sprintf("cannot cmp input types: %+v", err))
return nil, errwrap.Wrapf(err, "cannot cmp input types")
}
}
result, err := obj.V(args) // call it
if result == nil {
if err == nil {
return nil, fmt.Errorf("function returned nil result")
}
return nil, errwrap.Wrapf(err, "function returned nil result during error")
}
if err := result.Type().Cmp(obj.T.Out); err != nil {
panic(fmt.Sprintf("cannot cmp return types: %+v", err))
return nil, errwrap.Wrapf(err, "cannot cmp return types")
}
return result, err

4
lang/types/value_test.go
View File

@@ -586,3 +586,7 @@ func TestStruct2(t *testing.T) {
//t.Errorf("val2: %s", val2)
}
}
func TestValueOf0(t *testing.T) {
// TODO: implement testing of the ValueOf function
}