6 Commits

Author SHA1 Message Date
Lourenço Vales
cdc09f9c46 added partial cloudflare api integration 2025-10-02 16:04:21 +02:00
Lourenço Vales
65fac167cf added cmp function 2025-10-02 13:08:49 +02:00
Lourenço Vales
6c67acf5fe added CheckApply function; made some changes to structure 2025-10-02 13:08:49 +02:00
Lourenço Vales
ab69c29761 engine: resources: Add Cloudflare DNS resource 2025-10-02 13:08:49 +02:00
James Shubin
5f4ae05340 readme: We moved to matrix 2025-10-02 03:05:05 -04:00
James Shubin
c48b884d16 misc: Add fpm repo script 2025-09-30 02:55:29 -04:00
5 changed files with 704 additions and 12 deletions

1
.gitignore vendored
View File

@@ -15,6 +15,7 @@ tmp/
mgmt.iml
/rpmbuild/
/releases/
/repository/
/pprof/
/sites/
# vim swap files

View File

@@ -6,7 +6,6 @@
[![Build Status](https://github.com/purpleidea/mgmt/workflows/.github/workflows/test.yaml/badge.svg)](https://github.com/purpleidea/mgmt/actions/)
[![GoDoc](https://img.shields.io/badge/godoc-reference-5272B4.svg?style=flat-square)](https://godocs.io/github.com/purpleidea/mgmt)
[![Matrix](https://img.shields.io/badge/matrix-%23mgmtconfig-orange.svg?style=flat-square)](https://matrix.to/#/#mgmtconfig:matrix.org)
[![IRC](https://img.shields.io/badge/irc-%23mgmtconfig-orange.svg?style=flat-square)](https://web.libera.chat/?channels=#mgmtconfig)
[![Patreon](https://img.shields.io/badge/patreon-donate-yellow.svg?style=flat-square)](https://www.patreon.com/purpleidea)
[![Liberapay](https://img.shields.io/badge/liberapay-donate-yellow.svg?style=flat-square)](https://liberapay.com/purpleidea/donate)
@@ -73,7 +72,6 @@ Come join us in the `mgmt` community!
| Medium | Link |
|---|---|
| Matrix | [#mgmtconfig](https://matrix.to/#/#mgmtconfig:matrix.org) on Matrix.org |
| IRC | [#mgmtconfig](https://web.libera.chat/?channels=#mgmtconfig) on Libera.Chat |
| Twitter | [@mgmtconfig](https://twitter.com/mgmtconfig) & [#mgmtconfig](https://twitter.com/hashtag/mgmtconfig) |
| Mailing list | [looking for a new home, suggestions welcome](https://gitlab.freedesktop.org/freedesktop/freedesktop/-/issues/1082) |
| Patreon | [purpleidea](https://www.patreon.com/purpleidea) on Patreon |

View File

@@ -53,16 +53,13 @@ find a number of tutorials online.
3. Spend between four to six hours with the [golang tour](https://tour.golang.org/).
Skip over the longer problems, but try and get a solid overview of everything.
If you forget something, you can always go back and repeat those parts.
4. Connect to our [#mgmtconfig](https://web.libera.chat/?channels=#mgmtconfig)
IRC channel on the [Libera.Chat](https://libera.chat/) network. You can use any
IRC client that you'd like, but the [hosted web portal](https://web.libera.chat/?channels=#mgmtconfig)
will suffice if you don't know what else to use. [Here are a few suggestions for
alternative clients.](https://libera.chat/guides/clients)
4. Connect to our [#mgmtconfig](https://matrix.to/#/#mgmtconfig:matrix.org)
Matrix channel and hang out with us there out there.
5. Now it's time to try and starting writing a patch! We have tagged a bunch of
[open issues as #mgmtlove](https://github.com/purpleidea/mgmt/issues?q=is%3Aissue+is%3Aopen+label%3Amgmtlove)
for new users to have somewhere to get involved. Look through them to see if
something interests you. If you find one, let us know you're working on it by
leaving a comment in the ticket. We'll be around to answer questions in the IRC
leaving a comment in the ticket. We'll be around to answer questions in the
channel, and to create new issues if there wasn't something that fit your
interests. When you submit a patch, we'll review it and give you some feedback.
Over time, we hope you'll learn a lot while supporting the project! Now get
@@ -534,9 +531,7 @@ which definitely existed before the band did.
### You didn't answer my question, or I have a question!
It's best to ask on [IRC](https://web.libera.chat/?channels=#mgmtconfig)
to see if someone can help you. If you don't get a response from IRC, you can
contact me through my [technical blog](https://purpleidea.com/contact/) and I'll
do my best to help. If you have a good question, please add it as a patch to
It's best to ask on [Matrix](https://matrix.to/#/#mgmtconfig:matrix.org) to see
if someone can help. If you don't get a response there, you can send a patch to
this documentation. I'll merge your question, and add a patch with the answer!
For news and updates, subscribe to the [mailing list](https://www.redhat.com/mailman/listinfo/mgmtconfig-list).

View File

@@ -0,0 +1,512 @@
// Mgmt
// Copyright (C) James Shubin and the project contributors
// Written by James Shubin <james@shubin.ca> and the project contributors
//
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// This program is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
//
// You should have received a copy of the GNU General Public License
// along with this program. If not, see <https://www.gnu.org/licenses/>.
//
// Additional permission under GNU GPL version 3 section 7
//
// If you modify this program, or any covered work, by linking or combining it
// with embedded mcl code and modules (and that the embedded mcl code and
// modules which link with this program, contain a copy of their source code in
// the authoritative form) containing parts covered by the terms of any other
// license, the licensors of this program grant you additional permission to
// convey the resulting work. Furthermore, the licensors of this program grant
// the original author, James Shubin, additional permission to update this
// additional permission if he deems it necessary to achieve the goals of this
// additional permission.
package resources
import (
"context"
"fmt"
"github.com/purpleidea/mgmt/engine"
"github.com/purpleidea/mgmt/engine/traits"
"github.com/purpleidea/mgmt/util/errwrap"
"github.com/cloudflare/cloudflare-go/v6"
"github.com/cloudflare/cloudflare-go/v6/zones"
)
func init() {
engine.RegisterResource("cloudflare:dns", func() engine.Res { return &CloudflareDNSRes{} })
}
// TODO: description of cloudflare_dns resource
type CloudflareDNSRes struct {
traits.Base
init *engine.Init
APIToken string `lang:"apitoken"`
Comment string `lang:"comment"`
Content string `lang:"content"`
// using a *int64 here to help with disambiguating nil values
Priority *int64 `lang:"priority"`
// using a *bool here to help with disambiguating nil values
Proxied *bool `lang:"proxied"`
Purged bool `lang:"purged"`
RecordName string `lang:"record_name"`
State string `lang:"state"`
TTL int64 `lang:"ttl"`
Type string `lang:"type"`
Zone string `lang:"zone"`
client *cloudflare.Client
zoneID string
}
func (obj *CloudflareDNSRes) Default() engine.Res {
return &CloudflareDNSRes{
State: "exists",
TTL: 1, // this sets TTL to automatic
}
}
func (obj *CloudflareDNSRes) Validate() error {
if obj.RecordName == "" {
return fmt.Errorf("record name is required")
}
if obj.APIToken == "" {
return fmt.Errorf("API token is required")
}
if obj.Type == "" {
return fmt.Errorf("record type is required")
}
if (obj.TTL < 60 || obj.TTL > 86400) && obj.TTL != 1 { // API requirement
return fmt.Errorf("TTL must be between 60s and 86400s, or set to 1")
}
if obj.Zone == "" {
return fmt.Errorf("zone name is required")
}
if obj.State != "exists" && obj.State != "absent" && obj.State != "" {
return fmt.Errorf("state must be either 'exists', 'absent', or empty")
}
if obj.State == "exists" && obj.Content == "" && !obj.Purge {
return fmt.Errorf("content is required when state is 'exists'")
}
if obj.MetaParams().Poll == 0 {
return fmt.Errorf("cloudflare:dns requiers polling, set Meta:poll param (e.g., 60 seconds)")
}
return nil
}
func (obj *CloudflareDNSRes) Init(init *engine.Init) error {
obj.init = init
obj.client = cloudflare.NewClient(
option.WithAPIToken(obj.APIToken),
)
//TODO: does it make more sense to check it here or in CheckApply()?
//zoneListParams := zones.ZoneListParams{
// name: cloudflare.F(obj.Zone),
//}
//zoneList, err := obj.client.Zones.List(context.Background(), zoneListParams)
//if err != nil {
// return errwrap.Wrapf(err, "failed to list zones")
//}
//if len(zoneList.Result) == 0 {
// return fmt.Errorf("zone %s not found", obj.Zone)
//}
obj.zoneID = zoneList.Results[0].ID
return nil
}
func (obj *CloudflareDNSRes) Cleanup() error {
obj.APIToken = ""
obj.client = nil
obj.zoneID = ""
return nil
}
// Watch isn't implemented for this resource, since the Cloudflare API does not
// provide any event stream. Instead, always use polling.
func (obj *CloudflareDNSRes) Watch(context.Context) error {
return fmt.Errorf("invalid Watch call: requires poll metaparam")
}
func (obj *CloudflareDNSRes) CheckApply(ctx context.Context, apply bool) (bool, error) {
zone, err := obj.client.Zones.List(ctx, zones.ZoneListParams{
RecordName: cloudflare.F(obj.Zone),
})
if err != nil {
return false, fmt.Errorf(err)
}
if len(zone.Result) == 0 {
return false, fmt.Errorf("there's no zone registered with name %s", obj.Zone)
}
if len(zone.Result) > 1 {
return false, fmt.Errorf("there's more than one zone with name %s", obj.Zone)
}
// We start by checking the need for purging
if obj.Purge {
checkOK, err := obj.purgeCheckApply(ctx, apply)
if err != nil {
return false, err
}
if !checkOK {
return false, nil
}
}
// List existing records
listParams := dns.RecordListParams{
ZoneID: cloudflare.F(obj.zoneID),
Name: cloudflare.F(obj.RecordName),
Type: cloudflare.F(dns.RecordListParamsType(obj.Type)),
}
recordList, err := obj.client.DNS.Records.List(ctx, listParams)
if err != nil {
return false, errwrap.Wrapf(err, "failed to list DNS records")
}
recordExists := len(records.Result) > 0
var record dns.Record
if recordExists {
record = recordList.Result[0]
}
switch obj.State {
case "exists", "":
if !recordExists {
if !apply {
return false, nil
}
if err := obj.createRecord(ctx); err != nil {
return false, err
}
return true, nil
}
if obj.needsUpdate(record) {
if !apply {
return false, nil
}
if err := obj.updateRecord(ctx, record.ID); err != nil {
return false, err
}
return true, nil
}
case "absent":
if recordExists {
if !apply {
return false, nil
}
deleteParams := dns.RecordDeleteParams{
ZoneID: cloudflare.F(obj.zoneID),
}
_, err := obj.client.DNS.Reords.Delete(ctx, record.ID, deleteParams)
if err != nil {
return false, errwrap.Wrapf(err, "failed to delete DNS record")
}
return true, nil
}
}
return true, nil
}
func (obj *CloudflareDNSRes) Cmp(r engine.Res) error {
if obj == nil && r == nil {
return nil
}
if (obj == nil) != (r == nil) {
return fmt.Errorf("one resource is empty")
}
res, ok := r.(*CloudflareDNSRes)
if !ok {
return fmt.Errorf("not a %s", obj.Kind())
}
if obj.APIToken != res.APIToken {
return fmt.Errorf("apitoken differs")
}
// check how this being a pointer influences this check
if obj.Proxied != res.Proxied {
return fmt.Errorf("proxied values differ")
}
if obj.RecordName != res.RecordName {
return fmt.Errorf("record name differs")
}
if obj.Purged != res.Purged {
return fmt.Errorf("purge value differs")
}
if obj.State != res.State {
return fmt.Errorf("state differs")
}
if obj.TTL != res.TTL {
return fmt.Errorf("ttl differs")
}
if obj.Type != res.Type {
return fmt.Errorf("record type differs")
}
if obj.Zone != res.Zone {
return fmt.Errorf("zone differs")
}
if obj.zoneID != res.zoneID {
return fmt.Errorf("zoneid differs")
}
if obj.Content != res.Content {
return fmt.Errorf("content param differs")
}
// check how this being a pointer influences this check
if obj.Priority != res.Priority {
return fmt.Errorf("the priority param differs")
}
return nil
}
func (obj *CloudflareDNSRes) buildRecordParam() dns.RecordNewParamsBodyUnion {
ttl := dns.TTL(obj.TTL)
switch obj.Type {
case "A":
param := dns.ARecordParam{
Name: cloudflare.F(obj.RecordName),
Type: cloudflare.F(dns.ARecordTypeA),
Content: cloudflare.F(obj.Content),
TTL: cloudflare.F(ttl),
}
if obj.Proxied != nil {
param.Proxied = cloudflare.F(*obj.Proxied)
}
if obj.Comment != "" {
param.Comment = cloudflare.F(obj.Comment)
}
return param
case "AAAA":
param := dns.AAAARecordParam{
Name: cloudflare.F(obj.RecordName),
Type: cloudflare.F(dns.AAAARecordTypeAAAA),
Content: cloudflare.F(obj.Content),
TTL: cloudflare.F(ttl),
}
if obj.Proxied != nil {
param.Proxied = cloudflare.F(*obj.Proxied)
}
if obj.Comment != "" {
param.Comment = cloudflare.F(obj.Comment)
}
return param
case "CNAME":
param := dns.CNAMERecordParam{
Name: cloudflare.F(obj.RecordName),
Type: cloudflare.F(dns.CNAMERecordTypeCNAME),
Content: cloudflare.F(obj.Content),
TTL: cloudflare.F(ttl),
}
if obj.Proxied != nil {
param.Proxied = cloudflare.F(*obj.Proxied)
}
if obj.Comment != "" {
param.Comment = cloudflare.F(obj.Comment)
}
return param
case "MX":
param := dns.MXRecordParam{
Name: cloudflare.F(obj.RecordName),
Type: cloudflare.F(dns.MXRecordTypeMX),
Content: cloudflare.F(obj.Content),
TTL: cloudflare.F(ttl),
}
if obj.Proxied != nil {
param.Proxied = cloudflare.F(*obj.Proxied)
}
if obj.Priority != nil { // required for MX record
param.Priority = cloudflare.F(*obj.Priority)
}
if obj.Comment != "" {
param.Comment = cloudflare.F(obj.Comment)
}
return param
case "TXT":
param := dns.TXTRecordParam{
Name: cloudflare.F(obj.RecordName),
Type: cloudflare.F(dns.TXTRecordTypeTXT),
Content: cloudflare.F(obj.Content),
TTL: cloudflare.F(ttl),
}
if obj.Proxied != nil {
param.Proxied = cloudflare.F(*obj.Proxied)
}
if obj.Comment != "" {
param.Comment = cloudflare.F(obj.Comment)
}
return param
case "NS":
param := dns.NSRecordParam{
Name: cloudflare.F(obj.RecordName),
Type: cloudflare.F(dns.NSRecordTypeNS),
Content: cloudflare.F(obj.Content),
TTL: cloudflare.F(ttl),
}
if obj.Proxied != nil {
param.Proxied = cloudflare.F(*obj.Proxied)
}
if obj.Comment != "" {
param.Comment = cloudflare.F(obj.Comment)
}
return param
case "SRV":
param := dns.SRVRecordParam{
Name: cloudflare.F(obj.RecordName),
Type: cloudflare.F(dns.SRVRecordTypeSRV),
Content: cloudflare.F(obj.Content),
TTL: cloudflare.F(ttl),
}
if obj.Proxied != nil {
param.Proxied = cloudflare.F(*obj.Proxied)
}
if obj.Priority != nil {
param.Priority = cloudflare.F(*obj.Priority)
}
if obj.Comment != "" {
param.Comment = cloudflare.F(obj.Comment)
}
return param
case "PTR":
param := dns.PTRRecordParam{
Name: cloudflare.F(obj.RecordName),
Type: cloudflare.F(dns.PTRRecordTypePTR),
Content: cloudflare.F(obj.Content),
TTL: cloudflare.F(ttl),
}
if obj.Proxied != nil {
param.Proxied = cloudflare.F(*obj.Proxied)
}
if obj.Comment != "" {
param.Comment = cloudflare.F(obj.Comment)
}
return param
default: // we should return something else here, need to investigate
}
}
func (obj *CloudflareDNSRes) createRecord(ctx context.Context) error {
recordParams := obj.buildRecordParam()
createParams := dns.RecordNewParams{
ZoneID: cloudflare.F(obj.zoneID),
Body: recordParams,
}
_, err := obj.client.DNS.Records.New(ctx, createParams)
if err != nil {
return errwrap.Wrapf(err, "failed to create dns record")
}
return nil
}
func (obj *CloudflareDNSRes) updateRecord(ctx context.Context, recordID string) error {
recordParams := obj.buildRecordParam()
editParams := dns.RecordEditParams{
ZoneID: cloudflare.F(obj.zoneID),
Body: recordParams,
}
_, err := obj.client.DNS.Records.Edit(ctx, recordID, editParams)
if err != nil {
return errwrap.Wrapf(err, "failed to update dns record")
}
return nil
}
func (obj *CloudflareDNSRes) needsUpdate(record dns.Record) bool {
if obj.Content != record.Content {
return true
}
if obj.TTL != int64(record.TTL) {
return true
}
if obj.Proxied != nil && record.Proxied != nil {
if *obj.Proxied != *record.Proxied {
return true
}
}
if obj.Priority != nil && record.Priority != nil {
if *obj.Priority != *record.Priority {
return true
}
}
if obj.Comment != record.Comment {
return true
}
// TODO add more checks?
return false
}

186
misc/fpm-repo.sh Executable file
View File

@@ -0,0 +1,186 @@
#!/usr/bin/env bash
# This script makes packages of mgmt using fpm. It pulls in any binaries from:
# releases/$version/binary-linux-$arch/mgmt-linux-$arch-$version and builds to:
# repository/$distro-$version/$arch/ before it runs the appropriate createrepo
# commands.
# NOTE: run `gem install fpm` to update my ~/bin/fpm to the latest version.
# TODO: consider switching to https://github.com/goreleaser/nfpm
# the binary to package
BINARY="mgmt"
# maintainer email
MAINTAINER="fpm@mgmtconfig.com"
# project url
URL="https://github.com/purpleidea/mgmt/"
# project description
DESCRIPTION="Next generation distributed, event-driven, parallel config management!"
# project license
LICENSE="GPLv3"
# location to install the binary
PREFIX="/usr/bin"
# release directory
DIR="releases"
# repository directory
OUT="repository"
## make sure the distro is a known valid one
#if [[ "$DISTRO" == fedora-* ]]; then
# typ="rpm"
#elif [[ "$DISTRO" == centos-* ]]; then
# typ="rpm"
#elif [[ "$DISTRO" == debian-* ]]; then
# typ="deb"
#elif [[ "$DISTRO" == ubuntu-* ]]; then
# typ="deb"
#elif [[ "$DISTRO" == archlinux ]]; then
# typ="pacman"
#else
# echo "unknown distro: ${DISTRO}."
# exit 1
#fi
#if [ "$typ" != "rpm" ] && [ "$typ" != "deb" ] && [ "$typ" != "pacman" ]; then
# echo "invalid package type"
# exit 1
#fi
## assume the file extension
#ext="$typ"
#if [ "$typ" = "pacman" ]; then # archlinux is an exception
# ext="pkg.tar.xz"
#fi
# in case the `fpm` gem bin isn't in the $PATH
if command -v ruby >/dev/null && command -v gem >/dev/null && ! command -v fpm >/dev/null; then
PATH="$(ruby -r rubygems -e 'puts Gem.user_dir')/bin:$PATH"
fi
# skip putting these versions into the repos
skip_mgmt_version=()
skip_mgmt_version+=("0.0.25")
skip_mgmt_version+=("0.0.26")
skip_mgmt_version+=("0.0.27")
# from binary arch to repoarch
declare -A map_repoarch=(
[amd64]="x86_64"
[arm64]="aarch64"
)
declare -A map_distrotype=(
[fedora]="rpm"
[debian]="deb"
)
declare -A map_distro_version=(
["fedora-41"]="libvirt-devel augeas-devel"
["fedora-42"]="libvirt-devel augeas-devel"
["debian-13"]="libvirt-dev libaugeas-dev"
)
#echo releases:
#for dv in "fedora-41" "fedora-42" "debian-11" "archlinux-xx"; do
for dv in "${!map_distro_version[@]}"; do
distro=${dv%%-*};
version=${dv##*-}
deps=${map_distro_version[$dv]}
echo "distro-version: ${distro}-${version}"
mkdir -p ${OUT}/$dv/
type=${map_distrotype[$distro]}
# track the arches we see
declare -A repoarches=()
for chunk1 in ${DIR}/*; do
if [ ! -d "$chunk1" ]; then # check if it's a regular dir
continue
fi
package_version=$(basename "$chunk1")
#echo "package_version: $package_version"
if [[ " ${skip_mgmt_version[*]} " == *" $package_version "* ]]; then
echo "skip: ${package_version}"
continue
fi
for chunk2 in $chunk1/binary-linux-*; do
if [ ! -d "$chunk2" ]; then # check if it's a regular dir
continue
fi
arch=${chunk2##*-}
#echo "arch: $arch"
repoarch=${map_repoarch[$arch]}
#echo "repoarch: $repoarch"
repoarches["${repoarch}"]="${type}" # tag it
mkdir -p ${OUT}/${distro}-${version}/$repoarch/
#file $chunk2/mgmt-linux-$arch-$package_version # found it
output="${OUT}/${distro}-${version}/${repoarch}/mgmt-${package_version}.${repoarch}.${type}"
if [ -f "$output" ]; then
echo "skip: ${output}"
continue
fi
depends=""
for i in $deps; do
depends="$depends --depends $i"
done
# build the package
echo "fpm..."
echo "output: ${output}"
fpm \
--log error \
--name "$BINARY" \
--version "$package_version" \
--maintainer "$MAINTAINER" \
--url "$URL" \
--description "$DESCRIPTION" \
--license "$LICENSE" \
--input-type dir \
--output-type "$type" \
--package "${output}" \
${depends} \
"misc/mgmt.service"="/usr/lib/systemd/system/mgmt.service" \
"$BINARY"="$PREFIX/mgmt" \
|| rm "${output}" # if it fails, remove it...
done
done
# now run createrepo or similar
for key in "${!repoarches[@]}"; do
type=${repoarches[$key]}
outdir="${OUT}/$dv/${key}/"
if [ "$type" = "rpm" ]; then
echo "createrepo ${type} ${outdir}"
# TODO: use --deltas ?
createrepo_c --update "${outdir}"
fi
if [ "$type" = "deb" ]; then
cd ${outdir} > /dev/null
# don't regenerate unnecessarily
if [[ ! -f Packages.gz ]] || find . -name '*.deb' -newer Packages.gz | grep -q .; then
echo "dpkg-scanpackages ${type} ${outdir}"
dpkg-scanpackages --multiversion . /dev/null | gzip -9 > Packages.gz
fi
cd - > /dev/null # silence it
fi
done
done
USERNAME=$(cat ~/.config/copr 2>/dev/null | grep username | awk -F '=' '{print $2}' | tr -d ' ')
SERVER='dl.fedoraproject.org'
REMOTE_PATH="/srv/pub/alt/${USERNAME}/${BINARY}/repo/"
if [ "${USERNAME}" = "" ]; then
echo "empty username, can't rsync"
fi
rsync -avzSH --progress --delete ${OUT}/ ${SERVER}:${REMOTE_PATH}