diff --git a/docs/API.md b/docs/API.md index 0de1ea2..4ce0da1 100644 --- a/docs/API.md +++ b/docs/API.md @@ -23,8 +23,10 @@ Sitzungen enden nach 12 Stunden oder nach 1 Stunde Inaktivität (konfigurierbar) | Recht | Viewer | Operator | Admin | | --- | --- | --- | --- | -| Netze, Adressen, Lizenzstatus ansehen, exportieren, eigene Filter | ✓ | ✓ | ✓ | -| Netze und Adressen ändern, Import | | ✓ | ✓ | +| Netze, Adressen, Zonen, Records, Cluster, TSIG-Schlüssel (ohne Geheimnis), Lizenzstatus ansehen, exportieren, eigene Filter | ✓ | ✓ | ✓ | +| Netze, Adressen, Zonen und Records ändern, Import | | ✓ | ✓ | +| Revisionen erzeugen, ausrollen, zurückrollen | | ✓ | ✓ | +| Cluster und TSIG-Schlüssel verwalten, Geheimnis eines Schlüssels anzeigen, Agents, Generator-Einstellungen | | | ✓ | | Benutzer, SSO, Lizenz verwalten, Audit-Log | | | ✓ | `GET /auth/session` liefert die effektiven Rechte (`permissions`). @@ -57,7 +59,7 @@ Ist der Server schreibgeschützt (siehe [LICENSING.md](LICENSING.md)), antworten ## Gleichzeitige Änderungen -Netze und Adressen tragen eine `version`. Änderungen (`PUT`) müssen die gelesene Version mitsenden; hat jemand anderes inzwischen geändert, antwortet der Server mit `409 version_conflict`. +Netze, Adressen, Cluster, TSIG-Schlüssel, Zonen und Records tragen eine `version`. Änderungen (`PUT`) müssen die gelesene Version mitsenden; hat jemand anderes inzwischen geändert, antwortet der Server mit `409 version_conflict`. ## Adresstabelle @@ -74,6 +76,14 @@ Die Antwort enthält `items` und `total`. `GET /addresses/export` akzeptiert die `POST /addresses/preview` beschreibt vorab die Auswirkungen einer Änderung auf DNS und DHCP (z. B. `dhcp.add_reservation`). Massenänderungen (`/addresses/bulk`) und CSV-Import (`/addresses/import`, mit `dry_run=true` zur Prüfung) gelten ganz oder gar nicht; Probleme werden pro Adresse bzw. Zeile gemeldet. +## Geheimnisse + +TSIG-Schlüssel und HA-Passwörter erzeugt der Server selbst ([ADR 0023](adr/0023-cluster-und-zonen-m5.md)). Die API liefert sie nur an einer Stelle aus: `GET /tsig-keys/{id}/secret` (Recht `settings.admin`, jeder Abruf steht im Audit-Log – ohne das Geheimnis). Dateien mit Schlüsselmaterial erscheinen in Vorschau, Revision, Diff und offenen Änderungen mit `secret: true` und leerem Inhalt; Diffs nennen für sie nur den Status. Das HA-Passwort eines DHCP-Clusters wird nie angezeigt. + +## Ziele von Revisionen + +Dateien in Vorschau, Revision und Diff tragen ein `target`: leer für die Standardgruppe (Agents ohne Cluster), sonst die Agent-ID eines Cluster-Mitglieds. `targets` beschreibt die Ziele mit Rolle, Cluster und Mitgliedsrolle zum Zeitpunkt der Revision. Ein Agent erhält die Dateien seines Ziels, wenn die Revision eines hat, sonst die der Standardgruppe. + ## Betriebsendpunkte Außerhalb von `/api/v1`, ohne Anmeldung: diff --git a/docs/adr/0022-ausrollen-pull-push-m4.md b/docs/adr/0022-ausrollen-pull-push-m4.md index 592c5a9..9443fda 100644 --- a/docs/adr/0022-ausrollen-pull-push-m4.md +++ b/docs/adr/0022-ausrollen-pull-push-m4.md @@ -39,7 +39,7 @@ M3 rollt jede neue Revision sofort auf alle Agents aus. M4 verlangt laut Brief B ### Live-Status -- `GET /events` liefert einen **Server-Sent-Events**-Strom für angemeldete Benutzer (Recht `ipam.read`). Ereignisse enthalten nur Kennungen, keine Daten: `agent` (`id`), `deployment` (`id`, `agent_id`, `revision`), `revision` (`number`), `pending`. Die Oberfläche lädt daraufhin die betroffenen Abfragen neu und fällt bei Verbindungsabbruch auf Polling zurück. Alle 25 s ein Kommentar als Keepalive. +- `GET /events` liefert einen **Server-Sent-Events**-Strom für angemeldete Benutzer (Recht `ipam.read`). Ereignisse enthalten nur Kennungen, keine Daten: `agent` (`id`), `deployment` (`id`, `agent_id`, `revision`), `revision` (`number`), `pending`; seit M5 außerdem `addresses` (`network_ids`) für Adressen, die die Lease-Erfassung angelegt oder geändert hat (ADR 0023). Die Oberfläche lädt daraufhin die betroffenen Abfragen neu und fällt bei Verbindungsabbruch auf Polling zurück. Alle 25 s ein Kommentar als Keepalive. - SSE statt WebSocket: nur Server→Browser nötig, funktioniert mit Sitzungscookie, Proxys und `EventSource` ohne Zusatzbibliothek; der Endpunkt nutzt die bestehende Sitzungsprüfung. - Quelle ist ein prozessinterner Ereignisbus (`server/internal/events`), den Dienst und Agent-API bei Änderungen benachrichtigen. Mehrere Server-Instanzen sind nicht vorgesehen (ADR 0006). diff --git a/docs/adr/0023-cluster-und-zonen-m5.md b/docs/adr/0023-cluster-und-zonen-m5.md index 85a5030..840d54c 100644 --- a/docs/adr/0023-cluster-und-zonen-m5.md +++ b/docs/adr/0023-cluster-und-zonen-m5.md @@ -57,6 +57,15 @@ Bis M4 erhalten alle DNS-Agents dieselben Dateien (jede Zone als Primary) und al - Cluster und TSIG-Schlüssel: Lesen mit `ipam.read` (ohne Geheimnisse), Ändern mit `settings.admin`. Zonen und Records: Lesen mit `ipam.read`, Ändern mit `ipam.write`. Alle Änderungen schreiben Audit-Einträge und zählen zu den offenen Änderungen. +## Präzisierungen bei der Umsetzung (Server) + +- **Schema (Migration 00005):** `cluster_members` trägt neben der Mitgliedsrolle (`member_role`) die Rolle von Agent und Cluster (`role`); die Fremdschlüssel `(cluster_id, role)` und `(agent_id, role)` erzwingen gleiche Rollen. Zonen (`cluster_role`, fest `dns`) und Netze (`dhcp_cluster_role`, fest `dhcp`) verweisen ebenso über `(id, role)` auf Cluster. Der Transfer-Schlüssel gehört über `tsig_keys.cluster_id` (Löschen kaskadiert) seinem Cluster; `clusters.transfer_key_id` verweist auf `(id, cluster_id)` des Schlüssels, sodass er sich nicht allein löschen lässt. Adressen eines Clusters sind je Cluster eindeutig. Records sind (TTL ausgenommen) je Zone eindeutig, TXT-Daten über einen Hash im Index. +- **Geheimnisse:** TSIG-Geheimnisse liegen als Rohbytes versiegelt in `tsig_keys.secret` (zugehörige Daten: Tabelle, Spalte, ID), das HA-Passwort in `clusters.ha_password`. Der HA-Benutzer heißt `ddi-ha-<12 Hex-Zeichen der Cluster-ID>`, der Transfer-Schlüssel `transfer-`. `config_revisions.secrets` enthält `{"key:": "", "ha:": ""}`; fehlt ein benötigter Wert, ist ein Rollback `rollback_unavailable`. In die Generator-Eingabe gelangen nur Schlüssel, auf die Zonen oder Cluster verweisen; ein unbenutzter Schlüssel ändert die Konfiguration nicht. +- **Offene Änderungen:** Änderungen an Clustern, Zonen, Records und TSIG-Schlüsseln (`tsig_key.create`/`update`/`delete`/`rotate`) zählen dazu, das Anzeigen eines Geheimnisses (`tsig_key.read_secret`) nicht. +- **Dynamische Zonen** brauchen mindestens eine Quelle in `allow_update` oder einen Schlüssel in `update_key_ids` (`update_source_required`); bei managed Zonen werden beide Listen geleert. Reverse-Zonen werden auf gültige Oktette bzw. Nibbles geprüft. +- **Lease-Erfassung:** Ein Hostname wird nur übernommen, wenn er ein gültiges Label ist (sonst bleibt er leer bzw. unverändert); der FQDN folgt aus der Domain des Netzes. Lease-Felder (`last_seen`, `lease_state`, `lease_expires_at`) ändern die `version` nicht, eine geänderte MAC oder ein geänderter Hostname eines `dynamic`-Eintrags schon. Ist das Adresslimit der Lizenz erreicht, werden keine weiteren Einträge angelegt. Die Oberfläche erhält das Ereignis `addresses` mit `network_ids` (ADR 0022). +- **Fehlercodes:** `key_exists`, `invalid_key_name`, `key_in_use`, `key_owned_by_cluster`, `cluster_exists`, `cluster_role_mismatch`, `agent_in_other_cluster`, `cluster_primary_exists`, `cluster_member_limit`, `duplicate_member` (Agent oder Adresse doppelt), `unknown_agent`, `cluster_in_use` (Parameter `zones`, `networks`, `default`, `key_zones`), `zone_exists`, `zone_kind_mismatch`, `zone_not_empty` (`count`), `update_source_required`, `record_exists`, `cname_at_apex`, `field_not_allowed`, `invalid_label`, `invalid_txt`, `ip_family`, `not_found` (als Feldfehler für verwiesene Cluster und Schlüssel), `secrets_unavailable` (kein Master-Key). + ## Konsequenzen - Server ohne Cluster verhalten sich unverändert; Golden-Files von M2–M4 bleiben gültig. diff --git a/server/internal/agentapi/rpcs.go b/server/internal/agentapi/rpcs.go index ef01587..f611a9e 100644 --- a/server/internal/agentapi/rpcs.go +++ b/server/internal/agentapi/rpcs.go @@ -124,7 +124,7 @@ func (s *Server) GetDesiredState(ctx context.Context, _ *agentv1.GetDesiredState // for the calling agent and its role; equal input gives equal bytes. func (s *Server) FetchBundle(ctx context.Context, req *agentv1.FetchBundleRequest) (*agentv1.FetchBundleResponse, error) { c := callerFrom(ctx) - createdAt, arts, err := s.svc.RevisionArtifacts(ctx, req.GetRevision(), c.Role) + createdAt, arts, err := s.svc.RevisionArtifacts(ctx, req.GetRevision(), c.Role, c.AgentID) if errors.Is(err, service.ErrNotFound) { return nil, status.Errorf(codes.NotFound, "revision %d does not exist", req.GetRevision()) } diff --git a/server/internal/events/events.go b/server/internal/events/events.go index 50bb6da..7bb72e3 100644 --- a/server/internal/events/events.go +++ b/server/internal/events/events.go @@ -19,6 +19,9 @@ const ( TypeDeployment = "deployment" TypeRevision = "revision" TypePending = "pending" + // TypeAddresses: addresses were created or changed without a user, by + // the adoption of DHCP leases (ADR 0023). + TypeAddresses = "addresses" ) // Agent says that an agent changed (status, connection, mode, revocation, @@ -42,6 +45,12 @@ type Revision struct { // Pending says that data used by the generator changed. type Pending struct{} +// Addresses says that addresses of these networks were created or changed +// by the lease adoption (sorted network IDs). +type Addresses struct { + NetworkIDs []string `json:"network_ids"` +} + // Event is one published event with its JSON payload. type Event struct { Type string diff --git a/server/internal/httpapi/h_ipam.go b/server/internal/httpapi/h_ipam.go index f161f66..3b08f1b 100644 --- a/server/internal/httpapi/h_ipam.go +++ b/server/internal/httpapi/h_ipam.go @@ -82,6 +82,7 @@ type networkRequest struct { DNSServers []string `json:"dns_servers"` NTPServers []string `json:"ntp_servers"` Pools []poolRequest `json:"pools"` + DHCPClusterID *string `json:"dhcp_cluster_id"` Version int `json:"version"` } @@ -91,7 +92,7 @@ func (req *networkRequest) input(withCIDR bool) (service.NetworkInput, error) { Name: req.Name, Description: req.Description, VLAN: req.VLAN, Gateway: p.optAddr("gateway", req.Gateway), Domain: req.Domain, SearchDomains: req.SearchDomains, Tags: req.Tags, DHCPEnabled: req.DHCPEnabled, LeaseTime: req.LeaseTime, DNSServers: p.addrs("dns_servers", req.DNSServers), - NTPServers: p.addrs("ntp_servers", req.NTPServers), Version: req.Version, + NTPServers: p.addrs("ntp_servers", req.NTPServers), DHCPClusterID: req.DHCPClusterID, Version: req.Version, } if withCIDR { in.CIDR = p.prefix("cidr", req.CIDR) diff --git a/server/internal/httpapi/h_zones.go b/server/internal/httpapi/h_zones.go new file mode 100644 index 0000000..2a88840 --- /dev/null +++ b/server/internal/httpapi/h_zones.go @@ -0,0 +1,275 @@ +// SPDX-License-Identifier: LicenseRef-DDI-Proprietary + +package httpapi + +import ( + "net/http" + + "git.beugsiud.de/beug.k/DDI/server/internal/service" +) + +// Clusters, TSIG keys, zones and records (ADR 0023). + +type clusterMemberRequest struct { + AgentID string `json:"agent_id"` + Role string `json:"role"` + Address string `json:"address"` +} + +type clusterRequest struct { + Name string `json:"name"` + Role string `json:"role"` + Description string `json:"description"` + Members []clusterMemberRequest `json:"members"` + PrimaryNS string `json:"primary_ns"` + NameServers []service.GeneratorNS `json:"name_servers"` + HAMode string `json:"ha_mode"` + HAPort *int `json:"ha_port"` + Version int `json:"version"` +} + +func (req *clusterRequest) input() service.ClusterInput { + in := service.ClusterInput{Name: req.Name, Role: req.Role, Description: req.Description, PrimaryNS: req.PrimaryNS, + NameServers: req.NameServers, HAMode: req.HAMode, HAPort: req.HAPort, Version: req.Version} + for _, m := range req.Members { + in.Members = append(in.Members, service.ClusterMemberInput(m)) + } + return in +} + +func (a *API) listClusters(w http.ResponseWriter, r *http.Request, _ *reqCtx) error { + list, err := a.svc.ListClusters(r.Context(), r.URL.Query().Get("role")) + if err != nil { + return err + } + return writeJSON(w, http.StatusOK, listResponse[service.ClusterDTO]{Items: list}) +} + +func (a *API) createCluster(w http.ResponseWriter, r *http.Request, rc *reqCtx) error { + var req clusterRequest + if err := decode(w, r, &req); err != nil { + return err + } + c, err := a.svc.CreateCluster(r.Context(), rc.Actor, req.input()) + if err != nil { + return err + } + return writeJSON(w, http.StatusCreated, c) +} + +func (a *API) getCluster(w http.ResponseWriter, r *http.Request, _ *reqCtx) error { + c, err := a.svc.GetCluster(r.Context(), r.PathValue("id")) + if err != nil { + return err + } + return writeJSON(w, http.StatusOK, c) +} + +func (a *API) updateCluster(w http.ResponseWriter, r *http.Request, rc *reqCtx) error { + var req clusterRequest + if err := decode(w, r, &req); err != nil { + return err + } + c, err := a.svc.UpdateCluster(r.Context(), rc.Actor, r.PathValue("id"), req.input()) + if err != nil { + return err + } + return writeJSON(w, http.StatusOK, c) +} + +func (a *API) deleteCluster(w http.ResponseWriter, r *http.Request, rc *reqCtx) error { + if err := a.svc.DeleteCluster(r.Context(), rc.Actor, r.PathValue("id")); err != nil { + return err + } + return noContent(w) +} + +type tsigKeyRequest struct { + Name string `json:"name"` + Algorithm string `json:"algorithm"` + Description string `json:"description"` + Version int `json:"version"` +} + +func (req *tsigKeyRequest) input() service.TSIGKeyInput { return service.TSIGKeyInput(*req) } + +func (a *API) listTSIGKeys(w http.ResponseWriter, r *http.Request, _ *reqCtx) error { + list, err := a.svc.ListTSIGKeys(r.Context()) + if err != nil { + return err + } + return writeJSON(w, http.StatusOK, listResponse[service.TSIGKeyDTO]{Items: list}) +} + +func (a *API) createTSIGKey(w http.ResponseWriter, r *http.Request, rc *reqCtx) error { + var req tsigKeyRequest + if err := decode(w, r, &req); err != nil { + return err + } + k, err := a.svc.CreateTSIGKey(r.Context(), rc.Actor, req.input()) + if err != nil { + return err + } + return writeJSON(w, http.StatusCreated, k) +} + +func (a *API) updateTSIGKey(w http.ResponseWriter, r *http.Request, rc *reqCtx) error { + var req tsigKeyRequest + if err := decode(w, r, &req); err != nil { + return err + } + k, err := a.svc.UpdateTSIGKey(r.Context(), rc.Actor, r.PathValue("id"), req.input()) + if err != nil { + return err + } + return writeJSON(w, http.StatusOK, k) +} + +func (a *API) deleteTSIGKey(w http.ResponseWriter, r *http.Request, rc *reqCtx) error { + if err := a.svc.DeleteTSIGKey(r.Context(), rc.Actor, r.PathValue("id")); err != nil { + return err + } + return noContent(w) +} + +func (a *API) getTSIGKeySecret(w http.ResponseWriter, r *http.Request, rc *reqCtx) error { + k, err := a.svc.GetTSIGKeySecret(r.Context(), rc.Actor, r.PathValue("id")) + if err != nil { + return err + } + return writeJSON(w, http.StatusOK, k) +} + +func (a *API) rotateTSIGKey(w http.ResponseWriter, r *http.Request, rc *reqCtx) error { + k, err := a.svc.RotateTSIGKey(r.Context(), rc.Actor, r.PathValue("id")) + if err != nil { + return err + } + return writeJSON(w, http.StatusOK, k) +} + +type zoneRequest struct { + Name string `json:"name"` + Kind string `json:"kind"` + Mode string `json:"mode"` + ClusterID *string `json:"cluster_id"` + DefaultTTL *int `json:"default_ttl"` + AllowUpdate []string `json:"allow_update"` + UpdateKeyIDs []string `json:"update_key_ids"` + Description string `json:"description"` + Version int `json:"version"` +} + +func (req *zoneRequest) input() service.ZoneInput { return service.ZoneInput(*req) } + +func (a *API) listZones(w http.ResponseWriter, r *http.Request, _ *reqCtx) error { + list, err := a.svc.ListZones(r.Context()) + if err != nil { + return err + } + return writeJSON(w, http.StatusOK, listResponse[service.ZoneDTO]{Items: list}) +} + +func (a *API) listDerivedZones(w http.ResponseWriter, r *http.Request, _ *reqCtx) error { + list, err := a.svc.DerivedZones(r.Context()) + if err != nil { + return err + } + return writeJSON(w, http.StatusOK, listResponse[service.DerivedZoneDTO]{Items: list}) +} + +func (a *API) createZone(w http.ResponseWriter, r *http.Request, rc *reqCtx) error { + var req zoneRequest + if err := decode(w, r, &req); err != nil { + return err + } + z, err := a.svc.CreateZone(r.Context(), rc.Actor, req.input()) + if err != nil { + return err + } + return writeJSON(w, http.StatusCreated, z) +} + +func (a *API) getZone(w http.ResponseWriter, r *http.Request, _ *reqCtx) error { + z, err := a.svc.GetZone(r.Context(), r.PathValue("id")) + if err != nil { + return err + } + return writeJSON(w, http.StatusOK, z) +} + +func (a *API) updateZone(w http.ResponseWriter, r *http.Request, rc *reqCtx) error { + var req zoneRequest + if err := decode(w, r, &req); err != nil { + return err + } + z, err := a.svc.UpdateZone(r.Context(), rc.Actor, r.PathValue("id"), req.input()) + if err != nil { + return err + } + return writeJSON(w, http.StatusOK, z) +} + +func (a *API) deleteZone(w http.ResponseWriter, r *http.Request, rc *reqCtx) error { + force := r.URL.Query().Get("force") == "true" + if err := a.svc.DeleteZone(r.Context(), rc.Actor, r.PathValue("id"), force); err != nil { + return err + } + return noContent(w) +} + +type recordRequest struct { + Name string `json:"name"` + Type string `json:"type"` + TTL *int `json:"ttl"` + Address string `json:"address"` + Target string `json:"target"` + Preference int `json:"preference"` + Priority int `json:"priority"` + Weight int `json:"weight"` + Port int `json:"port"` + Text []string `json:"text"` + Description string `json:"description"` + Version int `json:"version"` +} + +func (req *recordRequest) input() service.RecordInput { return service.RecordInput(*req) } + +func (a *API) listRecords(w http.ResponseWriter, r *http.Request, _ *reqCtx) error { + list, err := a.svc.ListRecords(r.Context(), r.PathValue("id")) + if err != nil { + return err + } + return writeJSON(w, http.StatusOK, listResponse[service.RecordDTO]{Items: list}) +} + +func (a *API) createRecord(w http.ResponseWriter, r *http.Request, rc *reqCtx) error { + var req recordRequest + if err := decode(w, r, &req); err != nil { + return err + } + rec, err := a.svc.CreateRecord(r.Context(), rc.Actor, r.PathValue("id"), req.input()) + if err != nil { + return err + } + return writeJSON(w, http.StatusCreated, rec) +} + +func (a *API) updateRecord(w http.ResponseWriter, r *http.Request, rc *reqCtx) error { + var req recordRequest + if err := decode(w, r, &req); err != nil { + return err + } + rec, err := a.svc.UpdateRecord(r.Context(), rc.Actor, r.PathValue("id"), r.PathValue("recordId"), req.input()) + if err != nil { + return err + } + return writeJSON(w, http.StatusOK, rec) +} + +func (a *API) deleteRecord(w http.ResponseWriter, r *http.Request, rc *reqCtx) error { + if err := a.svc.DeleteRecord(r.Context(), rc.Actor, r.PathValue("id"), r.PathValue("recordId")); err != nil { + return err + } + return noContent(w) +} diff --git a/server/internal/httpapi/routes.go b/server/internal/httpapi/routes.go index 7d22f0a..4574218 100644 --- a/server/internal/httpapi/routes.go +++ b/server/internal/httpapi/routes.go @@ -104,6 +104,33 @@ func (a *API) Routes() []Route { perm(put, "/api/v1/addresses/{id}", auth.PermIPAMWrite, true, a.updateAddress), perm(del, "/api/v1/addresses/{id}", auth.PermIPAMWrite, true, a.deleteAddress), + // Clusters and TSIG keys (ADR 0023): read with ipam.read (never + // secrets), changed by administrators. Reading a key's secret + // changes nothing and stays available in read-only mode. + perm(get, "/api/v1/clusters", auth.PermIPAMRead, false, a.listClusters), + perm(post, "/api/v1/clusters", auth.PermSettingsAdm, true, a.createCluster), + perm(get, "/api/v1/clusters/{id}", auth.PermIPAMRead, false, a.getCluster), + perm(put, "/api/v1/clusters/{id}", auth.PermSettingsAdm, true, a.updateCluster), + perm(del, "/api/v1/clusters/{id}", auth.PermSettingsAdm, true, a.deleteCluster), + perm(get, "/api/v1/tsig-keys", auth.PermIPAMRead, false, a.listTSIGKeys), + perm(post, "/api/v1/tsig-keys", auth.PermSettingsAdm, true, a.createTSIGKey), + perm(put, "/api/v1/tsig-keys/{id}", auth.PermSettingsAdm, true, a.updateTSIGKey), + perm(del, "/api/v1/tsig-keys/{id}", auth.PermSettingsAdm, true, a.deleteTSIGKey), + perm(get, "/api/v1/tsig-keys/{id}/secret", auth.PermSettingsAdm, false, a.getTSIGKeySecret), + perm(post, "/api/v1/tsig-keys/{id}/rotate", auth.PermSettingsAdm, true, a.rotateTSIGKey), + + // Zones and records (ADR 0023). + perm(get, "/api/v1/zones", auth.PermIPAMRead, false, a.listZones), + perm(post, "/api/v1/zones", auth.PermIPAMWrite, true, a.createZone), + perm(get, "/api/v1/zones/derived", auth.PermIPAMRead, false, a.listDerivedZones), + perm(get, "/api/v1/zones/{id}", auth.PermIPAMRead, false, a.getZone), + perm(put, "/api/v1/zones/{id}", auth.PermIPAMWrite, true, a.updateZone), + perm(del, "/api/v1/zones/{id}", auth.PermIPAMWrite, true, a.deleteZone), + perm(get, "/api/v1/zones/{id}/records", auth.PermIPAMRead, false, a.listRecords), + perm(post, "/api/v1/zones/{id}/records", auth.PermIPAMWrite, true, a.createRecord), + perm(put, "/api/v1/zones/{id}/records/{recordId}", auth.PermIPAMWrite, true, a.updateRecord), + perm(del, "/api/v1/zones/{id}/records/{recordId}", auth.PermIPAMWrite, true, a.deleteRecord), + // Saved filters are personal and allowed in read-only mode. perm(get, "/api/v1/saved-filters", auth.PermIPAMRead, false, a.listSavedFilters), perm(post, "/api/v1/saved-filters", auth.PermIPAMRead, false, a.createSavedFilter), diff --git a/server/internal/httpapi/zones_test.go b/server/internal/httpapi/zones_test.go new file mode 100644 index 0000000..4cc9f11 --- /dev/null +++ b/server/internal/httpapi/zones_test.go @@ -0,0 +1,647 @@ +// SPDX-License-Identifier: LicenseRef-DDI-Proprietary + +package httpapi + +import ( + "context" + "encoding/base64" + "encoding/json" + "strings" + "testing" + "time" + + "git.beugsiud.de/beug.k/DDI/server/internal/events" + "git.beugsiud.de/beug.k/DDI/server/internal/service" +) + +type ( + clusterList = listResponse[service.ClusterDTO] + keyList = listResponse[service.TSIGKeyDTO] + zoneList = listResponse[service.ZoneDTO] + recordList = listResponse[service.RecordDTO] +) + +// problemCodes returns the top-level code and the codes of the problems. +func problemCodes(t *testing.T, r *response) []string { + t.Helper() + p := r.problem(t) + out := []string{p.Code} + for _, pr := range p.Problems { + out = append(out, pr.Code+"@"+pr.Field) + } + return out +} + +func wantCodes(t *testing.T, r *response, want ...string) { + t.Helper() + got := problemCodes(t, r) + if strings.Join(got, " ") != strings.Join(want, " ") { + t.Errorf("problem codes %v, want %v (%s)", got, want, r.Body) + } +} + +// auditJSON returns the whole audit log as JSON lines. +func (c *client) auditJSON() string { + c.e.t.Helper() + return string(c.mustStatus(200, "GET", "/api/v1/audit/export?format=json", nil).Body) +} + +func TestTSIGKeys(t *testing.T) { + e := newEnv(t) + admin := e.adminClient() + operator := e.userClient(admin, "otto", "operator") + viewer := e.userClient(admin, "vera", "viewer") + + var k service.TSIGKeyDTO + admin.mustStatus(201, "POST", "/api/v1/tsig-keys", map[string]any{"name": "DHCP-Update.Example.Test.", "description": "Kea"}).json(t, &k) + if k.Name != "dhcp-update.example.test" || k.Algorithm != "hmac-sha256" || k.ClusterID != nil || k.Version != 1 { + t.Fatalf("key %+v", k) + } + operator.mustStatus(403, "POST", "/api/v1/tsig-keys", map[string]any{"name": "x"}) + wantCodes(t, admin.mustStatus(409, "POST", "/api/v1/tsig-keys", map[string]any{"name": "dhcp-update.example.test"}), + "key_exists", "key_exists@name") + wantCodes(t, admin.mustStatus(422, "POST", "/api/v1/tsig-keys", map[string]any{"name": "bad name"}), + "validation_failed", "invalid_key_name@name") + + var list keyList + viewer.mustStatus(200, "GET", "/api/v1/tsig-keys", nil).json(t, &list) + if len(list.Items) != 1 { + t.Fatalf("keys %+v", list.Items) + } + viewer.mustStatus(403, "GET", "/api/v1/tsig-keys/"+k.ID+"/secret", nil) + operator.mustStatus(403, "GET", "/api/v1/tsig-keys/"+k.ID+"/secret", nil) + var sec service.TSIGKeySecretDTO + admin.mustStatus(200, "GET", "/api/v1/tsig-keys/"+k.ID+"/secret", nil).json(t, &sec) + raw, err := base64.StdEncoding.DecodeString(sec.Secret) + if err != nil || len(raw) != 32 || sec.Name != k.Name || sec.Algorithm != "hmac-sha256" { + t.Fatalf("secret %+v %v", sec, err) + } + var k512 service.TSIGKeyDTO + admin.mustStatus(201, "POST", "/api/v1/tsig-keys", map[string]any{"name": "big", "algorithm": "hmac-sha512"}).json(t, &k512) + var sec512 service.TSIGKeySecretDTO + admin.mustStatus(200, "GET", "/api/v1/tsig-keys/"+k512.ID+"/secret", nil).json(t, &sec512) + if raw, _ := base64.StdEncoding.DecodeString(sec512.Secret); len(raw) != 64 { + t.Errorf("hmac-sha512 secret has %d bytes", len(raw)) + } + + // Rotation replaces the secret. + e.advance(time.Minute) + var rotated service.TSIGKeyDTO + operator.mustStatus(403, "POST", "/api/v1/tsig-keys/"+k.ID+"/rotate", nil) + admin.mustStatus(200, "POST", "/api/v1/tsig-keys/"+k.ID+"/rotate", nil).json(t, &rotated) + if !rotated.RotatedAt.After(k.RotatedAt) || rotated.Version != 2 { + t.Errorf("rotated %+v", rotated) + } + var sec2 service.TSIGKeySecretDTO + admin.mustStatus(200, "GET", "/api/v1/tsig-keys/"+k.ID+"/secret", nil).json(t, &sec2) + if sec2.Secret == sec.Secret { + t.Error("rotation kept the secret") + } + + // Description only; stale versions fail. + var upd service.TSIGKeyDTO + admin.mustStatus(200, "PUT", "/api/v1/tsig-keys/"+k.ID, map[string]any{"name": "ignored", "description": "Kea DDNS", + "version": 2}).json(t, &upd) + if upd.Name != k.Name || upd.Description != "Kea DDNS" || upd.Version != 3 { + t.Errorf("updated %+v", upd) + } + admin.mustStatus(409, "PUT", "/api/v1/tsig-keys/"+k.ID, map[string]any{"description": "x", "version": 2}) + admin.mustStatus(404, "PUT", "/api/v1/tsig-keys/nonsense", map[string]any{"version": 1}) + + // Keys used by zones cannot be deleted. + var z service.ZoneDTO + admin.mustStatus(201, "POST", "/api/v1/zones", map[string]any{"name": "dyn.example.test", "mode": "dynamic", + "update_key_ids": []string{k.ID}}).json(t, &z) + wantCodes(t, admin.mustStatus(409, "DELETE", "/api/v1/tsig-keys/"+k.ID, nil), "key_in_use") + list = keyList{} + viewer.mustStatus(200, "GET", "/api/v1/tsig-keys", nil).json(t, &list) + if list.Items[1].Name != k.Name || list.Items[1].ZoneCount != 1 { + t.Errorf("zone count %+v", list.Items) + } + admin.mustStatus(204, "DELETE", "/api/v1/zones/"+z.ID, nil) + operator.mustStatus(403, "DELETE", "/api/v1/tsig-keys/"+k.ID, nil) + admin.mustStatus(204, "DELETE", "/api/v1/tsig-keys/"+k.ID, nil) + admin.mustStatus(404, "DELETE", "/api/v1/tsig-keys/"+k.ID, nil) + + // Every secret access is audited, no secret ever reaches the log. + audit := admin.auditJSON() + if strings.Count(audit, `"tsig_key.read_secret"`) != 3 { + t.Errorf("secret reads not audited:\n%s", audit) + } + for _, s := range []string{sec.Secret, sec2.Secret, sec512.Secret} { + if strings.Contains(audit, s) { + t.Fatal("secret in the audit log") + } + } + // Key changes count as pending changes, secret reads do not. + var p service.PendingChanges + viewer.mustStatus(200, "GET", "/api/v1/config/pending", nil).json(t, &p) + for _, c := range p.Changes { + if c.Action == "tsig_key.read_secret" { + t.Errorf("secret read counted as change") + } + } + if len(p.Changes) == 0 || p.Changes[0].Action != "tsig_key.delete" { + t.Errorf("changes %+v", p.Changes) + } +} + +func TestClusters(t *testing.T) { + e := newEnv(t) + admin := e.adminClient() + operator := e.userClient(admin, "otto", "operator") + viewer := e.userClient(admin, "vera", "viewer") + dns1 := e.enroll(admin.createToken("dns").Secret, "dns") + dns2 := e.enroll(admin.createToken("dns").Secret, "dns") + dns3 := e.enroll(admin.createToken("dns").Secret, "dns") + dns4 := e.enroll(admin.createToken("dns").Secret, "dns") + dhcp1 := e.enroll(admin.createToken("dhcp").Secret, "dhcp") + dhcp2 := e.enroll(admin.createToken("dhcp").Secret, "dhcp") + dhcp3 := e.enroll(admin.createToken("dhcp").Secret, "dhcp") + member := func(id, role, addr string) map[string]any { + return map[string]any{"agent_id": id, "role": role, "address": addr} + } + + operator.mustStatus(403, "POST", "/api/v1/clusters", map[string]any{"name": "x", "role": "dns"}) + var dnsC service.ClusterDTO + admin.mustStatus(201, "POST", "/api/v1/clusters", map[string]any{"name": "Office DNS", "role": "dns", + "members": []any{member(dns1, "primary", "192.0.2.53"), member(dns2, "secondary", "192.0.2.54")}, + "primary_ns": "NS1.Office.Test", "name_servers": []map[string]any{{"name": "ns1.office.test", "addresses": []string{"192.0.2.53"}}}, + "ha_mode": "load-balancing", "description": "Primary and secondary", + }).json(t, &dnsC) + if dnsC.Role != "dns" || len(dnsC.Members) != 2 || dnsC.Members[0].AgentID != dns1 || dnsC.Members[0].Role != "primary" || + dnsC.Members[0].Hostname != "host-dns" || dnsC.PrimaryNS != "ns1.office.test" || dnsC.TransferKey == nil || + dnsC.HAMode != "hot-standby" || dnsC.HAPort != 8001 || len(dnsC.NameServers) != 1 { + t.Fatalf("dns cluster %+v", dnsC) + } + // The transfer key is a TSIG key owned by the cluster. + var keys keyList + viewer.mustStatus(200, "GET", "/api/v1/tsig-keys", nil).json(t, &keys) + if len(keys.Items) != 1 || keys.Items[0].ID != dnsC.TransferKey.ID || keys.Items[0].ClusterID == nil || + *keys.Items[0].ClusterID != dnsC.ID { + t.Fatalf("transfer key %+v", keys.Items) + } + wantCodes(t, admin.mustStatus(409, "DELETE", "/api/v1/tsig-keys/"+dnsC.TransferKey.ID, nil), "key_owned_by_cluster") + + var dhcpC service.ClusterDTO + admin.mustStatus(201, "POST", "/api/v1/clusters", map[string]any{"name": "Office DHCP", "role": "dhcp", + "members": []any{member(dhcp1, "primary", "192.0.2.67"), member(dhcp2, "secondary", "192.0.2.68")}, + "ha_port": 8100}).json(t, &dhcpC) + if dhcpC.HAMode != "hot-standby" || dhcpC.HAPort != 8100 || dhcpC.TransferKey != nil || len(dhcpC.Members) != 2 { + t.Fatalf("dhcp cluster %+v", dhcpC) + } + + // Membership errors. + cases := []struct { + name string + members []any + role string + want []string + }{ + {"role mismatch", []any{member(dhcp3, "primary", "192.0.2.1")}, "dns", []string{"validation_failed", "cluster_role_mismatch@members[0].agent_id"}}, + {"other cluster", []any{member(dns1, "primary", "192.0.2.1")}, "dns", []string{"validation_failed", "agent_in_other_cluster@members[0].agent_id"}}, + {"second primary", []any{member(dns3, "primary", "192.0.2.1"), member(dns4, "primary", "192.0.2.2")}, "dns", + []string{"validation_failed", "cluster_primary_exists@members[1].role"}}, + {"unknown agent", []any{member(dhcp3, "secondary", "192.0.2.1"), member("00000000-0000-4000-8000-000000000001", "secondary", "192.0.2.2")}, "dhcp", + []string{"validation_failed", "unknown_agent@members[1].agent_id"}}, + {"invalid ip", []any{member(dns3, "primary", "not-an-ip")}, "dns", []string{"validation_failed", "invalid_ip@members[0].address"}}, + {"duplicate", []any{member(dns3, "primary", "192.0.2.1"), member(dns3, "secondary", "192.0.2.2")}, "dns", + []string{"validation_failed", "duplicate_member@members[1].agent_id"}}, + } + for _, c := range cases { + r := admin.mustStatus(422, "POST", "/api/v1/clusters", map[string]any{"name": "Bad " + c.name, "role": c.role, "members": c.members}) + wantCodes(t, r, c.want...) + } + wantCodes(t, admin.mustStatus(422, "POST", "/api/v1/clusters", map[string]any{"name": "DHCP 2", "role": "dhcp", + "members": []any{member(dhcp3, "primary", "192.0.2.1"), member(dhcp1, "primary", "192.0.2.2")}}), + "validation_failed", "agent_in_other_cluster@members[1].agent_id") + wantCodes(t, admin.mustStatus(409, "POST", "/api/v1/clusters", map[string]any{"name": "office dns", "role": "dns"}), + "cluster_exists", "cluster_exists@name") + + // A DHCP pair has at most one secondary. + var dhcpUpd service.ClusterDTO + r := admin.mustStatus(422, "PUT", "/api/v1/clusters/"+dhcpC.ID, map[string]any{"name": "Office DHCP", "version": dhcpC.Version, + "members": []any{member(dhcp1, "primary", "192.0.2.67"), member(dhcp2, "secondary", "192.0.2.68"), + member(dhcp3, "secondary", "192.0.2.69")}}) + wantCodes(t, r, "validation_failed", "cluster_member_limit@members[2].role") + + // Agents show their cluster. + if ag := viewer.agent(dns2); ag.Cluster == nil || ag.Cluster.ID != dnsC.ID || ag.Cluster.Role != "secondary" || + ag.Cluster.Address.String() != "192.0.2.54" || ag.Cluster.Name != "Office DNS" { + t.Errorf("agent cluster %+v", ag.Cluster) + } + if ag := viewer.agent(dns3); ag.Cluster != nil { + t.Errorf("agent without cluster %+v", ag.Cluster) + } + + // Update: the role is immutable, members are replaced. + viewer.mustStatus(403, "PUT", "/api/v1/clusters/"+dnsC.ID, map[string]any{"name": "x", "version": 1}) + admin.mustStatus(200, "PUT", "/api/v1/clusters/"+dhcpC.ID, map[string]any{"name": "Office DHCP", "role": "dns", + "version": dhcpC.Version, "members": []any{member(dhcp1, "primary", "192.0.2.67")}}).json(t, &dhcpUpd) + if dhcpUpd.Role != "dhcp" || len(dhcpUpd.Members) != 1 || dhcpUpd.Version != 2 || dhcpUpd.HAPort != 8001 { + t.Errorf("updated %+v", dhcpUpd) + } + admin.mustStatus(409, "PUT", "/api/v1/clusters/"+dhcpC.ID, map[string]any{"name": "x", "version": 1}) + if ag := viewer.agent(dhcp2); ag.Cluster != nil { + t.Errorf("removed member still in cluster: %+v", ag.Cluster) + } + + var list clusterList + viewer.mustStatus(200, "GET", "/api/v1/clusters", nil).json(t, &list) + if len(list.Items) != 2 || list.Items[0].Role != "dns" { + t.Errorf("clusters %+v", list.Items) + } + list = clusterList{} + viewer.mustStatus(200, "GET", "/api/v1/clusters?role=dhcp", nil).json(t, &list) + if len(list.Items) != 1 || list.Items[0].ID != dhcpC.ID { + t.Errorf("dhcp clusters %+v", list.Items) + } + + // Networks accept DHCP clusters only. + netReq := map[string]any{"cidr": "192.0.2.0/24", "name": "Office", "dhcp_cluster_id": dnsC.ID} + wantCodes(t, admin.mustStatus(422, "POST", "/api/v1/networks", netReq), "validation_failed", "cluster_role_mismatch@dhcp_cluster_id") + netReq["dhcp_cluster_id"] = "00000000-0000-4000-8000-000000000001" + wantCodes(t, admin.mustStatus(422, "POST", "/api/v1/networks", netReq), "validation_failed", "not_found@dhcp_cluster_id") + netReq["dhcp_cluster_id"] = dhcpC.ID + var netw service.NetworkDTO + admin.mustStatus(201, "POST", "/api/v1/networks", netReq).json(t, &netw) + if netw.DHCPClusterID == nil || *netw.DHCPClusterID != dhcpC.ID { + t.Errorf("network cluster %v", netw.DHCPClusterID) + } + + // Generator settings: default clusters must exist and match the role. + var g service.GeneratorSettings + admin.mustStatus(200, "GET", "/api/v1/settings/generator", nil).json(t, &g) + if g.DHCP.HAPasswordFile != "/etc/kea/ddi-suite-ha-password" || g.DNS.DefaultClusterID != nil { + t.Errorf("settings %+v", g) + } + g.DNS.DefaultClusterID = &dhcpC.ID + g.DHCP.HAPasswordFile = "relative/path" + r = admin.mustStatus(422, "PUT", "/api/v1/settings/generator", g) + wantCodes(t, r, "validation_failed", "invalid_path@dhcp.ha_password_file") + g.DHCP.HAPasswordFile = "/etc/kea/ha-password" + wantCodes(t, admin.mustStatus(422, "PUT", "/api/v1/settings/generator", g), "validation_failed", + "cluster_role_mismatch@dns.default_cluster_id") + g.DNS.DefaultClusterID, g.DHCP.DefaultClusterID = &dnsC.ID, &dhcpC.ID + g = service.GeneratorSettings{DNS: g.DNS, DHCP: g.DHCP} + admin.mustStatus(200, "PUT", "/api/v1/settings/generator", g).json(t, &g) + if g.DNS.DefaultClusterID == nil || *g.DNS.DefaultClusterID != dnsC.ID || g.DHCP.HAPasswordFile != "/etc/kea/ha-password" { + t.Errorf("saved settings %+v", g) + } + + // Clusters in use cannot be deleted. + r = admin.mustStatus(409, "DELETE", "/api/v1/clusters/"+dhcpC.ID, nil) + if p := r.problem(t); p.Code != "cluster_in_use" || p.Params["networks"] != "1" || p.Params["default"] != "true" { + t.Errorf("in use %+v", p) + } + g.DNS.DefaultClusterID, g.DHCP.DefaultClusterID = nil, nil + admin.mustStatus(200, "PUT", "/api/v1/settings/generator", g) + var z service.ZoneDTO + admin.mustStatus(201, "POST", "/api/v1/zones", map[string]any{"name": "office.test", "cluster_id": dnsC.ID}).json(t, &z) + r = admin.mustStatus(409, "DELETE", "/api/v1/clusters/"+dnsC.ID, nil) + if p := r.problem(t); p.Code != "cluster_in_use" || p.Params["zones"] != "1" { + t.Errorf("in use %+v", p) + } + admin.mustStatus(204, "DELETE", "/api/v1/zones/"+z.ID, nil) + viewer.mustStatus(403, "DELETE", "/api/v1/clusters/"+dnsC.ID, nil) + admin.mustStatus(204, "DELETE", "/api/v1/clusters/"+dnsC.ID, nil) + admin.mustStatus(404, "GET", "/api/v1/clusters/"+dnsC.ID, nil) + keys = keyList{} + viewer.mustStatus(200, "GET", "/api/v1/tsig-keys", nil).json(t, &keys) + if len(keys.Items) != 0 { + t.Errorf("transfer key survived its cluster: %+v", keys.Items) + } + if ag := viewer.agent(dns1); ag.Cluster != nil { + t.Errorf("member of deleted cluster %+v", ag.Cluster) + } + + // Audit: every change, never HA credentials; all count as changes. + audit := admin.auditJSON() + for _, want := range []string{`"cluster.create"`, `"cluster.update"`, `"cluster.delete"`, `"tsig_key.create"`} { + if !strings.Contains(audit, want) { + t.Errorf("audit misses %s", want) + } + } + for _, line := range strings.Split(strings.TrimSpace(audit), "\n") { + var entry map[string]any + if err := json.Unmarshal([]byte(line), &entry); err != nil { + t.Fatal(err) + } + if strings.Contains(line, "ha_user") || strings.Contains(line, "ha_password\"") || strings.Contains(line, "secret\"") { + t.Errorf("credentials in the audit log: %s", line) + } + } + var p service.PendingChanges + viewer.mustStatus(200, "GET", "/api/v1/config/pending", nil).json(t, &p) + if len(p.Changes) == 0 || p.Changes[0].Action != "cluster.delete" { + t.Errorf("pending %+v", p.Changes) + } +} + +func TestZonesAndRecords(t *testing.T) { + e := newEnv(t) + admin := e.adminClient() + operator := e.userClient(admin, "otto", "operator") + viewer := e.userClient(admin, "vera", "viewer") + + var z service.ZoneDTO + viewer.mustStatus(403, "POST", "/api/v1/zones", map[string]any{"name": "example.test"}) + operator.mustStatus(201, "POST", "/api/v1/zones", map[string]any{"name": "Example.TEST.", "default_ttl": 600, + "description": "Main", "allow_update": []string{"192.0.2.0/24"}}).json(t, &z) + if z.Name != "example.test" || z.Kind != "forward" || z.Mode != "managed" || z.DefaultTTL == nil || *z.DefaultTTL != 600 || + len(z.AllowUpdate) != 0 || z.ClusterID != nil { + t.Fatalf("zone %+v", z) + } + var rz service.ZoneDTO + operator.mustStatus(201, "POST", "/api/v1/zones", map[string]any{"name": "2.0.192.in-addr.arpa"}).json(t, &rz) + if rz.Kind != "reverse" { + t.Errorf("reverse zone %+v", rz) + } + wantCodes(t, operator.mustStatus(409, "POST", "/api/v1/zones", map[string]any{"name": "example.test"}), "zone_exists", "zone_exists@name") + wantCodes(t, operator.mustStatus(422, "POST", "/api/v1/zones", map[string]any{"name": "example.test", "kind": "reverse"}), + "validation_failed", "zone_kind_mismatch@kind") + wantCodes(t, operator.mustStatus(422, "POST", "/api/v1/zones", map[string]any{"name": "1.0.10.in-addr.arpa", "kind": "forward"}), + "validation_failed", "zone_kind_mismatch@kind") + wantCodes(t, operator.mustStatus(422, "POST", "/api/v1/zones", map[string]any{"name": "300.10.in-addr.arpa"}), + "validation_failed", "invalid_domain@name") + wantCodes(t, operator.mustStatus(422, "POST", "/api/v1/zones", map[string]any{"name": "_msdcs.example.test"}), + "validation_failed", "invalid_domain@name") + wantCodes(t, operator.mustStatus(422, "POST", "/api/v1/zones", map[string]any{"name": "ad.example.test", "mode": "dynamic"}), + "validation_failed", "update_source_required@allow_update") + wantCodes(t, operator.mustStatus(422, "POST", "/api/v1/zones", map[string]any{"name": "ad.example.test", "mode": "dynamic", + "update_key_ids": []string{"00000000-0000-4000-8000-000000000001"}}), "validation_failed", "not_found@update_key_ids") + wantCodes(t, operator.mustStatus(422, "POST", "/api/v1/zones", map[string]any{"name": "ad.example.test", + "cluster_id": "00000000-0000-4000-8000-000000000001"}), "validation_failed", "not_found@cluster_id") + var dhcpC service.ClusterDTO + admin.mustStatus(201, "POST", "/api/v1/clusters", map[string]any{"name": "DHCP", "role": "dhcp"}).json(t, &dhcpC) + wantCodes(t, operator.mustStatus(422, "POST", "/api/v1/zones", map[string]any{"name": "ad.example.test", "cluster_id": dhcpC.ID}), + "validation_failed", "cluster_role_mismatch@cluster_id") + + var ad service.ZoneDTO + operator.mustStatus(201, "POST", "/api/v1/zones", map[string]any{"name": "ad.example.test", "mode": "dynamic", + "allow_update": []string{"192.0.2.10/32", "192.0.2.10/32"}}).json(t, &ad) + if len(ad.AllowUpdate) != 1 || ad.AllowUpdate[0].String() != "192.0.2.10/32" { + t.Errorf("dynamic zone %+v", ad) + } + var adUpd service.ZoneDTO + operator.mustStatus(200, "PUT", "/api/v1/zones/"+ad.ID, map[string]any{"name": "ignored", "mode": "managed", + "allow_update": []string{"192.0.2.10/32"}, "version": 1}).json(t, &adUpd) + if adUpd.Name != "ad.example.test" || adUpd.Mode != "managed" || len(adUpd.AllowUpdate) != 0 || adUpd.Version != 2 { + t.Errorf("updated zone %+v", adUpd) + } + operator.mustStatus(409, "PUT", "/api/v1/zones/"+ad.ID, map[string]any{"version": 1}) + + var zones zoneList + viewer.mustStatus(200, "GET", "/api/v1/zones", nil).json(t, &zones) + if len(zones.Items) != 3 || zones.Items[0].Name != "2.0.192.in-addr.arpa" { + t.Errorf("zones %+v", zones.Items) + } + viewer.mustStatus(200, "GET", "/api/v1/zones/derived", nil) + viewer.mustStatus(404, "GET", "/api/v1/zones/nonsense", nil) + + // Records. + base := "/api/v1/zones/" + z.ID + "/records" + recs := []map[string]any{ + {"name": "www", "type": "A", "address": "192.0.2.10", "ttl": 300}, + {"name": "www", "type": "AAAA", "address": "2001:db8::10"}, + {"name": "@", "type": "MX", "target": "Mail.Example.Test.", "preference": 10}, + {"name": "_ldap._tcp.ad", "type": "SRV", "target": "dc1.example.test", "priority": 0, "weight": 100, "port": 389}, + {"name": "@", "type": "TXT", "text": []string{"v=spf1 -all", strings.Repeat("x", 300)}}, + {"name": "alias", "type": "CNAME", "target": "_acme.example.test"}, + {"name": "sub", "type": "NS", "target": "ns.sub.example.test"}, + } + viewer.mustStatus(403, "POST", base, recs[0]) + var created []service.RecordDTO + for _, rec := range recs { + var d service.RecordDTO + operator.mustStatus(201, "POST", base, rec).json(t, &d) + created = append(created, d) + } + if created[2].Target != "mail.example.test" || created[0].TTL == nil || *created[0].TTL != 300 || created[0].Address != "192.0.2.10" { + t.Errorf("records %+v", created) + } + wantCodes(t, operator.mustStatus(409, "POST", base, recs[0]), "record_exists", "record_exists@") + badCases := []struct { + rec map[string]any + want []string + }{ + {map[string]any{"name": "x", "type": "AAAA", "address": "192.0.2.1"}, []string{"validation_failed", "ip_family@address"}}, + {map[string]any{"name": "x", "type": "A", "address": "nonsense"}, []string{"validation_failed", "invalid_ip@address"}}, + {map[string]any{"name": "@", "type": "CNAME", "target": "x.example.test"}, []string{"validation_failed", "cname_at_apex@name"}}, + {map[string]any{"name": "www.example.test", "type": "A", "address": "192.0.2.1"}, []string{"validation_failed", "invalid_label@name"}}, + {map[string]any{"name": "a b", "type": "A", "address": "192.0.2.1"}, []string{"validation_failed", "invalid_label@name"}}, + {map[string]any{"name": "x", "type": "MX", "target": "mail.example.test", "port": 25}, []string{"validation_failed", "field_not_allowed@port"}}, + {map[string]any{"name": "x", "type": "SRV", "target": "_bad.example.test"}, []string{"validation_failed", "invalid_domain@target"}}, + {map[string]any{"name": "x", "type": "TXT"}, []string{"validation_failed", "invalid_txt@text"}}, + {map[string]any{"name": "x", "type": "TXT", "text": []string{strings.Repeat("y", 70000)}}, []string{"validation_failed", "invalid_txt@text"}}, + {map[string]any{"name": "x", "type": "A", "address": "192.0.2.1", "ttl": 0}, []string{"validation_failed", "invalid_ttl@ttl"}}, + } + for _, c := range badCases { + wantCodes(t, operator.mustStatus(422, "POST", base, c.rec, skipRequestValidation, "1"), c.want...) + } + + var list recordList + viewer.mustStatus(200, "GET", base, nil).json(t, &list) + if len(list.Items) != len(recs) || list.Items[0].Name != "@" { + t.Fatalf("records %+v", list.Items) + } + // Update and delete. + var upd service.RecordDTO + operator.mustStatus(200, "PUT", base+"/"+created[0].ID, map[string]any{"name": "web", "type": "A", "address": "192.0.2.11", + "version": 1}).json(t, &upd) + if upd.Name != "web" || upd.TTL != nil || upd.Version != 2 { + t.Errorf("updated record %+v", upd) + } + operator.mustStatus(409, "PUT", base+"/"+created[0].ID, map[string]any{"name": "web", "type": "A", "address": "192.0.2.11", "version": 1}) + operator.mustStatus(404, "PUT", "/api/v1/zones/"+rz.ID+"/records/"+created[0].ID, map[string]any{"name": "web", + "type": "A", "address": "192.0.2.11", "version": 2}) + operator.mustStatus(204, "DELETE", base+"/"+created[1].ID, nil) + operator.mustStatus(404, "DELETE", base+"/"+created[1].ID, nil) + + // A zone with records needs force. + r := operator.mustStatus(409, "DELETE", "/api/v1/zones/"+z.ID, nil) + if p := r.problem(t); p.Code != "zone_not_empty" || p.Params["count"] != "6" { + t.Errorf("not empty %+v", p) + } + operator.mustStatus(204, "DELETE", "/api/v1/zones/"+z.ID+"?force=true", nil) + operator.mustStatus(404, "GET", base, nil) + + // Every change is audited and pending. + var p service.PendingChanges + viewer.mustStatus(200, "GET", "/api/v1/config/pending", nil).json(t, &p) + actions := map[string]int{} + for _, c := range p.Changes { + actions[c.Action]++ + } + if actions["zone.create"] != 3 || actions["record.create"] != len(recs) || actions["record.update"] != 1 || + actions["record.delete"] != 1 || actions["zone.delete"] != 1 || actions["zone.update"] != 1 { + t.Errorf("pending changes %v", actions) + } +} + +func TestLeaseAdoption(t *testing.T) { + e := newEnv(t) + admin := e.adminClient() + viewer := e.userClient(admin, "vera", "viewer") + ctx := context.Background() + var netw service.NetworkDTO + admin.mustStatus(201, "POST", "/api/v1/networks", map[string]any{"cidr": "192.0.2.0/24", "name": "Office", + "domain": "office.test", "dhcp_enabled": true, "pools": []map[string]string{{"start": "192.0.2.100", "end": "192.0.2.199"}}, + }).json(t, &netw) + admin.mustStatus(201, "POST", "/api/v1/addresses", map[string]any{"ip": "192.0.2.120", "type": "manual_dhcp", + "mac": "00:11:22:33:44:99", "hostname": "printer"}) + ids, err := e.st.Q().SubnetIDs(ctx) + if err != nil { + t.Fatal(err) + } + subnet := ids[netw.ID] + agentID := e.enroll(admin.createToken("dhcp").Secret, "dhcp") + sub := e.bus.Subscribe(16) + defer sub.Close() + + exp := e.now.Add(time.Hour).UTC().Truncate(time.Second) + if _, err := e.svc.StoreLeaseBatch(ctx, agentID, 1, true, []service.LeaseInput{ + {IP: "192.0.2.100", MAC: "00-11-22-33-44-55", Hostname: "Laptop", State: "active", ExpiresAt: &exp, SubnetID: subnet}, + {IP: "192.0.2.101", MAC: "00:11:22:33:44:56", Hostname: "phone.example.com", State: "active", SubnetID: subnet}, + {IP: "192.0.2.102", MAC: "00:11:22:33:44:57", Hostname: "gone", State: "expired", SubnetID: subnet}, + {IP: "192.0.2.50", MAC: "00:11:22:33:44:58", Hostname: "outside", State: "active", SubnetID: subnet}, + {IP: "192.0.2.120", MAC: "00:11:22:33:44:59", Hostname: "other", State: "active", ExpiresAt: &exp, SubnetID: subnet}, + {IP: "198.51.100.7", State: "active", SubnetID: 9999}, + }); err != nil { + t.Fatal(err) + } + select { + case ev := <-sub.C(): + if ev.Type != events.TypeAddresses || !strings.Contains(string(ev.Data), netw.ID) { + t.Errorf("event %s %s", ev.Type, ev.Data) + } + case <-time.After(5 * time.Second): + t.Fatal("no addresses event") + } + + addrs := func() map[string]service.AddressDTO { + var page service.AddressPage + viewer.mustStatus(200, "GET", "/api/v1/addresses?network_id="+netw.ID, nil).json(t, &page) + out := map[string]service.AddressDTO{} + for _, a := range page.Items { + out[a.IP.String()] = a + } + return out + } + got := addrs() + if len(got) != 3 { + t.Fatalf("addresses %+v", got) + } + a := got["192.0.2.100"] + if a.Type != "dynamic" || a.MAC != "00:11:22:33:44:55" || a.Hostname != "laptop" || a.FQDN != "laptop.office.test" || + a.LeaseState != "active" || a.LeaseExpiresAt == nil || !a.LeaseExpiresAt.Equal(exp) || a.FirstSeen == nil || a.LastSeen == nil { + t.Errorf("adopted %+v", a) + } + if b := got["192.0.2.101"]; b.Type != "dynamic" || b.Hostname != "" || b.FQDN != "" { + t.Errorf("adopted without valid label %+v", b) + } + if p := got["192.0.2.120"]; p.Type != "manual_dhcp" || p.MAC != "00:11:22:33:44:99" || p.Hostname != "printer" || + p.LeaseState != "active" || p.LastSeen == nil || p.Version != 1 { + t.Errorf("reservation %+v", p) + } + + // Later reports update dynamic entries; expired leases only change the + // state, entries stay. + e.advance(time.Minute) + if _, err := e.svc.StoreLeaseBatch(ctx, agentID, 2, false, []service.LeaseInput{ + {IP: "192.0.2.100", MAC: "00:11:22:33:44:aa", Hostname: "laptop2", State: "active", ExpiresAt: &exp, SubnetID: subnet}, + {IP: "192.0.2.101", State: "expired", SubnetID: subnet}, + }); err != nil { + t.Fatal(err) + } + got = addrs() + if a := got["192.0.2.100"]; a.MAC != "00:11:22:33:44:aa" || a.Hostname != "laptop2" || !a.LastSeen.After(*got["192.0.2.101"].LastSeen) { + t.Errorf("updated %+v", a) + } + if b := got["192.0.2.101"]; b.LeaseState != "expired" || b.MAC != "00:11:22:33:44:56" { + t.Errorf("expired %+v", b) + } + // A full sync without the leases deletes nothing. + if _, err := e.svc.StoreLeaseBatch(ctx, agentID, 3, true, nil); err != nil { + t.Fatal(err) + } + if got := addrs(); len(got) != 3 { + t.Errorf("entries deleted: %+v", got) + } + + // No audit entries and no pending changes from leases. + audit := admin.auditJSON() + if strings.Count(audit, `"address.create"`) != 1 || strings.Contains(audit, "laptop") { + t.Errorf("lease adoption audited:\n%s", audit) + } + var p service.PendingChanges + viewer.mustStatus(200, "GET", "/api/v1/config/pending", nil).json(t, &p) + if len(p.Changes) != 2 { + t.Errorf("pending changes %+v", p.Changes) + } + // The adopted entry becomes a reservation by changing its type. + var res service.AddressDTO + admin.mustStatus(200, "PUT", "/api/v1/addresses/"+got["192.0.2.100"].ID, map[string]any{"ip": "192.0.2.100", + "type": "manual_dhcp", "mac": "00:11:22:33:44:aa", "hostname": "laptop2", "version": got["192.0.2.100"].Version}).json(t, &res) + if res.Type != "manual_dhcp" || res.LeaseState != "active" { + t.Errorf("reservation %+v", res) + } +} + +// TestRevisionSecretsAndRollback: the generator input of a revision is +// stored without secrets; they are sealed separately and restored for a +// rollback. +func TestRevisionSecretsAndRollback(t *testing.T) { + e := newEnv(t) + admin := e.adminClient() + viewer := e.userClient(admin, "vera", "viewer") + ctx := context.Background() + admin.configureDNS() + admin.officeNetwork() + var k service.TSIGKeyDTO + admin.mustStatus(201, "POST", "/api/v1/tsig-keys", map[string]any{"name": "ddns"}).json(t, &k) + admin.mustStatus(201, "POST", "/api/v1/zones", map[string]any{"name": "ad.office.test", "mode": "dynamic", + "update_key_ids": []string{k.ID}}) + var dhcpC service.ClusterDTO + admin.mustStatus(201, "POST", "/api/v1/clusters", map[string]any{"name": "DHCP", "role": "dhcp"}).json(t, &dhcpC) + var sec service.TSIGKeySecretDTO + admin.mustStatus(200, "GET", "/api/v1/tsig-keys/"+k.ID+"/secret", nil).json(t, &sec) + + rev := admin.newRevision() + if len(rev.Targets) < 2 || rev.Targets[0].Role != "dns" || rev.Targets[0].Target != "" || rev.Targets[0].ClusterID != nil { + t.Errorf("targets %+v", rev.Targets) + } + src, sealed, err := e.st.Q().RevisionSource(ctx, rev.Number) + if err != nil || len(sealed) == 0 { + t.Fatalf("source %v, secrets %d bytes", err, len(sealed)) + } + if strings.Contains(string(src), sec.Secret) || strings.Contains(string(sealed), sec.Secret) { + t.Fatal("secret stored in plain text") + } + var stored struct { + Version int + Keys []map[string]any + Zones []map[string]any + } + if err := json.Unmarshal(src, &stored); err != nil || stored.Version != 2 || len(stored.Keys) != 1 || len(stored.Zones) != 1 { + t.Errorf("stored source %+v %v", stored, err) + } + + // Rollback works after the key was rotated: the old secret comes back + // from the revision. + admin.mustStatus(200, "POST", "/api/v1/tsig-keys/"+k.ID+"/rotate", nil) + var rb service.RevisionDTO + viewer.mustStatus(403, "POST", "/api/v1/revisions/1/rollback", nil) + admin.mustStatus(201, "POST", "/api/v1/revisions/1/rollback", nil).json(t, &rb) + _, sealed2, err := e.st.Q().RevisionSource(ctx, rb.Number) + if err != nil || len(sealed2) == 0 { + t.Fatalf("rollback secrets %v", err) + } + var diff service.RevisionDiff + viewer.mustStatus(200, "GET", "/api/v1/revisions/2/diff", nil).json(t, &diff) + for _, f := range diff.Files { + if f.Target != "" { + t.Errorf("diff target %+v", f) + } + } +} diff --git a/server/internal/service/agentplane.go b/server/internal/service/agentplane.go index aa28dc7..241ead3 100644 --- a/server/internal/service/agentplane.go +++ b/server/internal/service/agentplane.go @@ -356,21 +356,38 @@ func (s *Service) AgentDesiredState(ctx context.Context, agentID string, remote return s.settings(ctx, q, a) } -// RevisionArtifacts returns the files of a revision for one role and the -// revision time (the bundle's created_at). -func (s *Service) RevisionArtifacts(ctx context.Context, number uint64, role string) (time.Time, []bundle.Artifact, error) { +// RevisionArtifacts returns the files of a revision for an agent and the +// revision time (the bundle's created_at): the files of the target with the +// agent's ID if the revision has one, otherwise those of the role's +// default group (ADR 0023). Secret files are decrypted; only the signed +// bundle carries them. +func (s *Service) RevisionArtifacts(ctx context.Context, number uint64, role, agentID string) (time.Time, []bundle.Artifact, error) { if number < 1 || number > store.MaxRevision { return time.Time{}, nil, ErrNotFound } q := s.st.Q() - r, err := q.GetRevision(ctx, int64(number)) + n := int64(number) + r, err := q.GetRevision(ctx, n) if err != nil { return time.Time{}, nil, translate(err) } - files, err := q.RevisionFiles(ctx, int64(number), role) + target := "" + if ValidUUID(agentID) { + ok, err := q.RevisionHasTarget(ctx, n, role, agentID) + if err != nil { + return time.Time{}, nil, translate(err) + } + if ok { + target = agentID + } + } + files, err := q.RevisionTargetFiles(ctx, n, role, target) if err != nil { return time.Time{}, nil, translate(err) } + if files, err = s.openFiles(n, files); err != nil { + return time.Time{}, nil, err + } arts := make([]bundle.Artifact, len(files)) for i, f := range files { arts[i] = bundle.Artifact{Path: f.Path, Mode: f.Mode, Content: f.Content, CreateOnly: f.CreateOnly} @@ -545,7 +562,11 @@ func (s *Service) StoreLeaseBatch(ctx context.Context, agentID string, seq uint6 return nil } res.Acknowledged = seq - return tx.StoreLeases(ctx, agentID, int64(seq), fullSync, leases, s.now()) //nolint:gosec // checked above + now := s.now().UTC().Truncate(time.Microsecond) + if err := tx.StoreLeases(ctx, agentID, int64(seq), fullSync, leases, now); err != nil { //nolint:gosec // checked above + return err + } + return s.adoptLeases(ctx, tx, leases, now) }) if err != nil { return nil, err diff --git a/server/internal/service/agents.go b/server/internal/service/agents.go index 4c06fd8..0bca486 100644 --- a/server/internal/service/agents.go +++ b/server/internal/service/agents.go @@ -11,6 +11,7 @@ import ( "encoding/json" "errors" "net" + "net/netip" "regexp" "slices" "strings" @@ -122,6 +123,24 @@ type AgentDTO struct { Health AgentHealth `json:"health"` RevokedAt *time.Time `json:"revoked_at,omitempty"` LastDeployment *DeploymentDTO `json:"last_deployment,omitempty"` + // Cluster is set for members of a cluster (ADR 0023). + Cluster *AgentClusterDTO `json:"cluster,omitempty"` +} + +// AgentClusterDTO is the cluster membership of an agent. +type AgentClusterDTO struct { + ID string `json:"id"` + Name string `json:"name"` + Role string `json:"role"` + Address netip.Addr `json:"address"` +} + +func agentCluster(ms map[string]store.Membership, agentID string) *AgentClusterDTO { + m, ok := ms[agentID] + if !ok { + return nil + } + return &AgentClusterDTO{ID: m.ClusterID, Name: m.ClusterName, Role: m.MemberRole, Address: m.Address} } // desiredRevision is the revision an agent should run (ADR 0022): the @@ -187,9 +206,15 @@ func (s *Service) ListAgents(ctx context.Context) ([]AgentDTO, error) { if err != nil { return nil, translate(err) } + ms, err := q.Memberships(ctx) + if err != nil { + return nil, translate(err) + } out := make([]AgentDTO, 0, len(agents)) for _, a := range agents { - out = append(out, s.agentDTO(a, desired, last[a.ID])) + d := s.agentDTO(a, desired, last[a.ID]) + d.Cluster = agentCluster(ms, a.ID) + out = append(out, d) } return out, nil } @@ -215,7 +240,13 @@ func (s *Service) loadAgentDTO(ctx context.Context, tx *store.Tx, id string) (Ag if err != nil && !errors.Is(err, store.ErrNotFound) { return AgentDTO{}, translate(err) } - return s.agentDTO(a, desired, last), nil + ms, err := tx.Memberships(ctx) + if err != nil { + return AgentDTO{}, translate(err) + } + d := s.agentDTO(a, desired, last) + d.Cluster = agentCluster(ms, id) + return d, nil } // RevokeAgent revokes the agent's certificate and closes its stream. Like diff --git a/server/internal/service/clusters.go b/server/internal/service/clusters.go new file mode 100644 index 0000000..9ccd53a --- /dev/null +++ b/server/internal/service/clusters.go @@ -0,0 +1,507 @@ +// SPDX-License-Identifier: LicenseRef-DDI-Proprietary + +package service + +import ( + "context" + "crypto/rand" + "encoding/base64" + "errors" + "net/netip" + "strconv" + "strings" + "time" + "unicode/utf8" + + "git.beugsiud.de/beug.k/DDI/server/internal/agenthub" + "git.beugsiud.de/beug.k/DDI/server/internal/configgen" + "git.beugsiud.de/beug.k/DDI/server/internal/ipam" + "git.beugsiud.de/beug.k/DDI/server/internal/secrets" + "git.beugsiud.de/beug.k/DDI/server/internal/store" +) + +// DNS and DHCP clusters (ADR 0023). + +// ErrClusterInUse is returned when a cluster to delete still serves zones +// or networks or is a default cluster of the generator settings. +var ErrClusterInUse = newError(KindConflict, "cluster_in_use", "the cluster is still in use") + +// Cluster defaults. +const ( + DefaultHAPort = 8001 + maxNameLen = 100 +) + +// ClusterMemberDTO is a member of a cluster. +type ClusterMemberDTO struct { + AgentID string `json:"agent_id"` + Role string `json:"role"` + Address netip.Addr `json:"address"` + Hostname string `json:"hostname"` + Connected bool `json:"connected"` +} + +// KeyRefDTO names a TSIG key. +type KeyRefDTO struct { + ID string `json:"id"` + Name string `json:"name"` +} + +// ClusterDTO is a cluster (without the HA credentials). +type ClusterDTO struct { + ID string `json:"id"` + Name string `json:"name"` + Role string `json:"role"` + Description string `json:"description"` + Members []ClusterMemberDTO `json:"members"` + PrimaryNS string `json:"primary_ns"` + NameServers []GeneratorNS `json:"name_servers"` + HAMode string `json:"ha_mode"` + HAPort int `json:"ha_port"` + TransferKey *KeyRefDTO `json:"transfer_key"` + ZoneCount int `json:"zone_count"` + NetworkCount int `json:"network_count"` + Version int `json:"version"` + CreatedAt time.Time `json:"created_at"` + UpdatedAt time.Time `json:"updated_at"` +} + +func (s *Service) clusterDTO(c *store.Cluster) ClusterDTO { + d := ClusterDTO{ID: c.ID, Name: c.Name, Role: c.Role, Description: c.Description, Members: []ClusterMemberDTO{}, + PrimaryNS: c.PrimaryNS, NameServers: []GeneratorNS{}, HAMode: c.HAMode, HAPort: c.HAPort, + ZoneCount: c.ZoneCount, NetworkCount: c.NetworkCount, Version: c.Version, CreatedAt: c.CreatedAt, + UpdatedAt: c.UpdatedAt} + for _, m := range c.Members { + md := ClusterMemberDTO{AgentID: m.AgentID, Role: m.MemberRole, Address: m.Address, Hostname: m.Hostname} + if s.opts.Hub != nil { + md.Connected = s.opts.Hub.Connected(m.AgentID) + } + d.Members = append(d.Members, md) + } + for _, ns := range c.NameServers { + d.NameServers = append(d.NameServers, GeneratorNS{Name: ns.Name, Addresses: orEmpty(ns.Addresses)}) + } + if c.TransferKeyID != nil { + d.TransferKey = &KeyRefDTO{ID: *c.TransferKeyID, Name: c.TransferKeyName} + } + return d +} + +// clusterAudit is the audit snapshot of a cluster (no connection state). +func (s *Service) clusterAudit(c *store.Cluster) ClusterDTO { + d := s.clusterDTO(c) + for i := range d.Members { + d.Members[i].Connected = false + } + return d +} + +// ClusterMemberInput is a member of a cluster request. +type ClusterMemberInput struct { + AgentID string + Role string + // Address is the textual IP (parsed and checked by the service). + Address string +} + +// ClusterInput creates or replaces a cluster. Role is used on create only. +type ClusterInput struct { + Name string + Role string + Description string + Members []ClusterMemberInput + PrimaryNS string + NameServers []GeneratorNS + HAMode string + HAPort *int + Version int +} + +func haAAD(clusterID string) []byte { return secrets.AAD("clusters", "ha_password", clusterID) } + +// normalizeCluster validates the plain fields of in for a cluster of role +// and writes them to c. +func normalizeCluster(in *ClusterInput, role string, c *store.Cluster) []Problem { + var problems []Problem + add := func(code, field string) { problems = append(problems, Problem{Code: code, Field: field}) } + c.Name = strings.TrimSpace(in.Name) + switch { + case c.Name == "": + add("name_required", "name") + case utf8.RuneCountInString(c.Name) > maxNameLen: + add("too_long", "name") + } + c.Description = strings.TrimSpace(in.Description) + if utf8.RuneCountInString(c.Description) > maxDescription { + add("too_long", "description") + } + c.PrimaryNS, c.NameServers = "", nil + c.HAMode, c.HAPort = string(configgen.HAHotStandby), DefaultHAPort + if role == RoleDNS { + if in.PrimaryNS != "" { + n, err := ipam.NormalizeDomain(in.PrimaryNS) + if err != nil { + add("invalid_domain", "primary_ns") + } + c.PrimaryNS = n + } + for _, ns := range in.NameServers { + n, err := ipam.NormalizeDomain(ns.Name) + if err != nil { + add("invalid_domain", "name_servers") + break + } + row := store.NameServerRow{Name: n, Addresses: []string{}} + for _, a := range ns.Addresses { + ip, err := netip.ParseAddr(strings.TrimSpace(a)) + if err != nil || ip.Zone() != "" { + add("invalid_ip", "name_servers") + break + } + row.Addresses = append(row.Addresses, ip.Unmap().String()) + } + c.NameServers = append(c.NameServers, row) + } + return problems + } + switch in.HAMode { + case "": + case string(configgen.HAHotStandby), string(configgen.HALoadBalancing): + c.HAMode = in.HAMode + default: + add("invalid_value", "ha_mode") + } + if in.HAPort != nil { + if *in.HAPort < 1 || *in.HAPort > 65535 { + add("out_of_range", "ha_port") + } + c.HAPort = *in.HAPort + } + return problems +} + +// checkMembers validates the member list of cluster c (role c.Role) and +// returns it sorted like the store returns it. +func (s *Service) checkMembers(ctx context.Context, tx *store.Tx, c *store.Cluster, in []ClusterMemberInput) ([]store.ClusterMember, error) { + memberships, err := tx.Memberships(ctx) + if err != nil { + return nil, err + } + var problems []Problem + var out []store.ClusterMember + seenAgent := map[string]bool{} + seenAddr := map[netip.Addr]bool{} + primaries, secondaries := 0, 0 + for i, m := range in { + params := map[string]string{"index": strconv.Itoa(i), "agent_id": m.AgentID} + add := func(code, sub string) { + problems = append(problems, Problem{Code: code, Field: "members[" + strconv.Itoa(i) + "]." + sub, Params: params}) + } + addr, err := netip.ParseAddr(strings.TrimSpace(m.Address)) + if err == nil { + addr = addr.Unmap() + } + switch { + case err != nil || addr.Zone() != "" || addr.IsUnspecified() || addr.IsMulticast(): + add("invalid_ip", "address") + continue + case m.Role != string(configgen.MemberPrimary) && m.Role != string(configgen.MemberSecondary): + add("invalid_value", "role") + continue + case seenAgent[m.AgentID]: + add("duplicate_member", "agent_id") + continue + case seenAddr[addr]: + params["address"] = addr.String() + add("duplicate_member", "address") + continue + } + seenAgent[m.AgentID], seenAddr[addr] = true, true + if !ValidUUID(m.AgentID) { + add("unknown_agent", "agent_id") + continue + } + ag, err := tx.GetAgent(ctx, m.AgentID, false) + if errors.Is(err, store.ErrNotFound) { + add("unknown_agent", "agent_id") + continue + } + if err != nil { + return nil, err + } + if ag.Role != c.Role { + add("cluster_role_mismatch", "agent_id") + continue + } + if ms, ok := memberships[m.AgentID]; ok && ms.ClusterID != c.ID { + params["cluster_id"], params["cluster_name"] = ms.ClusterID, ms.ClusterName + add("agent_in_other_cluster", "agent_id") + continue + } + if m.Role == string(configgen.MemberPrimary) { + primaries++ + if primaries > 1 { + add("cluster_primary_exists", "role") + continue + } + } else { + secondaries++ + if c.Role == RoleDHCP && secondaries > 1 { + add("cluster_member_limit", "role") + continue + } + } + out = append(out, store.ClusterMember{AgentID: m.AgentID, MemberRole: m.Role, Address: addr, Hostname: ag.Hostname}) + } + if len(problems) > 0 { + return nil, invalid(problems...) + } + return out, nil +} + +// ListClusters returns the clusters of a role ("" for all), sorted by role +// and name. +func (s *Service) ListClusters(ctx context.Context, role string) ([]ClusterDTO, error) { + if role != "" && !validRole(role) { + return nil, invalidField("invalid_role", "role") + } + list, err := s.st.Q().ListClusters(ctx, role) + if err != nil { + return nil, translate(err) + } + out := make([]ClusterDTO, 0, len(list)) + for _, c := range list { + out = append(out, s.clusterDTO(c)) + } + return out, nil +} + +// GetCluster returns one cluster. +func (s *Service) GetCluster(ctx context.Context, id string) (ClusterDTO, error) { + if !ValidUUID(id) { + return ClusterDTO{}, ErrNotFound + } + c, err := s.st.Q().GetCluster(ctx, id, false) + if err != nil { + return ClusterDTO{}, translate(err) + } + return s.clusterDTO(c), nil +} + +// CreateCluster creates a cluster with its members. A DNS cluster gets a +// transfer key, a DHCP cluster an HA user and password. +func (s *Service) CreateCluster(ctx context.Context, a *Actor, in ClusterInput) (ClusterDTO, error) { + if err := s.requireWritable(); err != nil { + return ClusterDTO{}, err + } + if !validRole(in.Role) { + return ClusterDTO{}, invalidField("invalid_role", "role") + } + c := &store.Cluster{ID: agenthub.NewID(), Role: in.Role} + if problems := normalizeCluster(&in, in.Role, c); len(problems) > 0 { + return ClusterDTO{}, invalid(problems...) + } + var key *store.TSIGKey + var err error + if c.Role == RoleDHCP { + c.HAUser = "ddi-ha-" + strings.ReplaceAll(c.ID, "-", "")[:12] + if c.HAPassword, err = s.newHAPassword(c.ID); err != nil { + return ClusterDTO{}, err + } + } else { + key = &store.TSIGKey{ID: agenthub.NewID(), Name: "transfer-" + c.ID, Algorithm: DefaultTSIGAlgorithm, + Description: "Transfer key of cluster " + c.Name, ClusterID: &c.ID} + if key.Secret, err = s.newTSIGSecret(key.ID, key.Algorithm); err != nil { + return ClusterDTO{}, err + } + } + var out ClusterDTO + err = s.inTx(ctx, func(tx *store.Tx) error { + members, err := s.checkMembers(ctx, tx, c, in.Members) + if err != nil { + return err + } + if err := tx.InsertCluster(ctx, c); err != nil { + return err + } + if key != nil { + if err := tx.InsertTSIGKey(ctx, key); err != nil { + return err + } + if err := tx.SetClusterTransferKey(ctx, c.ID, key.ID); err != nil { + return err + } + if err := s.audit(ctx, tx, a, "tsig_key.create", "tsig_key", key.ID, key.Name, nil, tsigKeyDTO(key)); err != nil { + return err + } + } + if err := tx.ReplaceClusterMembers(ctx, c.ID, c.Role, members); err != nil { + return err + } + stored, err := tx.GetCluster(ctx, c.ID, false) + if err != nil { + return err + } + out = s.clusterDTO(stored) + return s.audit(ctx, tx, a, "cluster.create", "cluster", c.ID, c.Name, nil, s.clusterAudit(stored)) + }) + if err != nil { + return ClusterDTO{}, err + } + s.publishMembers(out.Members) + return out, nil +} + +// newHAPassword returns a random HA password sealed for the cluster row. +func (s *Service) newHAPassword(clusterID string) ([]byte, error) { + var b [32]byte + if _, err := rand.Read(b[:]); err != nil { + return nil, err + } + return s.seal([]byte(base64.RawURLEncoding.EncodeToString(b[:])), haAAD(clusterID)) +} + +// publishMembers tells the UI that the agents' cluster changed. +func (s *Service) publishMembers(members ...[]ClusterMemberDTO) { + seen := map[string]bool{} + for _, list := range members { + for _, m := range list { + if !seen[m.AgentID] { + seen[m.AgentID] = true + s.publishAgent(m.AgentID) + } + } + } +} + +// UpdateCluster replaces the mutable fields and the member list. +func (s *Service) UpdateCluster(ctx context.Context, a *Actor, id string, in ClusterInput) (ClusterDTO, error) { + if err := s.requireWritable(); err != nil { + return ClusterDTO{}, err + } + if !ValidUUID(id) { + return ClusterDTO{}, ErrNotFound + } + var out, before ClusterDTO + err := s.inTx(ctx, func(tx *store.Tx) error { + c, err := tx.GetCluster(ctx, id, true) + if err != nil { + return err + } + if c.Version != in.Version { + return ErrVersionConflict + } + before = s.clusterAudit(c) + if problems := normalizeCluster(&in, c.Role, c); len(problems) > 0 { + return invalid(problems...) + } + members, err := s.checkMembers(ctx, tx, c, in.Members) + if err != nil { + return err + } + if err := tx.UpdateCluster(ctx, c); err != nil { + return err + } + if err := tx.ReplaceClusterMembers(ctx, c.ID, c.Role, members); err != nil { + return err + } + stored, err := tx.GetCluster(ctx, c.ID, false) + if err != nil { + return err + } + out = s.clusterDTO(stored) + return s.audit(ctx, tx, a, "cluster.update", "cluster", c.ID, c.Name, before, s.clusterAudit(stored)) + }) + if err != nil { + return ClusterDTO{}, err + } + s.publishMembers(before.Members, out.Members) + return out, nil +} + +// DeleteCluster removes a cluster with its members and transfer key. +func (s *Service) DeleteCluster(ctx context.Context, a *Actor, id string) error { + if err := s.requireWritable(); err != nil { + return err + } + if !ValidUUID(id) { + return ErrNotFound + } + var before ClusterDTO + err := s.inTx(ctx, func(tx *store.Tx) error { + c, err := tx.GetCluster(ctx, id, true) + if err != nil { + return err + } + if err := tx.LockGeneratorSettings(ctx); err != nil { + return err + } + settings, err := s.generatorSettings(ctx, tx) + if err != nil { + return err + } + params := map[string]string{} + if c.ZoneCount > 0 { + params["zones"] = itoa(c.ZoneCount) + } + if c.NetworkCount > 0 { + params["networks"] = itoa(c.NetworkCount) + } + if isDefault(settings.DNS.DefaultClusterID, id) || isDefault(settings.DHCP.DefaultClusterID, id) { + params["default"] = "true" + } + if c.TransferKeyID != nil { + k, err := tx.GetTSIGKey(ctx, *c.TransferKeyID, false) + if err != nil { + return err + } + if k.ZoneCount > 0 { + params["key_zones"] = itoa(k.ZoneCount) + } + } + if len(params) > 0 { + e := *ErrClusterInUse + e.Params = params + return &e + } + before = s.clusterAudit(c) + if err := tx.DeleteCluster(ctx, id); err != nil { + return err + } + if c.TransferKeyID != nil { + if err := s.audit(ctx, tx, a, "tsig_key.delete", "tsig_key", *c.TransferKeyID, c.TransferKeyName, + KeyRefDTO{ID: *c.TransferKeyID, Name: c.TransferKeyName}, nil); err != nil { + return err + } + } + return s.audit(ctx, tx, a, "cluster.delete", "cluster", c.ID, c.Name, before, nil) + }) + if err != nil { + return err + } + s.publishMembers(before.Members) + return nil +} + +func isDefault(setting *string, id string) bool { return setting != nil && *setting == id } + +// requireCluster checks that a referenced cluster exists and has role; it +// locks the cluster row so that it cannot be deleted concurrently. +func requireCluster(ctx context.Context, tx *store.Tx, id, role, field string) error { + if !ValidUUID(id) { + return invalidField("not_found", field) + } + c, err := tx.GetCluster(ctx, id, true) + if errors.Is(err, store.ErrNotFound) { + return invalidField("not_found", field) + } + if err != nil { + return err + } + if c.Role != role { + e := invalidField("cluster_role_mismatch", field) + e.Problems[0].Params = map[string]string{"role": c.Role} + return e + } + return nil +} diff --git a/server/internal/service/configpreview.go b/server/internal/service/configpreview.go index fca7c31..38ead10 100644 --- a/server/internal/service/configpreview.go +++ b/server/internal/service/configpreview.go @@ -3,25 +3,44 @@ package service import ( + "cmp" "context" + "encoding/base64" "errors" + "slices" "git.beugsiud.de/beug.k/DDI/server/internal/configgen" "git.beugsiud.de/beug.k/DDI/server/internal/configgen/bindgen" - "git.beugsiud.de/beug.k/DDI/server/internal/configgen/derive" "git.beugsiud.de/beug.k/DDI/server/internal/configgen/keagen" "git.beugsiud.de/beug.k/DDI/server/internal/ipam" + "git.beugsiud.de/beug.k/DDI/server/internal/store" ) // previewSerial is the SOA serial shown in the preview; revisions use their // own number. const previewSerial = 1 -// GeneratedFile is one generated file. +// GeneratedFile is one generated file. The content of secret files is +// never shown (ADR 0023). type GeneratedFile struct { Path string `json:"path"` Content string `json:"content"` CreateOnly bool `json:"create_only"` + // Target is "" for the default group, otherwise the agent ID of a + // cluster member. + Target string `json:"target"` + Secret bool `json:"secret"` +} + +// RevisionTarget describes the target of a set of files: the default group +// of a role or one cluster member, with the names at the time of the +// revision. +type RevisionTarget struct { + Role string `json:"role"` + Target string `json:"target"` + ClusterID *string `json:"cluster_id"` + ClusterName string `json:"cluster_name"` + MemberRole string `json:"member_role"` } // GeneratorProblem explains why a part could not be generated. @@ -35,12 +54,13 @@ type ConfigPreview struct { DNS []GeneratedFile `json:"dns"` DHCP []GeneratedFile `json:"dhcp"` Warnings []configgen.Warning `json:"warnings"` + Targets []RevisionTarget `json:"targets"` DNSError *GeneratorProblem `json:"dns_error,omitempty"` DHCPError *GeneratorProblem `json:"dhcp_error,omitempty"` } -// loadSource reads everything the generator needs. serial becomes the SOA -// serial of every zone. +// loadSource reads everything the generator needs, secrets included. +// serial becomes the SOA serial of every zone. func (s *Service) loadSource(ctx context.Context, serial uint32) (configgen.Source, error) { tx := s.st.Q() nets, err := tx.ListNetworks(ctx) @@ -55,76 +75,199 @@ func (s *Service) loadSource(ctx context.Context, serial uint32) (configgen.Sour if err != nil { return configgen.Source{}, translate(err) } - settings, err := s.GetGeneratorSettings(ctx) + settings, err := s.generatorSettings(ctx, tx) if err != nil { return configgen.Source{}, err } src := configgen.Source{ + Version: configgen.SourceVersion, Serial: serial, Addresses: addrs, SubnetIDs: ids, DNS: settings.dnsSettings(), - DHCP: configgen.DHCPSettings{ - Interfaces: settings.DHCP.Interfaces, DirectClients: settings.DHCP.DirectClients, - ControlSocket: settings.DHCP.ControlSocket, LeaseFile: settings.DHCP.LeaseFile, - HooksDir: settings.DHCP.HooksDir, ValidLifetime: settings.DHCP.ValidLifetime, - }, + DHCP: settings.dhcpSettings(), } src.Networks = make([]ipam.Network, len(nets)) for i, n := range nets { src.Networks[i] = *n } + if err := s.loadClustersAndZones(ctx, tx, &src); err != nil { + return configgen.Source{}, err + } return src, nil } -func files(arts []configgen.Artifact) []GeneratedFile { - out := make([]GeneratedFile, len(arts)) // never nil: the API returns [] - for i, a := range arts { - out[i] = GeneratedFile{Path: a.Path, Content: string(a.Content), CreateOnly: a.CreateOnly} +// loadClustersAndZones fills the M5 part of the generator input: zones, +// records, clusters and the TSIG keys they refer to (ADR 0023). +func (s *Service) loadClustersAndZones(ctx context.Context, tx *store.Tx, src *configgen.Source) error { + zones, err := tx.ListZones(ctx) + if err != nil { + return translate(err) } - return out + records, err := tx.AllRecords(ctx) + if err != nil { + return translate(err) + } + clusters, err := tx.ListClusters(ctx, "") + if err != nil { + return translate(err) + } + var keyIDs []string + for _, z := range zones { + zs := configgen.ZoneSource{ID: z.ID, Name: z.Name, Kind: configgen.ZoneKind(z.Kind), Mode: configgen.ZoneMode(z.Mode), + ClusterID: deref(z.ClusterID), AllowUpdate: z.AllowUpdate, UpdateKeyIDs: z.UpdateKeyIDs} + if z.DefaultTTL != nil { + zs.DefaultTTL = uint32(*z.DefaultTTL) //nolint:gosec // 1..2^31-1 by constraint + } + src.Zones = append(src.Zones, zs) + keyIDs = append(keyIDs, z.UpdateKeyIDs...) + } + for _, r := range records { + src.Records = append(src.Records, recordSource(r)) + } + for _, c := range clusters { + cs := configgen.ClusterSource{ID: c.ID, Name: c.Name, Role: c.Role, PrimaryNS: c.PrimaryNS, + HAMode: configgen.HAMode(c.HAMode), HAPort: uint16(c.HAPort), HAUser: c.HAUser} //nolint:gosec // 1..65535 by constraint + for _, m := range c.Members { + cs.Members = append(cs.Members, configgen.ClusterMember{AgentID: m.AgentID, Role: configgen.MemberRole(m.MemberRole), + Addr: m.Address}) + } + for _, ns := range c.NameServers { + cs.NameServers = append(cs.NameServers, nameServer(GeneratorNS{Name: ns.Name, Addresses: ns.Addresses})) + } + if c.TransferKeyID != nil { + cs.TransferKeyID = *c.TransferKeyID + keyIDs = append(keyIDs, *c.TransferKeyID) + } + if c.HAPassword != nil { + pw, err := s.unseal(c.HAPassword, haAAD(c.ID)) + if err != nil { + return err + } + cs.HAPassword = string(pw) + } + src.Clusters = append(src.Clusters, cs) + } + slices.Sort(keyIDs) + keys, err := tx.TSIGKeysByID(ctx, slices.Compact(keyIDs)) + if err != nil { + return translate(err) + } + for _, k := range keys { + raw, err := s.unseal(k.Secret, tsigAAD(k.ID)) + if err != nil { + return err + } + src.Keys = append(src.Keys, configgen.KeySource{ID: k.ID, Name: k.Name, Algorithm: k.Algorithm, + Secret: base64.StdEncoding.EncodeToString(raw)}) + } + slices.SortFunc(src.Keys, func(a, b configgen.KeySource) int { return cmp.Compare(a.Name, b.Name) }) + return nil } -func generatorProblem(err error) *GeneratorProblem { +func generatorProblem(err error, target string) *GeneratorProblem { + var p *GeneratorProblem var ve *configgen.ValidationError if errors.As(err, &ve) { - return &GeneratorProblem{Code: "config_invalid", Params: map[string]string{"path": ve.Path, "message": ve.Message}} + p = &GeneratorProblem{Code: "config_invalid", Params: map[string]string{"path": ve.Path, "message": ve.Message}} + } else { + p = &GeneratorProblem{Code: "config_failed", Params: map[string]string{"message": err.Error()}} } - return &GeneratorProblem{Code: "config_failed", Params: map[string]string{"message": err.Error()}} + if target != "" { + p.Params["target"] = target + } + return p } // PreviewConfig generates the BIND and Kea configuration for the current // data without storing or deploying anything. func (s *Service) PreviewConfig(ctx context.Context) (*ConfigPreview, error) { - g, err := s.generate(ctx, previewSerial) + src, err := s.loadSource(ctx, previewSerial) if err != nil { return nil, err } - out := &ConfigPreview{ - DNS: files(g.DNS), DHCP: files(g.DHCP), Warnings: g.Warnings, - DNSError: g.DNSError, DHCPError: g.DHCPError, + g := render(src) + files := revisionFiles(g) + out := &ConfigPreview{DNS: []GeneratedFile{}, DHCP: []GeneratedFile{}, Warnings: g.Warnings, + Targets: revisionTargets(files, src.Clusters), DNSError: g.DNSError, DHCPError: g.DHCPError} + for _, f := range files { + if f.Role == RoleDHCP { + out.DHCP = append(out.DHCP, generatedFile(f)) + } else { + out.DNS = append(out.DNS, generatedFile(f)) + } } return out, nil } -// Generated is the output of one generator run. A part that could not be -// generated has no artifacts and a problem instead. +// generatedFile converts a file for the API: secret content is left out. +func generatedFile(f store.RevisionFile) GeneratedFile { + g := GeneratedFile{Path: f.Path, CreateOnly: f.CreateOnly, Target: f.Target, Secret: f.Secret} + if !f.Secret { + g.Content = string(f.Content) + } + return g +} + +// revisionTargets describes the targets of files (in their order) with the +// clusters of the generator input they were rendered from. +func revisionTargets(files []store.RevisionFile, clusters []configgen.ClusterSource) []RevisionTarget { + out := []RevisionTarget{} + type key struct{ role, target string } + seen := map[key]bool{} + for _, f := range files { + k := key{f.Role, f.Target} + if seen[k] { + continue + } + seen[k] = true + t := RevisionTarget{Role: f.Role, Target: f.Target} + if f.Target != configgen.DefaultTarget { + for _, c := range clusters { + if c.Role != f.Role { + continue + } + for _, m := range c.Members { + if m.AgentID == f.Target { + id := c.ID + t.ClusterID, t.ClusterName, t.MemberRole = &id, c.Name, string(m.Role) + } + } + } + } + out = append(out, t) + } + return out +} + +// GeneratedTarget is the output for one target of a role. +type GeneratedTarget struct { + Role string + Target string + ClusterID string + MemberRole configgen.MemberRole + Artifacts []configgen.Artifact +} + +// Generated is the output of one generator run: the targets of both roles, +// DNS first, each role's default target first. A role that could not be +// generated has no targets and a problem instead. type Generated struct { - DNS []configgen.Artifact - DHCP []configgen.Artifact + Targets []GeneratedTarget Warnings []configgen.Warning DNSError *GeneratorProblem DHCPError *GeneratorProblem } -// generate runs the generator on the current data (read outside a -// transaction; callers that need a consistent snapshot hold their own). -func (s *Service) generate(ctx context.Context, serial uint32) (*Generated, error) { - src, err := s.loadSource(ctx, serial) - if err != nil { - return nil, err +// fileCount returns the number of files of a role. +func (g *Generated) fileCount(role string) int { + n := 0 + for _, t := range g.Targets { + if t.Role == role { + n += len(t.Artifacts) + } } - return render(src), nil + return n } // render runs the generator on src (a pure function: equal input gives @@ -135,21 +278,33 @@ func render(src configgen.Source) *Generated { if src.DNS.SOA.PrimaryNS == "" || src.DNS.SOA.Hostmaster == "" || len(src.DNS.NameServers) == 0 { out.DNSError = &GeneratorProblem{Code: "dns_settings_missing"} } else { - dnsCfg, warnings := derive.BuildDNS(src) + targets, warnings := buildDNS(src) out.Warnings = append(out.Warnings, warnings...) - if arts, err := bindgen.Render(dnsCfg); err != nil { - out.DNSError = generatorProblem(err) - } else { - out.DNS = arts + var gts []GeneratedTarget + for _, t := range targets { + arts, err := bindgen.Render(t.Config) + if err != nil { + out.DNSError, gts = generatorProblem(err, t.Target), nil + break + } + gts = append(gts, GeneratedTarget{Role: RoleDNS, Target: t.Target, ClusterID: t.ClusterID, MemberRole: t.Role, + Artifacts: arts}) } + out.Targets = append(out.Targets, gts...) } - dhcpCfg, warnings := derive.BuildDHCP(src) + targets, warnings := buildDHCP(src) out.Warnings = append(out.Warnings, warnings...) - if arts, err := keagen.Render(dhcpCfg); err != nil { - out.DHCPError = generatorProblem(err) - } else { - out.DHCP = arts + var gts []GeneratedTarget + for _, t := range targets { + arts, err := keagen.Render(t.Config) + if err != nil { + out.DHCPError, gts = generatorProblem(err, t.Target), nil + break + } + gts = append(gts, GeneratedTarget{Role: RoleDHCP, Target: t.Target, ClusterID: t.ClusterID, MemberRole: t.Role, + Artifacts: arts}) } + out.Targets = append(out.Targets, gts...) return out } diff --git a/server/internal/service/deploy.go b/server/internal/service/deploy.go index 90a3cdb..ad34726 100644 --- a/server/internal/service/deploy.go +++ b/server/internal/service/deploy.go @@ -51,6 +51,8 @@ func (s *Service) storeRevision(ctx context.Context, tx *store.Tx, a *Actor, src if err != nil { return RevisionDTO{}, nil, err } + // The JSON of the source has no secrets (json:"-"); they are stored + // sealed next to it. source, err := json.Marshal(src) if err != nil { return RevisionDTO{}, nil, err @@ -58,12 +60,24 @@ func (s *Service) storeRevision(ctx context.Context, tx *store.Tx, a *Actor, src now := s.now().UTC().Truncate(time.Microsecond) r := &store.Revision{Number: int64(src.Serial), CreatedAt: now, CreatedByName: a.Username, Comment: comment, Warnings: warnings, Source: source} + if r.Secrets, err = s.sealSourceSecrets(r.Number, src); err != nil { + return RevisionDTO{}, nil, err + } if a.UserID != "" { uid := a.UserID r.CreatedBy = &uid } files := revisionFiles(g) - if err := tx.InsertRevision(ctx, r, files); err != nil { + stored := make([]store.RevisionFile, len(files)) + for i, f := range files { + if f.Secret { + if f.Content, err = s.seal(f.Content, revisionFileAAD(r.Number, &f)); err != nil { + return RevisionDTO{}, nil, err + } + } + stored[i] = f + } + if err := tx.InsertRevision(ctx, r, stored); err != nil { return RevisionDTO{}, nil, err } superseded, err := tx.SupersedeDeployments(ctx, store.SupersedeFilter{AutoPullOnly: true, KeepRevision: r.Number}, now) @@ -71,7 +85,7 @@ func (s *Service) storeRevision(ctx context.Context, tx *store.Tx, a *Actor, src return RevisionDTO{}, nil, err } summary := revisionAudit{RevisionSummaryDTO: RevisionSummaryDTO{Number: r.Number, CreatedAt: r.CreatedAt, - CreatedByName: r.CreatedByName, Comment: r.Comment, DNSFiles: len(g.DNS), DHCPFiles: len(g.DHCP), + CreatedByName: r.CreatedByName, Comment: r.Comment, DNSFiles: g.fileCount(RoleDNS), DHCPFiles: g.fileCount(RoleDHCP), Warnings: len(g.Warnings)}, RollbackOf: rollbackOf} action := "revision.create" if rollbackOf > 0 { @@ -81,7 +95,73 @@ func (s *Service) storeRevision(ctx context.Context, tx *store.Tx, a *Actor, src if err := s.audit(ctx, tx, a, action, "revision", num, "#"+num, nil, summary); err != nil { return RevisionDTO{}, nil, err } - return revisionDTO(r, files), superseded, nil + return revisionDTO(r, files, src.Clusters), superseded, nil +} + +// Keys of the secrets map of a revision (ADR 0023). +const ( + secretKeyPrefix = "key:" + secretHAPrefix = "ha:" +) + +// sealSourceSecrets seals the secrets of src (TSIG secrets, HA passwords) +// as one JSON object; nil if there are none. +func (s *Service) sealSourceSecrets(number int64, src configgen.Source) ([]byte, error) { + m := map[string]string{} + for _, k := range src.Keys { + m[secretKeyPrefix+k.ID] = k.Secret + } + for _, c := range src.Clusters { + if c.HAPassword != "" { + m[secretHAPrefix+c.ID] = c.HAPassword + } + } + if len(m) == 0 { + return nil, nil + } + raw, err := json.Marshal(m) + if err != nil { + return nil, err + } + return s.seal(raw, revisionSecretsAAD(number)) +} + +// restoreSourceSecrets puts the sealed secrets of revision number back into +// its generator input. A secret that is missing makes the revision +// unusable for a rollback. +func (s *Service) restoreSourceSecrets(number int64, sealed []byte, src *configgen.Source) error { + m := map[string]string{} + if sealed != nil { + raw, err := s.unseal(sealed, revisionSecretsAAD(number)) + if err != nil { + e := *ErrRollbackUnavailable + e.err = err + return &e + } + if err := json.Unmarshal(raw, &m); err != nil { + e := *ErrRollbackUnavailable + e.err = err + return &e + } + } + for i := range src.Keys { + v, ok := m[secretKeyPrefix+src.Keys[i].ID] + if !ok { + return ErrRollbackUnavailable + } + src.Keys[i].Secret = v + } + for i := range src.Clusters { + if src.Clusters[i].Role != RoleDHCP { + continue + } + v, ok := m[secretHAPrefix+src.Clusters[i].ID] + if !ok { + return ErrRollbackUnavailable + } + src.Clusters[i].HAPassword = v + } + return nil } // revisionCreated notifies agents and the UI after a new revision was @@ -143,12 +223,16 @@ func (s *Service) CreateRevision(ctx context.Context, a *Actor, comment string) if err := generationProblem(g); err != nil { return err } - stored, err := tx.RevisionFiles(ctx, latest.Number, "") + stored, err := s.loadRevisionFiles(ctx, tx, latest.Number, "") if err != nil { return err } if sameFiles(revisionFiles(g), stored) { - out = revisionDTO(latest, stored) + clusters, err := revisionClusters(ctx, tx, latest.Number) + if err != nil { + return err + } + out = revisionDTO(latest, stored, clusters) return nil } if latest.Number >= store.MaxRevision { @@ -198,7 +282,7 @@ func (s *Service) RollbackRevision(ctx context.Context, a *Actor, n int64, comme if err := tx.LockRevisions(ctx); err != nil { return err } - raw, err := tx.RevisionSource(ctx, n) + raw, sealed, err := tx.RevisionSource(ctx, n) if err != nil { return err } @@ -211,6 +295,9 @@ func (s *Service) RollbackRevision(ctx context.Context, a *Actor, n int64, comme e.err = err return &e } + if err := s.restoreSourceSecrets(n, sealed, &src); err != nil { + return err + } latest, err := tx.LatestRevision(ctx) if err != nil { return err @@ -233,10 +320,13 @@ func (s *Service) RollbackRevision(ctx context.Context, a *Actor, n int64, comme return &out, nil } -// FileDiff is the change of one generated file. +// FileDiff is the change of one generated file. Secret files only report +// their status (no diff, no line counts). type FileDiff struct { Role string `json:"role"` + Target string `json:"target"` Path string `json:"path"` + Secret bool `json:"secret"` Status string `json:"status"` // added, removed, modified Diff string `json:"diff"` Inserted int `json:"inserted"` @@ -257,28 +347,30 @@ func roleOrder(role string) int { return 1 } -// diffFiles compares two file sets; unchanged files are left out. The -// result is ordered by role (DNS first) and path. +// diffFiles compares two file sets (secret content decrypted); unchanged +// files are left out. The result is ordered by role (DNS first), target +// ("" first) and path. func diffFiles(older, newer []store.RevisionFile) []FileDiff { - type key struct{ role, path string } + type key struct{ role, target, path string } oldBy := make(map[key]store.RevisionFile, len(older)) for _, f := range older { - oldBy[key{f.Role, f.Path}] = f + oldBy[key{f.Role, f.Target, f.Path}] = f } newBy := make(map[key]store.RevisionFile, len(newer)) keys := make([]key, 0, len(older)+len(newer)) for _, f := range newer { - k := key{f.Role, f.Path} + k := key{f.Role, f.Target, f.Path} newBy[k] = f keys = append(keys, k) } for _, f := range older { - if k := (key{f.Role, f.Path}); !hasKey(newBy, k) { + if k := (key{f.Role, f.Target, f.Path}); !hasKey(newBy, k) { keys = append(keys, k) } } slices.SortFunc(keys, func(a, b key) int { - return cmp.Or(cmp.Compare(roleOrder(a.role), roleOrder(b.role)), strings.Compare(a.path, b.path)) + return cmp.Or(cmp.Compare(roleOrder(a.role), roleOrder(b.role)), strings.Compare(a.target, b.target), + strings.Compare(a.path, b.path)) }) keys = slices.Compact(keys) @@ -286,20 +378,23 @@ func diffFiles(older, newer []store.RevisionFile) []FileDiff { for _, k := range keys { o, inOld := oldBy[k] n, inNew := newBy[k] - d := FileDiff{Role: k.role, Path: k.path} + d := FileDiff{Role: k.role, Target: k.target, Path: k.path} switch { case !inOld: d.Status = "added" case !inNew: d.Status = "removed" - case string(o.Content) == string(n.Content) && o.Mode == n.Mode && o.CreateOnly == n.CreateOnly: + case string(o.Content) == string(n.Content) && o.Mode == n.Mode && o.CreateOnly == n.CreateOnly && o.Secret == n.Secret: continue default: d.Status = "modified" } - a, b := string(o.Content), string(n.Content) - d.Diff = textdiff.Unified(a, b, diffContext) - d.Inserted, d.Deleted = textdiff.Stats(textdiff.Lines(a, b)) + d.Secret = o.Secret || n.Secret + if !d.Secret { + a, b := string(o.Content), string(n.Content) + d.Diff = textdiff.Unified(a, b, diffContext) + d.Inserted, d.Deleted = textdiff.Stats(textdiff.Lines(a, b)) + } out = append(out, d) } return out @@ -327,7 +422,7 @@ func (s *Service) DiffRevisions(ctx context.Context, n int64, against *int64) (* if _, err := q.GetRevision(ctx, n); err != nil { return nil, translate(err) } - newer, err := q.RevisionFiles(ctx, n, "") + newer, err := s.loadRevisionFiles(ctx, q, n, "") if err != nil { return nil, translate(err) } @@ -336,7 +431,7 @@ func (s *Service) DiffRevisions(ctx context.Context, n int64, against *int64) (* if _, err := q.GetRevision(ctx, m); err != nil { return nil, translate(err) } - if older, err = q.RevisionFiles(ctx, m, ""); err != nil { + if older, err = s.loadRevisionFiles(ctx, q, m, ""); err != nil { return nil, translate(err) } } @@ -376,7 +471,7 @@ func (s *Service) PendingChanges(ctx context.Context) (*PendingChanges, error) { serial = uint32(latest.Number) //nolint:gosec // 1..2^32-1 by constraint out.LatestRevision = latest.Number filter.AfterRevision, filter.Since = latest.Number, latest.CreatedAt - if stored, err = q.RevisionFiles(ctx, latest.Number, ""); err != nil { + if stored, err = s.loadRevisionFiles(ctx, q, latest.Number, ""); err != nil { return nil, translate(err) } } diff --git a/server/internal/service/derive_shim.go b/server/internal/service/derive_shim.go new file mode 100644 index 0000000..4c61990 --- /dev/null +++ b/server/internal/service/derive_shim.go @@ -0,0 +1,23 @@ +// SPDX-License-Identifier: LicenseRef-DDI-Proprietary + +package service + +import ( + "git.beugsiud.de/beug.k/DDI/server/internal/configgen" + "git.beugsiud.de/beug.k/DDI/server/internal/configgen/derive" +) + +// Temporary adapter to the M4 derivation until derive implements the M5 +// contract (configgen/model.go). To be removed. + +func buildDNS(src configgen.Source) ([]configgen.DNSTarget, []configgen.Warning) { + cfg, w := derive.BuildDNS(src) + return []configgen.DNSTarget{{Target: configgen.DefaultTarget, Config: cfg}}, w +} + +func buildDHCP(src configgen.Source) ([]configgen.DHCPTarget, []configgen.Warning) { + cfg, w := derive.BuildDHCP(src) + return []configgen.DHCPTarget{{Target: configgen.DefaultTarget, Config: cfg}}, w +} + +func derivedZones(configgen.Source) []configgen.DerivedZone { return nil } diff --git a/server/internal/service/dto.go b/server/internal/service/dto.go index 54faf46..2b86ff5 100644 --- a/server/internal/service/dto.go +++ b/server/internal/service/dto.go @@ -63,6 +63,7 @@ type NetworkDTO struct { DNSServers []netip.Addr `json:"dns_servers"` NTPServers []netip.Addr `json:"ntp_servers"` Pools []PoolDTO `json:"pools"` + DHCPClusterID *string `json:"dhcp_cluster_id"` Version int `json:"version"` CreatedAt time.Time `json:"created_at"` UpdatedAt time.Time `json:"updated_at"` @@ -86,6 +87,10 @@ func networkDTO(n *ipam.Network) NetworkDTO { for _, p := range n.Pools { d.Pools = append(d.Pools, PoolDTO(p)) } + if n.DHCPClusterID != "" { + id := n.DHCPClusterID + d.DHCPClusterID = &id + } return d } diff --git a/server/internal/service/errors.go b/server/internal/service/errors.go index b112079..9bea506 100644 --- a/server/internal/service/errors.go +++ b/server/internal/service/errors.go @@ -110,6 +110,22 @@ var constraintCodes = map[string]struct { "users_username_format": {"invalid_username", "username"}, "users_oidc_unique": {"oidc_identity_taken", ""}, "saved_filters_unique": {"filter_name_taken", "name"}, + + // M5 (ADR 0023). The service checks these before writing; the + // constraints catch concurrent changes. + "tsig_keys_name_unique": {"key_exists", "name"}, + "zone_update_keys_key_fk": {"key_in_use", ""}, + "clusters_transfer_key_fk": {"key_owned_by_cluster", ""}, + "clusters_name_unique": {"cluster_exists", "name"}, + "cluster_members_pkey": {"agent_in_other_cluster", "members"}, + "cluster_members_agent_fk": {"cluster_role_mismatch", "members"}, + "cluster_members_one_primary": {"cluster_primary_exists", "members"}, + "cluster_members_dhcp_one_secondary": {"cluster_member_limit", "members"}, + "cluster_members_address_unique": {"duplicate_member", "members"}, + "zones_name_unique": {"zone_exists", "name"}, + "zones_cluster_fk": {"cluster_in_use", "cluster_id"}, + "networks_dhcp_cluster_fk": {"cluster_in_use", "dhcp_cluster_id"}, + "dns_records_unique": {"record_exists", ""}, } // asUserError translates err and reports whether it is a classified error diff --git a/server/internal/service/generator.go b/server/internal/service/generator.go index d17d6e4..767d574 100644 --- a/server/internal/service/generator.go +++ b/server/internal/service/generator.go @@ -42,6 +42,9 @@ type GeneratorDNS struct { // ZoneDir is the directory on the DNS hosts that holds the zone files // (must match the DNS agent configuration). ZoneDir string `json:"zone_dir"` + // DefaultClusterID is the DNS cluster of zones without own cluster; + // nil for the default group (ADR 0023). + DefaultClusterID *string `json:"default_cluster_id"` } // GeneratorNS is a name server with optional glue addresses. @@ -61,8 +64,16 @@ type GeneratorDHCP struct { LeaseFile string `json:"lease_file"` // HooksDir is empty to load hooks from the directory Kea was built with. HooksDir string `json:"hooks_dir"` + // DefaultClusterID is the DHCP cluster of networks without own + // cluster; nil for the default group (ADR 0023). + DefaultClusterID *string `json:"default_cluster_id"` + // HAPasswordFile is the path of the Kea HA password file. + HAPasswordFile string `json:"ha_password_file"` } +// DefaultHAPasswordFile is the default path of the Kea HA password file. +const DefaultHAPasswordFile = "/etc/kea/ddi-suite-ha-password" //nolint:gosec // a path, not a credential + // DefaultGeneratorSettings are used until an administrator saves settings. func DefaultGeneratorSettings() GeneratorSettings { return GeneratorSettings{ @@ -73,17 +84,27 @@ func DefaultGeneratorSettings() GeneratorSettings { DHCP: GeneratorDHCP{ Interfaces: []string{"*"}, DirectClients: true, ValidLifetime: 3600, ControlSocket: "/run/kea/kea4-ctrl-socket", LeaseFile: "/var/lib/kea/kea-leases4.csv", + HAPasswordFile: DefaultHAPasswordFile, }, } } // GetGeneratorSettings returns the stored settings or the defaults. func (s *Service) GetGeneratorSettings(ctx context.Context) (GeneratorSettings, error) { + return s.generatorSettings(ctx, s.st.Q()) +} + +// generatorSettings reads the settings within tx. Settings stored by an +// older version get the defaults of new fields. +func (s *Service) generatorSettings(ctx context.Context, tx *store.Tx) (GeneratorSettings, error) { cfg := DefaultGeneratorSettings() - err := s.st.Q().GetSetting(ctx, store.SettingGenerator, &cfg) + err := tx.GetSetting(ctx, store.SettingGenerator, &cfg) if err != nil && !errors.Is(err, store.ErrNotFound) { return GeneratorSettings{}, translate(err) } + if cfg.DHCP.HAPasswordFile == "" { + cfg.DHCP.HAPasswordFile = DefaultHAPasswordFile + } return cfg, nil } @@ -185,6 +206,18 @@ func normalizeGenerator(g *GeneratorSettings) *Error { if g.DHCP.HooksDir != "" && !validHostPath(g.DHCP.HooksDir, 255) { add("invalid_path", "dhcp.hooks_dir") } + if g.DHCP.HAPasswordFile == "" { + g.DHCP.HAPasswordFile = DefaultHAPasswordFile + } + if !validHostPath(g.DHCP.HAPasswordFile, 255) { + add("invalid_path", "dhcp.ha_password_file") + } + // An empty ID means the default group, like null. + for _, id := range []**string{&d.DefaultClusterID, &g.DHCP.DefaultClusterID} { + if *id != nil && **id == "" { + *id = nil + } + } if d.NameServers == nil { d.NameServers = []GeneratorNS{} } @@ -209,7 +242,21 @@ func (s *Service) SetGeneratorSettings(ctx context.Context, a *Actor, g Generato return GeneratorSettings{}, e } err := s.inTx(ctx, func(tx *store.Tx) error { - before, err := s.GetGeneratorSettings(ctx) + for _, c := range []struct { + id *string + role, field string + }{{g.DNS.DefaultClusterID, RoleDNS, "dns.default_cluster_id"}, {g.DHCP.DefaultClusterID, RoleDHCP, "dhcp.default_cluster_id"}} { + if c.id == nil { + continue + } + if err := requireCluster(ctx, tx, *c.id, c.role, c.field); err != nil { + return err + } + } + if err := tx.LockGeneratorSettings(ctx); err != nil { + return err + } + before, err := s.generatorSettings(ctx, tx) if err != nil { return err } @@ -229,20 +276,15 @@ func (s *Service) SetGeneratorSettings(ctx context.Context, a *Actor, g Generato func (g GeneratorSettings) dnsSettings() configgen.DNSSettings { d := g.DNS out := configgen.DNSSettings{ - ZoneDir: d.ZoneDir, - DefaultTTL: d.DefaultTTL, + DefaultClusterID: deref(d.DefaultClusterID), + ZoneDir: d.ZoneDir, + DefaultTTL: d.DefaultTTL, SOA: configgen.SOA{PrimaryNS: d.PrimaryNS, Hostmaster: d.Hostmaster, Refresh: d.Refresh, Retry: d.Retry, Expire: d.Expire, Minimum: d.Minimum}, Options: configgen.DNSOptions{Recursion: d.Recursion}, } for _, ns := range d.NameServers { - n := configgen.NameServer{Name: ns.Name} - for _, a := range ns.Addresses { - if ip, err := netip.ParseAddr(a); err == nil { - n.Addrs = append(n.Addrs, ip.Unmap()) - } - } - out.NameServers = append(out.NameServers, n) + out.NameServers = append(out.NameServers, nameServer(ns)) } for _, p := range d.AllowRecursion { if pf, err := netip.ParsePrefix(p); err == nil { @@ -256,3 +298,31 @@ func (g GeneratorSettings) dnsSettings() configgen.DNSSettings { } return out } + +// nameServer converts a validated name server into generator input. +func nameServer(ns GeneratorNS) configgen.NameServer { + n := configgen.NameServer{Name: ns.Name} + for _, a := range ns.Addresses { + if ip, err := netip.ParseAddr(a); err == nil { + n.Addrs = append(n.Addrs, ip.Unmap()) + } + } + return n +} + +// dhcpSettings converts the settings into generator input. +func (g GeneratorSettings) dhcpSettings() configgen.DHCPSettings { + d := g.DHCP + return configgen.DHCPSettings{ + DefaultClusterID: deref(d.DefaultClusterID), Interfaces: d.Interfaces, DirectClients: d.DirectClients, + ControlSocket: d.ControlSocket, LeaseFile: d.LeaseFile, HooksDir: d.HooksDir, HAPasswordFile: d.HAPasswordFile, + ValidLifetime: d.ValidLifetime, + } +} + +func deref(s *string) string { + if s == nil { + return "" + } + return *s +} diff --git a/server/internal/service/leaseadopt.go b/server/internal/service/leaseadopt.go new file mode 100644 index 0000000..fb582f4 --- /dev/null +++ b/server/internal/service/leaseadopt.go @@ -0,0 +1,166 @@ +// SPDX-License-Identifier: LicenseRef-DDI-Proprietary + +package service + +import ( + "context" + "net/netip" + "slices" + "time" + + "git.beugsiud.de/beug.k/DDI/server/internal/events" + "git.beugsiud.de/beug.k/DDI/server/internal/ipam" + "git.beugsiud.de/beug.k/DDI/server/internal/store" +) + +// adoptLeases matches reported leases against the address table (ADR 0023, +// "Lease-Erfassung"), in the transaction that stores the leases: +// +// - an active lease on an IP without entry inside a pool of its network +// creates a dynamic entry (MAC, hostname if it is a valid label, +// first/last seen, lease state and expiry); +// - an active lease on an existing entry updates last seen, lease state +// and expiry; dynamic entries also take MAC and hostname; +// - other lease states only change the lease state of existing entries. +// +// Entries are never deleted. There is no audit entry per lease (dynamic +// entries produce no DNS or DHCP configuration); the UI receives an +// addresses event after the commit. +func (s *Service) adoptLeases(ctx context.Context, tx *store.Tx, leases []store.Lease, at time.Time) error { + if len(leases) == 0 { + return nil + } + ips := make([]netip.Addr, len(leases)) + for i, l := range leases { + ips[i] = l.IP + } + existing, err := tx.AddressesByIP(ctx, ips) + if err != nil { + return err + } + byIP := make(map[netip.Addr]*ipam.Address, len(existing)) + networkOf := make(map[string]string, len(existing)) + for i := range existing { + byIP[existing[i].IP] = &existing[i] + networkOf[existing[i].ID] = existing[i].NetworkID + } + var nets []*ipam.Network + var creates []ipam.Address + var updates []store.LeaseUpdate + for _, l := range leases { + active := l.State == "active" + if a := byIP[l.IP]; a != nil { + u := store.LeaseUpdate{AddressID: a.ID, State: l.State, Active: active, ExpiresAt: l.ExpiresAt} + if active && a.Type == ipam.TypeDynamic { + n, err := s.networkByID(ctx, tx, &nets, a.NetworkID) + if err != nil { + return err + } + u.SetIdentity = true + u.MAC, u.Hostname = a.MAC, a.Hostname + if mac, err := ipam.NormalizeMAC(l.MAC); err == nil { + u.MAC = mac + } + if h, err := ipam.NormalizeHostname(l.Hostname); err == nil { + u.Hostname = h + } + u.FQDN = ipam.FQDN(u.Hostname, n.Domain) + } + updates = append(updates, u) + continue + } + if !active { + continue + } + if nets == nil { + if nets, err = tx.ListNetworks(ctx); err != nil { + return err + } + } + n := networkFor(nets, l.IP) + if n == nil { + continue + } + a := ipam.Address{NetworkID: n.ID, IP: l.IP, Type: ipam.TypeDynamic, LeaseState: l.State, + LeaseExpiresAt: l.ExpiresAt} + if mac, err := ipam.NormalizeMAC(l.MAC); err == nil { + a.MAC = mac + } + if h, err := ipam.NormalizeHostname(l.Hostname); err == nil { + a.Hostname = h + } + // The same rules as for entries created by users: inside a pool of + // a DHCP network, a usable host address. + if errs := ipam.NormalizeAddress(&a, n); len(errs) > 0 { + continue + } + creates = append(creates, a) + } + if creates, err = s.withinAddressLimit(ctx, tx, creates); err != nil { + return err + } + res, err := tx.ApplyLeaseAdoption(ctx, creates, updates, networkOf, at) + if err != nil { + return err + } + if len(res.NetworkIDs) == 0 { + return nil + } + ids := make([]string, 0, len(res.NetworkIDs)) + for id := range res.NetworkIDs { + ids = append(ids, id) + } + slices.Sort(ids) + tx.AfterCommit("", func() { s.publish(events.TypeAddresses, events.Addresses{NetworkIDs: ids}) }) + return nil +} + +// networkByID returns a network from the cached list (loaded on first use). +func (s *Service) networkByID(ctx context.Context, tx *store.Tx, nets *[]*ipam.Network, id string) (*ipam.Network, error) { + if *nets == nil { + list, err := tx.ListNetworks(ctx) + if err != nil { + return nil, err + } + *nets = list + } + for _, n := range *nets { + if n.ID == id { + return n, nil + } + } + return &ipam.Network{ID: id}, nil +} + +// networkFor returns the network containing ip. +func networkFor(nets []*ipam.Network, ip netip.Addr) *ipam.Network { + for _, n := range nets { + if n.CIDR.Contains(ip) { + return n + } + } + return nil +} + +// withinAddressLimit drops creations beyond the license limit of +// addresses; leases cannot be refused, so the entries are simply not +// created. +func (s *Service) withinAddressLimit(ctx context.Context, tx *store.Tx, creates []ipam.Address) ([]ipam.Address, error) { + if len(creates) == 0 || s.opts.License == nil { + return creates, nil + } + limit := s.opts.License.Limits().MaxAddresses + if limit == nil { + return creates, nil + } + n, err := tx.CountAddresses(ctx) + if err != nil { + return nil, err + } + room := max(*limit-n, 0) + if len(creates) > room { + s.log.Warn("lease adoption: address limit of the license reached", "limit", *limit, "skipped", len(creates)-room) + creates = creates[:room] + } + return creates, nil +} diff --git a/server/internal/service/networks.go b/server/internal/service/networks.go index 038bb2a..06e780b 100644 --- a/server/internal/service/networks.go +++ b/server/internal/service/networks.go @@ -26,6 +26,8 @@ type NetworkInput struct { DNSServers []netip.Addr NTPServers []netip.Addr Pools []PoolDTO + // DHCPClusterID assigns a DHCP cluster; nil or "" for the default. + DHCPClusterID *string // Version must match the stored version on update. Version int } @@ -34,12 +36,21 @@ func (in *NetworkInput) apply(n *ipam.Network) { n.Name, n.Description, n.VLAN, n.Gateway = in.Name, in.Description, in.VLAN, in.Gateway n.Domain, n.SearchDomains, n.Tags = in.Domain, in.SearchDomains, in.Tags n.DHCPEnabled, n.LeaseTime, n.DNSServers, n.NTPServers = in.DHCPEnabled, in.LeaseTime, in.DNSServers, in.NTPServers + n.DHCPClusterID = deref(in.DHCPClusterID) n.Pools = nil for _, p := range in.Pools { n.Pools = append(n.Pools, ipam.Pool(p)) } } +// checkNetworkCluster verifies the DHCP cluster of a network. +func checkNetworkCluster(ctx context.Context, tx *store.Tx, n *ipam.Network) error { + if n.DHCPClusterID == "" { + return nil + } + return requireCluster(ctx, tx, n.DHCPClusterID, RoleDHCP, "dhcp_cluster_id") +} + // ListNetworks returns all networks with utilization. func (s *Service) ListNetworks(ctx context.Context) ([]NetworkDTO, error) { tx := s.st.Q() @@ -105,6 +116,9 @@ func (s *Service) CreateNetwork(ctx context.Context, a *Actor, in NetworkInput) return NetworkDTO{}, fromIPAM(errs) } err := s.inTx(ctx, func(tx *store.Tx) error { + if err := checkNetworkCluster(ctx, tx, n); err != nil { + return err + } if err := tx.CreateNetwork(ctx, n); err != nil { return err } @@ -135,6 +149,9 @@ func (s *Service) UpdateNetwork(ctx context.Context, a *Actor, id string, in Net if errs := ipam.NormalizeNetwork(n); len(errs) > 0 { return fromIPAM(errs) } + if err := checkNetworkCluster(ctx, tx, n); err != nil { + return err + } addrs, err := tx.AddressesInNetwork(ctx, id) if err != nil { return err diff --git a/server/internal/service/revisions.go b/server/internal/service/revisions.go index a2892f9..d5ac97d 100644 --- a/server/internal/service/revisions.go +++ b/server/internal/service/revisions.go @@ -7,9 +7,11 @@ import ( "context" "encoding/json" "net/netip" + "strconv" "time" "git.beugsiud.de/beug.k/DDI/server/internal/configgen" + "git.beugsiud.de/beug.k/DDI/server/internal/secrets" "git.beugsiud.de/beug.k/DDI/server/internal/store" ) @@ -45,6 +47,7 @@ type RevisionDTO struct { DNS []GeneratedFile `json:"dns"` DHCP []GeneratedFile `json:"dhcp"` Warnings []configgen.Warning `json:"warnings"` + Targets []RevisionTarget `json:"targets"` } // RevisionPage is a page of revisions, newest first. @@ -53,9 +56,12 @@ type RevisionPage struct { NextBefore *int64 `json:"next_before"` } -func revisionDTO(r *store.Revision, files []store.RevisionFile) RevisionDTO { +// revisionDTO converts a revision with its files (secret content is left +// out); clusters come from the revision's generator input. +func revisionDTO(r *store.Revision, files []store.RevisionFile, clusters []configgen.ClusterSource) RevisionDTO { d := RevisionDTO{Number: r.Number, CreatedAt: r.CreatedAt, CreatedByName: r.CreatedByName, Comment: r.Comment, - DNS: []GeneratedFile{}, DHCP: []GeneratedFile{}, Warnings: []configgen.Warning{}} + DNS: []GeneratedFile{}, DHCP: []GeneratedFile{}, Warnings: []configgen.Warning{}, + Targets: revisionTargets(files, clusters)} if len(r.Warnings) > 0 { _ = json.Unmarshal(r.Warnings, &d.Warnings) // written by us; a broken value shows no warnings if d.Warnings == nil { @@ -63,25 +69,25 @@ func revisionDTO(r *store.Revision, files []store.RevisionFile) RevisionDTO { } } for _, f := range files { - g := GeneratedFile{Path: f.Path, Content: string(f.Content), CreateOnly: f.CreateOnly} if f.Role == RoleDHCP { - d.DHCP = append(d.DHCP, g) + d.DHCP = append(d.DHCP, generatedFile(f)) } else { - d.DNS = append(d.DNS, g) + d.DNS = append(d.DNS, generatedFile(f)) } } return d } -// revisionFiles flattens a generator run into stored files (DNS first, as -// the database returns them). +// revisionFiles flattens a generator run into files in the order the +// database returns them: by role (DNS first), target and generator order. +// Secret files hold their plain content. func revisionFiles(g *Generated) []store.RevisionFile { - out := make([]store.RevisionFile, 0, len(g.DNS)+len(g.DHCP)) - for _, a := range g.DNS { - out = append(out, store.RevisionFile{Role: RoleDNS, Path: a.Path, Mode: a.Mode, CreateOnly: a.CreateOnly, Content: a.Content}) - } - for _, a := range g.DHCP { - out = append(out, store.RevisionFile{Role: RoleDHCP, Path: a.Path, Mode: a.Mode, CreateOnly: a.CreateOnly, Content: a.Content}) + var out []store.RevisionFile + for _, t := range g.Targets { + for _, a := range t.Artifacts { + out = append(out, store.RevisionFile{Role: t.Role, Target: t.Target, Path: a.Path, Mode: a.Mode, + CreateOnly: a.CreateOnly, Secret: a.Secret, Content: a.Content}) + } } return out } @@ -92,13 +98,64 @@ func sameFiles(a, b []store.RevisionFile) bool { } for i := range a { x, y := a[i], b[i] - if x.Role != y.Role || x.Path != y.Path || x.Mode != y.Mode || x.CreateOnly != y.CreateOnly || !bytes.Equal(x.Content, y.Content) { + if x.Role != y.Role || x.Target != y.Target || x.Path != y.Path || x.Mode != y.Mode || x.CreateOnly != y.CreateOnly || + x.Secret != y.Secret || !bytes.Equal(x.Content, y.Content) { return false } } return true } +// revisionFileAAD binds the sealed content of a secret file to its row. +func revisionFileAAD(number int64, f *store.RevisionFile) []byte { + return secrets.AAD("config_revision_files", "content", + strconv.FormatInt(number, 10)+"/"+f.Role+"/"+f.Target+"/"+f.Path) +} + +// revisionSecretsAAD binds the sealed secrets of a revision to its row. +func revisionSecretsAAD(number int64) []byte { + return secrets.AAD("config_revisions", "secrets", strconv.FormatInt(number, 10)) +} + +// loadRevisionFiles returns the files of a revision (role "" for both) with +// the content of secret files decrypted. +func (s *Service) loadRevisionFiles(ctx context.Context, tx *store.Tx, number int64, role string) ([]store.RevisionFile, error) { + files, err := tx.RevisionFiles(ctx, number, role) + if err != nil { + return nil, err + } + return s.openFiles(number, files) +} + +// openFiles decrypts the content of secret files in place. +func (s *Service) openFiles(number int64, files []store.RevisionFile) ([]store.RevisionFile, error) { + for i := range files { + if !files[i].Secret { + continue + } + plain, err := s.unseal(files[i].Content, revisionFileAAD(number, &files[i])) + if err != nil { + return nil, err + } + files[i].Content = plain + } + return files, nil +} + +// revisionClusters returns the clusters of a revision's stored generator +// input (none for revisions before M5). +func revisionClusters(ctx context.Context, tx *store.Tx, number int64) ([]configgen.ClusterSource, error) { + raw, _, err := tx.RevisionSource(ctx, number) + if err != nil || raw == nil { + return nil, err + } + var src struct{ Clusters []configgen.ClusterSource } + if err := json.Unmarshal(raw, &src); err != nil { + return nil, nil //nolint:nilerr // written by us; a broken value shows targets without cluster names + } + return src.Clusters, nil +} + // generationProblem reports the first part that could not be generated. func generationProblem(g *Generated) error { for _, p := range []struct { @@ -155,7 +212,11 @@ func (s *Service) GetRevision(ctx context.Context, number int64) (*RevisionDTO, if err != nil { return nil, translate(err) } - d := revisionDTO(r, files) + clusters, err := revisionClusters(ctx, q, number) + if err != nil { + return nil, translate(err) + } + d := revisionDTO(r, files, clusters) return &d, nil } diff --git a/server/internal/service/service.go b/server/internal/service/service.go index 84c2306..23936ea 100644 --- a/server/internal/service/service.go +++ b/server/internal/service/service.go @@ -145,8 +145,10 @@ func (s *Service) audit(ctx context.Context, tx *store.Tx, a *Actor, action, obj // configObjectTypes and configActions identify audit entries of data the // generator uses (pending changes, ADR 0022). var ( - configObjectTypes = []string{"network", "address"} - configActions = []string{"settings.generator"} + configObjectTypes = []string{"network", "address", "cluster", "zone", "record"} + // TSIG key changes count, reading a secret does not. + configActions = []string{"settings.generator", "tsig_key.create", "tsig_key.update", "tsig_key.delete", + "tsig_key.rotate"} ) func affectsConfig(action, objType string) bool { diff --git a/server/internal/service/source_test.go b/server/internal/service/source_test.go index 0f93325..fd7da60 100644 --- a/server/internal/service/source_test.go +++ b/server/internal/service/source_test.go @@ -6,6 +6,7 @@ import ( "bytes" "context" "encoding/json" + "errors" "net/netip" "reflect" "testing" @@ -13,6 +14,7 @@ import ( "git.beugsiud.de/beug.k/DDI/server/internal/configgen" "git.beugsiud.de/beug.k/DDI/server/internal/ipam" + "git.beugsiud.de/beug.k/DDI/server/internal/secrets" "git.beugsiud.de/beug.k/DDI/server/internal/store/storetest" ) @@ -75,7 +77,8 @@ func sameOutput(t *testing.T, a, b *Generated) { func TestSourceJSONRoundTrip(t *testing.T) { src := testSource(7) orig := render(src) - if orig.DNSError != nil || orig.DHCPError != nil || len(orig.DNS) == 0 || len(orig.DHCP) == 0 || len(orig.Warnings) == 0 { + if orig.DNSError != nil || orig.DHCPError != nil || orig.fileCount(RoleDNS) == 0 || orig.fileCount(RoleDHCP) == 0 || + len(orig.Warnings) == 0 { t.Fatalf("test source does not exercise the generator: %+v", orig) } raw, err := json.Marshal(src) @@ -108,7 +111,7 @@ func TestSourceJSONRoundTrip(t *testing.T) { func TestStoredSourceReRendersIdentically(t *testing.T) { st := storetest.New(t) ctx := context.Background() - svc := New(Options{Store: st}) + svc := New(Options{Store: st, Keyring: testKeyring(t)}) a := SystemActor g := DefaultGeneratorSettings() @@ -146,7 +149,7 @@ func TestStoredSourceReRendersIdentically(t *testing.T) { if err != nil || !created { t.Fatalf("revision: %v %v", created, err) } - raw, err := st.Q().RevisionSource(ctx, rev.Number) + raw, sealed, err := st.Q().RevisionSource(ctx, rev.Number) if err != nil || raw == nil { t.Fatalf("source %s %v", raw, err) } @@ -154,10 +157,13 @@ func TestStoredSourceReRendersIdentically(t *testing.T) { if err := json.Unmarshal(raw, &src); err != nil { t.Fatal(err) } - if src.Serial != uint32(rev.Number) { //nolint:gosec // small - t.Errorf("stored serial %d", src.Serial) + if src.Serial != uint32(rev.Number) || src.Version != configgen.SourceVersion { //nolint:gosec // small + t.Errorf("stored serial %d, version %d", src.Serial, src.Version) } - stored, err := st.Q().RevisionFiles(ctx, rev.Number, "") + if err := svc.restoreSourceSecrets(rev.Number, sealed, &src); err != nil { + t.Fatal(err) + } + stored, err := svc.loadRevisionFiles(ctx, st.Q(), rev.Number, "") if err != nil { t.Fatal(err) } @@ -165,3 +171,70 @@ func TestStoredSourceReRendersIdentically(t *testing.T) { t.Error("the stored source does not render the stored files") } } + +func testKeyring(t *testing.T) *secrets.Keyring { + t.Helper() + data, err := secrets.GenerateKeyFile("k1") + if err != nil { + t.Fatal(err) + } + kr, err := secrets.ParseKeyFile(data) + if err != nil { + t.Fatal(err) + } + return kr +} + +// TestSourceSecrets: secrets never enter the JSON of the source; they are +// sealed separately and restored for rollbacks. +func TestSourceSecrets(t *testing.T) { + svc := New(Options{Keyring: testKeyring(t)}) + src := testSource(3) + src.Keys = []configgen.KeySource{{ID: "k1", Name: "transfer", Algorithm: "hmac-sha256", Secret: "c2VjcmV0LWtleQ=="}} + src.Clusters = []configgen.ClusterSource{ + {ID: "c1", Name: "DNS", Role: RoleDNS, TransferKeyID: "k1"}, + {ID: "c2", Name: "DHCP", Role: RoleDHCP, HAMode: configgen.HAHotStandby, HAPort: 8001, HAUser: "ha", + HAPassword: "ha-password-value"}, + } + raw, err := json.Marshal(src) + if err != nil { + t.Fatal(err) + } + if bytes.Contains(raw, []byte("c2VjcmV0LWtleQ==")) || bytes.Contains(raw, []byte("ha-password-value")) { + t.Fatalf("secret in the source JSON: %s", raw) + } + sealed, err := svc.sealSourceSecrets(3, src) + if err != nil || bytes.Contains(sealed, []byte("ha-password-value")) { + t.Fatalf("seal: %v", err) + } + var back configgen.Source + if err := json.Unmarshal(raw, &back); err != nil { + t.Fatal(err) + } + if err := svc.restoreSourceSecrets(3, sealed, &back); err != nil { + t.Fatal(err) + } + if back.Keys[0].Secret != src.Keys[0].Secret || back.Clusters[1].HAPassword != src.Clusters[1].HAPassword { + t.Errorf("restored %+v %+v", back.Keys, back.Clusters) + } + unavailable := func(err error) bool { + var se *Error + return errors.As(err, &se) && se.Code == ErrRollbackUnavailable.Code + } + // Bound to the revision number. + if err := svc.restoreSourceSecrets(4, sealed, &back); !unavailable(err) { + t.Errorf("other revision: %v", err) + } + // Missing secrets make the rollback impossible. + if err := svc.restoreSourceSecrets(3, nil, &back); !unavailable(err) { + t.Errorf("no secrets: %v", err) + } + // Sources before M5 have none and need none. + old := testSource(3) + if err := svc.restoreSourceSecrets(3, nil, &old); err != nil { + t.Errorf("M4 source: %v", err) + } + if s, err := svc.sealSourceSecrets(3, old); s != nil || err != nil { + t.Errorf("M4 source sealed %v %v", s, err) + } +} diff --git a/server/internal/service/targets_test.go b/server/internal/service/targets_test.go new file mode 100644 index 0000000..be5736d --- /dev/null +++ b/server/internal/service/targets_test.go @@ -0,0 +1,125 @@ +// SPDX-License-Identifier: LicenseRef-DDI-Proprietary + +package service + +import ( + "bytes" + "context" + "strings" + "testing" + + "git.beugsiud.de/beug.k/DDI/server/internal/configgen" + "git.beugsiud.de/beug.k/DDI/server/internal/store" + "git.beugsiud.de/beug.k/DDI/server/internal/store/storetest" +) + +const ( + primaryAgent = "00000000-0000-4000-8000-0000000000a1" + otherAgent = "00000000-0000-4000-8000-0000000000a2" +) + +// targetOutput is a generator run with a default group and one cluster +// member, each with a secret keys file. +func targetOutput(secret string) *Generated { + art := func(path, content string, secretFile bool) configgen.Artifact { + mode := uint32(0o644) + if secretFile { + mode = 0o640 + } + return configgen.Artifact{Path: path, Mode: mode, Content: []byte(content), Secret: secretFile} + } + return &Generated{Warnings: []configgen.Warning{}, Targets: []GeneratedTarget{ + {Role: RoleDNS, Artifacts: []configgen.Artifact{art("ddi-suite.conf", "default\n", false), + art("ddi-suite-keys.conf", "# no keys\n", true)}}, + {Role: RoleDNS, Target: primaryAgent, ClusterID: "c1", MemberRole: configgen.MemberPrimary, + Artifacts: []configgen.Artifact{art("ddi-suite.conf", "primary\n", false), + art("ddi-suite-keys.conf", "key \"t\" { secret \""+secret+"\"; };\n", true)}}, + {Role: RoleDHCP, Artifacts: []configgen.Artifact{art("kea-dhcp4.json", "{}\n", false)}}, + }} +} + +func TestRevisionTargetsAndSecretFiles(t *testing.T) { + st := storetest.New(t) + ctx := context.Background() + svc := New(Options{Store: st, Keyring: testKeyring(t)}) + store1 := func(serial uint32, g *Generated) { + t.Helper() + src := testSource(serial) + src.Clusters = []configgen.ClusterSource{{ID: "c1", Name: "Pair", Role: RoleDNS, + Members: []configgen.ClusterMember{{AgentID: primaryAgent, Role: configgen.MemberPrimary}}}} + if err := st.InTx(ctx, func(tx *store.Tx) error { + _, _, err := svc.storeRevision(ctx, tx, SystemActor, src, g, "", 0) + return err + }); err != nil { + t.Fatal(err) + } + } + store1(1, targetOutput("c2VjcmV0LW9uZQ==")) + + // The database holds secret files sealed. + raw, err := st.Q().RevisionFiles(ctx, 1, RoleDNS) + if err != nil || len(raw) != 4 { + t.Fatalf("files %+v %v", raw, err) + } + for _, f := range raw { + if f.Secret && (bytes.Contains(f.Content, []byte("c2VjcmV0LW9uZQ==")) || bytes.Contains(f.Content, []byte("no keys"))) { + t.Errorf("secret file stored in plain text: %s", f.Path) + } + } + + // Bundles: the agent's own target, otherwise the default group. + _, arts, err := svc.RevisionArtifacts(ctx, 1, RoleDNS, primaryAgent) + if err != nil || len(arts) != 2 || string(arts[0].Content) != "primary\n" || + !strings.Contains(string(arts[1].Content), "c2VjcmV0LW9uZQ==") || arts[1].Mode != 0o640 { + t.Fatalf("member bundle %+v %v", arts, err) + } + _, arts, err = svc.RevisionArtifacts(ctx, 1, RoleDNS, otherAgent) + if err != nil || len(arts) != 2 || string(arts[0].Content) != "default\n" || string(arts[1].Content) != "# no keys\n" { + t.Fatalf("default bundle %+v %v", arts, err) + } + _, arts, err = svc.RevisionArtifacts(ctx, 1, RoleDHCP, primaryAgent) + if err != nil || len(arts) != 1 || arts[0].Path != "kea-dhcp4.json" { + t.Fatalf("dhcp bundle %+v %v", arts, err) + } + + // The API shows targets, never secret content. + rev, err := svc.GetRevision(ctx, 1) + if err != nil { + t.Fatal(err) + } + if len(rev.DNS) != 4 || len(rev.DHCP) != 1 || len(rev.Targets) != 3 { + t.Fatalf("revision %+v", rev) + } + for _, f := range rev.DNS { + if f.Secret != (f.Path == "ddi-suite-keys.conf") || (f.Secret && f.Content != "") { + t.Errorf("file %+v", f) + } + } + tg := rev.Targets[1] + if rev.Targets[0].Target != "" || rev.Targets[0].ClusterID != nil || tg.Target != primaryAgent || tg.ClusterID == nil || + *tg.ClusterID != "c1" || tg.ClusterName != "Pair" || tg.MemberRole != "primary" || rev.Targets[2].Role != RoleDHCP { + t.Errorf("targets %+v", rev.Targets) + } + + // A changed secret appears in the diff with its status only. + store1(2, targetOutput("c2VjcmV0LXR3bw==")) + d, err := svc.DiffRevisions(ctx, 2, nil) + if err != nil { + t.Fatal(err) + } + if len(d.Files) != 1 || d.Files[0].Path != "ddi-suite-keys.conf" || d.Files[0].Target != primaryAgent || !d.Files[0].Secret || + d.Files[0].Status != "modified" || d.Files[0].Diff != "" || d.Files[0].Inserted != 0 || d.Files[0].Deleted != 0 { + t.Errorf("diff %+v", d.Files) + } + d, err = svc.DiffRevisions(ctx, 2, ptrInt64(0)) + if err != nil || len(d.Files) != 5 || d.Files[0].Target != "" || d.Files[2].Target != primaryAgent || + d.Files[4].Role != RoleDHCP { + t.Errorf("diff against nothing %+v %v", d, err) + } + page, err := svc.ListRevisions(ctx, 10, 0) + if err != nil || page.Items[0].DNSFiles != 4 || page.Items[0].DHCPFiles != 1 { + t.Errorf("summary %+v %v", page, err) + } +} + +func ptrInt64(v int64) *int64 { return &v } diff --git a/server/internal/service/tsigkeys.go b/server/internal/service/tsigkeys.go new file mode 100644 index 0000000..f5e4f24 --- /dev/null +++ b/server/internal/service/tsigkeys.go @@ -0,0 +1,283 @@ +// SPDX-License-Identifier: LicenseRef-DDI-Proprietary + +package service + +import ( + "context" + "crypto/rand" + "encoding/base64" + "strings" + "time" + "unicode/utf8" + + "git.beugsiud.de/beug.k/DDI/server/internal/agenthub" + "git.beugsiud.de/beug.k/DDI/server/internal/ipam" + "git.beugsiud.de/beug.k/DDI/server/internal/secrets" + "git.beugsiud.de/beug.k/DDI/server/internal/store" +) + +// TSIG keys (ADR 0023). Secrets are generated by the server, stored sealed +// and leave it only through GetTSIGKeySecret (audited) and the signed +// bundles of the DNS agents. + +// Errors of the TSIG key operations. +var ( + ErrKeyExists = newError(KindConflict, "key_exists", "a TSIG key with this name exists already") + ErrKeyInUse = newError(KindConflict, "key_in_use", "zones still accept updates with this key") + ErrKeyOwnedByCluster = newError(KindConflict, "key_owned_by_cluster", "the transfer key of a cluster is deleted with the cluster") + ErrSecretsUnavailable = newError(KindUnavailable, "secrets_unavailable", "no master key is configured") +) + +// keyLengths are the secret lengths (= hash lengths) of the algorithms. +var keyLengths = map[string]int{"hmac-sha256": 32, "hmac-sha384": 48, "hmac-sha512": 64} + +// DefaultTSIGAlgorithm is used when a request names none. +const DefaultTSIGAlgorithm = "hmac-sha256" + +const maxDescription = 2000 + +// TSIGKeyDTO is a key without its secret. +type TSIGKeyDTO struct { + ID string `json:"id"` + Name string `json:"name"` + Algorithm string `json:"algorithm"` + Description string `json:"description"` + ClusterID *string `json:"cluster_id"` + ZoneCount int `json:"zone_count"` + Version int `json:"version"` + CreatedAt time.Time `json:"created_at"` + RotatedAt time.Time `json:"rotated_at"` +} + +func tsigKeyDTO(k *store.TSIGKey) TSIGKeyDTO { + return TSIGKeyDTO{ID: k.ID, Name: k.Name, Algorithm: k.Algorithm, Description: k.Description, ClusterID: k.ClusterID, + ZoneCount: k.ZoneCount, Version: k.Version, CreatedAt: k.CreatedAt, RotatedAt: k.RotatedAt} +} + +// TSIGKeySecretDTO is a key with its secret (base64). +type TSIGKeySecretDTO struct { + Name string `json:"name"` + Algorithm string `json:"algorithm"` + Secret string `json:"secret"` +} + +// TSIGKeyInput creates a key or changes its description. Name and +// algorithm are used on create only. +type TSIGKeyInput struct { + Name string + Algorithm string + Description string + Version int +} + +func tsigAAD(id string) []byte { return secrets.AAD("tsig_keys", "secret", id) } + +// seal encrypts a secret with the master key. +func (s *Service) seal(plain, aad []byte) ([]byte, error) { + if s.opts.Keyring == nil { + return nil, ErrSecretsUnavailable + } + return s.opts.Keyring.Seal(plain, aad) +} + +// unseal decrypts a secret sealed with seal. +func (s *Service) unseal(sealed, aad []byte) ([]byte, error) { + if s.opts.Keyring == nil { + return nil, ErrSecretsUnavailable + } + return s.opts.Keyring.Open(sealed, aad) +} + +// newTSIGSecret returns a random secret of the algorithm's hash length, +// sealed for the key row id. +func (s *Service) newTSIGSecret(id, algorithm string) ([]byte, error) { + raw := make([]byte, keyLengths[algorithm]) + if _, err := rand.Read(raw); err != nil { + return nil, err + } + return s.seal(raw, tsigAAD(id)) +} + +// tsigSecret returns the base64 secret of a key. +func (s *Service) tsigSecret(k *store.TSIGKey) (string, error) { + raw, err := s.unseal(k.Secret, tsigAAD(k.ID)) + if err != nil { + return "", err + } + return base64.StdEncoding.EncodeToString(raw), nil +} + +// normalizeKeyName validates a TSIG key name (lower-case domain name). +func normalizeKeyName(name string) (string, error) { + n, err := ipam.NormalizeDomain(name) + if err != nil { + return "", invalidField("invalid_key_name", "name") + } + return n, nil +} + +func checkDescription(d string) (string, *Problem) { + if utf8.RuneCountInString(d) > maxDescription { + return d, &Problem{Code: "too_long", Field: "description"} + } + return d, nil +} + +// ListTSIGKeys returns all keys (without secrets) sorted by name. +func (s *Service) ListTSIGKeys(ctx context.Context) ([]TSIGKeyDTO, error) { + keys, err := s.st.Q().ListTSIGKeys(ctx) + if err != nil { + return nil, translate(err) + } + out := make([]TSIGKeyDTO, 0, len(keys)) + for _, k := range keys { + out = append(out, tsigKeyDTO(k)) + } + return out, nil +} + +// CreateTSIGKey creates a key with a random secret. +func (s *Service) CreateTSIGKey(ctx context.Context, a *Actor, in TSIGKeyInput) (TSIGKeyDTO, error) { + if err := s.requireWritable(); err != nil { + return TSIGKeyDTO{}, err + } + name, err := normalizeKeyName(in.Name) + if err != nil { + return TSIGKeyDTO{}, err + } + alg := in.Algorithm + if alg == "" { + alg = DefaultTSIGAlgorithm + } + if _, ok := keyLengths[alg]; !ok { + return TSIGKeyDTO{}, invalidField("invalid_value", "algorithm") + } + desc, p := checkDescription(strings.TrimSpace(in.Description)) + if p != nil { + return TSIGKeyDTO{}, invalid(*p) + } + k := &store.TSIGKey{ID: agenthub.NewID(), Name: name, Algorithm: alg, Description: desc} + if k.Secret, err = s.newTSIGSecret(k.ID, alg); err != nil { + return TSIGKeyDTO{}, err + } + var out TSIGKeyDTO + err = s.inTx(ctx, func(tx *store.Tx) error { + if err := tx.InsertTSIGKey(ctx, k); err != nil { + return err + } + out = tsigKeyDTO(k) + return s.audit(ctx, tx, a, "tsig_key.create", "tsig_key", k.ID, k.Name, nil, out) + }) + return out, err +} + +// UpdateTSIGKey changes the description of a key. +func (s *Service) UpdateTSIGKey(ctx context.Context, a *Actor, id string, in TSIGKeyInput) (TSIGKeyDTO, error) { + if err := s.requireWritable(); err != nil { + return TSIGKeyDTO{}, err + } + if !ValidUUID(id) { + return TSIGKeyDTO{}, ErrNotFound + } + desc, p := checkDescription(strings.TrimSpace(in.Description)) + if p != nil { + return TSIGKeyDTO{}, invalid(*p) + } + var out TSIGKeyDTO + err := s.inTx(ctx, func(tx *store.Tx) error { + k, err := tx.GetTSIGKey(ctx, id, true) + if err != nil { + return err + } + if k.Version != in.Version { + return ErrVersionConflict + } + before := tsigKeyDTO(k) + k.Description = desc + if err := tx.UpdateTSIGKey(ctx, k); err != nil { + return err + } + out = tsigKeyDTO(k) + return s.audit(ctx, tx, a, "tsig_key.update", "tsig_key", k.ID, k.Name, before, out) + }) + return out, err +} + +// DeleteTSIGKey removes a key that no zone uses and that is no transfer +// key. +func (s *Service) DeleteTSIGKey(ctx context.Context, a *Actor, id string) error { + if err := s.requireWritable(); err != nil { + return err + } + if !ValidUUID(id) { + return ErrNotFound + } + return s.inTx(ctx, func(tx *store.Tx) error { + k, err := tx.GetTSIGKey(ctx, id, true) + if err != nil { + return err + } + if k.ClusterID != nil { + return ErrKeyOwnedByCluster + } + if k.ZoneCount > 0 { + e := *ErrKeyInUse + e.Params = map[string]string{"zones": itoa(k.ZoneCount)} + return &e + } + if err := tx.DeleteTSIGKey(ctx, id); err != nil { + return err + } + return s.audit(ctx, tx, a, "tsig_key.delete", "tsig_key", k.ID, k.Name, tsigKeyDTO(k), nil) + }) +} + +// GetTSIGKeySecret returns the secret of a key. Every call is audited +// (without the secret); it works in read-only mode, since it changes +// nothing. +func (s *Service) GetTSIGKeySecret(ctx context.Context, a *Actor, id string) (TSIGKeySecretDTO, error) { + if !ValidUUID(id) { + return TSIGKeySecretDTO{}, ErrNotFound + } + var out TSIGKeySecretDTO + err := s.inTx(ctx, func(tx *store.Tx) error { + k, err := tx.GetTSIGKey(ctx, id, false) + if err != nil { + return err + } + secret, err := s.tsigSecret(k) + if err != nil { + return err + } + out = TSIGKeySecretDTO{Name: k.Name, Algorithm: k.Algorithm, Secret: secret} + return s.audit(ctx, tx, a, "tsig_key.read_secret", "tsig_key", k.ID, k.Name, nil, nil) + }) + return out, err +} + +// RotateTSIGKey replaces the secret of a key with a new random one. +func (s *Service) RotateTSIGKey(ctx context.Context, a *Actor, id string) (TSIGKeyDTO, error) { + if err := s.requireWritable(); err != nil { + return TSIGKeyDTO{}, err + } + if !ValidUUID(id) { + return TSIGKeyDTO{}, ErrNotFound + } + var out TSIGKeyDTO + err := s.inTx(ctx, func(tx *store.Tx) error { + k, err := tx.GetTSIGKey(ctx, id, true) + if err != nil { + return err + } + before := tsigKeyDTO(k) + if k.Secret, err = s.newTSIGSecret(k.ID, k.Algorithm); err != nil { + return err + } + if err := tx.RotateTSIGKey(ctx, k, s.now().UTC().Truncate(time.Microsecond)); err != nil { + return err + } + out = tsigKeyDTO(k) + return s.audit(ctx, tx, a, "tsig_key.rotate", "tsig_key", k.ID, k.Name, before, out) + }) + return out, err +} diff --git a/server/internal/service/zones.go b/server/internal/service/zones.go new file mode 100644 index 0000000..8de7701 --- /dev/null +++ b/server/internal/service/zones.go @@ -0,0 +1,643 @@ +// SPDX-License-Identifier: LicenseRef-DDI-Proprietary + +package service + +import ( + "context" + "net/netip" + "slices" + "strconv" + "strings" + "time" + + "git.beugsiud.de/beug.k/DDI/server/internal/configgen" + "git.beugsiud.de/beug.k/DDI/server/internal/ipam" + "git.beugsiud.de/beug.k/DDI/server/internal/store" +) + +// Explicit zones and their records (ADR 0023). + +// Errors of the zone operations. +var ( + ErrZoneNotEmpty = newError(KindConflict, "zone_not_empty", "the zone still has records") +) + +// ZoneDTO is an explicit zone. +type ZoneDTO struct { + ID string `json:"id"` + Name string `json:"name"` + Kind string `json:"kind"` + Mode string `json:"mode"` + ClusterID *string `json:"cluster_id"` + DefaultTTL *int `json:"default_ttl"` + AllowUpdate []netip.Prefix `json:"allow_update"` + UpdateKeyIDs []string `json:"update_key_ids"` + Description string `json:"description"` + RecordCount int `json:"record_count"` + Version int `json:"version"` + CreatedAt time.Time `json:"created_at"` + UpdatedAt time.Time `json:"updated_at"` +} + +func zoneDTO(z *store.Zone) ZoneDTO { + return ZoneDTO{ID: z.ID, Name: z.Name, Kind: z.Kind, Mode: z.Mode, ClusterID: z.ClusterID, DefaultTTL: z.DefaultTTL, + AllowUpdate: orEmpty(z.AllowUpdate), UpdateKeyIDs: orEmpty(z.UpdateKeyIDs), Description: z.Description, + RecordCount: z.RecordCount, Version: z.Version, CreatedAt: z.CreatedAt, UpdatedAt: z.UpdatedAt} +} + +// ZoneInput creates or replaces a zone. Name and Kind are used on create +// only; Kind defaults to the kind the name implies. +type ZoneInput struct { + Name string + Kind string + Mode string + ClusterID *string + DefaultTTL *int + AllowUpdate []string + UpdateKeyIDs []string + Description string + Version int +} + +// reverseZoneName reports whether name is a reverse zone name. +func reverseZoneName(name string) bool { + return strings.HasSuffix(name, ".in-addr.arpa") || strings.HasSuffix(name, ".ip6.arpa") +} + +// validReverseLabels checks the labels in front of in-addr.arpa (at most +// four octets 0-255) or ip6.arpa (at most 32 hex nibbles). +func validReverseLabels(name string) bool { + if rest, ok := strings.CutSuffix(name, ".in-addr.arpa"); ok { + labels := strings.Split(rest, ".") + if len(labels) > 4 { + return false + } + for _, l := range labels { + n, err := strconv.Atoi(l) + if err != nil || n < 0 || n > 255 || strconv.Itoa(n) != l { + return false + } + } + return true + } + rest := strings.TrimSuffix(name, ".ip6.arpa") + labels := strings.Split(rest, ".") + if len(labels) > 32 { + return false + } + for _, l := range labels { + if len(l) != 1 || !strings.Contains("0123456789abcdef", l) { + return false + } + } + return true +} + +// normalizeZoneName validates the name and kind of a new zone. +func normalizeZoneName(in *ZoneInput) (name, kind string, err error) { + name, nerr := ipam.NormalizeDomain(in.Name) + if nerr != nil { + return "", "", invalidField("invalid_domain", "name") + } + reverse := reverseZoneName(name) + kind = in.Kind + if kind == "" { + kind = string(configgen.ZoneForward) + if reverse { + kind = string(configgen.ZoneReverse) + } + } + switch kind { + case string(configgen.ZoneForward): + if name == "arpa" || strings.HasSuffix(name, ".arpa") { + return "", "", invalidField("zone_kind_mismatch", "kind") + } + case string(configgen.ZoneReverse): + if !reverse { + return "", "", invalidField("zone_kind_mismatch", "kind") + } + if !validReverseLabels(name) { + return "", "", invalidField("invalid_domain", "name") + } + default: + return "", "", invalidField("invalid_value", "kind") + } + return name, kind, nil +} + +// applyZone validates the mutable fields of in and writes them to z. The +// cluster and keys are checked against the database. +func (s *Service) applyZone(ctx context.Context, tx *store.Tx, in *ZoneInput, z *store.Zone) error { + var problems []Problem + add := func(code, field string) { problems = append(problems, Problem{Code: code, Field: field}) } + z.Mode = in.Mode + if z.Mode == "" { + z.Mode = string(configgen.ZoneManaged) + } + if z.Mode != string(configgen.ZoneManaged) && z.Mode != string(configgen.ZoneDynamic) { + add("invalid_value", "mode") + } + z.DefaultTTL = in.DefaultTTL + if z.DefaultTTL != nil && (*z.DefaultTTL < 1 || *z.DefaultTTL > maxTTL) { + add("invalid_ttl", "default_ttl") + } + var desc string + var p *Problem + if desc, p = checkDescription(strings.TrimSpace(in.Description)); p != nil { + problems = append(problems, *p) + } + z.Description = desc + z.AllowUpdate, z.UpdateKeyIDs = nil, nil + if z.Mode == string(configgen.ZoneDynamic) { + for _, c := range in.AllowUpdate { + pf, err := netip.ParsePrefix(strings.TrimSpace(c)) + if err != nil || pf.Masked() != pf || pf.Addr().Zone() != "" || pf.Addr().Is4In6() { + add("invalid_cidr", "allow_update") + break + } + if !slices.Contains(z.AllowUpdate, pf) { + z.AllowUpdate = append(z.AllowUpdate, pf) + } + } + for _, id := range in.UpdateKeyIDs { + if !slices.Contains(z.UpdateKeyIDs, id) { + z.UpdateKeyIDs = append(z.UpdateKeyIDs, id) + } + } + slices.Sort(z.UpdateKeyIDs) + if len(z.AllowUpdate) == 0 && len(z.UpdateKeyIDs) == 0 { + add("update_source_required", "allow_update") + } + } + if len(problems) > 0 { + return invalid(problems...) + } + z.ClusterID = nil + if in.ClusterID != nil && *in.ClusterID != "" { + if err := requireCluster(ctx, tx, *in.ClusterID, RoleDNS, "cluster_id"); err != nil { + return err + } + id := *in.ClusterID + z.ClusterID = &id + } + var valid []string + for _, id := range z.UpdateKeyIDs { + if ValidUUID(id) { + valid = append(valid, id) + } + } + keys, err := tx.TSIGKeysByID(ctx, valid) + if err != nil { + return err + } + for _, id := range z.UpdateKeyIDs { + if keys[id] == nil { + return invalid(Problem{Code: "not_found", Field: "update_key_ids", Params: map[string]string{"key_id": id}}) + } + } + return nil +} + +// ListZones returns the explicit zones sorted by name. +func (s *Service) ListZones(ctx context.Context) ([]ZoneDTO, error) { + list, err := s.st.Q().ListZones(ctx) + if err != nil { + return nil, translate(err) + } + out := make([]ZoneDTO, 0, len(list)) + for _, z := range list { + out = append(out, zoneDTO(z)) + } + return out, nil +} + +// GetZone returns one zone. +func (s *Service) GetZone(ctx context.Context, id string) (ZoneDTO, error) { + if !ValidUUID(id) { + return ZoneDTO{}, ErrNotFound + } + z, err := s.st.Q().GetZone(ctx, id, false) + if err != nil { + return ZoneDTO{}, translate(err) + } + return zoneDTO(z), nil +} + +// CreateZone creates an explicit zone. +func (s *Service) CreateZone(ctx context.Context, a *Actor, in ZoneInput) (ZoneDTO, error) { + if err := s.requireWritable(); err != nil { + return ZoneDTO{}, err + } + name, kind, err := normalizeZoneName(&in) + if err != nil { + return ZoneDTO{}, err + } + z := &store.Zone{Name: name, Kind: kind} + var out ZoneDTO + err = s.inTx(ctx, func(tx *store.Tx) error { + if err := s.applyZone(ctx, tx, &in, z); err != nil { + return err + } + if err := tx.InsertZone(ctx, z); err != nil { + return err + } + out = zoneDTO(z) + return s.audit(ctx, tx, a, "zone.create", "zone", z.ID, z.Name, nil, out) + }) + return out, err +} + +// UpdateZone replaces the mutable fields of a zone. +func (s *Service) UpdateZone(ctx context.Context, a *Actor, id string, in ZoneInput) (ZoneDTO, error) { + if err := s.requireWritable(); err != nil { + return ZoneDTO{}, err + } + if !ValidUUID(id) { + return ZoneDTO{}, ErrNotFound + } + var out ZoneDTO + err := s.inTx(ctx, func(tx *store.Tx) error { + z, err := tx.GetZone(ctx, id, true) + if err != nil { + return err + } + if z.Version != in.Version { + return ErrVersionConflict + } + before := zoneDTO(z) + if err := s.applyZone(ctx, tx, &in, z); err != nil { + return err + } + if err := tx.UpdateZone(ctx, z); err != nil { + return err + } + out = zoneDTO(z) + return s.audit(ctx, tx, a, "zone.update", "zone", z.ID, z.Name, before, out) + }) + return out, err +} + +// DeleteZone removes a zone. With records it requires force. +func (s *Service) DeleteZone(ctx context.Context, a *Actor, id string, force bool) error { + if err := s.requireWritable(); err != nil { + return err + } + if !ValidUUID(id) { + return ErrNotFound + } + return s.inTx(ctx, func(tx *store.Tx) error { + z, err := tx.GetZone(ctx, id, true) + if err != nil { + return err + } + if z.RecordCount > 0 && !force { + e := *ErrZoneNotEmpty + e.Params = map[string]string{"count": itoa(z.RecordCount)} + return &e + } + if err := tx.DeleteZone(ctx, id); err != nil { + return err + } + return s.audit(ctx, tx, a, "zone.delete", "zone", z.ID, z.Name, zoneDTO(z), + map[string]int{"deleted_records": z.RecordCount}) + }) +} + +// DerivedZoneDTO is a zone derived from network data. +type DerivedZoneDTO struct { + Name string `json:"name"` + Kind string `json:"kind"` + NetworkIDs []string `json:"network_ids"` +} + +// DerivedZones lists the zones derived from networks that no explicit zone +// covers. +func (s *Service) DerivedZones(ctx context.Context) ([]DerivedZoneDTO, error) { + src, err := s.loadSource(ctx, previewSerial) + if err != nil { + return nil, err + } + out := []DerivedZoneDTO{} + for _, z := range derivedZones(src) { + out = append(out, DerivedZoneDTO{Name: z.Name, Kind: string(z.Kind), NetworkIDs: orEmpty(z.NetworkIDs)}) + } + return out, nil +} + +// --- Records --------------------------------------------------------------- + +// RecordDTO is a record of an explicit zone. +type RecordDTO struct { + ID string `json:"id"` + ZoneID string `json:"zone_id"` + Name string `json:"name"` + Type string `json:"type"` + TTL *int `json:"ttl"` + Address string `json:"address"` + Target string `json:"target"` + Preference int `json:"preference"` + Priority int `json:"priority"` + Weight int `json:"weight"` + Port int `json:"port"` + Text []string `json:"text"` + Description string `json:"description"` + Version int `json:"version"` + CreatedAt time.Time `json:"created_at"` + UpdatedAt time.Time `json:"updated_at"` +} + +func recordDTO(r *store.Record) RecordDTO { + d := RecordDTO{ID: r.ID, ZoneID: r.ZoneID, Name: r.Name, Type: r.Type, TTL: r.TTL, Target: r.Target, + Preference: r.Preference, Priority: r.Priority, Weight: r.Weight, Port: r.Port, Text: orEmpty(r.Text), + Description: r.Description, Version: r.Version, CreatedAt: r.CreatedAt, UpdatedAt: r.UpdatedAt} + if r.Address != nil { + d.Address = r.Address.String() + } + return d +} + +// RecordInput creates or replaces a record. Fields that do not belong to +// the type must be empty or 0. +type RecordInput struct { + Name string + Type string + TTL *int + Address string + Target string + Preference int + Priority int + Weight int + Port int + Text []string + Description string + Version int +} + +// DNS limits shared with the renderer (bindgen). +const ( + maxTTL = 1<<31 - 1 + maxDNSName = 253 + maxDNSLabel = 63 + maxRDataLen = 65535 + maxCharString = 255 +) + +// targetTypes are the record types with a target name. +var targetTypes = map[configgen.RRType]bool{ + configgen.TypeCNAME: true, configgen.TypeMX: true, configgen.TypeSRV: true, configgen.TypeNS: true, configgen.TypePTR: true, +} + +// validDNSLabel reports whether l is a lower-case LDH label, with +// underscore optionally prefixed by '_' (as the renderer accepts it). +func validDNSLabel(l string, underscore bool) bool { + if underscore && strings.HasPrefix(l, "_") { + l = l[1:] + } + if l == "" || len(l) > maxDNSLabel || l[0] == '-' || l[len(l)-1] == '-' { + return false + } + for i := 0; i < len(l); i++ { + c := l[i] + if (c < 'a' || c > 'z') && (c < '0' || c > '9') && c != '-' { + return false + } + } + return true +} + +func validDNSName(name string, underscore bool) bool { + if name == "" || len(name) > maxDNSName { + return false + } + for _, l := range strings.Split(name, ".") { + if !validDNSLabel(l, underscore) { + return false + } + } + return true +} + +// txtWireLength is the RDATA length of TXT strings split into 255-byte +// character-strings. +func txtWireLength(texts []string) int { + n := 0 + for _, s := range texts { + chunks := (len(s) + maxCharString - 1) / maxCharString + if chunks == 0 { + chunks = 1 + } + n += chunks + len(s) + } + return n +} + +// normalizeRecord validates in for zone (the renderer's rules) and returns +// the record to store. +func normalizeRecord(in *RecordInput, zone string) (*store.Record, error) { + var problems []Problem + add := func(code, field string) { problems = append(problems, Problem{Code: code, Field: field}) } + r := &store.Record{Type: in.Type, TTL: in.TTL} + + name := strings.ToLower(strings.TrimSpace(in.Name)) + switch { + case name == "@": + case name == "" || strings.HasSuffix(name, ".") || name == zone || strings.HasSuffix(name, "."+zone) || + !validDNSName(name, true) || len(name)+1+len(zone) > maxDNSName: + add("invalid_label", "name") + } + r.Name = name + if r.TTL != nil && (*r.TTL < 1 || *r.TTL > maxTTL) { + add("invalid_ttl", "ttl") + } + if desc, p := checkDescription(strings.TrimSpace(in.Description)); p != nil { + problems = append(problems, *p) + } else { + r.Description = desc + } + + typ := configgen.RRType(in.Type) + hasTarget := targetTypes[typ] + notAllowed := func(field string, set bool) { + if set { + add("field_not_allowed", field) + } + } + notAllowed("address", strings.TrimSpace(in.Address) != "" && typ != configgen.TypeA && typ != configgen.TypeAAAA) + notAllowed("target", strings.TrimSpace(in.Target) != "" && !hasTarget) + notAllowed("preference", in.Preference != 0 && typ != configgen.TypeMX) + notAllowed("priority", in.Priority != 0 && typ != configgen.TypeSRV) + notAllowed("weight", in.Weight != 0 && typ != configgen.TypeSRV) + notAllowed("port", in.Port != 0 && typ != configgen.TypeSRV) + notAllowed("text", len(in.Text) > 0 && typ != configgen.TypeTXT) + for field, v := range map[string]int{"preference": in.Preference, "priority": in.Priority, "weight": in.Weight, "port": in.Port} { + if v < 0 || v > 65535 { + add("out_of_range", field) + } + } + + switch typ { + case configgen.TypeA, configgen.TypeAAAA: + ip, err := netip.ParseAddr(strings.TrimSpace(in.Address)) + switch { + case err != nil || ip.Zone() != "" || ip.Is4In6(): + add("invalid_ip", "address") + case typ == configgen.TypeA && !ip.Is4(), typ == configgen.TypeAAAA && !ip.Is6(): + add("ip_family", "address") + default: + r.Address = &ip + } + case configgen.TypeCNAME, configgen.TypeMX, configgen.TypeSRV, configgen.TypeNS, configgen.TypePTR: + target := strings.TrimSuffix(strings.ToLower(strings.TrimSpace(in.Target)), ".") + // CNAME targets may be service names (DKIM, ACME delegation). + if !validDNSName(target, typ == configgen.TypeCNAME) { + add("invalid_domain", "target") + } + if typ == configgen.TypeCNAME && name == "@" { + add("cname_at_apex", "name") + } + r.Target = target + r.Preference, r.Priority, r.Weight, r.Port = in.Preference, in.Priority, in.Weight, in.Port + case configgen.TypeTXT: + if len(in.Text) == 0 || txtWireLength(in.Text) > maxRDataLen { + add("invalid_txt", "text") + } + r.Text = in.Text + default: + add("invalid_value", "type") + } + if len(problems) > 0 { + return nil, invalid(problems...) + } + return r, nil +} + +// ListRecords returns the records of a zone. +func (s *Service) ListRecords(ctx context.Context, zoneID string) ([]RecordDTO, error) { + if !ValidUUID(zoneID) { + return nil, ErrNotFound + } + q := s.st.Q() + if _, err := q.GetZone(ctx, zoneID, false); err != nil { + return nil, translate(err) + } + list, err := q.ListRecords(ctx, zoneID) + if err != nil { + return nil, translate(err) + } + out := make([]RecordDTO, 0, len(list)) + for _, r := range list { + out = append(out, recordDTO(r)) + } + return out, nil +} + +// recordAuditName names a record in the audit log ("www.example.test A"). +func recordAuditName(r *store.Record, zone string) string { + owner := zone + if r.Name != "@" { + owner = r.Name + "." + zone + } + return owner + " " + r.Type +} + +// CreateRecord adds a record to a zone. +func (s *Service) CreateRecord(ctx context.Context, a *Actor, zoneID string, in RecordInput) (RecordDTO, error) { + if err := s.requireWritable(); err != nil { + return RecordDTO{}, err + } + if !ValidUUID(zoneID) { + return RecordDTO{}, ErrNotFound + } + var out RecordDTO + err := s.inTx(ctx, func(tx *store.Tx) error { + z, err := tx.GetZone(ctx, zoneID, true) + if err != nil { + return err + } + r, err := normalizeRecord(&in, z.Name) + if err != nil { + return err + } + r.ZoneID = z.ID + if err := tx.InsertRecord(ctx, r); err != nil { + return err + } + out = recordDTO(r) + return s.audit(ctx, tx, a, "record.create", "record", r.ID, recordAuditName(r, z.Name), nil, out) + }) + return out, err +} + +// UpdateRecord replaces a record. +func (s *Service) UpdateRecord(ctx context.Context, a *Actor, zoneID, id string, in RecordInput) (RecordDTO, error) { + if err := s.requireWritable(); err != nil { + return RecordDTO{}, err + } + if !ValidUUID(zoneID) || !ValidUUID(id) { + return RecordDTO{}, ErrNotFound + } + var out RecordDTO + err := s.inTx(ctx, func(tx *store.Tx) error { + z, err := tx.GetZone(ctx, zoneID, true) + if err != nil { + return err + } + cur, err := tx.GetRecord(ctx, zoneID, id, true) + if err != nil { + return err + } + if cur.Version != in.Version { + return ErrVersionConflict + } + r, err := normalizeRecord(&in, z.Name) + if err != nil { + return err + } + r.ID, r.ZoneID, r.Version, r.CreatedAt = cur.ID, cur.ZoneID, cur.Version, cur.CreatedAt + if err := tx.UpdateRecord(ctx, r); err != nil { + return err + } + out = recordDTO(r) + return s.audit(ctx, tx, a, "record.update", "record", r.ID, recordAuditName(r, z.Name), recordDTO(cur), out) + }) + return out, err +} + +// DeleteRecord removes a record. +func (s *Service) DeleteRecord(ctx context.Context, a *Actor, zoneID, id string) error { + if err := s.requireWritable(); err != nil { + return err + } + if !ValidUUID(zoneID) || !ValidUUID(id) { + return ErrNotFound + } + return s.inTx(ctx, func(tx *store.Tx) error { + z, err := tx.GetZone(ctx, zoneID, true) + if err != nil { + return err + } + cur, err := tx.GetRecord(ctx, zoneID, id, true) + if err != nil { + return err + } + if err := tx.DeleteRecord(ctx, zoneID, id); err != nil { + return err + } + return s.audit(ctx, tx, a, "record.delete", "record", cur.ID, recordAuditName(cur, z.Name), recordDTO(cur), nil) + }) +} + +// recordSource converts a stored record into generator input. +func recordSource(r *store.Record) configgen.RecordSource { + rec := configgen.Record{Name: r.Name, Type: configgen.RRType(r.Type), Target: r.Target, + Preference: uint16(r.Preference), Priority: uint16(r.Priority), //nolint:gosec // 0..65535 by constraint + Weight: uint16(r.Weight), Port: uint16(r.Port), //nolint:gosec // 0..65535 by constraint + Text: r.Text} + if r.TTL != nil { + rec.TTL = uint32(*r.TTL) //nolint:gosec // 1..2^31-1 by constraint + } + if r.Address != nil { + rec.Addr = *r.Address + } + return configgen.RecordSource{ZoneID: r.ZoneID, Record: rec} +} diff --git a/server/internal/store/clusters.go b/server/internal/store/clusters.go new file mode 100644 index 0000000..8f5b7fe --- /dev/null +++ b/server/internal/store/clusters.go @@ -0,0 +1,369 @@ +// SPDX-License-Identifier: LicenseRef-DDI-Proprietary + +package store + +import ( + "context" + "encoding/json" + "errors" + "net/netip" + "time" + + "github.com/jackc/pgx/v5" +) + +// TSIGKey is a row of tsig_keys. Secret is sealed with the master key. +type TSIGKey struct { + ID string + Name string + Algorithm string + Secret []byte + Description string + // ClusterID is set for the transfer key of a DNS cluster. + ClusterID *string + Version int + CreatedAt time.Time + RotatedAt time.Time + // ZoneCount is the number of zones that accept updates with the key + // (filled by ListTSIGKeys and GetTSIGKey). + ZoneCount int +} + +const tsigKeyColumns = `k.id, k.name, k.algorithm, k.secret, k.description, k.cluster_id, k.version, k.created_at, k.rotated_at, + (SELECT count(*) FROM zone_update_keys u WHERE u.key_id = k.id)` + +func scanTSIGKey(row pgx.Row) (*TSIGKey, error) { + var k TSIGKey + if err := row.Scan(&k.ID, &k.Name, &k.Algorithm, &k.Secret, &k.Description, &k.ClusterID, &k.Version, + &k.CreatedAt, &k.RotatedAt, &k.ZoneCount); err != nil { + return nil, mapErr(err) + } + return &k, nil +} + +// ListTSIGKeys returns all keys ordered by name. +func (t *Tx) ListTSIGKeys(ctx context.Context) ([]*TSIGKey, error) { + rows, err := t.q.Query(ctx, `SELECT `+tsigKeyColumns+` FROM tsig_keys k ORDER BY k.name`) + if err != nil { + return nil, mapErr(err) + } + defer rows.Close() + var out []*TSIGKey + for rows.Next() { + k, err := scanTSIGKey(rows) + if err != nil { + return nil, err + } + out = append(out, k) + } + return out, mapErr(rows.Err()) +} + +// GetTSIGKey returns one key; forUpdate locks the row. +func (t *Tx) GetTSIGKey(ctx context.Context, id string, forUpdate bool) (*TSIGKey, error) { + sql := `SELECT ` + tsigKeyColumns + ` FROM tsig_keys k WHERE k.id = $1` + if forUpdate { + sql += ` FOR UPDATE OF k` + } + return scanTSIGKey(t.q.QueryRow(ctx, sql, id)) +} + +// InsertTSIGKey stores a key whose ID and sealed secret are set; version +// and times are filled in. +func (t *Tx) InsertTSIGKey(ctx context.Context, k *TSIGKey) error { + return mapErr(t.q.QueryRow(ctx, `INSERT INTO tsig_keys (id, name, algorithm, secret, description, cluster_id) + VALUES ($1, $2, $3, $4, $5, $6) RETURNING version, created_at, rotated_at`, + k.ID, k.Name, k.Algorithm, k.Secret, k.Description, k.ClusterID).Scan(&k.Version, &k.CreatedAt, &k.RotatedAt)) +} + +// UpdateTSIGKey writes the description if the stored version equals +// k.Version. +func (t *Tx) UpdateTSIGKey(ctx context.Context, k *TSIGKey) error { + err := t.q.QueryRow(ctx, `UPDATE tsig_keys SET description = $3, version = version + 1 + WHERE id = $1 AND version = $2 RETURNING version`, k.ID, k.Version, k.Description).Scan(&k.Version) + if err = mapErr(err); errors.Is(err, ErrNotFound) { + return ErrConflict + } + return err +} + +// RotateTSIGKey replaces the sealed secret. +func (t *Tx) RotateTSIGKey(ctx context.Context, k *TSIGKey, at time.Time) error { + return mapErr(t.q.QueryRow(ctx, `UPDATE tsig_keys SET secret = $2, rotated_at = $3, version = version + 1 + WHERE id = $1 RETURNING version, rotated_at`, k.ID, k.Secret, at).Scan(&k.Version, &k.RotatedAt)) +} + +// DeleteTSIGKey removes a key. +func (t *Tx) DeleteTSIGKey(ctx context.Context, id string) error { + tag, err := t.q.Exec(ctx, `DELETE FROM tsig_keys WHERE id = $1`, id) + if err != nil { + return mapErr(err) + } + if tag.RowsAffected() == 0 { + return ErrNotFound + } + return nil +} + +// TSIGKeysByID returns the keys with the given IDs (unknown IDs are +// missing from the map). +func (t *Tx) TSIGKeysByID(ctx context.Context, ids []string) (map[string]*TSIGKey, error) { + out := map[string]*TSIGKey{} + if len(ids) == 0 { + return out, nil + } + rows, err := t.q.Query(ctx, `SELECT `+tsigKeyColumns+` FROM tsig_keys k WHERE k.id = ANY($1::uuid[])`, ids) + if err != nil { + return nil, mapErr(err) + } + defer rows.Close() + for rows.Next() { + k, err := scanTSIGKey(rows) + if err != nil { + return nil, err + } + out[k.ID] = k + } + return out, mapErr(rows.Err()) +} + +// NameServerRow is a name server override of a DNS cluster (JSON column). +type NameServerRow struct { + Name string `json:"name"` + Addresses []string `json:"addresses"` +} + +// Cluster is a row of clusters with its members. HAPassword is sealed. +type Cluster struct { + ID string + Name string + Role string + Description string + PrimaryNS string + NameServers []NameServerRow + HAMode string + HAPort int + HAUser string + HAPassword []byte + TransferKeyID *string + // TransferKeyName is the name of the transfer key (read only). + TransferKeyName string + Version int + CreatedAt time.Time + UpdatedAt time.Time + Members []ClusterMember + // ZoneCount and NetworkCount are the zones and networks assigned to the + // cluster (read only). + ZoneCount int + NetworkCount int +} + +// ClusterMember is a row of cluster_members with the agent's hostname. +type ClusterMember struct { + AgentID string + MemberRole string + Address netip.Addr + // Hostname of the agent (read only). + Hostname string +} + +const clusterColumns = `c.id, c.name, c.role, c.description, c.primary_ns, c.name_servers, c.ha_mode, c.ha_port, c.ha_user, + c.ha_password, c.transfer_key_id, coalesce((SELECT k.name FROM tsig_keys k WHERE k.id = c.transfer_key_id), ''), + c.version, c.created_at, c.updated_at, + (SELECT count(*) FROM zones z WHERE z.cluster_id = c.id), + (SELECT count(*) FROM networks n WHERE n.dhcp_cluster_id = c.id)` + +func scanCluster(row pgx.Row) (*Cluster, error) { + var c Cluster + var ns []byte + if err := row.Scan(&c.ID, &c.Name, &c.Role, &c.Description, &c.PrimaryNS, &ns, &c.HAMode, &c.HAPort, &c.HAUser, + &c.HAPassword, &c.TransferKeyID, &c.TransferKeyName, &c.Version, &c.CreatedAt, &c.UpdatedAt, &c.ZoneCount, + &c.NetworkCount); err != nil { + return nil, mapErr(err) + } + if err := json.Unmarshal(ns, &c.NameServers); err != nil { + return nil, err + } + return &c, nil +} + +// loadMembers attaches the members (primary first, then by address) to the +// given clusters. +func (t *Tx) loadMembers(ctx context.Context, cs ...*Cluster) error { + if len(cs) == 0 { + return nil + } + ids := make([]string, len(cs)) + byID := make(map[string]*Cluster, len(cs)) + for i, c := range cs { + ids[i] = c.ID + byID[c.ID] = c + c.Members = []ClusterMember{} + } + rows, err := t.q.Query(ctx, `SELECT m.cluster_id, m.agent_id, m.member_role, m.address, a.hostname + FROM cluster_members m JOIN agents a ON a.id = m.agent_id + WHERE m.cluster_id = ANY($1::uuid[]) ORDER BY m.member_role, m.address, m.agent_id`, ids) + if err != nil { + return mapErr(err) + } + defer rows.Close() + for rows.Next() { + var cid string + var m ClusterMember + if err := rows.Scan(&cid, &m.AgentID, &m.MemberRole, &m.Address, &m.Hostname); err != nil { + return mapErr(err) + } + m.Address = m.Address.Unmap() + byID[cid].Members = append(byID[cid].Members, m) + } + return mapErr(rows.Err()) +} + +// ListClusters returns the clusters of a role ("" for all) ordered by role +// and name, with members. +func (t *Tx) ListClusters(ctx context.Context, role string) ([]*Cluster, error) { + sql := `SELECT ` + clusterColumns + ` FROM clusters c` + var args []any + if role != "" { + sql += ` WHERE c.role = $1::agent_role` + args = append(args, role) + } + rows, err := t.q.Query(ctx, sql+` ORDER BY c.role, lower(c.name), c.id`, args...) + if err != nil { + return nil, mapErr(err) + } + var out []*Cluster + for rows.Next() { + c, err := scanCluster(rows) + if err != nil { + rows.Close() + return nil, err + } + out = append(out, c) + } + rows.Close() + if err := rows.Err(); err != nil { + return nil, mapErr(err) + } + return out, t.loadMembers(ctx, out...) +} + +// GetCluster returns one cluster with members; forUpdate locks the row. +func (t *Tx) GetCluster(ctx context.Context, id string, forUpdate bool) (*Cluster, error) { + sql := `SELECT ` + clusterColumns + ` FROM clusters c WHERE c.id = $1` + if forUpdate { + sql += ` FOR UPDATE OF c` + } + c, err := scanCluster(t.q.QueryRow(ctx, sql, id)) + if err != nil { + return nil, err + } + return c, t.loadMembers(ctx, c) +} + +func nameServersJSON(ns []NameServerRow) ([]byte, error) { + if ns == nil { + ns = []NameServerRow{} + } + return json.Marshal(ns) +} + +// InsertCluster stores a cluster whose ID is set (members and transfer key +// are written separately); version and times are filled in. +func (t *Tx) InsertCluster(ctx context.Context, c *Cluster) error { + ns, err := nameServersJSON(c.NameServers) + if err != nil { + return err + } + return mapErr(t.q.QueryRow(ctx, `INSERT INTO clusters (id, name, role, description, primary_ns, name_servers, ha_mode, + ha_port, ha_user, ha_password) + VALUES ($1, $2, $3::agent_role, $4, $5, $6, $7, $8, $9, $10) RETURNING version, created_at, updated_at`, + c.ID, c.Name, c.Role, c.Description, c.PrimaryNS, ns, c.HAMode, c.HAPort, c.HAUser, c.HAPassword). + Scan(&c.Version, &c.CreatedAt, &c.UpdatedAt)) +} + +// UpdateCluster writes the mutable fields if the stored version equals +// c.Version. +func (t *Tx) UpdateCluster(ctx context.Context, c *Cluster) error { + ns, err := nameServersJSON(c.NameServers) + if err != nil { + return err + } + err = t.q.QueryRow(ctx, `UPDATE clusters SET name = $3, description = $4, primary_ns = $5, name_servers = $6, + ha_mode = $7, ha_port = $8, version = version + 1, updated_at = now() + WHERE id = $1 AND version = $2 RETURNING version, updated_at`, + c.ID, c.Version, c.Name, c.Description, c.PrimaryNS, ns, c.HAMode, c.HAPort).Scan(&c.Version, &c.UpdatedAt) + if err = mapErr(err); errors.Is(err, ErrNotFound) { + return ErrConflict + } + return err +} + +// SetClusterTransferKey records the transfer key of a DNS cluster. +func (t *Tx) SetClusterTransferKey(ctx context.Context, clusterID, keyID string) error { + _, err := t.q.Exec(ctx, `UPDATE clusters SET transfer_key_id = $2 WHERE id = $1`, clusterID, keyID) + return mapErr(err) +} + +// ReplaceClusterMembers replaces the member list of a cluster. +func (t *Tx) ReplaceClusterMembers(ctx context.Context, clusterID, role string, members []ClusterMember) error { + if _, err := t.q.Exec(ctx, `DELETE FROM cluster_members WHERE cluster_id = $1`, clusterID); err != nil { + return mapErr(err) + } + b := &pgx.Batch{} + for _, m := range members { + b.Queue(`INSERT INTO cluster_members (agent_id, cluster_id, role, member_role, address) + VALUES ($1, $2, $3::agent_role, $4::cluster_member_role, $5)`, m.AgentID, clusterID, role, m.MemberRole, m.Address) + } + return t.execBatch(ctx, b) +} + +// DeleteCluster removes a cluster with its members and transfer key. +func (t *Tx) DeleteCluster(ctx context.Context, id string) error { + tag, err := t.q.Exec(ctx, `DELETE FROM clusters WHERE id = $1`, id) + if err != nil { + return mapErr(err) + } + if tag.RowsAffected() == 0 { + return ErrNotFound + } + return nil +} + +// Membership is the cluster an agent belongs to. +type Membership struct { + ClusterID string + ClusterName string + ClusterRole string + MemberRole string + Address netip.Addr +} + +// Memberships returns the cluster membership of every agent in a cluster, +// keyed by agent ID. +func (t *Tx) Memberships(ctx context.Context) (map[string]Membership, error) { + rows, err := t.q.Query(ctx, `SELECT m.agent_id, c.id, c.name, c.role, m.member_role, m.address + FROM cluster_members m JOIN clusters c ON c.id = m.cluster_id`) + if err != nil { + return nil, mapErr(err) + } + defer rows.Close() + out := map[string]Membership{} + for rows.Next() { + var agentID string + var m Membership + if err := rows.Scan(&agentID, &m.ClusterID, &m.ClusterName, &m.ClusterRole, &m.MemberRole, &m.Address); err != nil { + return nil, mapErr(err) + } + m.Address = m.Address.Unmap() + out[agentID] = m + } + return out, mapErr(rows.Err()) +} + +// LockGeneratorSettings locks the generator settings row (if it exists), so +// that checks against the default clusters cannot race with a change. +func (t *Tx) LockGeneratorSettings(ctx context.Context) error { + _, err := t.q.Exec(ctx, `SELECT 1 FROM app_settings WHERE key = $1 FOR UPDATE`, SettingGenerator) + return mapErr(err) +} diff --git a/server/internal/store/clusters_test.go b/server/internal/store/clusters_test.go new file mode 100644 index 0000000..64bbfb3 --- /dev/null +++ b/server/internal/store/clusters_test.go @@ -0,0 +1,353 @@ +// SPDX-License-Identifier: LicenseRef-DDI-Proprietary + +package store_test + +import ( + "context" + "errors" + "net/netip" + "testing" + "time" + + "git.beugsiud.de/beug.k/DDI/server/internal/ipam" + "git.beugsiud.de/beug.k/DDI/server/internal/store" + "git.beugsiud.de/beug.k/DDI/server/internal/store/storetest" +) + +const ( + clusterDNS = "00000000-0000-4000-8000-0000000000c1" + clusterDHCP = "00000000-0000-4000-8000-0000000000c2" + keyID = "00000000-0000-4000-8000-0000000000e1" + agentC = "00000000-0000-4000-8000-00000000000c" + agentD = "00000000-0000-4000-8000-00000000000d" +) + +func mustCluster(t *testing.T, tx *store.Tx, id, name, role string) *store.Cluster { + t.Helper() + c := &store.Cluster{ID: id, Name: name, Role: role, HAMode: "hot-standby", HAPort: 8001} + if role == "dhcp" { + c.HAUser, c.HAPassword = "ha", []byte("sealed") + } + if err := tx.InsertCluster(context.Background(), c); err != nil { + t.Fatalf("insert cluster: %v", err) + } + return c +} + +func member(agent, role, addr string) store.ClusterMember { + return store.ClusterMember{AgentID: agent, MemberRole: role, Address: netip.MustParseAddr(addr)} +} + +func TestClusterConstraints(t *testing.T) { + st := storetest.New(t) + ctx := context.Background() + newAgent(t, st, agentA, "dns", "ns1", "a1") + newAgent(t, st, agentB, "dns", "ns2", "b1") + newAgent(t, st, agentC, "dhcp", "dhcp1", "c1") + newAgent(t, st, agentD, "dhcp", "dhcp2", "d1") + q := st.Q() + mustCluster(t, q, clusterDNS, "DNS", "dns") + mustCluster(t, q, clusterDHCP, "DHCP", "dhcp") + + nested := func(fn func(tx *store.Tx) error) error { + return st.InTx(ctx, fn) + } + replace := func(cluster, role string, ms ...store.ClusterMember) error { + return nested(func(tx *store.Tx) error { return tx.ReplaceClusterMembers(ctx, cluster, role, ms) }) + } + + // Duplicate name (case-insensitive). + if err := q.InsertCluster(ctx, &store.Cluster{ID: "00000000-0000-4000-8000-0000000000c9", Name: "dns", Role: "dns", + HAMode: "hot-standby", HAPort: 8001}); !store.IsConstraint(err, "clusters_name_unique") { + t.Errorf("duplicate name: %v", err) + } + // A DHCP agent in a DNS cluster: the composite foreign key refuses it. + if err := replace(clusterDNS, "dns", member(agentC, "primary", "192.0.2.1")); !store.IsConstraint(err, "cluster_members_agent_fk") { + t.Errorf("role mismatch: %v", err) + } + if err := replace(clusterDNS, "dns", member(agentA, "primary", "192.0.2.1"), member(agentB, "primary", "192.0.2.2")); !store.IsConstraint(err, "cluster_members_one_primary") { + t.Errorf("second primary: %v", err) + } + if err := replace(clusterDNS, "dns", member(agentA, "primary", "192.0.2.1"), member(agentB, "secondary", "192.0.2.1")); !store.IsConstraint(err, "cluster_members_address_unique") { + t.Errorf("same address: %v", err) + } + if err := replace(clusterDNS, "dns", member(agentA, "primary", "192.0.2.1"), member(agentB, "secondary", "192.0.2.2")); err != nil { + t.Fatal(err) + } + if err := replace(clusterDHCP, "dhcp", member(agentC, "secondary", "192.0.2.3"), member(agentD, "secondary", "192.0.2.4")); !store.IsConstraint(err, "cluster_members_dhcp_one_secondary") { + t.Errorf("two DHCP secondaries: %v", err) + } + if err := replace(clusterDHCP, "dhcp", member(agentC, "primary", "192.0.2.3"), member(agentD, "secondary", "192.0.2.4")); err != nil { + t.Fatal(err) + } + // An agent is in one cluster only. + if _, err := st.Pool().Exec(ctx, `INSERT INTO cluster_members (agent_id, cluster_id, role, member_role, address) + VALUES ($1, $2, 'dhcp', 'secondary', '192.0.2.9')`, agentC, clusterDHCP); err == nil { + t.Error("agent added twice") + } + + c, err := q.GetCluster(ctx, clusterDNS, false) + if err != nil || len(c.Members) != 2 || c.Members[0].AgentID != agentA || c.Members[0].MemberRole != "primary" || + c.Members[0].Hostname != "ns1" { + t.Fatalf("cluster %+v %v", c, err) + } + ms, err := q.Memberships(ctx) + if err != nil || len(ms) != 4 || ms[agentD].ClusterName != "DHCP" || ms[agentD].MemberRole != "secondary" { + t.Errorf("memberships %+v %v", ms, err) + } + + // Transfer key: owned by the cluster, deleted with it, not alone. + k := &store.TSIGKey{ID: keyID, Name: "transfer.dns", Algorithm: "hmac-sha256", Secret: []byte("sealed"), + ClusterID: &c.ID} + if err := q.InsertTSIGKey(ctx, k); err != nil { + t.Fatal(err) + } + if err := q.SetClusterTransferKey(ctx, clusterDNS, keyID); err != nil { + t.Fatal(err) + } + if err := q.SetClusterTransferKey(ctx, clusterDHCP, keyID); err == nil { + t.Error("transfer key of another cluster accepted") + } + if err := q.DeleteTSIGKey(ctx, keyID); !store.IsConstraint(err, "clusters_transfer_key_fk") { + t.Errorf("delete transfer key: %v", err) + } + if err := q.InsertTSIGKey(ctx, &store.TSIGKey{ID: "00000000-0000-4000-8000-0000000000e2", Name: "Bad Name", + Algorithm: "hmac-sha256", Secret: []byte("x")}); !store.IsConstraint(err, "tsig_keys_name_format") { + t.Errorf("bad key name: %v", err) + } + if err := q.InsertTSIGKey(ctx, &store.TSIGKey{ID: "00000000-0000-4000-8000-0000000000e3", Name: "k", + Algorithm: "hmac-md5", Secret: []byte("x")}); !store.IsConstraint(err, "tsig_keys_algorithm") { + t.Errorf("bad algorithm: %v", err) + } + + // Zones accept DNS clusters only; networks DHCP clusters only. + z := &store.Zone{Name: "example.test", Kind: "forward", Mode: "managed", ClusterID: ptr(clusterDHCP)} + if err := q.InsertZone(ctx, z); !store.IsConstraint(err, "zones_cluster_fk") { + t.Errorf("zone in DHCP cluster: %v", err) + } + z.ClusterID = ptr(clusterDNS) + if err := q.InsertZone(ctx, z); err != nil { + t.Fatal(err) + } + if err := q.InsertZone(ctx, &store.Zone{Name: "1.0.10.in-addr.arpa", Kind: "forward", Mode: "managed"}); !store.IsConstraint(err, "zones_kind_name") { + t.Errorf("kind mismatch: %v", err) + } + n := mustNetwork(t, q, "192.0.2.0/24", "Office", "office.test") + n.DHCPClusterID = clusterDNS + if err := q.UpdateNetwork(ctx, n); !store.IsConstraint(err, "networks_dhcp_cluster_fk") { + t.Errorf("network in DNS cluster: %v", err) + } + n.DHCPClusterID = clusterDHCP + if err := q.UpdateNetwork(ctx, n); err != nil { + t.Fatal(err) + } + if got, _ := q.GetNetwork(ctx, n.ID, false); got.DHCPClusterID != clusterDHCP { + t.Errorf("network cluster %q", got.DHCPClusterID) + } + if c, _ := q.GetCluster(ctx, clusterDHCP, false); c.NetworkCount != 1 { + t.Errorf("network count %d", c.NetworkCount) + } + + // Clusters in use cannot be deleted. + if err := q.DeleteCluster(ctx, clusterDNS); !store.IsConstraint(err, "zones_cluster_fk") { + t.Errorf("delete DNS cluster in use: %v", err) + } + if err := q.DeleteZone(ctx, z.ID); err != nil { + t.Fatal(err) + } + if err := q.DeleteCluster(ctx, clusterDNS); err != nil { + t.Fatalf("delete cluster: %v", err) + } + if _, err := q.GetTSIGKey(ctx, keyID, false); !errors.Is(err, store.ErrNotFound) { + t.Errorf("transfer key survived: %v", err) + } + if ms, _ := q.Memberships(ctx); len(ms) != 2 { + t.Errorf("members survived: %+v", ms) + } + // Deleting an agent removes its membership. + if err := q.DeleteAgent(ctx, agentD); err != nil { + t.Fatal(err) + } + if c, _ := q.GetCluster(ctx, clusterDHCP, false); len(c.Members) != 1 { + t.Errorf("members after agent delete: %+v", c.Members) + } +} + +func TestZonesAndRecords(t *testing.T) { + st := storetest.New(t) + ctx := context.Background() + q := st.Q() + k := &store.TSIGKey{ID: keyID, Name: "ad.update", Algorithm: "hmac-sha512", Secret: []byte("sealed")} + if err := q.InsertTSIGKey(ctx, k); err != nil { + t.Fatal(err) + } + ttl := 600 + z := &store.Zone{Name: "ad.example.test", Kind: "forward", Mode: "dynamic", DefaultTTL: &ttl, + AllowUpdate: []netip.Prefix{netip.MustParsePrefix("192.0.2.10/32")}, UpdateKeyIDs: []string{keyID}} + if err := q.InsertZone(ctx, z); err != nil { + t.Fatal(err) + } + if err := q.InsertZone(ctx, &store.Zone{Name: "ad.example.test", Kind: "forward", Mode: "managed"}); !store.IsConstraint(err, "zones_name_unique") { + t.Errorf("duplicate zone: %v", err) + } + if err := q.InsertZone(ctx, &store.Zone{Name: "m.example.test", Kind: "forward", Mode: "managed", + AllowUpdate: []netip.Prefix{netip.MustParsePrefix("192.0.2.0/24")}}); !store.IsConstraint(err, "zones_allow_update_dynamic") { + t.Errorf("allow_update on managed zone: %v", err) + } + got, err := q.GetZone(ctx, z.ID, false) + if err != nil || got.Mode != "dynamic" || *got.DefaultTTL != 600 || len(got.AllowUpdate) != 1 || len(got.UpdateKeyIDs) != 1 { + t.Fatalf("zone %+v %v", got, err) + } + if k, _ := q.GetTSIGKey(ctx, keyID, false); k.ZoneCount != 1 { + t.Errorf("zone count %d", k.ZoneCount) + } + if err := q.DeleteTSIGKey(ctx, keyID); !store.IsConstraint(err, "zone_update_keys_key_fk") { + t.Errorf("delete key in use: %v", err) + } + + addr := netip.MustParseAddr("192.0.2.10") + recs := []*store.Record{ + {ZoneID: z.ID, Name: "dc1", Type: "A", Address: &addr}, + {ZoneID: z.ID, Name: "_ldap._tcp", Type: "SRV", Target: "dc1.ad.example.test", Priority: 0, Weight: 100, Port: 389}, + {ZoneID: z.ID, Name: "@", Type: "TXT", Text: []string{"v=spf1 -all"}}, + {ZoneID: z.ID, Name: "@", Type: "MX", Target: "mail.example.test", Preference: 10}, + } + for _, r := range recs { + if err := q.InsertRecord(ctx, r); err != nil { + t.Fatalf("insert %+v: %v", r, err) + } + } + dup := *recs[2] + if err := q.InsertRecord(ctx, &dup); !store.IsConstraint(err, "dns_records_unique") { + t.Errorf("duplicate TXT: %v", err) + } + dup.Text = []string{"v=spf1", " -all"} + if err := q.InsertRecord(ctx, &dup); err != nil { + t.Errorf("different TXT strings: %v", err) + } + bad := []*store.Record{ + {ZoneID: z.ID, Name: "x", Type: "A"}, + {ZoneID: z.ID, Name: "x", Type: "AAAA", Address: &addr}, + {ZoneID: z.ID, Name: "x", Type: "CNAME"}, + {ZoneID: z.ID, Name: "x", Type: "A", Address: &addr, Preference: 1}, + {ZoneID: z.ID, Name: "x", Type: "TXT"}, + {ZoneID: z.ID, Name: "X", Type: "A", Address: &addr}, + } + for _, r := range bad { + if err := q.InsertRecord(ctx, r); err == nil { + t.Errorf("accepted %+v", r) + } + } + list, err := q.ListRecords(ctx, z.ID) + if err != nil || len(list) != 5 || list[0].Name != "@" || list[len(list)-1].Name != "dc1" || *list[len(list)-1].Address != addr { + t.Errorf("records %+v %v", list, err) + } + r := list[len(list)-1] + r.Name = "dc2" + if err := q.UpdateRecord(ctx, r); err != nil { + t.Fatal(err) + } + r.Version-- + if err := q.UpdateRecord(ctx, r); !errors.Is(err, store.ErrConflict) { + t.Errorf("stale update: %v", err) + } + if got, _ := q.GetZone(ctx, z.ID, false); got.RecordCount != 5 { + t.Errorf("record count %d", got.RecordCount) + } + if err := q.DeleteZone(ctx, z.ID); err != nil { + t.Fatal(err) + } + if all, _ := q.AllRecords(ctx); len(all) != 0 { + t.Errorf("records survived the zone: %d", len(all)) + } + if err := q.DeleteTSIGKey(ctx, keyID); err != nil { + t.Errorf("delete unused key: %v", err) + } +} + +func TestRevisionFilesPerTarget(t *testing.T) { + st := storetest.New(t) + ctx := context.Background() + q := st.Q() + files := []store.RevisionFile{ + {Role: "dns", Path: "ddi-suite.conf", Mode: 0o644, Content: []byte("default")}, + {Role: "dns", Target: agentB, Path: "ddi-suite.conf", Mode: 0o644, Content: []byte("b")}, + {Role: "dns", Target: agentB, Path: "ddi-suite-keys.conf", Mode: 0o640, Secret: true, Content: []byte("sealed")}, + {Role: "dns", Target: agentA, Path: "ddi-suite.conf", Mode: 0o644, Content: []byte("a")}, + {Role: "dhcp", Path: "kea.conf", Mode: 0o644, Content: []byte("{}")}, + } + if err := q.InsertRevision(ctx, &store.Revision{Number: 1, CreatedAt: time.Now(), CreatedByName: "admin"}, files); err != nil { + t.Fatal(err) + } + all, err := q.RevisionFiles(ctx, 1, "") + if err != nil || len(all) != 5 || all[0].Target != "" || all[1].Target != agentA || all[2].Target != agentB || + all[3].Path != "ddi-suite-keys.conf" || !all[3].Secret || all[4].Role != "dhcp" { + t.Fatalf("files %+v %v", all, err) + } + b, err := q.RevisionTargetFiles(ctx, 1, "dns", agentB) + if err != nil || len(b) != 2 || string(b[0].Content) != "b" { + t.Errorf("target files %+v %v", b, err) + } + if ok, _ := q.RevisionHasTarget(ctx, 1, "dhcp", agentB); ok { + t.Error("dhcp target of agent B") + } + if ok, _ := q.RevisionHasTarget(ctx, 1, "dns", agentA); !ok { + t.Error("dns target of agent A missing") + } + list, _ := q.ListRevisions(ctx, 10, 0) + if len(list) != 1 || list[0].DNSFiles != 4 || list[0].DHCPFiles != 1 { + t.Errorf("counts %+v", list[0]) + } + if err := q.InsertRevision(ctx, &store.Revision{Number: 2, CreatedAt: time.Now(), CreatedByName: "admin"}, + []store.RevisionFile{{Role: "dns", Target: "not-a-uuid", Path: "x", Content: []byte("x")}}); !store.IsConstraint(err, "config_revision_files_target") { + t.Errorf("bad target: %v", err) + } +} + +func TestLeaseAdoptionStore(t *testing.T) { + st := storetest.New(t) + ctx := context.Background() + q := st.Q() + n := mustNetwork(t, q, "192.0.2.0/24", "Office", "office.test") + existing := mustAddress(t, q, n, "192.0.2.3", ipam.TypeDynamic, "", "") + exp := time.Now().Add(time.Hour).UTC().Truncate(time.Microsecond) + at := time.Now().UTC().Truncate(time.Microsecond) + res, err := q.ApplyLeaseAdoption(ctx, + []ipam.Address{{NetworkID: n.ID, IP: netip.MustParseAddr("192.0.2.4"), MAC: "00:11:22:33:44:55", Hostname: "pc", + FQDN: "pc.office.test", LeaseState: "active", LeaseExpiresAt: &exp}, + // Exists already: skipped. + {NetworkID: n.ID, IP: netip.MustParseAddr("192.0.2.3"), LeaseState: "active"}}, + []store.LeaseUpdate{{AddressID: existing.ID, State: "active", Active: true, ExpiresAt: &exp, SetIdentity: true, + MAC: "00:11:22:33:44:66", Hostname: "laptop", FQDN: "laptop.office.test"}}, + map[string]string{existing.ID: n.ID}, at) + if err != nil || res.Created != 1 || res.Updated != 1 || !res.NetworkIDs[n.ID] { + t.Fatalf("adoption %+v %v", res, err) + } + got, _ := q.GetAddress(ctx, existing.ID, false) + if got.MAC != "00:11:22:33:44:66" || got.Hostname != "laptop" || got.LeaseState != "active" || got.Version != 2 || + got.LastSeen == nil || got.FirstSeen == nil { + t.Errorf("updated %+v", got.Address) + } + // Same data again: only the lease fields change, the version stays. + res, err = q.ApplyLeaseAdoption(ctx, nil, []store.LeaseUpdate{{AddressID: existing.ID, State: "active", Active: true, + SetIdentity: true, MAC: "00:11:22:33:44:66", Hostname: "laptop", FQDN: "laptop.office.test"}}, + map[string]string{existing.ID: n.ID}, at) + if err != nil || res.Updated != 1 { + t.Fatalf("second adoption %+v %v", res, err) + } + if got, _ := q.GetAddress(ctx, existing.ID, false); got.Version != 2 { + t.Errorf("version changed: %d", got.Version) + } + // Expired: only the state; unchanged state changes nothing. + upd := []store.LeaseUpdate{{AddressID: existing.ID, State: "expired"}} + if res, _ := q.ApplyLeaseAdoption(ctx, nil, upd, map[string]string{existing.ID: n.ID}, at); res.Updated != 1 { + t.Errorf("expire: %+v", res) + } + if res, _ := q.ApplyLeaseAdoption(ctx, nil, upd, map[string]string{existing.ID: n.ID}, at); res.Updated != 0 { + t.Errorf("expire again: %+v", res) + } + byIP, err := q.AddressesByIP(ctx, []netip.Addr{netip.MustParseAddr("192.0.2.4"), netip.MustParseAddr("192.0.2.99")}) + if err != nil || len(byIP) != 1 || byIP[0].Type != ipam.TypeDynamic || byIP[0].FQDN != "pc.office.test" { + t.Errorf("by IP %+v %v", byIP, err) + } +} diff --git a/server/internal/store/export_test.go b/server/internal/store/export_test.go new file mode 100644 index 0000000..1b05544 --- /dev/null +++ b/server/internal/store/export_test.go @@ -0,0 +1,17 @@ +// SPDX-License-Identifier: LicenseRef-DDI-Proprietary + +package store + +import "context" + +// MigrateDownTo rolls the schema back to version (tests of the Down +// sections). +func (s *Store) MigrateDownTo(ctx context.Context, version int64) error { + p, closeDB, err := s.newProvider() + if err != nil { + return err + } + defer closeDB() //nolint:errcheck // closing the wrapper does not close the pool + _, err = p.DownTo(ctx, version) + return err +} diff --git a/server/internal/store/leaseadopt.go b/server/internal/store/leaseadopt.go new file mode 100644 index 0000000..0c1fe81 --- /dev/null +++ b/server/internal/store/leaseadopt.go @@ -0,0 +1,126 @@ +// SPDX-License-Identifier: LicenseRef-DDI-Proprietary + +package store + +import ( + "context" + "net/netip" + "time" + + "github.com/jackc/pgx/v5" + + "git.beugsiud.de/beug.k/DDI/server/internal/ipam" +) + +// AddressesByIP returns the addresses with one of the given IPs. +func (t *Tx) AddressesByIP(ctx context.Context, ips []netip.Addr) ([]ipam.Address, error) { + if len(ips) == 0 { + return nil, nil + } + rows, err := t.q.Query(ctx, `SELECT `+addressColumns+` FROM addresses a WHERE a.ip = ANY($1::inet[])`, ips) + if err != nil { + return nil, mapErr(err) + } + defer rows.Close() + var out []ipam.Address + for rows.Next() { + a, err := scanAddress(rows) + if err != nil { + return nil, err + } + out = append(out, *a) + } + return out, mapErr(rows.Err()) +} + +// LeaseUpdate changes the lease fields of an existing address (ADR 0023). +type LeaseUpdate struct { + AddressID string + State string + // Active leases also set the expiry and the time the address was seen; + // other states change only the lease state. + Active bool + ExpiresAt *time.Time + // Dynamic entries take MAC and hostname from the lease; SetIdentity + // enables this (the update applies only while the entry is dynamic). + SetIdentity bool + MAC string + Hostname string + FQDN string +} + +// LeaseAdoption is the outcome of ApplyLeaseAdoption. +type LeaseAdoption struct { + Created int + Updated int + // NetworkIDs are the networks with created or changed addresses. + NetworkIDs map[string]bool +} + +// ApplyLeaseAdoption creates dynamic addresses for new leases and updates +// the lease fields of existing ones. Creations of IPs that got an address +// in the meantime are skipped. The lease fields are not user data: they do +// not change the version; a changed MAC or hostname of a dynamic entry does. +func (t *Tx) ApplyLeaseAdoption(ctx context.Context, creates []ipam.Address, updates []LeaseUpdate, + networkOf map[string]string, at time.Time, +) (*LeaseAdoption, error) { + res := &LeaseAdoption{NetworkIDs: map[string]bool{}} + if len(creates) == 0 && len(updates) == 0 { + return res, nil + } + b := &pgx.Batch{} + for _, a := range creates { + b.Queue(`INSERT INTO addresses (network_id, ip, type, mac, hostname, fqdn, first_seen, last_seen, lease_state, + lease_expires_at) + VALUES ($1, $2, 'dynamic', $3::macaddr, $4, $5, $6, $6, $7::lease_state, $8) + ON CONFLICT (network_id, ip) DO NOTHING`, + a.NetworkID, a.IP, nullMAC(a.MAC), nullString(a.Hostname), nullString(a.FQDN), at, a.LeaseState, a.LeaseExpiresAt) + } + for _, u := range updates { + if u.Active { + b.Queue(`UPDATE addresses SET lease_state = $2::lease_state, lease_expires_at = $3, last_seen = $4, + first_seen = coalesce(first_seen, $4) WHERE id = $1`, u.AddressID, u.State, u.ExpiresAt, at) + } else { + b.Queue(`UPDATE addresses SET lease_state = $2::lease_state WHERE id = $1 + AND lease_state IS DISTINCT FROM $2::lease_state`, u.AddressID, u.State) + } + if u.SetIdentity { + b.Queue(`UPDATE addresses SET mac = $2::macaddr, hostname = $3, fqdn = $4, version = version + 1, updated_at = now() + WHERE id = $1 AND type = 'dynamic' + AND (mac IS DISTINCT FROM $2::macaddr OR hostname IS DISTINCT FROM $3 OR fqdn IS DISTINCT FROM $4)`, + u.AddressID, nullMAC(u.MAC), nullString(u.Hostname), nullString(u.FQDN)) + } + } + br := t.q.SendBatch(ctx, b) + for _, a := range creates { + tag, err := br.Exec() + if err != nil { + _ = br.Close() + return nil, mapErr(err) + } + if tag.RowsAffected() > 0 { + res.Created++ + res.NetworkIDs[a.NetworkID] = true + } + } + for _, u := range updates { + changed := false + n := 1 + if u.SetIdentity { + n = 2 + } + for range n { + tag, err := br.Exec() + if err != nil { + _ = br.Close() + return nil, mapErr(err) + } + changed = changed || tag.RowsAffected() > 0 + } + if changed { + res.Updated++ + res.NetworkIDs[networkOf[u.AddressID]] = true + } + } + return res, mapErr(br.Close()) +} diff --git a/server/internal/store/migrations/00005_clusters_zones.sql b/server/internal/store/migrations/00005_clusters_zones.sql new file mode 100644 index 0000000..ca19272 --- /dev/null +++ b/server/internal/store/migrations/00005_clusters_zones.sql @@ -0,0 +1,233 @@ +-- SPDX-License-Identifier: LicenseRef-DDI-Proprietary +-- Clusters and zones (M5, ADR 0023): DNS/DHCP clusters with members, TSIG +-- keys, explicit zones with records, the DHCP cluster of a network and +-- revision files per target with encrypted secrets. + +-- +goose Up + +-- Composite foreign keys on (id, role) make sure that agents, clusters, +-- zones and networks of different roles are never combined. +ALTER TABLE agents ADD CONSTRAINT agents_id_role_unique UNIQUE (id, role); + +CREATE TABLE clusters ( + id uuid PRIMARY KEY, + name text NOT NULL, + role agent_role NOT NULL, + description text NOT NULL DEFAULT '', + -- DNS: overrides of the generator settings for the cluster's zones + -- ('' and [] = use the settings). + primary_ns text NOT NULL DEFAULT '', + name_servers jsonb NOT NULL DEFAULT '[]', + -- DHCP: Kea HA. The password is sealed with the master key; the + -- associated data binds it to the row (ADR 0015). + ha_mode text NOT NULL DEFAULT 'hot-standby', + ha_port integer NOT NULL DEFAULT 8001, + ha_user text NOT NULL DEFAULT '', + ha_password bytea, + -- DNS: the key that secures NOTIFY and zone transfers (set right after + -- the cluster is created; see the foreign key below). + transfer_key_id uuid, + version integer NOT NULL DEFAULT 1, + created_at timestamptz NOT NULL DEFAULT now(), + updated_at timestamptz NOT NULL DEFAULT now(), + CONSTRAINT clusters_id_role_unique UNIQUE (id, role), + CONSTRAINT clusters_name_not_blank CHECK (btrim(name) <> '' AND length(name) <= 100), + CONSTRAINT clusters_name_servers_array CHECK (jsonb_typeof(name_servers) = 'array'), + CONSTRAINT clusters_ha_mode CHECK (ha_mode IN ('hot-standby', 'load-balancing')), + CONSTRAINT clusters_ha_port CHECK (ha_port BETWEEN 1 AND 65535), + CONSTRAINT clusters_dns_only CHECK (role = 'dns' OR (primary_ns = '' AND name_servers = '[]' AND transfer_key_id IS NULL)), + CONSTRAINT clusters_dhcp_only CHECK ((role = 'dhcp') = (ha_password IS NOT NULL AND ha_user <> '')) +); +CREATE UNIQUE INDEX clusters_name_unique ON clusters (lower(name)); + +CREATE TYPE cluster_member_role AS ENUM ('primary', 'secondary'); + +CREATE TABLE cluster_members ( + -- An agent belongs to at most one cluster. + agent_id uuid PRIMARY KEY, + cluster_id uuid NOT NULL, + -- Role of the agent and the cluster: both foreign keys include it, so + -- the roles must match. + role agent_role NOT NULL, + member_role cluster_member_role NOT NULL, + -- Where the other members reach this one (DNS: port 53; DHCP: the HA + -- control socket). + address inet NOT NULL, + CONSTRAINT cluster_members_cluster_fk FOREIGN KEY (cluster_id, role) REFERENCES clusters (id, role) ON DELETE CASCADE, + CONSTRAINT cluster_members_agent_fk FOREIGN KEY (agent_id, role) REFERENCES agents (id, role) ON DELETE CASCADE, + CONSTRAINT cluster_members_host_address CHECK (masklen(address) = CASE family(address) WHEN 4 THEN 32 ELSE 128 END), + CONSTRAINT cluster_members_address_unique UNIQUE (cluster_id, address) +); +CREATE INDEX cluster_members_cluster ON cluster_members (cluster_id); +-- One primary per cluster; a DHCP cluster (HA pair) has at most one +-- secondary as well. +CREATE UNIQUE INDEX cluster_members_one_primary ON cluster_members (cluster_id) WHERE member_role = 'primary'; +CREATE UNIQUE INDEX cluster_members_dhcp_one_secondary ON cluster_members (cluster_id) + WHERE member_role = 'secondary' AND role = 'dhcp'; + +CREATE TABLE tsig_keys ( + id uuid PRIMARY KEY, + -- Lower-case domain name without trailing dot. + name text NOT NULL, + algorithm text NOT NULL, + -- Random secret of the hash length, sealed with the master key; the + -- associated data binds it to the row (ADR 0015). + secret bytea NOT NULL, + description text NOT NULL DEFAULT '', + -- Set for the transfer key of a DNS cluster; deleted with it. + cluster_id uuid REFERENCES clusters (id) ON DELETE CASCADE, + version integer NOT NULL DEFAULT 1, + created_at timestamptz NOT NULL DEFAULT now(), + rotated_at timestamptz NOT NULL DEFAULT now(), + CONSTRAINT tsig_keys_name_format CHECK (length(name) <= 253 + AND name ~ '^[a-z0-9]([a-z0-9-]*[a-z0-9])?(\.[a-z0-9]([a-z0-9-]*[a-z0-9])?)*$'), + CONSTRAINT tsig_keys_algorithm CHECK (algorithm IN ('hmac-sha256', 'hmac-sha384', 'hmac-sha512')), + CONSTRAINT tsig_keys_id_cluster_unique UNIQUE (id, cluster_id) +); +CREATE UNIQUE INDEX tsig_keys_name_unique ON tsig_keys (name); +-- One transfer key per cluster. +CREATE UNIQUE INDEX tsig_keys_cluster_unique ON tsig_keys (cluster_id) WHERE cluster_id IS NOT NULL; + +-- The transfer key of a cluster must belong to that cluster. The check runs +-- at the end of each statement, so deleting a cluster (which cascades to +-- its key) works, while deleting the key alone fails. +ALTER TABLE clusters ADD CONSTRAINT clusters_transfer_key_fk + FOREIGN KEY (transfer_key_id, id) REFERENCES tsig_keys (id, cluster_id); + +CREATE TYPE zone_kind AS ENUM ('forward', 'reverse'); +CREATE TYPE zone_mode AS ENUM ('managed', 'dynamic'); + +CREATE TABLE zones ( + id uuid PRIMARY KEY DEFAULT gen_random_uuid(), + -- Zone apex: lower case, no trailing dot. + name text NOT NULL, + kind zone_kind NOT NULL, + mode zone_mode NOT NULL DEFAULT 'managed', + -- Serving DNS cluster; NULL = generator setting dns.default_cluster_id. + -- cluster_role is fixed: the foreign key only accepts DNS clusters. + cluster_id uuid, + cluster_role agent_role NOT NULL DEFAULT 'dns', + default_ttl integer, + -- Dynamic zones: sources allowed to send RFC 2136 updates. + allow_update cidr[] NOT NULL DEFAULT '{}', + description text NOT NULL DEFAULT '', + version integer NOT NULL DEFAULT 1, + created_at timestamptz NOT NULL DEFAULT now(), + updated_at timestamptz NOT NULL DEFAULT now(), + CONSTRAINT zones_cluster_fk FOREIGN KEY (cluster_id, cluster_role) REFERENCES clusters (id, role), + CONSTRAINT zones_cluster_role CHECK (cluster_role = 'dns'), + CONSTRAINT zones_name_format CHECK (length(name) <= 253 + AND name ~ '^[a-z0-9]([a-z0-9-]*[a-z0-9])?(\.[a-z0-9]([a-z0-9-]*[a-z0-9])?)*$'), + CONSTRAINT zones_kind_name CHECK ((kind = 'reverse') = (name ~ '\.(in-addr|ip6)\.arpa$') + AND (kind = 'reverse' OR name !~ '(^|\.)arpa$')), + CONSTRAINT zones_default_ttl CHECK (default_ttl IS NULL OR default_ttl BETWEEN 1 AND 2147483647), + CONSTRAINT zones_allow_update_dynamic CHECK (mode = 'dynamic' OR allow_update = '{}') +); +CREATE UNIQUE INDEX zones_name_unique ON zones (name); +CREATE INDEX zones_cluster ON zones (cluster_id); + +-- TSIG keys allowed to send dynamic updates to a zone. A key in use cannot +-- be deleted. +CREATE TABLE zone_update_keys ( + zone_id uuid NOT NULL REFERENCES zones (id) ON DELETE CASCADE, + key_id uuid NOT NULL CONSTRAINT zone_update_keys_key_fk REFERENCES tsig_keys (id), + PRIMARY KEY (zone_id, key_id) +); +CREATE INDEX zone_update_keys_key ON zone_update_keys (key_id); + +CREATE TYPE dns_record_type AS ENUM ('A', 'AAAA', 'CNAME', 'MX', 'TXT', 'SRV', 'NS', 'PTR'); + +CREATE TABLE dns_records ( + id uuid PRIMARY KEY DEFAULT gen_random_uuid(), + zone_id uuid NOT NULL REFERENCES zones (id) ON DELETE CASCADE, + -- Owner relative to the zone, '@' for the apex. + name text NOT NULL, + type dns_record_type NOT NULL, + ttl integer, + address inet, + target text NOT NULL DEFAULT '', + preference integer NOT NULL DEFAULT 0, + priority integer NOT NULL DEFAULT 0, + weight integer NOT NULL DEFAULT 0, + port integer NOT NULL DEFAULT 0, + text text[] NOT NULL DEFAULT '{}', + description text NOT NULL DEFAULT '', + version integer NOT NULL DEFAULT 1, + created_at timestamptz NOT NULL DEFAULT now(), + updated_at timestamptz NOT NULL DEFAULT now(), + CONSTRAINT dns_records_name CHECK (name <> '' AND length(name) <= 253 AND name !~ '\.$' AND name = lower(name)), + CONSTRAINT dns_records_ttl CHECK (ttl IS NULL OR ttl BETWEEN 1 AND 2147483647), + CONSTRAINT dns_records_address CHECK ((type IN ('A', 'AAAA')) = (address IS NOT NULL) + AND (address IS NULL OR (masklen(address) = CASE family(address) WHEN 4 THEN 32 ELSE 128 END + AND family(address) = CASE type WHEN 'A' THEN 4 ELSE 6 END))), + CONSTRAINT dns_records_target CHECK ((type IN ('CNAME', 'MX', 'SRV', 'NS', 'PTR')) = (target <> '') + AND length(target) <= 253 AND target = lower(target)), + CONSTRAINT dns_records_numbers CHECK (preference BETWEEN 0 AND 65535 AND priority BETWEEN 0 AND 65535 + AND weight BETWEEN 0 AND 65535 AND port BETWEEN 0 AND 65535 + AND (type = 'MX' OR preference = 0) AND (type = 'SRV' OR (priority = 0 AND weight = 0 AND port = 0))), + CONSTRAINT dns_records_text CHECK ((type = 'TXT') = (cardinality(text) > 0)) +); + +-- The same record (TTL aside) exists only once per zone. TXT strings enter +-- the index as a hash, since index entries are limited in size. +-- +goose StatementBegin +CREATE FUNCTION dns_records_text_key(t text[]) RETURNS text +LANGUAGE sql IMMUTABLE PARALLEL SAFE AS $$ SELECT md5(t::text) $$; +-- +goose StatementEnd +CREATE UNIQUE INDEX dns_records_unique ON dns_records (zone_id, name, type, coalesce(host(address), ''), target, + preference, priority, weight, port, dns_records_text_key(text)); + +-- DHCP cluster of a network; NULL = generator setting +-- dhcp.default_cluster_id. Only DHCP clusters are accepted. +ALTER TABLE networks ADD COLUMN dhcp_cluster_id uuid; +ALTER TABLE networks ADD COLUMN dhcp_cluster_role agent_role NOT NULL DEFAULT 'dhcp'; +ALTER TABLE networks ADD CONSTRAINT networks_dhcp_cluster_role CHECK (dhcp_cluster_role = 'dhcp'); +ALTER TABLE networks ADD CONSTRAINT networks_dhcp_cluster_fk + FOREIGN KEY (dhcp_cluster_id, dhcp_cluster_role) REFERENCES clusters (id, role); +CREATE INDEX networks_dhcp_cluster ON networks (dhcp_cluster_id); + +-- Revision files per target: '' is the default group (agents without +-- cluster), otherwise the agent ID of a cluster member. Secret files hold +-- key material; their content is sealed (associated data: revision, role, +-- target, path). Adding columns does not fire the immutability trigger. +ALTER TABLE config_revision_files ADD COLUMN target text NOT NULL DEFAULT ''; +ALTER TABLE config_revision_files ADD COLUMN secret boolean NOT NULL DEFAULT false; +ALTER TABLE config_revision_files ADD CONSTRAINT config_revision_files_target + CHECK (target = '' OR target ~ '^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$'); +ALTER TABLE config_revision_files DROP CONSTRAINT config_revision_files_path_unique; +ALTER TABLE config_revision_files DROP CONSTRAINT config_revision_files_pkey; +ALTER TABLE config_revision_files ADD PRIMARY KEY (revision, role, target, ordinal); +ALTER TABLE config_revision_files ADD CONSTRAINT config_revision_files_path_unique UNIQUE (revision, role, target, path); + +-- Secrets of the generator input (TSIG secrets, HA passwords) as a sealed +-- JSON object; config_revisions.source never contains them. +ALTER TABLE config_revisions ADD COLUMN secrets bytea; + +-- +goose Down +ALTER TABLE config_revisions DROP COLUMN secrets; +DELETE FROM config_revision_files WHERE target <> ''; +ALTER TABLE config_revision_files DROP CONSTRAINT config_revision_files_path_unique; +ALTER TABLE config_revision_files DROP CONSTRAINT config_revision_files_pkey; +ALTER TABLE config_revision_files ADD PRIMARY KEY (revision, role, ordinal); +ALTER TABLE config_revision_files ADD CONSTRAINT config_revision_files_path_unique UNIQUE (revision, role, path); +ALTER TABLE config_revision_files DROP CONSTRAINT config_revision_files_target; +ALTER TABLE config_revision_files DROP COLUMN secret; +ALTER TABLE config_revision_files DROP COLUMN target; +DROP INDEX networks_dhcp_cluster; +ALTER TABLE networks DROP CONSTRAINT networks_dhcp_cluster_fk; +ALTER TABLE networks DROP CONSTRAINT networks_dhcp_cluster_role; +ALTER TABLE networks DROP COLUMN dhcp_cluster_role; +ALTER TABLE networks DROP COLUMN dhcp_cluster_id; +DROP TABLE dns_records; +DROP FUNCTION dns_records_text_key(text[]); +DROP TYPE dns_record_type; +DROP TABLE zone_update_keys; +DROP TABLE zones; +DROP TYPE zone_mode; +DROP TYPE zone_kind; +ALTER TABLE clusters DROP CONSTRAINT clusters_transfer_key_fk; +DROP TABLE tsig_keys; +DROP TABLE cluster_members; +DROP TYPE cluster_member_role; +DROP TABLE clusters; +ALTER TABLE agents DROP CONSTRAINT agents_id_role_unique; diff --git a/server/internal/store/networks.go b/server/internal/store/networks.go index a988223..bc08041 100644 --- a/server/internal/store/networks.go +++ b/server/internal/store/networks.go @@ -13,12 +13,13 @@ import ( ) const networkColumns = `id, cidr, name, description, vlan, gateway, coalesce(domain, ''), search_domains, tags, - dhcp_enabled, lease_time, dns_servers, ntp_servers, version, created_at, updated_at` + dhcp_enabled, lease_time, dns_servers, ntp_servers, coalesce(dhcp_cluster_id::text, ''), version, created_at, updated_at` func scanNetwork(row pgx.Row) (*ipam.Network, error) { var n ipam.Network err := row.Scan(&n.ID, &n.CIDR, &n.Name, &n.Description, &n.VLAN, &n.Gateway, &n.Domain, &n.SearchDomains, - &n.Tags, &n.DHCPEnabled, &n.LeaseTime, &n.DNSServers, &n.NTPServers, &n.Version, &n.CreatedAt, &n.UpdatedAt) + &n.Tags, &n.DHCPEnabled, &n.LeaseTime, &n.DNSServers, &n.NTPServers, &n.DHCPClusterID, &n.Version, &n.CreatedAt, + &n.UpdatedAt) if err != nil { return nil, mapErr(err) } @@ -129,11 +130,12 @@ func (t *Tx) replacePools(ctx context.Context, n *ipam.Network) error { func (t *Tx) CreateNetwork(ctx context.Context, n *ipam.Network) error { row := t.q.QueryRow(ctx, ` INSERT INTO networks (cidr, name, description, vlan, gateway, domain, search_domains, tags, - dhcp_enabled, lease_time, dns_servers, ntp_servers) - VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12) + dhcp_enabled, lease_time, dns_servers, ntp_servers, dhcp_cluster_id) + VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13) RETURNING id, version, created_at, updated_at`, n.CIDR, n.Name, n.Description, n.VLAN, n.Gateway, nullString(n.Domain), emptyIfNil(n.SearchDomains), - emptyIfNil(n.Tags), n.DHCPEnabled, n.LeaseTime, emptyIfNil(n.DNSServers), emptyIfNil(n.NTPServers)) + emptyIfNil(n.Tags), n.DHCPEnabled, n.LeaseTime, emptyIfNil(n.DNSServers), emptyIfNil(n.NTPServers), + nullString(n.DHCPClusterID)) if err := row.Scan(&n.ID, &n.Version, &n.CreatedAt, &n.UpdatedAt); err != nil { return mapErr(err) } @@ -146,11 +148,12 @@ func (t *Tx) UpdateNetwork(ctx context.Context, n *ipam.Network) error { row := t.q.QueryRow(ctx, ` UPDATE networks SET name = $3, description = $4, vlan = $5, gateway = $6, domain = $7, search_domains = $8, tags = $9, dhcp_enabled = $10, lease_time = $11, dns_servers = $12, - ntp_servers = $13, version = version + 1, updated_at = now() + ntp_servers = $13, dhcp_cluster_id = $14, version = version + 1, updated_at = now() WHERE id = $1 AND version = $2 RETURNING version, updated_at`, n.ID, n.Version, n.Name, n.Description, n.VLAN, n.Gateway, nullString(n.Domain), emptyIfNil(n.SearchDomains), - emptyIfNil(n.Tags), n.DHCPEnabled, n.LeaseTime, emptyIfNil(n.DNSServers), emptyIfNil(n.NTPServers)) + emptyIfNil(n.Tags), n.DHCPEnabled, n.LeaseTime, emptyIfNil(n.DNSServers), emptyIfNil(n.NTPServers), + nullString(n.DHCPClusterID)) if err := row.Scan(&n.Version, &n.UpdatedAt); err != nil { if err = mapErr(err); errors.Is(err, ErrNotFound) { return ErrConflict diff --git a/server/internal/store/push_test.go b/server/internal/store/push_test.go index c84c683..fecfc20 100644 --- a/server/internal/store/push_test.go +++ b/server/internal/store/push_test.go @@ -51,13 +51,14 @@ func TestRevisionSource(t *testing.T) { ctx := context.Background() now := time.Now().UTC().Truncate(time.Microsecond) src := json.RawMessage(`{"Serial":1,"SubnetIDs":{"a":7},"Networks":null}`) - if err := st.Q().InsertRevision(ctx, &store.Revision{Number: 1, CreatedAt: now, CreatedByName: "admin", Source: src}, nil); err != nil { + if err := st.Q().InsertRevision(ctx, &store.Revision{Number: 1, CreatedAt: now, CreatedByName: "admin", Source: src, + Secrets: []byte("sealed")}, nil); err != nil { t.Fatal(err) } if err := st.Q().InsertRevision(ctx, &store.Revision{Number: 2, CreatedAt: now, CreatedByName: "admin"}, nil); err != nil { t.Fatal(err) } - got, err := st.Q().RevisionSource(ctx, 1) + got, secrets, err := st.Q().RevisionSource(ctx, 1) if err != nil { t.Fatal(err) } @@ -65,10 +66,13 @@ func TestRevisionSource(t *testing.T) { if json.Unmarshal(got, &a) != nil || json.Unmarshal(src, &b) != nil || len(a) != 3 || a["SubnetIDs"].(map[string]any)["a"] != b["SubnetIDs"].(map[string]any)["a"] { t.Errorf("source %s", got) } - if got, err := st.Q().RevisionSource(ctx, 2); err != nil || got != nil { + if string(secrets) != "sealed" { + t.Errorf("secrets %q", secrets) + } + if got, secrets, err := st.Q().RevisionSource(ctx, 2); err != nil || got != nil || secrets != nil { t.Errorf("revision without source: %s %v", got, err) } - if _, err := st.Q().RevisionSource(ctx, 3); !errors.Is(err, store.ErrNotFound) { + if _, _, err := st.Q().RevisionSource(ctx, 3); !errors.Is(err, store.ErrNotFound) { t.Errorf("unknown revision: %v", err) } if err := st.Q().InsertRevision(ctx, &store.Revision{Number: 3, CreatedAt: now, CreatedByName: "admin", diff --git a/server/internal/store/revisions.go b/server/internal/store/revisions.go index e41db47..18af885 100644 --- a/server/internal/store/revisions.go +++ b/server/internal/store/revisions.go @@ -30,17 +30,25 @@ type Revision struct { DHCPFiles int WarningCount int // Source is the generator input (JSON object) the revision was rendered - // from. It is written by InsertRevision and read only by - // RevisionSource; nil for revisions created before M4. + // from, without secrets. It is written by InsertRevision and read only + // by RevisionSource; nil for revisions created before M4. Source json.RawMessage + // Secrets are the secrets of the generator input, sealed (ADR 0023); + // written by InsertRevision and read only by RevisionSource. + Secrets []byte } -// RevisionFile is one artifact of a revision. +// RevisionFile is one artifact of a revision. Content of secret files is +// sealed in the database; the caller seals and opens it. type RevisionFile struct { - Role string + Role string + // Target is "" for the default group, otherwise the agent ID of a + // cluster member (ADR 0023). + Target string Path string Mode uint32 CreateOnly bool + Secret bool Content []byte } @@ -73,53 +81,78 @@ func (t *Tx) getRevision(ctx context.Context, cond string, args ...any) (*Revisi return &r, nil } -// RevisionSource returns the generator input of a revision: ErrNotFound for -// an unknown revision, nil for one created before M4. -func (t *Tx) RevisionSource(ctx context.Context, number int64) (json.RawMessage, error) { +// RevisionSource returns the generator input of a revision and its sealed +// secrets: ErrNotFound for an unknown revision, a nil source for one created +// before M4, nil secrets for one without. +func (t *Tx) RevisionSource(ctx context.Context, number int64) (source json.RawMessage, secrets []byte, err error) { var src []byte - if err := t.q.QueryRow(ctx, `SELECT source FROM config_revisions WHERE number = $1`, number).Scan(&src); err != nil { - return nil, mapErr(err) + if err := t.q.QueryRow(ctx, `SELECT source, secrets FROM config_revisions WHERE number = $1`, number). + Scan(&src, &secrets); err != nil { + return nil, nil, mapErr(err) } if src == nil { - return nil, nil + return nil, secrets, nil } - return src, nil + return src, secrets, nil } // InsertRevision stores a revision with its files (in the given order per -// role). CreatedAt is taken from r. +// role and target). CreatedAt is taken from r. func (t *Tx) InsertRevision(ctx context.Context, r *Revision, files []RevisionFile) error { warnings := r.Warnings if len(warnings) == 0 { warnings = json.RawMessage("[]") } - if _, err := t.q.Exec(ctx, `INSERT INTO config_revisions (number, created_at, created_by, created_by_name, comment, warnings, source) - VALUES ($1, $2, $3, $4, $5, $6, $7)`, r.Number, r.CreatedAt, r.CreatedBy, r.CreatedByName, r.Comment, []byte(warnings), - nullJSON(r.Source)); err != nil { + if _, err := t.q.Exec(ctx, `INSERT INTO config_revisions (number, created_at, created_by, created_by_name, comment, warnings, + source, secrets) + VALUES ($1, $2, $3, $4, $5, $6, $7, $8)`, r.Number, r.CreatedAt, r.CreatedBy, r.CreatedByName, r.Comment, + []byte(warnings), nullJSON(r.Source), r.Secrets); err != nil { return mapErr(err) } b := &pgx.Batch{} - ordinal := map[string]int32{} + type group struct{ role, target string } + ordinal := map[group]int32{} for _, f := range files { - n := ordinal[f.Role] - ordinal[f.Role] = n + 1 - b.Queue(`INSERT INTO config_revision_files (revision, role, ordinal, path, mode, create_only, content) - VALUES ($1, $2::agent_role, $3, $4, $5, $6, $7)`, - r.Number, f.Role, n, f.Path, int32(f.Mode), f.CreateOnly, f.Content) //nolint:gosec // modes are at most 0o777 (checked by the database) + g := group{f.Role, f.Target} + n := ordinal[g] + ordinal[g] = n + 1 + b.Queue(`INSERT INTO config_revision_files (revision, role, target, ordinal, path, mode, create_only, secret, content) + VALUES ($1, $2::agent_role, $3, $4, $5, $6, $7, $8, $9)`, + r.Number, f.Role, f.Target, n, f.Path, int32(f.Mode), f.CreateOnly, f.Secret, f.Content) //nolint:gosec // modes are at most 0o777 (checked by the database) } return t.execBatch(ctx, b) } -// RevisionFiles returns the files of a revision ordered by role and -// generator order. role "" returns both roles. +// RevisionFiles returns the files of a revision ordered by role (DNS +// first), target ("" first) and generator order. role "" returns both +// roles. func (t *Tx) RevisionFiles(ctx context.Context, number int64, role string) ([]RevisionFile, error) { - sql := `SELECT role, path, mode, create_only, content FROM config_revision_files WHERE revision = $1` + sql := `SELECT role, target, path, mode, create_only, secret, content FROM config_revision_files WHERE revision = $1` args := []any{number} if role != "" { sql += ` AND role = $2::agent_role` args = append(args, role) } - rows, err := t.q.Query(ctx, sql+` ORDER BY role, ordinal`, args...) + return t.queryRevisionFiles(ctx, sql+` ORDER BY role, target COLLATE "C", ordinal`, args...) +} + +// RevisionTargetFiles returns the files of one role and target of a +// revision in generator order. +func (t *Tx) RevisionTargetFiles(ctx context.Context, number int64, role, target string) ([]RevisionFile, error) { + return t.queryRevisionFiles(ctx, `SELECT role, target, path, mode, create_only, secret, content FROM config_revision_files + WHERE revision = $1 AND role = $2::agent_role AND target = $3 ORDER BY ordinal`, number, role, target) +} + +// RevisionHasTarget reports whether a revision has files for the target. +func (t *Tx) RevisionHasTarget(ctx context.Context, number int64, role, target string) (bool, error) { + var ok bool + err := t.q.QueryRow(ctx, `SELECT EXISTS (SELECT 1 FROM config_revision_files + WHERE revision = $1 AND role = $2::agent_role AND target = $3)`, number, role, target).Scan(&ok) + return ok, mapErr(err) +} + +func (t *Tx) queryRevisionFiles(ctx context.Context, sql string, args ...any) ([]RevisionFile, error) { + rows, err := t.q.Query(ctx, sql, args...) if err != nil { return nil, mapErr(err) } @@ -128,7 +161,7 @@ func (t *Tx) RevisionFiles(ctx context.Context, number int64, role string) ([]Re for rows.Next() { var f RevisionFile var mode int32 - if err := rows.Scan(&f.Role, &f.Path, &mode, &f.CreateOnly, &f.Content); err != nil { + if err := rows.Scan(&f.Role, &f.Target, &f.Path, &mode, &f.CreateOnly, &f.Secret, &f.Content); err != nil { return nil, mapErr(err) } f.Mode = uint32(mode) //nolint:gosec // 0..511 by constraint @@ -138,7 +171,7 @@ func (t *Tx) RevisionFiles(ctx context.Context, number int64, role string) ([]Re } // ListRevisions returns up to limit revisions with a number below before -// (0 = no bound), newest first. +// (0 = no bound), newest first. The file counts include all targets. func (t *Tx) ListRevisions(ctx context.Context, limit int, before int64) ([]*Revision, error) { sql := `SELECT r.number, r.created_at, r.created_by, r.created_by_name, r.comment, jsonb_array_length(r.warnings), (SELECT count(*) FROM config_revision_files f WHERE f.revision = r.number AND f.role = 'dns'), diff --git a/server/internal/store/store_test.go b/server/internal/store/store_test.go index aff3c09..9e5ae82 100644 --- a/server/internal/store/store_test.go +++ b/server/internal/store/store_test.go @@ -29,6 +29,33 @@ func TestMigrateIsIdempotent(t *testing.T) { } } +// TestMigrateDownAndUp runs the Down sections of the migrations from M3 on +// with data in the new tables, then migrates up again. +func TestMigrateDownAndUp(t *testing.T) { + st := storetest.New(t) + ctx := context.Background() + for _, sql := range []string{ + `INSERT INTO agents (id, role, cert_serial, cert_not_after) VALUES ('00000000-0000-4000-8000-00000000000a', 'dns', 'a1', now())`, + `INSERT INTO clusters (id, name, role) VALUES ('00000000-0000-4000-8000-0000000000c1', 'c', 'dns')`, + `INSERT INTO cluster_members (agent_id, cluster_id, role, member_role, address) + VALUES ('00000000-0000-4000-8000-00000000000a', '00000000-0000-4000-8000-0000000000c1', 'dns', 'primary', '192.0.2.1')`, + `INSERT INTO zones (name, kind, cluster_id) VALUES ('example.test', 'forward', '00000000-0000-4000-8000-0000000000c1')`, + `INSERT INTO config_revisions (number, created_by_name) VALUES (1, 'admin')`, + `INSERT INTO config_revision_files (revision, role, target, ordinal, path, mode, content) + VALUES (1, 'dns', '', 0, 'a', 420, ''), (1, 'dns', '00000000-0000-4000-8000-00000000000a', 0, 'a', 420, '')`, + } { + if _, err := st.Pool().Exec(ctx, sql); err != nil { + t.Fatalf("%s: %v", sql, err) + } + } + if err := st.MigrateDownTo(ctx, 2); err != nil { + t.Fatalf("down: %v", err) + } + if err := st.Migrate(ctx, slog.New(slog.NewTextHandler(io.Discard, nil))); err != nil { + t.Fatalf("up again: %v", err) + } +} + func TestMigrateRefusesNewerSchema(t *testing.T) { st := storetest.New(t) ctx := context.Background() diff --git a/server/internal/store/zones.go b/server/internal/store/zones.go new file mode 100644 index 0000000..440e752 --- /dev/null +++ b/server/internal/store/zones.go @@ -0,0 +1,234 @@ +// SPDX-License-Identifier: LicenseRef-DDI-Proprietary + +package store + +import ( + "context" + "errors" + "net/netip" + "time" + + "github.com/jackc/pgx/v5" +) + +// Zone is a row of zones with its update keys. +type Zone struct { + ID string + Name string + Kind string + Mode string + ClusterID *string + DefaultTTL *int + AllowUpdate []netip.Prefix + // UpdateKeyIDs are sorted. + UpdateKeyIDs []string + Description string + Version int + CreatedAt time.Time + UpdatedAt time.Time + // RecordCount is filled by the read methods. + RecordCount int +} + +const zoneColumns = `z.id, z.name, z.kind, z.mode, z.cluster_id, z.default_ttl, z.allow_update, z.description, z.version, + z.created_at, z.updated_at, (SELECT count(*) FROM dns_records r WHERE r.zone_id = z.id), + coalesce((SELECT array_agg(u.key_id::text ORDER BY u.key_id::text) FROM zone_update_keys u WHERE u.zone_id = z.id), '{}')` + +func scanZone(row pgx.Row) (*Zone, error) { + var z Zone + if err := row.Scan(&z.ID, &z.Name, &z.Kind, &z.Mode, &z.ClusterID, &z.DefaultTTL, &z.AllowUpdate, &z.Description, + &z.Version, &z.CreatedAt, &z.UpdatedAt, &z.RecordCount, &z.UpdateKeyIDs); err != nil { + return nil, mapErr(err) + } + return &z, nil +} + +// ListZones returns all zones ordered by name. +func (t *Tx) ListZones(ctx context.Context) ([]*Zone, error) { + rows, err := t.q.Query(ctx, `SELECT `+zoneColumns+` FROM zones z ORDER BY z.name`) + if err != nil { + return nil, mapErr(err) + } + defer rows.Close() + var out []*Zone + for rows.Next() { + z, err := scanZone(rows) + if err != nil { + return nil, err + } + out = append(out, z) + } + return out, mapErr(rows.Err()) +} + +// GetZone returns one zone; forUpdate locks the row (which serializes the +// changes to its records). +func (t *Tx) GetZone(ctx context.Context, id string, forUpdate bool) (*Zone, error) { + sql := `SELECT ` + zoneColumns + ` FROM zones z WHERE z.id = $1` + if forUpdate { + sql += ` FOR UPDATE OF z` + } + return scanZone(t.q.QueryRow(ctx, sql, id)) +} + +func (t *Tx) replaceZoneKeys(ctx context.Context, z *Zone) error { + if _, err := t.q.Exec(ctx, `DELETE FROM zone_update_keys WHERE zone_id = $1`, z.ID); err != nil { + return mapErr(err) + } + if len(z.UpdateKeyIDs) == 0 { + return nil + } + _, err := t.q.Exec(ctx, `INSERT INTO zone_update_keys (zone_id, key_id) SELECT $1, unnest($2::uuid[])`, z.ID, z.UpdateKeyIDs) + return mapErr(err) +} + +// InsertZone stores z with its update keys; ID, version and times are set. +func (t *Tx) InsertZone(ctx context.Context, z *Zone) error { + if err := mapErr(t.q.QueryRow(ctx, `INSERT INTO zones (name, kind, mode, cluster_id, default_ttl, allow_update, description) + VALUES ($1, $2::zone_kind, $3::zone_mode, $4, $5, $6, $7) RETURNING id, version, created_at, updated_at`, + z.Name, z.Kind, z.Mode, z.ClusterID, z.DefaultTTL, emptyIfNil(z.AllowUpdate), z.Description). + Scan(&z.ID, &z.Version, &z.CreatedAt, &z.UpdatedAt)); err != nil { + return err + } + return t.replaceZoneKeys(ctx, z) +} + +// UpdateZone writes the mutable fields if the stored version equals +// z.Version and replaces the update keys. +func (t *Tx) UpdateZone(ctx context.Context, z *Zone) error { + err := t.q.QueryRow(ctx, `UPDATE zones SET mode = $3::zone_mode, cluster_id = $4, default_ttl = $5, allow_update = $6, + description = $7, version = version + 1, updated_at = now() + WHERE id = $1 AND version = $2 RETURNING version, updated_at`, + z.ID, z.Version, z.Mode, z.ClusterID, z.DefaultTTL, emptyIfNil(z.AllowUpdate), z.Description). + Scan(&z.Version, &z.UpdatedAt) + if err = mapErr(err); errors.Is(err, ErrNotFound) { + return ErrConflict + } + if err != nil { + return err + } + return t.replaceZoneKeys(ctx, z) +} + +// DeleteZone removes a zone with its records. +func (t *Tx) DeleteZone(ctx context.Context, id string) error { + tag, err := t.q.Exec(ctx, `DELETE FROM zones WHERE id = $1`, id) + if err != nil { + return mapErr(err) + } + if tag.RowsAffected() == 0 { + return ErrNotFound + } + return nil +} + +// Record is a row of dns_records. +type Record struct { + ID string + ZoneID string + Name string + Type string + TTL *int + Address *netip.Addr + Target string + Preference int + Priority int + Weight int + Port int + Text []string + Description string + Version int + CreatedAt time.Time + UpdatedAt time.Time +} + +const recordColumns = `r.id, r.zone_id, r.name, r.type, r.ttl, r.address, r.target, r.preference, r.priority, r.weight, + r.port, r.text, r.description, r.version, r.created_at, r.updated_at` + +func scanRecord(row pgx.Row) (*Record, error) { + var r Record + if err := row.Scan(&r.ID, &r.ZoneID, &r.Name, &r.Type, &r.TTL, &r.Address, &r.Target, &r.Preference, &r.Priority, + &r.Weight, &r.Port, &r.Text, &r.Description, &r.Version, &r.CreatedAt, &r.UpdatedAt); err != nil { + return nil, mapErr(err) + } + if r.Address != nil { + a := r.Address.Unmap() + r.Address = &a + } + return &r, nil +} + +// recordOrder sorts by name, type and data. +const recordOrder = ` ORDER BY r.zone_id, r.name, r.type, r.address, r.target, r.preference, r.priority, r.weight, r.port, r.text, r.id` + +func (t *Tx) queryRecords(ctx context.Context, sql string, args ...any) ([]*Record, error) { + rows, err := t.q.Query(ctx, sql, args...) + if err != nil { + return nil, mapErr(err) + } + defer rows.Close() + out := []*Record{} + for rows.Next() { + r, err := scanRecord(rows) + if err != nil { + return nil, err + } + out = append(out, r) + } + return out, mapErr(rows.Err()) +} + +// ListRecords returns the records of a zone sorted by name, type and data. +func (t *Tx) ListRecords(ctx context.Context, zoneID string) ([]*Record, error) { + return t.queryRecords(ctx, `SELECT `+recordColumns+` FROM dns_records r WHERE r.zone_id = $1`+recordOrder, zoneID) +} + +// AllRecords returns the records of all zones. +func (t *Tx) AllRecords(ctx context.Context) ([]*Record, error) { + return t.queryRecords(ctx, `SELECT `+recordColumns+` FROM dns_records r`+recordOrder) +} + +// GetRecord returns a record of a zone; forUpdate locks the row. +func (t *Tx) GetRecord(ctx context.Context, zoneID, id string, forUpdate bool) (*Record, error) { + sql := `SELECT ` + recordColumns + ` FROM dns_records r WHERE r.id = $1 AND r.zone_id = $2` + if forUpdate { + sql += ` FOR UPDATE OF r` + } + return scanRecord(t.q.QueryRow(ctx, sql, id, zoneID)) +} + +// InsertRecord stores r; ID, version and times are set. +func (t *Tx) InsertRecord(ctx context.Context, r *Record) error { + return mapErr(t.q.QueryRow(ctx, `INSERT INTO dns_records (zone_id, name, type, ttl, address, target, preference, + priority, weight, port, text, description) + VALUES ($1, $2, $3::dns_record_type, $4, $5, $6, $7, $8, $9, $10, $11, $12) + RETURNING id, version, created_at, updated_at`, + r.ZoneID, r.Name, r.Type, r.TTL, r.Address, r.Target, r.Preference, r.Priority, r.Weight, r.Port, + emptyIfNil(r.Text), r.Description).Scan(&r.ID, &r.Version, &r.CreatedAt, &r.UpdatedAt)) +} + +// UpdateRecord writes r if the stored version equals r.Version. +func (t *Tx) UpdateRecord(ctx context.Context, r *Record) error { + err := t.q.QueryRow(ctx, `UPDATE dns_records SET name = $3, type = $4::dns_record_type, ttl = $5, address = $6, + target = $7, preference = $8, priority = $9, weight = $10, port = $11, text = $12, description = $13, + version = version + 1, updated_at = now() + WHERE id = $1 AND version = $2 RETURNING version, updated_at`, + r.ID, r.Version, r.Name, r.Type, r.TTL, r.Address, r.Target, r.Preference, r.Priority, r.Weight, r.Port, + emptyIfNil(r.Text), r.Description).Scan(&r.Version, &r.UpdatedAt) + if err = mapErr(err); errors.Is(err, ErrNotFound) { + return ErrConflict + } + return err +} + +// DeleteRecord removes a record of a zone. +func (t *Tx) DeleteRecord(ctx context.Context, zoneID, id string) error { + tag, err := t.q.Exec(ctx, `DELETE FROM dns_records WHERE id = $1 AND zone_id = $2`, id, zoneID) + if err != nil { + return mapErr(err) + } + if tag.RowsAffected() == 0 { + return ErrNotFound + } + return nil +}