Migration Guide

This document collects the breaking-change migrations for Bindy, newest first.

Scout routes publish every VIP of their Gateway (ADR-0028)

A --gateway-service target written as namespace/<label-selector> now publishes the addresses of every LoadBalancer Service the selector matches, not the first one with an address (issue #569). Routes also re-resolve when their Gateway or its Services change. No CRD, RBAC or admission change: Scout's ClusterRole already allows watching Gateways.

• Check what each selector matches before upgrading Scout. Every LoadBalancer Service listed, with an address, will be published for the routes of that class:

kubectl get svc -n <namespace> -l '<label-selector>' \
  -o custom-columns=NAME:.metadata.name,TYPE:.spec.type,IPS:.status.loadBalancer.ingress[*].ip

If it lists Services you do not want in DNS (the Traefik chart, for example, labels its extra internal Services app.kubernetes.io/name: traefik too), label the public VIP Services and add that label to the selector, or switch to a namespace/name target. - Several Gateways of one class. When the matched Services carry gateway.networking.k8s.io/gateway-name (Envoy Gateway, Cilium, Istio), a route now publishes only its own Gateway's Services. Two Gateways of the same name in different namespaces behind one selector are not told apart. - Selectors are validated at startup. Only equality-based selectors (k=v, k==v, k!=v, k, !k) are accepted; a set-based one (k in (a,b)) or a malformed one now fails Scout at startup instead of matching nothing at runtime. - With --allowed-ip-ranges, a route address discovered from a Gateway or gateway Service outside the ranges is now dropped with a warning instead of stopping the route publishing. Annotated and default addresses outside the ranges still stop it. - Contributions are sorted. Each source's entry in bindy.firestoned.io/scout-contributors is now written with its addresses sorted numerically, so the first reconcile after the upgrade may rewrite entries once. The record's addresses are unchanged.

Scout records of two clusters no longer overwrite each other (ADR-0027)

When two edge clusters registered for one zone publish the same name, the cluster that published it first now keeps it, and the other cluster's record is held back (Ready=False, reason RRsetConflict) instead of the two replacing each other's addresses. No CRD, RBAC or admission change.

• Check for names two clusters publish today before upgrading the operator. After the upgrade, the older of two published records keeps the name, so the other cluster's address stops being served:

kubectl get arecords -n bindy-system -l bindy.firestoned.io/managed-by=scout \
  -o jsonpath='{range .items[*]}{.metadata.labels.bindy\.firestoned\.io/zone}{" "}{.spec.name}{" "}{.metadata.labels.bindy\.firestoned\.io/source-cluster}{"\n"}{end}' \
  | sort -u | awk '{k=$1" "$2; c[k]++; v[k]=v[k]" "$3} END {for (k in c) if (c[k] > 1) print k ":" v[k]}'

Each line is a zone and name with the clusters publishing it. Move each one to a single cluster (change the record-name or host on the others). - Register both names before renaming a cluster. The operator tells a rename from two clusters by the registry: register the old and new names on the account (bindy bootstrap mc --cluster-name new,old ...) first, as the rename steps already say. A rename without that step leaves the new records held back while the old ones keep serving. - Find held-back records:

kubectl get arecords -n bindy-system -o json \
  | jq -r '.items[] | select(any(.status.conditions[]?; .type == "Ready" and .reason == "RRsetConflict")) | .metadata.name'

Scout runs one pod at a time (ADR-0002 addendum)

The Scout Deployment now uses the Recreate strategy and Scout drains on SIGTERM, so an upgrade never runs two Scout versions side by side (they would both publish, and a source changed in that window could keep a record the new version never revisits).

• bindy bootstrap scout: re-running it switches an existing Scout Deployment to Recreate automatically (a merge patch before the apply). No action needed.
• deploy/scout/deployment.yaml with kubectl apply: the new manifest carries strategy.type: Recreate; a client-side apply switches the Deployment on its own.
• Helm, Kustomize or GitOps with server-side apply: set spec.strategy.type: Recreate in your manifest. The API server refuses that apply while the Deployment still has its defaulted rollingUpdate block; remove it once first:

kubectl patch deployment bindy-scout -n bindy-system --type=merge \
  -p '{"spec":{"strategy":{"type":"Recreate","rollingUpdate":null}}}'

• Expect one longer pause, once. The Scout being replaced by this upgrade does not yet drain on SIGTERM, so publishing pauses for its 30-second grace period. Later upgrades pause for about a second. DNS keeps serving everything already published throughout.
• Run exactly one replica. More than one Scout replica per cluster is not supported.

Scout publishes one record per DNS name (ADR-0026)

Scout used to write one ARecord per resource host. Several resources that publish the same DNS name (one LoadBalancer Service per availability zone, all at the apex, as in issue #556) wrote one record each, the records replaced each other's addresses, and BIND9 served whichever was written last. Scout now writes one record per DNS name with every resource's addresses. No CRD, RBAC, flag or admission-policy change.

• Run an operator that includes ADR-0022 first. The handover below briefly has two records for one name; that operator keeps the name served when the old record is deleted (ADR-0022, Decision 5).
• Every Scout ARecord is replaced once, on the first reconcile after the upgrade. The new record is named after the DNS name, scout-{cluster}-{dns-name}-{zone}-{hash}, and carries managed-by, source-cluster, zone, one contributor.scout.bindy.firestoned.io/<hash> label per resource, and the bindy.firestoned.io/scout-contributors annotation. Scout writes it first and deletes the old per-resource record only once the operator reports the new one bound to its zone and Ready, rechecking every 10 seconds, so the name keeps resolving. Every source reconciles when Scout starts, so the migration completes shortly after startup. kubectl get arecords -n bindy-system -l bindy.firestoned.io/managed-by=scout,bindy.firestoned.io/source-kind lists the per-resource records still waiting.
• Resources that share a name are merged. Before upgrading, check that every shared name is meant to be shared. Two resources that publish the same name in one zone, in any namespace and of any kind, now publish all of their addresses instead of overwriting each other. This lists the names more than one current record publishes:

kubectl get arecords -n bindy-system -l bindy.firestoned.io/managed-by=scout \
  -o jsonpath='{range .items[*]}{.metadata.labels.bindy\.firestoned\.io/source-cluster}{" "}{.metadata.labels.bindy\.firestoned\.io/zone}{" "}{.spec.name}{"\n"}{end}' \
  | sort | uniq -d

Give a resource that should not share its name a different bindy.firestoned.io/record-name, or opt it out, before upgrading. - Addresses are listed in numeric order, each once, instead of in the order of the bindy.firestoned.io/ip annotation or the load-balancer status. DNS clients do not depend on that order. - Different TTLs on one name publish the lowest. Scout logs a warning for each change. - Tools that select Scout records by source-name must change. The new records carry no source-kind, source-namespace or source-name labels. Select a resource's records by its contributor label, the first 16 hex characters of the SHA-256 of its kind, namespace and name joined by NUL bytes:

kubectl get arecords -n bindy-system -l \
  "contributor.scout.bindy.firestoned.io/$(printf '%s\0%s\0%s' Ingress team-a web | sha256sum | cut -c1-16)"

• Do not edit the contributor annotation. Scout rebuilds the addresses and TTL from it on every write, and refuses to write a record whose annotation it cannot parse.

Rollback. An older Scout does not know the per-name records. It recreates its per-resource records next to them, and the two compete for the name. After rolling Scout back, wait for its records to be Ready, then delete the per-name records:

kubectl get arecords -n bindy-system -l bindy.firestoned.io/managed-by=scout -o json \
  | jq -r '.items[] | select(.metadata.annotations["bindy.firestoned.io/scout-contributors"]) | .metadata.name' \
  | xargs -r kubectl delete arecord -n bindy-system

Scout records bound to their cluster and zones (ADR-0025)

No CRD change, and nothing changes until you opt in, with one exception (the zone label rule below). The full step-by-step procedure, with pre-checks, a warn-only phase, verification commands and rollback, is in Migrating to Scout Source Guardrails. In short, per edge cluster:

• Limit what each Scout publishes. Set --allowed-zones (or BINDY_SCOUT_ALLOWED_ZONES) to the zones the cluster owns, and --allowed-ip-ranges (BINDY_SCOUT_ALLOWED_IP_RANGES) to its load-balancer ranges; bindy bootstrap scout takes both flags. A Scout that has neither logs a startup warning. A --default-zone or --default-ips outside them fails Scout at startup.

• Register each Scout's ServiceAccount on the bindy cluster. Re-run bindy bootstrap mc with the account's existing --service-account plus --cluster-name <the Scout's --cluster-name> and --allowed-zones. It re-applies the same ServiceAccount and RBAC and adds the account's keys to the bindy-scout-sources ConfigMap. Then install policies 21 and 22 (make admission-policies-install, or the release's admission-policies.yaml). An account that is not registered stays unconstrained. The default bindy-scout-remote account shared by several clusters can only be registered for all of their names and zones at once; give each cluster its own account first.

• Check DNSZone selectors that pick up Scout records (behavior change). A DNSZone now takes a record labelled bindy.firestoned.io/managed-by: scout only when the record's bindy.firestoned.io/zone label equals its spec.zoneName. Scout always writes that label with the zone it authorized against, so its records are unaffected. A hand-made record carrying managed-by: scout with a different or missing zone label stops being served by the zone; the operator logs a warning naming it. Fix its label, or drop managed-by: scout from records Scout did not write.

• Renaming a registered cluster now takes two steps: register the new name next to the old one, restart Scout with the new name and let it clean up, then drop the old name. See Changing the Cluster Name.

Secret operations are audited (ADR-0024)

No CRD, RBAC or flag change.

• RUST_LOG no longer hides the Secret audit trail. The target bindy::audit is added at info unless RUST_LOG names it, so RUST_LOG=warn keeps it. To turn it off, set it explicitly: RUST_LOG="warn,bindy::audit=off". See Secret Audit Log.
• New log lines and a new metric. One event per Secret write, per key load on a cache miss, and per source read; a counter bindy_firestoned_io_secret_operations_total. Route fields.kind == "audit" (with RUST_LOG_FORMAT=json) to a retention-controlled sink.
• secretRef mirrors gain an annotation. Each {instance}-rndc-key mirror is rewritten once after the upgrade to record its source's resourceVersion (bindy.firestoned.io/rndc-source-version). The key is unchanged, so the pods do not roll.

RNDC secretRef keys are mirrored; their Secret must start with bindy- (ADR-0023)

Applies to every instance whose RNDC key comes from rndcKey.secretRef (on the instance or its cluster's primary / secondary role) or from the deprecated rndcSecretRef. Instances with an operator-generated key are unaffected.

• Rename the referenced Secret to start with bindy- before upgrading. A reference to any other name is rejected by the bindy-pod-shape-validation admission policy and refused by the operator. Such an instance keeps its running pods but stops reconciling until the reference is fixed. Copy the Secret under a bindy- name, point the secretRef at it, then delete the old one:

kubectl get secret my-rndc -n bindy-system -o json \
  | jq '.metadata |= {name: "bindy-my-rndc", namespace: .namespace}' \
  | kubectl apply -f -

To find what needs changing: kubectl get bind9instances,bind9clusters,clusterbind9providers -A -o yaml | grep -A1 -E 'secretRef:|rndcSecretRef:'. - The pods now mount {instance}-rndc-key, the operator's mirror of the referenced key. Each such instance rolls its pods once after the upgrade. The referenced Secret is no longer mounted. - secretKey now works; keyNameKey is deprecated. The key is read from the secretKey entry (the algorithm from the Secret's algorithm entry, else secretRef.algorithm), or from the rndc.key entry when it is absent. In BIND9 the key is always named bindy-operator, the only name the shared named.conf control channel accepts, so a rndc.key under any other name now works too (it used to crash-loop named). An external tool that signs updates with the same key must use the name bindy-operator. - Editing the referenced Secret rolls the pods. Before, the operator switched to the new key at once while the pods kept the old one, and every write failed until the pods were restarted by hand. - A missing or invalid referenced Secret stops reconciliation but keeps DNS up. The mirror is left in place, so the running pods keep their key. A key that is not standard base64 is refused.

Scout records gain a source-kind label and new names (ADR-0022)

Scout's five source kinds now share one reconcile pipeline. No CRD, RBAC or flag change.

• Upgrade the operator before Scout. The new operator keeps a DNS name served while two records declare it and one is deleted (ADR-0022, Decision 5). An older operator deletes the shared RRset with the old record, and the name stays off the air until the new record next changes.

• Every Scout ARecord is renamed once, on the first reconcile after the upgrade. Names change from scout-{cluster}-{namespace}-{name}-{index} (scout-{cluster}-{namespace}-{service} for a Service) to scout-{cluster}-{namespace}-{name}-{hash}-{index}, where {hash} is 8 hex characters derived from the cluster, kind, namespace and name. Each record also gains the bindy.firestoned.io/source-kind label (Ingress, Service, HTTPRoute, TLSRoute or TCPRoute). Scout applies the new record first and deletes the old one only once the operator reports the new one bound to its zone (status.zoneRef) and Ready, rechecking every 10 seconds until then, so the name keeps resolving throughout. Every source reconciles when Scout starts, so the migration completes shortly after startup; a source whose new record never becomes Ready keeps its old record, and kubectl get arecords -l '!bindy.firestoned.io/source-kind' lists what is still waiting.

• The published DNS names do not change. Only the ARecord object names do; spec.name, the addresses and the TTL are the same.
• Tools that select Scout records by name pattern must use the labels. Select by bindy.firestoned.io/managed-by=scout plus source-cluster, source-kind, source-namespace and source-name, for example kubectl get arecords -n bindy-system -l bindy.firestoned.io/source-kind=Ingress,bindy.firestoned.io/source-name=web.
• Same-named sources of different kinds no longer share records. An Ingress and an HTTPRoute both called web in one namespace used to overwrite each other's ARecord, and releasing a same-named Service deleted them all. Each now owns its own records. If two kinds shared a name before the upgrade, the first to reconcile removes the shared legacy record and the other recreates its own on its next reconcile.
• An invalid TTL annotation now stops publication for that source. A bindy.firestoned.io/ttl that is not a positive integer ("abc", "0", "-5") used to be ignored (or, if negative, accepted). Scout now logs a warning, publishes nothing for that resource and retries every 30s, like an invalid record-name override; records already published stay. An invalid record-name override on an Ingress or route also no longer deletes the resource's existing records.
• A misconfigured Scout fails at startup. A setting that is present but empty (BINDY_SCOUT_DEFAULT_ZONE=""), a malformed --gateway-service / BINDY_SCOUT_GATEWAY_SERVICES entry, or a non-IPv4 --default-ips entry now stops Scout with an error naming the setting, instead of being silently ignored. Check the Scout Deployment's env and args before upgrading.
• Status addresses that are not IPv4 are skipped. An IPv6 address in an Ingress or Service status.loadBalancer, or in a Gateway's status.addresses, is no longer written into an ARecord.
• A Gateway or Service lookup that fails (other than 404) is retried. Before, the failure was logged at debug level and Scout fell back to --default-ips; now the reconcile fails and retries, so a transient API error cannot publish the fallback IP.

bindy bootstrap mc writes the kubeconfig Secret to a file

The Secret holds the remote ServiceAccount token, so bindy bootstrap mc no longer prints it to stdout. It writes it to a file created with mode 0600: --output <path> (-o), by default <service-account>-kubeconfig.yaml in the current directory.

• Scripts that pipe the output stop working. bindy bootstrap mc | kubectl apply -f - now writes the file and pipes nothing, and --output - is refused. Apply the file, then delete it:

bindy bootstrap mc -o cluster-a.yaml
kubectl --context=child-cluster-a apply -f cluster-a.yaml && rm cluster-a.yaml

• The Secret is now named <service-account>-kubeconfig (was <service-account>-remote-kubeconfig). With the default service account that is bindy-scout-remote-kubeconfig, the name the static deploy/scout/secrets-reader-rbac.yaml and the docs always used; before, the default produced bindy-scout-remote-remote-kubeconfig, which that Role does not let the scout read. A Secret already applied to a child cluster keeps working under its old name as long as BINDY_SCOUT_REMOTE_SECRET and the secrets-reader Role name it; when you re-run bindy bootstrap mc, point both at the new name. The Secret's type and contents are unchanged.

Zone transfer peers follow the pods (ADR-0019)

Apply the regenerated DNSZone CRD before the new operator: the status gains transferPeers (additive; an old CRD prunes it and every reconcile would push the peers again). No flag or RBAC change.

• First reconcile after the upgrade pushes every zone's peers once. Zones reconciled by an older operator have no status.transferPeers, so every primary's allow-transfer / also-notify is rewritten in place and every secondary zone is replaced (deleted, created again with the current primaries, retransferred). Each secondary answers SERVFAIL for a zone for the length of one AXFR while it is replaced; the primaries keep answering. This is also what repairs a cluster that hit the v0.8.0-rc.7 bug (a secondary with no copy of the zone).
• also-notify now targets the secondary instance's Service (its ClusterIP, port 53) instead of the secondary pods on 5353, so it survives secondary pod replacement and can be rewritten in place by bindcar 0.9.0. Zones created by an older operator keep their old also-notify until their primary pod is replaced: roll the BIND9 pods once after upgrading (kubectl rollout restart deployment -n <ns> -l app=bind9; the rollouts are staggered).
• A zone is Ready only when every secondary has it loaded. A secondary with the zone configured but no data makes the zone Degraded=True / Ready=False with reason SecondaryNotLoaded. Alerts on DNSZone readiness may fire for a few seconds after a secondary pod is replaced, until its transfer completes.
• A bindcar env var removed from bindcarConfig now leaves the pods. Removing one rolls the instances (staggered); before, the removal never took effect.
• Record and zone deletions wait for every pod that holds the zone. A deletion while a primary's container restarts now completes a few seconds later instead of leaving the record served. A pod that never becomes ready again blocks the deletion (RecordDeletionPending) until it is deleted. Record writes made while such a pod is down are retried until it is back.
• The leader election has its own Kubernetes client (not counted in BINDY_KUBE_QPS) with an 8 s request deadline, and renews the lease one retry period after each renewal. Lease API traffic rises from about one request every 13 s to one every 2 s per operator, the cadence of client-go's leader election.

Rollouts hand over at termination and are staggered across instances (ADR-0017 amendment, ADR-0018)

No CRD, flag or RBAC change (the pods/status grant of ADR-0017 covers the new write). Two behaviour changes:

• A terminating BIND9 pod's gate closes from its first second. The operator sets its bindy.firestoned.io/zones-loaded condition to False (reason PodTerminating) as soon as the pod is deleted (ADR-0017, decision 6). The pod's Ready follows when the kubelet next syncs its status (about 18 s on v0.8.0-rc.6), not at once, so this does not move traffic earlier by itself. Expect kubectl get pods to show the old pod not Ready for the end of its termination. Alerts that count not-Ready BIND9 pods should ignore terminating ones.
• Instances that share a zone or a cluster roll one at a time. A change that would roll several instances (a shared cluster ConfigMap change, an image change at the cluster or provider level, a bindy upgrade that changes the rendered configuration or the pod template) now rolls them in turn (ADR-0018). A cluster of N instances takes about N times one instance's rollout (roughly 20 to 30 s each with one replica). Waiting instances show Rollout=False, reason: RolloutQueued and keep serving; their status.observedGeneration stays at the last generation applied until their turn. Creating instances, changing replicas and RNDC key rotation are not delayed. Upgrading the operator to this version is itself such a change only if it alters the rendered pod template; the new operator staggers it.

If you run MetalLB in layer-2 mode with externalTrafficPolicy: Local, read LoadBalancer Services: each handover of a one-replica instance keeps a gap of MetalLB's re-announcement time.

BIND9 pods are Ready only once their zones are loaded (ADR-0017)

Every BIND9 pod now carries the readiness gate bindy.firestoned.io/zones-loaded (ADR-0017). A replacement pod stays out of its Service until the operator has loaded every live zone (and, on a primary, every record) onto it, so a rollout no longer leaves a nameserver answering REFUSED.

Required: apply the new RBAC with, or before, the new operator. The operator needs get and patch on pods/status in every namespace that runs BIND9 pods:

• cluster-wide install: re-apply deploy/operator/rbac/role.yaml;
• namespace-restricted install (BINDY_WATCH_NAMESPACES): re-apply deploy/operator/rbac/namespaced/role.yaml in every watched namespace.

Without it the gate cannot be opened: new pods stay not Ready (the old pods keep serving) and the operator logs 403 Forbidden on pods/status.

Every BIND9 Deployment rolls once. The gate is part of the pod template. Pods created before the upgrade have no gate and are not touched. An ADR-0013 stage 3 upgrade rolls the pods anyway, so this costs no extra rollout when both land together.

Behaviour to be aware of:

• kubectl rollout status for a BIND9 Deployment completes when the new pod's zones are loaded, not when named starts listening.
• If the operator is stopped, or its version predates the gate while the pod template has it, new BIND9 pods stay not Ready. Old pods keep serving (maxUnavailable 0). This is the intended fail-safe.
• A Service with publishNotReadyAddresses: true (a user override of the instance's Service spec) routes to pods regardless of readiness and so bypasses the gate.
• Diagnose a pod that stays not Ready from its bindy.firestoned.io/zones-loaded condition; see Troubleshooting.

No periodic resync: out-of-band BIND9 changes are no longer reverted (ADR-0016)

Every controller (records, DNSZone, Bind9Instance, Bind9Cluster, ClusterBind9Provider) used to requeue itself after every successful reconcile: every 5 minutes when Ready, every 30 seconds when not. That timer is gone (ADR-0016). No CRD, flag or RBAC change; the behaviour change is:

• A change made directly inside a running BIND9 pod stands until the next event for the resource that owns the data (spec, label or annotation change, pod restart, operator restart). It used to be reverted within 5 minutes. If you relied on that, force the repair by annotating the resource: kubectl annotate <kind> <name> bindy.firestoned.io/reconcile-trigger="$(date +%s)" --overwrite. See Troubleshooting.
• A resource stuck on a failure retries with backoff, from 2 s up to 60 s (a record write BIND9 rejected: never sooner than 30 s), instead of a flat 30 s.
• At rest the operator makes no reconciles and no API calls beyond its watches. Dashboards that expected a steady reconcile rate per object (bindy_firestoned_io_reconciliations_total) will show it drop to zero between changes.
• RNDC key rotation wakes the instance at the instant its key falls due instead of being noticed by the next 5-minute pass.

BIND9 configuration written by hornet (ADR-0013 stage 3)

named.conf and named.conf.options are now written by hornet's writer instead of text templates (ADR-0013).

Every BIND9 Deployment rolls once after the upgrade. The rendered text changes (layout, and the explanatory comments are gone), so each ConfigMap's content hash changes and its Deployment rolls, honouring the instance's PodDisruptionBudget. Plan the upgrade as you would any operand rollout.

dnssec.validation: true now renders dnssec-validation auto (was yes). yes needs trust-anchors, which bindy does not configure: on BIND 9.18 the server validated nothing, and BIND 9.20 refused to load the configuration at all. auto validates with BIND's built-in root trust anchor, which is what the field documents.

• How to detect: kubectl get bind9instances,bind9clusters,clusterbind9providers -A -o yaml | grep -B3 'validation: true'. On 9.18 those servers start validating after the upgrade; recursive answers for zones with a broken DNSSEC chain become SERVFAIL.
• Remediation: none needed for correct chains. To keep the old effective behaviour (no validation), set validation: false.

A DNSSEC key lifetime BIND does not accept is refused. kskLifetime: 1y (there is no y unit) used to reach named, which then refused the whole configuration. The operator now refuses to render it and the resource reports the error; use P1Y, 365d or 8760h.

Migrating to bindcar 0.9.0

Bindy now provisions bindcar 0.9.0 (ghcr.io/firestoned/bindcar:v0.9.0). For bindy deployments this is not breaking:

• Zone commands on a missing zone return 404. DELETE and the reload/freeze/thaw/notify/retransfer actions used to answer 500 when the zone was not loaded, which made bindy's zone deletion retry for minutes per instance (bindy bug-192). Bindy also checks a zone's status before deleting it, so it behaves the same against older bindcar images.
• TSIG is SHA-2 only. bindcar 0.9.0 rejects hmac-md5 and hmac-sha1 keys for dynamic updates, matching the RNDC policy it has enforced since 0.7.0. Bindy uses each instance's RNDC key for both, the CRD already rejects hmac-md5, and the default is hmac-sha256, so an instance this affects was already failing RNDC. To check: kubectl get bind9instances,bind9clusters,clusterbind9providers -A -o yaml | grep -i 'hmac-sha1'.
• bindcar 0.9.0 also adds post-quantum hybrid key exchange for its own TLS (bindcar ADR-0002); nothing to configure on the bindy side.

Migrating to bindcar 0.8.2

Bindy now provisions bindcar 0.8.2 (ghcr.io/firestoned/bindcar:v0.8.2), the live-zone DNSSEC lifecycle release (bindcar roadmap 07 / bindcar ADR-0001). Not breaking: no env vars, defaults, or API contracts consumed by bindy changed. What bindy gains:

• DNSZone.status.dnssec.nextKeyRollover is now populated for signed zones from the sidecar's rndc dnssec -status parsing (the field existed since ADR-0006 but had no data source). lastKeyRollover remains null: bindcar 0.8.x exposes the next scheduled event and current key states, not history.
• The v0.8.1 tag of bindcar carries a Cargo.toml version of 0.8.0 (tag/version divergence); v0.8.2 is the corrected release. Deployments should skip v0.8.1 images: they self-report 0.8.0.

Options rendering: explicit dnssec.validation always honored; cluster-level transfers denied by default (ADR-0007)

Two named.conf.options rendering asymmetries preserved by the roadmap-03 refactor are fixed (ADR-0007). Both change the generated ConfigMaps, so operand pods pick the change up on their next config rollout.

1. dnssec-validation is now rendered whenever dnssec is configured. Previously, a Bind9Instance with no spec.config block ignored a cluster-global dnssec.validation: false and emitted no directive, and an absent directive means auto to named, i.e. validation ON. Explicit configuration is now always honored.

• How to detect: Bind9Cluster has spec.global.dnssec.validation: false and instances without a spec.config block. Those operands validated upstream responses before; after the upgrade they will not.
• Remediation: if you actually wanted validation, set spec.global.dnssec.validation: true (or remove the dnssec block entirely to get named's default).

2. Cluster-level options now deny zone transfers by default. The cluster-level named.conf.options builder previously emitted no allow-transfer directive when spec.global.allowTransfer was unset, and BIND 9.18's own default allows AXFR to any host. It now renders allow-transfer { none; };, matching the instance-level builder (#466).

• How to detect: anything performing ad-hoc AXFR (monitoring probes, scripts using dig axfr) against operands configured via the cluster-level ConfigMap will start receiving transfer refusals. Operator-managed secondaries are unaffected: per-zone ACLs with the secondaries' IPs override the options-level default.
• Remediation: add the consumers' CIDRs to Bind9Cluster.spec.global.allowTransfer.

DNSZone gains status.recordsResyncPending (record replay after a pod is wiped)

Fixes #486. BIND9 operand pods hold zone data in ephemeral storage, so any pod replacement (an operator upgrade, a placement change that rolls the Deployment, an eviction, a node reboot, a kubectl delete pod) brings the pod back with no zones. The operator recreated the zone from spec (SOA and NS records only) but never replayed the record CRs, leaving the server authoritative for an empty zone: authoritative NXDOMAIN, or, with global.recursion + global.forwarders, the silently returned public answer for an internal name.

The zone reconciler now replays every record CR a zone selects whenever it creates the zone on a server, and records that intent in the new DNSZone.status.recordsResyncPending field so an incomplete replay is retried rather than forgotten.

Required: re-apply the CRDs

status.recordsResyncPending is a new field in the dnszones CRD schema. Until the CRD is updated, the API server prunes it silently (the status patch returns HTTP 200 and nothing persists). The operator still replays records on every zone recreation, but it cannot remember an incomplete replay across reconciliations, so a partial failure would not be retried and the zone would not stay out of Ready.

kubectl replace --force -f deploy/operator/crds/

⚠️ Use replace --force, not apply. The Bind9Instance, Bind9Cluster and ClusterBind9Provider CRDs exceed the 256KB last-applied-configuration annotation limit, so a plain kubectl apply is rejected. (kubectl apply --server-side also works.)

This is a status-only schema addition: it does not touch any spec, so existing DNSZone, record and instance resources are unaffected and no resource needs to be recreated. Rolling the CRDs does not interrupt DNS serving.

Rollout order

1. Re-apply the CRDs (above).
2. Roll out the new operator image.
3. Verify that a healthy zone reports the field as false:

kubectl get dnszone <zone-name> -n bindy-system \
  -o jsonpath='{.status.recordsResyncPending}{"\n"}'
# false

New behaviour to be aware of

A DNSZone with an outstanding record replay now reports Ready=False / Degraded=True with reason RecordsResyncPending. If you have automation or alerting that gates on DNSZone readiness, it will now (correctly) fire while a zone's records are being restored after a pod restart, typically for a few seconds. A zone stuck in that state means the replay is failing rather than pending; the Degraded message names each record that could not be pushed. See Common Issues.

named moves to the unprivileged port 5353

named now binds the unprivileged container port 5353 instead of 53, so the operand container drops the NET_BIND_SERVICE capability entirely (it now adds no capabilities, the strictest posture under PSA restricted). The DNS Service is unchanged from a client's view: it still exposes 53 and forwards to targetPort: 5353, so dig @<service> on port 53 keeps working.

This reverses the port decision from the bindcar 0.7.0 migration below. It is now possible because bindcar 0.7.2 accepts port-qualified transfer endpoints (<ip>:<port>): the operator publishes secondary primaries and primary also-notify as <ip>:5353, and sets NSUPDATE_PORT=5353 on the bindcar sidecar so dynamic updates reach named. allow-transfer remains a bare-IP ACL.

Area	Before	After
Operand DNS port	named on 53 with NET_BIND_SERVICE	named on 5353, no added capabilities
DNS Service	port 53 → targetPort 53	port 53 → targetPort 5353 (clients unchanged)
Secondary primaries	plain <ip> (port 53)	port-qualified <ip>:5353
Primary also-notify	plain <ip> (port 53)	port-qualified <ip>:5353
bindcar DDNS	nsupdate to 53	NSUPDATE_PORT=5353

⚠️ Rollout-blocking for multi-instance clusters. Primary and secondary operands must both run this version, since transfers now target 5353. A mixed fleet (some pods on 53, some on 5353) will fail zone transfers until fully rolled out. Requires bindcar 0.7.2+. NetworkPolicy ingress and peer egress must allow 5353 (see deploy/pod-hardening.yaml).

Migrating to bindcar 0.8.0

Bindy now provisions bindcar 0.8.0 (ghcr.io/firestoned/bindcar:v0.8.0).

No action is required to upgrade. Everything bindy relies on is unchanged, and bindcar 0.8.0 is backward compatible with the 0.7.x configuration bindy already emits.

What changed in bindcar

• TLS and mutual TLS are available for bindcar's REST API (BIND_TLS_CERT / BIND_TLS_KEY / BIND_TLS_CLIENT_CA), with certificate hot-reload (BIND_TLS_RELOAD_INTERVAL, default 60s). TLS is opt-in; bindcar serves plaintext unless configured otherwise, so this changes nothing until bindy grows a way to configure it.
• Rate-limit defaults were raised at v0.7.4: RATE_LIMIT_REQUESTS 100 → 600 and the burst 10 → 50. This is a fix, not a regression: the old burst of 10 guaranteed HTTP 429 during the record replay that follows a BIND9 pod restart, turning a ~30s recovery into ~130s. Bindy does not set these, so it simply inherits the better defaults.
• The crate is feature-gated. Bindy now depends on bindcar = { version = "0.8", default-features = false }, which drops bindcar's HTTP and TLS stacks from bindy's dependency graph: 31 crates, including utoipa, utoipa-swagger-ui, rust-embed, tower-http, tower_governor and governor. Bindy imports only the data types (ZoneConfig, SoaRecord, DnsRecord, ZoneResponse, ZONE_TYPE_*), which are unaffected.

Metrics

bindcar_zones_managed_total was renamed to bindcar_zones_managed in bindcar v0.7.3 (the _total suffix is reserved for counters, and this is a gauge). Bindy's bundled Grafana dashboard and metrics reference have been updated. If you maintain your own dashboards or PrometheusRules against the old name, they are silently returning no data. Rename them. During a rolling upgrade, bindcar_zones_managed or bindcar_zones_managed_total covers both.

Still outstanding

Serving the API over TLS requires a way to express the scheme and CA bundle on Bind9Instance / Bind9Cluster / ClusterBind9Provider. build_api_url already honours an explicit https://, so only the configuration surface is missing. Until that lands, the operator's ServiceAccount token still crosses the pod network in cleartext (audit finding P2-4).

---

Migrating to bindcar 0.7.0 (Mode B / TokenReview)

Bindy now provisions and consumes bindcar 0.7.2 (ghcr.io/firestoned/bindcar:v0.7.2), which enforces real authentication at startup, validates request payloads strictly, and requires a Pod Security Admission restricted-compliant pod. Bindy uses Mode B (TokenReview): the operator authenticates to the bindcar API with an audience-scoped ServiceAccount token, which bindcar validates via a Kubernetes TokenReview.

⚠️ This upgrade is rollout-blocking. A v0.6.0-shaped deployment will not come up under bindcar 0.7.0; follow every step below before rolling out.

What changed in Bindy

Area	v0.6.0 behavior	0.7.0 behavior
Auth	Presence-only Bearer token	TokenReview with enforced bindcar audience; allow-list names the operator SA (system:serviceaccount:<operator-ns>:bindy)
DNS port	named listened on 5353 (non-privileged)	named listens on 53 with NET_BIND_SERVICE (the one capability PSA restricted allows); Service targetPort follows
Secondary zones	primaries sent as "<ip> port 5353"	Plain IPs only; bindcar rejects non-IP entries with HTTP 400
RNDC keys	hmac-sha1 accepted	SHA-2 only: hmac-md5/hmac-sha1 rejected by both Bindy's parsers and bindcar
Sidecar pod shape	Writable rootfs, no seccomp	readOnlyRootFilesystem: true, RuntimeDefault seccomp, memory-backed emptyDir at /tmp + TMPDIR
Swagger/OpenAPI	Served by default	Off unless BIND_ENABLE_DOCS=true (set it via bindcarConfig.envVars for dev only; /api/v1/health, /api/v1/ready, /metrics unchanged)

Pre-upgrade checklist

1. Apply the TokenReview RBAC (the bindcar sidecar runs as the operand bind9 SA and must be able to create TokenReviews):

kubectl apply -f deploy/operator/rbac/tokenreview-clusterrole.yaml
kubectl apply -f deploy/operator/rbac/tokenreview-clusterrolebinding.yaml

If you run Bind9Instances outside bindy-system, add one binding subject per operand namespace's bind9 ServiceAccount.

1. Redeploy the operator with the updated deploy/operator/deployment.yaml, which projects a bindcar-audience token at /var/run/secrets/bindcar/token. Without it the operator falls back to its default SA token, which bindcar 0.7.0 rejects (wrong audience).

2. Rotate any hmac-sha1 RNDC keys to hmac-sha256. Operator-generated keys are already SHA-256; only external/user-managed Secrets are affected. Bindy fails closed at parse time on hmac-md5 and hmac-sha1.

3. NetworkPolicy (recommended): apply deploy/pod-hardening.yaml and replace the fail-closed 10.0.0.1/32 placeholder with your real kube-apiserver endpoint/CIDR: TokenReview (and therefore every authenticated bindcar request) is denied without it.

4. PSA restricted (recommended): label the operand namespace(s):

kubectl label ns bindy-system \
  pod-security.kubernetes.io/enforce=restricted --overwrite

The operator-built pods comply (non-root, drop-ALL + NET_BIND_SERVICE on named only, seccomp RuntimeDefault, read-only sidecar rootfs).

1. Admission policies: apply the new record-value policy (and the now-recommended RNDC-strict policy):

make admission-policies-install
kubectl apply -f deploy/admission-policies/05-bindy-rndc-strict-policy.yaml
kubectl apply -f deploy/admission-policies/06-bindy-rndc-strict-binding.yaml

1. DNS port change: anything that reached the operand pods on 5353 directly (custom NetworkPolicies, external monitors, firewall rules) must move to 53. The Service-facing port was already 53 and is unchanged.

2. Record hygiene: CNAMERecord.spec.target, MXRecord.spec.mailServer, NSRecord.spec.nameserver, SRVRecord.spec.target, and PTRRecord.spec.target must be absolute FQDNs ending with a trailing dot (e.g. mail.example.com.). The new bindy-record-value-validation admission policy rejects violations at the API server.

Rollout order

1. RBAC + admission policies (steps 1, 6): safe to apply ahead of time.
2. Operator Deployment (step 2).
3. Operand rollout: the operator recreates Bind9Instance pods with the 0.7.0 sidecar, port 53, and the hardened pod shape on its next reconcile.
4. NetworkPolicy + PSA labels (steps 4, 5) after confirming the operand pods are healthy.

Verification

# bindcar started and passed its auth startup guard
kubectl logs -n bindy-system deploy/<instance> -c api | head

# named is serving on 53
kubectl get svc -n bindy-system <instance> -o jsonpath='{.spec.ports}'

# zone ops authenticate (no 401/403 in operator logs)
kubectl logs -n bindy-system deploy/bindy | grep -iE 'unauthorized|forbidden' || echo OK

---

Migration Guide: v0.2.x → v0.3.x

This document explains how to migrate from Bindy v0.2.x to v0.3.x, which introduces breaking changes to how DNS records are associated with zones.

Breaking Changes in v0.3.0

1. Records Now Use Label Selectors (Breaking Change)

What Changed: The mechanism for associating DNS records with zones has changed from explicit references to label-based selection.

Before (v0.2.x): Explicit zoneRef References

# DNSZone
apiVersion: bindy.firestoned.io/v1beta1
kind: DNSZone
metadata:
  name: example-com
  namespace: bindy-system
spec:
  zoneName: example.com
  clusterRef: my-dns

---
# Record with explicit zone reference
apiVersion: bindy.firestoned.io/v1beta1
kind: ARecord
metadata:
  name: www
  namespace: bindy-system
spec:
  zoneRef: example-com  # ❌ This field no longer exists!
  name: www
  ipv4Addresses:
    - "192.0.2.1"

After (v0.3.0+): Label-Based Selection

# DNSZone selects records via labels
apiVersion: bindy.firestoned.io/v1beta1
kind: DNSZone
metadata:
  name: example-com
  namespace: bindy-system
spec:
  zoneName: example.com
  clusterRef: my-dns
  recordsFrom:  # ✅ New: Label selectors
    - selector:
        matchLabels:
          zone: example.com

---
# Record with matching labels
apiVersion: bindy.firestoned.io/v1beta1
kind: ARecord
metadata:
  name: www
  namespace: bindy-system
  labels:  # ✅ New: Labels for selection
    zone: example.com
spec:
  # NO zoneRef field - selection is via labels
  name: www
  ipv4Addresses:
    - "192.0.2.1"

Why This Change?

Problems with v0.2.x explicit references: 1. Tight coupling: Records hardcoded the zone name 2. Limited flexibility: Couldn't dynamically group records 3. No multi-environment support: Records couldn't belong to multiple zones 4. Manual management: Had to update every record when changing zones

Benefits of v0.3.0 label selectors: 1. Decoupled: Zones select records, not vice versa 2. Flexible: Use any label combination for selection 3. Dynamic: New records automatically picked up by matching zones 4. Multi-tenant: Isolate records by team, environment, application 5. Kubernetes-native: Uses standard label selector pattern

Migration Steps

Step 1: Backup Your Configuration

# Backup all DNS resources
kubectl get bind9clusters,dnszones -A -o yaml > clusters-zones-backup.yaml
kubectl get arecords,aaaarecords,cnamerecords,mxrecords,txtrecords,nsrecords,srvrecords,caarecords,ptrrecords -A -o yaml > records-backup.yaml

Step 2: Update CRDs

# Update to v0.3.0 CRDs
kubectl apply -f https://github.com/firestoned/bindy/releases/download/v0.3.0/crds.yaml

Or if installing from source:

kubectl apply --server-side -f deploy/operator/crds/

IMPORTANT: Use kubectl replace --force instead of kubectl apply to avoid the 256KB annotation size limit.

Step 3: Update DNSZone Resources

Add recordsFrom selectors to all DNSZone resources:

# For each DNSZone, add recordsFrom selector
kubectl edit dnszone example-com -n bindy-system

Add this to the spec:

spec:
  recordsFrom:
    - selector:
        matchLabels:
          zone: example.com  # Use the zone name as the label

Automation Script:

#!/bin/bash
# auto-migrate-zones.sh

# Get all DNSZones
kubectl get dnszones -A -o json | jq -r '.items[] | "\(.metadata.namespace) \(.metadata.name) \(.spec.zoneName)"' | while read ns name zonename; do
  echo "Migrating DNSZone $ns/$name (zone: $zonename)"

  # Patch the DNSZone with recordsFrom selector
  kubectl patch dnszone "$name" -n "$ns" --type=merge -p "{
    \"spec\": {
      \"recordsFrom\": [
        {
          \"selector\": {
            \"matchLabels\": {
              \"zone\": \"$zonename\"
            }
          }
        }
      ]
    }
  }"
done

Step 4: Update DNS Record Resources

Add labels to all DNS records matching the zone they belong to:

# For each record, add the zone label
kubectl label arecord www -n bindy-system zone=example.com

Automation Script:

#!/bin/bash
# auto-migrate-records.sh

RECORD_TYPES="arecords aaaarecords cnamerecords mxrecords txtrecords nsrecords srvrecords caarecords ptrrecords"

for record_type in $RECORD_TYPES; do
  echo "Migrating $record_type..."

  # Get all records and their old zoneRef
  kubectl get $record_type -A -o json | jq -r '.items[] | select(.spec.zoneRef != null) | "\(.metadata.namespace) \(.metadata.name) \(.spec.zoneRef)"' | while read ns name zoneref; do
    # Get the zone's zoneName
    zonename=$(kubectl get dnszone "$zoneref" -n "$ns" -o jsonpath='{.spec.zoneName}' 2>/dev/null)

    if [ -n "$zonename" ]; then
      echo "  Labeling $ns/$name with zone=$zonename"
      kubectl label $record_type "$name" -n "$ns" "zone=$zonename" --overwrite
    else
      echo "  WARNING: Could not find DNSZone $zoneref in namespace $ns"
    fi
  done
done

Step 5: Remove Old zoneRef Fields

The spec.zoneRef field no longer exists in v0.3.0 CRDs. After migration, you can optionally clean up your YAML files by removing these fields (they're already ignored by the new CRD).

Step 6: Upgrade the Operator

# Update the Bindy operator to v0.3.0
kubectl set image deployment/bindy bindy=ghcr.io/firestoned/bindy:v0.3.0 -n bindy-system

# Or apply the new deployment
kubectl apply -f https://github.com/firestoned/bindy/releases/download/v0.3.0/bindy.yaml

Step 7: Verify Migration

# Check DNSZone status - should show selected records
kubectl get dnszones -A -o jsonpath='{range .items[*]}{.metadata.name}{"\t"}{.status.recordsCount}{"\n"}{end}'

# Check that records have zone labels
kubectl get arecords,cnamerecords,mxrecords -A --show-labels

# Verify DNS resolution still works
kubectl port-forward -n bindy-system svc/<bind9-service> 5353:53
dig @localhost -p 5353 www.example.com

Advanced Migration Patterns

Multi-Environment Records

Use labels to support records in multiple environments:

# DNSZone for dev
apiVersion: bindy.firestoned.io/v1beta1
kind: DNSZone
metadata:
  name: example-com-dev
spec:
  zoneName: dev.example.com
  recordsFrom:
    - selector:
        matchLabels:
          app: myapp
          environment: dev

---
# DNSZone for prod
apiVersion: bindy.firestoned.io/v1beta1
kind: DNSZone
metadata:
  name: example-com-prod
spec:
  zoneName: prod.example.com
  recordsFrom:
    - selector:
        matchLabels:
          app: myapp
          environment: prod

---
# Record selected by dev zone
apiVersion: bindy.firestoned.io/v1beta1
kind: ARecord
metadata:
  name: myapp-dev
  labels:
    app: myapp
    environment: dev
spec:
  name: api
  ipv4Addresses:
    - "192.0.2.10"

Team-Based Isolation

Use team labels for multi-tenancy:

# Team A's zone
spec:
  recordsFrom:
    - selector:
        matchLabels:
          team: team-a

# Team A's record
metadata:
  labels:
    team: team-a

Troubleshooting

Records Not Appearing in Zone

Symptom: Records exist but don't show up in DNS queries.

Diagnosis:

# Check if zone selected the records
kubectl get dnszone example-com -n bindy-system -o jsonpath='{.status.recordsCount}'

# Check if records have the right labels
kubectl get arecords -n bindy-system --show-labels

Solution: - Ensure record labels match the zone's recordsFrom selector - Check that records are in the same namespace as the zone

Zone Shows Zero Records

Symptom: status.recordsCount is 0 or missing.

Diagnosis:

# Check the zone's selector
kubectl get dnszone example-com -n bindy-system -o yaml | grep -A 5 recordsFrom

Solution: - Add recordsFrom selector to the DNSZone spec - Ensure the selector matches at least one record's labels

Old zoneRef Field Error

Symptom: kubectl apply fails with "unknown field spec.zoneRef"

Solution: - Update the CRDs to v0.3.0: kubectl apply --server-side -f deploy/operator/crds/ - Remove spec.zoneRef from your YAML files (field no longer exists)

Rollback Procedure

If you need to rollback to v0.2.x:

1. Restore old CRDs:

   kubectl apply -f https://github.com/firestoned/bindy/releases/download/v0.2.x/crds.yaml
   

2. Restore backups:

   kubectl apply -f clusters-zones-backup.yaml
   kubectl apply -f records-backup.yaml
   

3. Downgrade operator:

   kubectl set image deployment/bindy bindy=ghcr.io/firestoned/bindy:v0.2.x -n bindy-system
   

Migrating from nameServerIps to nameServers (v0.4.0)

What Changed: The nameServerIps field is deprecated in favor of the new nameServers field, which provides better structure and clarity.

Why This Change?

Problems with nameServerIps: 1. Misleading name: Suggests only glue records, but actually defines authoritative nameservers 2. Limited structure: HashMap format doesn't support IPv6 addresses 3. No IPv6 glue records: Can't specify AAAA records for nameservers 4. Unclear purpose: Name doesn't convey that NS records are auto-generated

Benefits of nameServers: 1. Clear intent: Explicitly represents authoritative nameservers 2. Structured data: Separate fields for hostname, IPv4, and IPv6 3. Dual-stack support: Both IPv4 and IPv6 glue records 4. Better documentation: Field name matches DNS terminology

Old Format (Deprecated)

apiVersion: bindy.firestoned.io/v1beta1
kind: DNSZone
metadata:
  name: example-com
  namespace: bindy-system
spec:
  zoneName: example.com
  clusterRef: production-dns
  soaRecord:
    primaryNs: ns1.example.com.
    adminEmail: admin.example.com.
    serial: 2025012101
    refresh: 3600
    retry: 600
    expire: 604800
    negativeTtl: 86400
  # OLD: nameServerIps (deprecated in v0.4.0)
  nameServerIps:
    ns2.example.com.: "192.0.2.2"
    ns3.example.com.: "192.0.2.3"
  ttl: 3600

New Format (v0.4.0+)

apiVersion: bindy.firestoned.io/v1beta1
kind: DNSZone
metadata:
  name: example-com
  namespace: bindy-system
spec:
  zoneName: example.com
  clusterRef: production-dns
  soaRecord:
    primaryNs: ns1.example.com.
    adminEmail: admin.example.com.
    serial: 2025012101
    refresh: 3600
    retry: 600
    expire: 604800
    negativeTtl: 86400
  # NEW: nameServers (v0.4.0+)
  nameServers:
    - hostname: ns2.example.com.
      ipv4Address: "192.0.2.2"
    - hostname: ns3.example.com.
      ipv4Address: "192.0.2.3"
      ipv6Address: "2001:db8::3"  # Now supports IPv6!
  ttl: 3600

Migration Steps

1. Update CRDs to v0.4.0+

# Update CRDs from latest release
kubectl apply -f https://github.com/firestoned/bindy/releases/latest/download/crds.yaml

# Or from source
make crds-combined
kubectl replace --force -f deploy/crds.yaml

2. Identify Zones Using nameServerIps

# Find all DNSZones with nameServerIps field
kubectl get dnszones -A -o json | jq -r '.items[] | select(.spec.nameServerIps != null) | "\(.metadata.namespace)/\(.metadata.name)"'

3. Convert Each Zone

For each zone found, update the spec:

Manual Conversion:

# Edit the zone
kubectl edit dnszone example-com -n bindy-system

Then change the format from HashMap to array:

# Remove old field
nameServerIps:
  ns2.example.com.: "192.0.2.2"
  ns3.example.com.: "192.0.2.3"

# Add new field
nameServers:
  - hostname: ns2.example.com.
    ipv4Address: "192.0.2.2"
  - hostname: ns3.example.com.
    ipv4Address: "192.0.2.3"

Automated Conversion Script:

#!/bin/bash
# migrate-nameserverips.sh

kubectl get dnszones -A -o json | jq -r '.items[] | select(.spec.nameServerIps != null) | "\(.metadata.namespace) \(.metadata.name)"' | while read ns name; do
  echo "Migrating DNSZone $ns/$name"

  # Get current nameServerIps as JSON
  old_ips=$(kubectl get dnszone "$name" -n "$ns" -o json | jq -c '.spec.nameServerIps // {}')

  # Convert to new nameServers format
  new_servers=$(echo "$old_ips" | jq -c 'to_entries | map({hostname: .key, ipv4Address: .value})')

  # Patch the DNSZone
  kubectl patch dnszone "$name" -n "$ns" --type=json -p "[
    {\"op\": \"add\", \"path\": \"/spec/nameServers\", \"value\": $new_servers},
    {\"op\": \"remove\", \"path\": \"/spec/nameServerIps\"}
  ]"

  echo "  ✓ Migrated $ns/$name"
done

4. Add IPv6 Glue Records (Optional)

If you have dual-stack nameservers, add IPv6 addresses:

nameServers:
  - hostname: ns2.example.com.
    ipv4Address: "192.0.2.2"
    ipv6Address: "2001:db8::2"  # Add IPv6 glue record

5. Verify Migration

# Check that zones no longer use nameServerIps
kubectl get dnszones -A -o json | jq '.items[] | select(.spec.nameServerIps != null) | "\(.metadata.namespace)/\(.metadata.name)"'

# Should return nothing if migration is complete

# Verify nameServers field is present
kubectl get dnszone example-com -n bindy-system -o jsonpath='{.spec.nameServers}'

6. Check Operator Logs

After migration, ensure no deprecation warnings:

kubectl logs -n bindy-system -l app=bindy-operator -f | grep nameServerIps

You should NOT see any deprecation warnings after migration.

Backward Compatibility

Good News: Existing zones using nameServerIps will continue to work in v0.4.0+. The operator automatically converts the old format internally.

// In crates/bindy-controller-zone/src/dnszone.rs
fn get_effective_name_servers(spec: &DNSZoneSpec) -> Option<Vec<NameServer>> {
    if let Some(ref new_servers) = spec.name_servers {
        // New field takes precedence
        return Some(new_servers.clone());
    }

    // Fallback to deprecated field with automatic conversion
    if let Some(ref old_ips) = spec.name_server_ips {
        warn!("DNSZone uses deprecated `nameServerIps` field. Migrate to `nameServers`.");
        let servers: Vec<NameServer> = old_ips
            .iter()
            .map(|(hostname, ip)| NameServer {
                hostname: hostname.clone(),
                ipv4_address: Some(ip.clone()),
                ipv6_address: None,
            })
            .collect();
        return Some(servers);
    }

    None
}

Deprecation Timeline: - v0.4.0: nameServerIps deprecated, still functional with warnings - v1.0.0: nameServerIps will be removed entirely

Troubleshooting

Deprecation Warnings in Logs

Symptom: Operator logs show warnings about nameServerIps:

WARN DNSZone uses deprecated `nameServerIps` field. Migrate to `nameServers`.

Solution: Follow migration steps above to convert to nameServers format.

Zone Still Has Both Fields

Symptom: Zone has both nameServers and nameServerIps defined.

Diagnosis:

kubectl get dnszone example-com -n bindy-system -o yaml | grep -A 10 "nameServer"

Solution: Remove nameServerIps field - the new nameServers takes precedence:

kubectl patch dnszone example-com -n bindy-system --type=json -p '[{"op": "remove", "path": "/spec/nameServerIps"}]'

Missing IPv6 Glue Records

Symptom: Dual-stack nameservers only have A records, no AAAA records.

Solution: Add ipv6Address to nameServers entries:

nameServers:
  - hostname: ns2.example.com.
    ipv4Address: "192.0.2.2"
    ipv6Address: "2001:db8::2"  # Add this

Security hardening (RED-team remediation)

This release tightens multi-tenant isolation. Three changes reject inputs that were previously accepted. Review each before upgrading.

1. Scout no longer publishes cross-namespace by default (Breaking)

What changed: Scout (the Ingress/Service/Route watcher) previously created DNS records in any existing zone. It now requires the target DNSZone to authorize the source object's namespace; otherwise the record is skipped and Scout logs ... namespace not authorized for zone ....

Who is affected: anyone running Scout where the Ingress/Service/Route lives in a different namespace than the DNSZone (the common case; DNSZones usually live in bindy-system).

How to migrate: annotate each DNSZone with the namespaces allowed to publish into it (comma-separated), or * to restore the previous publish-from-anywhere behavior:

apiVersion: bindy.firestoned.io/v1beta1
kind: DNSZone
metadata:
  name: example-com
  namespace: bindy-system
  annotations:
    # allow these namespaces' Ingresses/Services/Routes to publish into this zone
    bindy.firestoned.io/allow-zone-namespaces: "tenant-a,tenant-b"
    # ...or "*" for the old behavior (NOT recommended on multi-tenant clusters)
spec:
  zoneName: example.com

A DNSZone in the same namespace as the source object is always authorized and needs no annotation.

2. DNSSEC keysFrom.secretRef must be a bindy- Secret (Breaking)

What changed: spec.dnssec.signing.keysFrom.secretRef.name must now start with bindy-, matching the prefix already required for user secret: volumes. This stops a tenant from mounting another tenant's Secret (e.g. an RNDC/TSIG key) into their BIND pod.

How to migrate: rename the DNSSEC key Secret to a bindy- prefix and update the reference:

spec:
  dnssec:
    signing:
      enabled: true
      keysFrom:
        secretRef:
          name: bindy-example-dnssec-keys   # was: example-dnssec-keys

3. volumeMount.mountPath may not contain ..

What changed: user volumeMounts on Bind9Instance/Bind9Cluster are rejected if mountPath contains a .. segment (e.g. /data/../etc/bind/named.conf), which previously slipped past the prefix allow-list. Legitimate mounts under /data/ or /var/log/bind/ are unaffected; only paths using .. need to be rewritten to their intended absolute location.

All three are enforced both in the operator and, if installed, by the bindy-pod-shape-validation / Scout admission policies (see deploy/admission-policies/).

See Also

• DNSZone Concepts - Detailed explanation of label selectors
• DNSZone Specification - Complete field reference
• Multi-Nameserver Example - Working example
• Architecture Overview - Event-driven reconciliation
• API Reference - Complete CRD schemas
• GitHub Release Notes - Full changelog

Support

For issues or questions: - GitHub Issues - Documentation