Skip to content

ADR-0024: Ansible repository layout and provisioning model

  • Status: Accepted. §4 superseded in part by ADR-0042 (the static inventory lives in its own private repository, which the runner commits to)
  • Date: 2026-08-17
  • Deciders: El, Lachlan

Context

Ansible is the provisioning mechanism (ADR-0007, ADR-0014), but its repository layout was never specified. That layout is not a directory preference: it is the mechanism that makes tenant two cheap. Get the inventory structure wrong and the failure mode is per-client forks, which is exactly the drift ADR-0003 exists to prevent.

The structural problem: a tenant is not a host. Several tenants share one box (ADR-0025), so Ansible's host-centric inventory does not map onto the unit of work.

Decision

  1. A tenant is an inventory host, and ansible_host points at the physical box.
tenants:
  hosts:
    greenhills:
      ansible_host: 10.x.x.x
      tenant_slug: greenhills
      region: au-mel
      services: [core, n8n, metabase]

--limit greenhills then targets one client naturally. Moving a tenant to a dedicated box (ADR-0025) is one line changed rather than a restructure. And the slug is the same immutable key across inventory, vault ID, database name, DNS and directory paths (ADR-0022). 2. Host-level tasks run against a separate servers group, covering hardening, Docker and Caddy, so they execute once per machine rather than once per tenant. 3. Layout, under infra/ in this repository:

inventory/hosts.yml
tenants/{slug}/vars.yml
tenants/{slug}/vault.yml        # --vault-id {slug}
roles/  common hardening tailscale docker caddy
        core_stack tenant_stack metabase backup dns
playbooks/  host-hardening.yml tailnet.yml dns.yml edge.yml
            provision-tenant.yml deploy-tenant.yml verify-restore.yml

Same repository as the ADRs that govern it, because splitting them means two review paths for one change. Vault files are encrypted, so co-location exposes nothing. 4. Static inventory in git, not dynamic inventory from the registry. The registry runs on the box being provisioned, so dynamic inventory would put the control plane in the recovery path: rebuilding after total loss would require the thing being rebuilt. Static inventory also gives diff, review and rollback on infrastructure changes. 5. The resulting two records of the tenant list are ADR-0010 tiering, not duplication. Git holds infrastructure facts (host, region, enabled services), and the registry holds business facts (rules, thresholds, taxonomy). They overlap only on the tenant list, which is exactly what the ADR-0006 reconciliation pass exists to police. Drift is detected rather than discovered. 6. Registry pre-flight gate. provision-tenant.yml fails closed unless the registry holds a matching tenant record whose region matches inventory. This prevents a tenant existing in infrastructure but not in the map, which is the state in which nothing knows who owns the data. After provisioning, the tenant is registered and reconciled. The same check runs as a scheduled job, because drift appears between runs rather than during them. 7. --force exists for genuine recovery and requires a logged reason, making a bypass a deliberate, recorded act. The accepted cost is that a registry outage blocks new provisioning. 8. Secrets are per-tenant. A separate --vault-id per tenant plus one for core, so one vault password does not open every client. Values are encrypted inline with ansible-vault encrypt_string, so variable names stay readable and a secrets change is reviewable in a PR without being decrypted (ADR-0027). 9. Metabase is a conditional role driven by services (ADR-0023), and its pinned version has a scheduled bump task. 10. DNS record creation is automated via the dns role using a zone-scoped Cloudflare token. Deletion is never automated: deprovisioning marks records for removal and a human executes (ADR-0022). 11. Provisioning a tenant includes confirming its new vault password is in the shared credential store (ADR-0027) before the tenant is considered provisioned.

Scope note: the gate in §10 is justified by blast radius, not by the "no review, no go" commitment. That commitment governs AI-generated actions. Deterministic automation, such as scheduled syncs and provisioning runs, executes without approval.

Consequences

Adding a tenant is an inventory entry, a vars file, an encrypted vault file and a playbook run. Promoting a tenant to a dedicated host is a one-line change. Per-client forks have no natural place to form, because variation lives in vars.yml and registry config rather than in roles. Static inventory keeps recovery independent of the control plane, at the cost of a tenant list in two places that must be reconciled.

Alternatives considered

  • Dynamic inventory from the registry is rejected: a single source of truth is appealing, but it puts the control plane in the recovery path and removes diff and rollback from infrastructure changes.
  • One inventory host per physical box, with tenants as a variable list, is rejected: --limit no longer targets a single client, and moving a tenant to its own box becomes a restructure.
  • A separate Ansible repository is rejected: two review paths for one change.
  • Automating DNS deletion for a one-command deprovision is rejected on blast radius.