Skip to content

ADR-0024: Ansible repository layout and provisioning model

  • Status: Accepted
  • Date: 2026-08-17
  • Deciders: El, Lachlan

Context

Ansible is the provisioning mechanism (ADR-0007, ADR-0014), but its repository layout was never specified. That layout is not a directory preference — it is the mechanism that makes tenant two cheap. Get the inventory structure wrong and the failure mode is per-client forks, which is exactly the drift ADR-0003 exists to prevent.

The structural problem: a tenant is not a host. Several tenants share one box (ADR-0025), so Ansible's host-centric inventory does not map onto the unit of work.

Decision

  1. A tenant is an inventory host; ansible_host points at the physical box.
tenants:
  hosts:
    greenhills:
      ansible_host: 10.x.x.x
      tenant_slug: greenhills
      region: au-mel
      services: [core, n8n, metabase]

--limit greenhills targets one client naturally; moving a tenant to a dedicated box (ADR-0025) is one line changed, not a restructure; and the slug is the same immutable key across inventory, vault ID, database name, DNS and directory paths (ADR-0022). 2. Host-level tasks run against a separate servers group — hardening, Docker, Caddy — so they execute once per machine, not once per tenant. 3. Layout, under ansible/ in this repository:

inventory/hosts.yml
tenants/{slug}/vars.yml
tenants/{slug}/vault.yml        # --vault-id {slug}
roles/  base docker caddy core_stack tenant_stack metabase backup dns
playbooks/  provision-host.yml provision-tenant.yml
            deploy-tenant.yml verify-restore.yml

Same repository as the ADRs that govern it — splitting them means two review paths for one change. Vault files are encrypted, so co-location exposes nothing. 4. Static inventory in git, not dynamic inventory from the registry. The registry runs on the box being provisioned, so dynamic inventory would put the control plane in the recovery path: rebuilding after total loss would require the thing being rebuilt. Static inventory also gives diff, review and rollback on infrastructure changes. 5. The resulting two records of the tenant list are ADR-0010 tiering, not duplication. Git holds infrastructure facts (host, region, enabled services); the registry holds business facts (rules, thresholds, taxonomy). They overlap only on the tenant list, which is precisely what the ADR-0006 reconciliation pass polices. Drift is detected, not discovered. 6. Registry pre-flight gate. provision-tenant.yml fails closed unless the registry holds a matching tenant record whose region matches inventory. This prevents a tenant existing in infrastructure but not in the map — the state in which nothing knows who owns the data. After provisioning, the tenant is registered and reconciled. The same check runs as a scheduled job, because drift appears between runs, not during them. 7. --force exists for genuine recovery and requires a logged reason, making a bypass a deliberate, recorded act. Accepted cost: a registry outage blocks new provisioning. 8. Secrets are per-tenant. A separate --vault-id per tenant plus one for core, so one vault password does not open every client. Values encrypted inline with ansible-vault encrypt_string so variable names stay readable and a secrets change is reviewable in a PR without being decrypted (ADR-0027). 9. Metabase is a conditional role driven by services (ADR-0023), and its pinned version has a scheduled bump task. 10. DNS record creation is automated via the dns role using a zone-scoped Cloudflare token. Deletion is never automated — deprovisioning marks records for removal and a human executes (ADR-0022). 11. Provisioning a tenant includes confirming its new vault password is in the shared credential store (ADR-0027) before the tenant is considered provisioned.

Scope note: the gate in §10 is justified by blast radius, not by the "no review, no go" commitment. That commitment governs AI-generated actions. Deterministic automation — scheduled syncs, provisioning runs — executes without approval.

Consequences

Adding a tenant is an inventory entry, a vars file, an encrypted vault file and a playbook run. Promoting a tenant to a dedicated host is a one-line change. Per-client forks have no natural place to form, because variation lives in vars.yml and registry config rather than in roles. Static inventory keeps recovery independent of the control plane, at the cost of a tenant list in two places that must be reconciled.

Alternatives considered

  • Dynamic inventory from the registry — rejected: puts the control plane in the recovery path and removes diff and rollback from infrastructure changes.
  • One inventory host per box, tenants as a variable list — rejected: --limit no longer targets one client, and moving a tenant becomes a restructure.
  • A separate Ansible repository — rejected: two review paths for one change.
  • Automating DNS deletion for a one-command deprovision — rejected on blast radius.