Skip to content

ADR-0042: Inventory repository, tenant secrets and new vault passwords

  • Status: Accepted
  • Date: 2026-10-15
  • Deciders: El
  • Supersedes: ADR-0024 §4 in part (the static inventory lives in its own repository, which the runner may commit to); ADR-0039 §7 in part (the runner also checks out the inventory repository, at a recorded commit); ADR-0006 in part (the Console, not Ansible, creates a tenant record, at the approval of create tenant); ADR-0027 §11 in part (how a new tenant's vault password reaches Bitwarden)
  • Relates to: ADR-0037, ADR-0040; OI-108, OI-124, OI-126; the phase 4 plan

Context

Create tenant (ADR-0039 §2) makes a vault ID, an encrypted vault file and an inventory entry. Two accepted decisions stop the runner writing either:

  • ADR-0024 §4 keeps the inventory static and in git, for diff, review, rollback, and a recovery that does not need the control plane.
  • ADR-0039 §7 runs the runner from a clean checkout of main, so it cannot change the repository it runs from, and must not: merging to main is how code changes, and it is reviewed.

In practice the inventory is already outside git: infra/inventory/hosts.yml is ignored, so it exists only on octo-ctl. Only group_vars/all is tracked. And several tenants share one host (ADR-0024 §1), so a tenant's secrets cannot live in its host's file (OI-124).

The runner also creates the new tenant's vault password, and ADR-0027 §11 requires it in Bitwarden before the tenant counts as provisioned. OI-108 asks for a custody step that never sends the password to the registry.

Decision

flowchart LR
    OP(["Approver"]) -->|"approve: passkey step-up<br/>+ one-time public key"| CON["Console"]
    R["Runner<br/>octo-ctl"] -->|"pulls job"| CON
    R -->|"code at main, exact commit"| CODE[("octopodia-abos")]
    R <-->|"writes tenant files,<br/>commits, pushes"| INV[("octopodia-abos-inventory<br/>private")]
    R -->|"ansible-playbook"| H["Host"]
    R -->|"new vault password,<br/>encrypted to the approver's key"| CON
    CON -->|"ciphertext, once,<br/>after a second step-up"| OP
    OP -->|"stores it"| BW[("Bitwarden")]

1. The inventory lives in its own private repository

octopodia-abos-inventory, private on GitHub, both partners as admins. It holds everything Ansible reads as inventory:

hosts.yml                       tenants as inventory hosts, ansible_host the box (ADR-0024 §1)
group_vars/all/main.yml         shared, not secret (admin keys, ports)
group_vars/all/vault.yml        only what every host shares (the tailnet auth key), --vault-id core
host_vars/<host>/vault.yml      one machine's secrets: tunnel token, alerts secret, registry passwords; core vault ID
tenants/<slug>/vars.yml         one tenant's settings for Ansible
tenants/<slug>/vault.yml        one tenant's secrets: n8n encryption key, link and detail tokens, AI key; --vault-id <slug>

Values are encrypted inline with encrypt_string (ADR-0027 §4), so a diff shows which secret changed without showing it. The code repository keeps the roles, playbooks, job definitions and the example inventories, and no live inventory: infra/inventory/group_vars/all moves to the new repository.

2. The runner commits to it; people may too

  • The runner is the only automated writer. It writes a tenant's files, commits to main with the job ID and the approver in the message, and pushes. Its credential is a deploy key with write access to this repository only. Force pushes are refused; a mistake is undone by a revert, which is itself a commit.
  • People commit too: break-glass and hand changes, through a pull request when there is time.
  • Every job records two commits: the code commit it ran (ADR-0039 §7) and the inventory commit it read or wrote.
  • Create tenant writes and pushes before it runs a playbook. If the push fails, nothing runs.

3. Both checkouts are clean, and recovery needs neither the registry nor the runner

The runner keeps its own clean clones of both repositories and refuses to run from a dirty tree. El's hand runs use a clone of the inventory repository beside the code (ansible.cfg points to ../octopodia-abos-inventory/hosts.yml). The inventory is on GitHub and in every clone, so rebuilding after a total loss does not need the control plane, which is ADR-0024 §4's reason for static inventory, kept.

4. The Console creates the tenant record at approval

When create tenant is approved, the Console writes the tenant record from the validated inputs, marked provisioning, and records it in History. The playbook's pre-flight gate (ADR-0024 §6) checks that record through the runner's Console token and fails closed without it. When the job succeeds, the Console marks the tenant provisioned; when it fails, the record stays as provisioning failed, visible, never deleted (nothing destructive in v1, ADR-0039 §3). This replaces ADR-0006's "Ansible creates the tenant record at provisioning", and matches the adding a tenant runbook. The console role's seed list stays for a Console's first tenants by hand (break-glass), as lite-dev's today.

5. A new vault password is shown once to the approver, and never reaches the Console in plain text

  1. At approval, the approver's browser makes a one-time key pair (Web Crypto, the private key not extractable, kept in that browser). The public key goes with the approval, and the passkey step-up (ADR-0040 §4) signs it with the job's ID and inputs, so it cannot be swapped.
  2. The runner makes the vault password, encrypts the tenant's vault file with it, keeps it in its own vault password file (ADR-0039 §5), and uploads only the password encrypted to that public key.
  3. To reveal it, the approver opens the job on the same browser and makes a second passkey step-up, bound to "reveal, job N". The browser fetches the ciphertext, decrypts it locally and shows it once. The Console deletes the ciphertext on that first fetch, or after 24 hours if never fetched.
  4. The custody task on the job stays open until the approver confirms it is stored in Bitwarden. The tenant is provisioned only then (ADR-0027 §11). History records who it was shown to and when, never the value.
  5. If it is lost (browser closed, cleared or never revealed), the fallback is break-glass on octo-ctl: a person reads it from the runner's file with sudo, and History records that as break-glass.

The registry and the Console hold only ciphertext they cannot read, for at most 24 hours. Job output never contains the password (no_log, OI-112).

6. What this does not change

The runner still holds every tenant's vault password unattended (ADR-0039 §5, OI-108). This ADR settles how a new one reaches Bitwarden; the concentration stays recorded on OI-108, accepted for v1.

Consequences

Create tenant can run end to end from the Console, and every inventory change still has a diff, an author (the runner, with the job and approver) and a revert. Recovery needs only GitHub or any clone. The cost is a second repository to clone, a write key for the runner, and an approver who must reveal the password on the device that approved. A person who approves on a phone reveals on that phone.

Alternatives considered

  • The runner writes on a branch and a person merges before the playbook runs. Keeps ADR-0024 and ADR-0039 unchanged, but adds a step outside the Console and a write key on the main repository. Rejected: the acceptance test is "all started from the Console".
  • Inventory only in the runner's folder on octo-ctl. Simplest, but no history, review or second copy. Rejected.
  • Dynamic inventory from the registry. Already rejected by ADR-0024: it puts the control plane in its own recovery path.
  • The new vault password shown by the Console in plain text. Rejected: the Console would hold a vault password, even briefly, and anything with the Console's database or logs could read it.
  • The password only on octo-ctl, copied by a person each time. Works, and is kept as the fallback, but makes every new tenant a terminal session.