Skip to content

Architecture: overview

Scope of this section: how the system is shaped, and why. System-wide, and it changes rarely.

Three pages, three zoom levels:

  • This page: the logical view. Planes, engines, surfaces, and the rules that override convenience.
  • Deployment topology: the physical view. Containers, networks, ports and hostnames on an actual host.
  • Stack and sizing: every component, what it does, and what it costs in RAM, CPU and disk.

It deliberately does not contain per-product detail. A data model for one product is a recipe over this architecture, not part of it, and lives under Products. If a document only makes sense for one product, it does not belong here.

In one line

A shared, stateless brain plus per-tenant isolated stacks, driven by config resolved at runtime, acting on the tools clients already run. Own the spine (state, decisions, memory) and orchestrate the body.

The two planes

Everything follows from this split.

flowchart LR
    subgraph CP ["Control plane: shared"]
        REG[("Registry<br/>tenant records, resolved config,<br/>connector manifests, metadata")]
    end

    subgraph BR ["The brain: shared, stateless"]
        E1["Capture"]
        E2["Progress"]
        E3["Bill"]
        E4["Remember"]
    end

    subgraph TP ["Tenant plane: one per client"]
        T1[("Client A spine")]
        T2[("Client B spine")]
    end

    REG -. "read-only" .-> BR
    BR --> T1
    BR --> T2

The control plane holds the map, never the content. Tenant records, resolved config, connector manifests, metadata. No emails, no invoices, no client data. Ever (ADR-0006).

The tenant plane holds the content. Each client's own Postgres is the spine, plus their own n8n instance. Isolation is the container and volume boundary, not a tenant_id column (ADR-0023).

The brain is shared and stateless. It reads config from the registry and acts on tenant data. It never writes the registry.

Two writers only: Ansible creates the tenant record at bootstrap, the BOS Console edits config. Nothing else (ADR-0006).

Engines vs products

Engines are machinery, shared by everyone. A work item flows through them:

flowchart LR
    IN(["Inbound<br/>email · form · webhook"]) --> CAP["<b>Capture</b><br/>interpret it"]
    CAP --> PROG["<b>Progress</b><br/>work-item<br/>state machine"]
    PROG --> BILL["<b>Bill</b><br/>money<br/>state machine"]
    REM[("<b>Remember</b><br/>retrieval + memory<br/>pgvector")]
    REM -.->|"context"| CAP
    REM -.->|"context"| PROG
    PROG --> GATE{"AI-generated<br/>action?"}
    GATE -->|"yes"| APPROVE["⏸️ Human approval<br/><i>no review, no go</i>"]
    GATE -->|"no, deterministic"| ACT(["Execute"])
    APPROVE -->|"approved"| ACT
    APPROVE -->|"expired"| PROG

Bill runs parallel to Progress, not after it. Money and work have separate lifecycles, and conflating them is how a paid-but-incomplete job becomes invisible.

An expired approval re-queues, it never auto-approves (ADR-0026).

Engine Does
Capture Interpret inbound: email, form, webhook
Progress Work-item state machine
Bill Money state machine, parallel to Progress
Remember Retrieval and memory, pgvector

Products are switch-on patterns over those engines, such as email triage and invoicing. An engine is never built "for" one product or one client (ADR-0002).

flowchart TB
    subgraph P ["Products: switch-on patterns"]
        P1["Email triage"]
        P2["Invoicing"]
        P3["…"]
    end
    subgraph E ["Engines: shared machinery"]
        E1["Capture"]
        E2["Progress"]
        E3["Bill"]
        E4["Remember"]
    end
    P1 --> E1
    P1 --> E2
    P2 --> E2
    P2 --> E3
    P3 -.-> E4

This is what makes client specificity config rather than code. New behaviour is a config change. If it genuinely needs new logic, the shared vocabulary is extended so every tenant benefits, then switched on as config (ADR-0003).

Surfaces

Who may write what. The arrows are the whole rule:

flowchart LR
    OPS(["Operators<br/>El · Lachlan"]) --> CONSOLE["BOS Console"]
    CLIENT(["Client users"]) --> APP["Client review app"]
    ANS(["Ansible"]) -->|"creates tenant record<br/>at bootstrap"| REG
    CONSOLE -->|"edits config"| REG[("Control plane")]
    APP -->|"approvals"| TEN[("Own tenant data only")]
    APP -.->|"never"| REG
    BRAIN["The brain"] -.->|"read-only"| REG
Surface Audience May write
BOS Console Operators only Control plane
Client review app One tenant's users That tenant's data only
Documentation Us Nothing, authored in git

Two writers only to the control plane: Ansible at bootstrap, the Console thereafter. The brain reads it and never writes it (ADR-0006).

The Console is its own application at console.octopodia.com.au, behind Cloudflare Access (ADR-0037, ADR-0040). The client app never touches the control plane.

portal is reserved for the Internal Portal and is used for neither (ADR-0022 §4).

Where a problem goes

Applies to every product. Three destinations, and conflating them is how a client ends up looking at our outages.

flowchart LR
    P["Something<br/>went wrong"] --> Q{"Whose problem<br/>is it?"}
    Q -->|"needs a human decision"| R(["Client review app"])
    Q -->|"needs us to fix it"| C(["BOS Console"])
Goes to Who fixes it Client sees
Escalated by design Client human, in the review app Nobody. It is working as intended The item
Escalated, cannot determine Client human, in the review app Us, eventually. It is a quality signal The item
System failure Us, in the Console Us, now Nothing. It retries

A system failure never reaches the client. A database being unreachable, a provider outage, an expired token: none of those are a decision anyone can make from a queue. The work stays unprocessed and the fix is ours.

The two escalations look identical to the person receiving them and mean opposite things in reporting, which is why the work item's status distinguishes them rather than the reason alone. One is the product working. The other is a measurable gap.

Some failures are genuinely both. A model response that will not parse still needs a human to handle the work today, and signals a regression only we can fix. Those escalate to the client and alert us.

Rules that override convenience

Rule ADR
Config, not code, for client specificity ADR-0003
Structured predicates only. No expression language, no eval ADR-0004
Rule precedence: guardrails → vertical template → tenant authored → learned ADR-0005
Control plane holds the map, never client content ADR-0006
Config applies to new work items only, never retroactively ADR-0006
Data residency is hard: AU infrastructure, region a first-class field ADR-0008
Human approval before any AI-generated action executes ADR-0026

Where to go next