Prompts¶

Rules that apply to every prompt in ABOS, whichever product or engine it belongs to.

Prompts are the only place model behaviour is specified, so they are treated like code: versioned, reviewed, and recorded against every decision they produced.

The prompt itself is not in these docs. It is a template in the brain's source, versioned and reviewed in a PR. A copy here would be a second version that drifts, and the prettier one is the one people would read. These pages carry the reasoning, which a code comment cannot.

Where the template lives is not yet decided

Which directory, how prompt_version is generated, and whether prompts ship with the brain or independently are all open (OI-076).

Worth settling before the brain is built: retrofitting a versioning scheme makes every prompt_version recorded before the fix meaningless.

Anatomy¶

Every prompt has the same ten sections in the same order. What differs is the content, not the shape.

#	Section	Block	Cached	Contents	Comes from
1	Role	System	✅	What it does, and what it explicitly does not	Template
2	Vocabulary	System	✅	The closed sets it may use, with definitions and exclusions	Tenant config
3	Tie-breaks	System	✅	How to resolve genuinely adjacent choices	Vertical template
4	Escape hatch	System	✅	When to say "I cannot tell", and that it is legitimate	Template
5	Output contract	System	✅	The schema, and that nothing else may be returned	Template
6	Examples	System	✅	Six to ten real, labelled, chosen for coverage	Tenant config
7	Facts	User	❌	Deterministic extraction for this item	Per item
8	State	User	❌	Turn, status, and what has happened so far	Per item
9	Permitted actions	User	❌	What is still allowed, after guardrails	Per item
10	The content	User	❌	The masked message itself	Per item

Sections 1 to 6 are identical on every call for a given tenant and prompt version, so the provider can cache them. Sections 7 to 10 vary every call. That difference is the whole reason for the split.

flowchart LR
    S["<b>System</b><br/>1 to 6<br/><i>identical every call</i>"] --> API["One API call"]
    U["<b>User</b><br/>7 to 10<br/><i>different every call</i>"] --> API
    API --> R["Response"]
    classDef llm fill:#7c3aed,stroke:#5b21b6,color:#ffffff
    class API llm

The rules¶

One prompt, tenant config injected¶

Not one prompt per tenant. One structure, with vocabulary, examples and voice injected from config. Otherwise you maintain N prompts and a fix has to be applied N times.

Same discipline as everywhere else: behaviour is built once in code, and giving it to a client is config (ADR-0003).

Nothing per-item may leak into the system block¶

Sections 1 to 6 stay byte-identical between calls, which is what makes them cacheable by the provider and a large saving on input tokens.

Even things that feel like configuration

The permitted action list is the clearest case. It looks like config, but guardrails narrow it per email, so putting it in the system block would break the cache on every single call.

The test is not "does this feel like configuration". It is "can this differ between two calls for the same tenant".

Version recorded, not pinned¶

	Config and catalogue	Prompt
Frozen for the item's life	Yes	No
Recorded per decision	Yes	Yes, as `decision.prompt_version`

Config is policy, being what the client agreed to, so freezing it means a long conversation behaves consistently (ADR-0006).

A prompt change is an improvement, not a policy change. You want a fix to reach in-flight items rather than leaving them on a known-worse version. Recording the version means any past decision can still be explained.

State the negative¶

Every prompt says what it is not doing, not only what it is.

Without it, models drift toward being helpful: suggesting next steps, explaining their reasoning, adding caveats. All of that is cost with no consumer, and it pollutes structured output.

Structured output is enforced in code¶

Where a prompt returns JSON, the instruction reduces failures and Pydantic prevents them (parsing to Pydantic). Never a regular expression, never a bare json.loads.

The escape hatch is mandatory¶

Every prompt that chooses from a closed set must offer a way to say "I cannot tell", presented as a legitimate answer rather than a failure.

A model given a list will always pick from it, and confidently. Without an explicit escape, a silent misclassification replaces a visible one.

Examples are chosen for coverage, not typicality¶

Include the adjacent cases, at least one escape-hatch case, and at least one edge. Examples of the easy cases teach nothing.

Masking happens inside `llm-client`¶

Not in the prompt, and not in the caller. It is the only path to a model, so it cannot be bypassed by a future engine that forgets.

Log usage, never content¶

Token counts and model version per call. Prompt content never goes to incidental surfaces: Sentry, application logs, or the Console.

It is stored deliberately, on the decision row in the tenant's own database, because a wrong classification cannot be diagnosed without the input and the raw response, and there is nothing to calibrate a prompt against. That is a different thing from content leaking into a surface with different rules.

Destination	Prompt content?
Sentry, application logs, the Console	Never
Tenant Postgres `decision`	Yes, deliberately. Same residency, backup, access and exit rules as everything else there
Langfuse	Short retention, debugging only. Never the durable record

Only the user block is stored. The system block is reconstructible from prompt_version, so storing it per row would duplicate the same six sections thousands of times.

Tracing¶

Langfuse, self-hosted, one shared instance with a project per tenant.

Why not per tenant¶

The isolation instinct is right, and the footprint makes it unaffordable. Langfuse v3 needs six services (web, worker, ClickHouse, MinIO, Redis, Postgres) at roughly 6 to 8 GB. That is more than an entire tenant stack plus Metabase combined.

Why shared is acceptable here, when it was not for Metabase¶

	Metabase	Langfuse
Who uses it	The client	Only us
Third-party CVE surface	Exposed to clients	Internal only
Needs credentials to tenant databases	Yes	No. The brain pushes to it
Crosses tenants	Would be a new boundary	The brain already does (ADR-0032 §5)

That last row is the argument. The brain is already a cross-tenant component by design, so its observability tool holding traces from it does not widen a blast radius that already exists. A shared Metabase would have created one.

It is a convenience, not infrastructure¶

The durable operational record is Postgres: the stage trace, the counters, the stored prompt and response, and the approval chain. Losing Langfuse costs nothing permanent, which is what makes short retention and deferred deployment reasonable.

	Postgres	Langfuse
Answers	What happened to this item	Why was this call slow
Retention	Years, with a floor on approvals	Weeks
Losing it	Unacceptable	An inconvenience

Deployment¶

The Ansible role is written and deployment is conditional, the same pattern as Metabase. Infrastructure ready, not running. Turning it on is a config change plus a playbook run rather than a build.

Not deployed for client-zero. The volumes are tiny and the brain barely exists. Add it when there is something worth observing.

Two different tasks, and mixing them produces a prompt that does neither well.

	Classification	Generation
Task	Analytical, structured output	Generative, voice-matched
Needs	Facts, vocabulary, permitted actions	Approved examples, tone, context
History	One line per turn	Recent messages in full
Output	JSON to a schema	Prose
Model	Cheap	Better
Improves via	Scored eval	Retrieval of approved examples

Classification runs every turn. Generation runs only when the chosen action needs content, and only after rules have confirmed the action is permitted.

flowchart LR
    C["Classify<br/><i>cheap</i>"] --> R["Rules validate"]
    R -->|"allowed"| D["Generate<br/><i>better model</i>"]
    R -->|"forbidden"| E(["escalate_item"])
    classDef llm fill:#7c3aed,stroke:#5b21b6,color:#ffffff
    class C,D llm

You never pay to generate content that a rule then forbids. That ordering is why generation sits after validation.

It also qualifies "one model call per turn" (ADR-0032): one classification call per turn, plus a generation call only for items that produce content.

This is the same split as the two feedback stores: triage feedback improves classification, generation feedback improves generation, and mixing them would teach each the wrong lesson.

Adding a prompt¶

Decide which engine it belongs to, and what it is not doing.
Decide the model tier. Analytical and structured means cheap. Generative and client-facing means better.
Define the output contract, and a Pydantic model if it returns JSON.
Fill the ten sections. Check nothing per-item leaked into sections 1 to 6.
Source examples from real client data, chosen for coverage.
Add it to the product's prompt inventory, and to the model routing table.
Record prompt_version on every decision it produces.

Per-product prompts¶

Product	Prompts
Email triage	Inventory