Email triage: data dictionary¶

Every fact, observation and enumerated value, with what it means and what it explicitly excludes.

This page grows as the design does. It exists before the first extraction rather than after, because otherwise the prompt defines the meaning implicitly and two people reading sender_known disagree about whether a domain match counts.

The excludes column is the one that prevents drift. Most ambiguity is about edge cases rather than the central meaning.

How to read it¶

Column	Means
Produced by	`code` is deterministic and testable. `model` needs judgement and needs eval
Type	The Pydantic type. `Literal[...]` means a closed set
Excludes	What this deliberately does not cover

Facts and observations are kept in separate objects (ADR-0032 §6d). Facts are objectively right or wrong and can be scored without a human. Observations need judgement.

Facts, produced by code¶

Deterministic. Same input, same output, every time. These form the regression test suite.

Threading¶

Field	Type	Means	Excludes
`thread_has_work_item`	`bool`	A `Message-ID` in this email's `References` or `In-Reply-To` matches one we hold	Provider thread ID matches. That is a separate, weaker field
`thread_matched_on`	`Literal["rfc_headers", "provider_thread_id", "none"]`	Which mechanism found the item
`thread_depth`	`int`	Count of entries in `References`. How deep in a conversation this is	Our `brain_calls` count, which is a different thing
`is_reply`	`bool`	`In-Reply-To` is present	Subject beginning "Re:". That is unreliable
`is_forward`	`bool`	Headers indicate a forward	Subject beginning "Fwd:" alone
`was_chased`	`bool`	An inbound message arrived while the item was already `awaiting_approval`	Ageing, which is time passing. A chase is evidence from the customer
`chase_count`	`int`	How many times

Sender¶

Field	Type	Means	Excludes
`sender_known`	`bool`	The exact email address matches a contact, or the client has emailed it in the last twelve months (the Sent folder). Only when `sender_authenticated`	A domain match. That is `sender_domain_known`
`sender_domain_known`	`bool`	The domain matches a known customer organisation, or one the client has emailed	Free mail domains such as `gmail.com`. One past email to a Gmail address must not make every Gmail sender known
`sender_authenticated`	`bool`	Gmail's (or the provider's) own check says the sender proved who they are. Only the result the receiving provider added (for Gmail, the first `Authentication-Results` from `mx.google.com`), never one written by the sender or an earlier server	A forged "From". A forged sender never counts as known
`has_unsubscribe_header`	`bool`	A `List-Unsubscribe` header is present, so the email went to a list	Whether it is marketing. Order confirmations and platform notices carry one too
`sender_is_internal`	`bool`	The sender is inside the tenant's own organisation
`customer_id`	`str \\| None`	Resolved customer, if any	A guess. Null when unresolved
`customer_open_jobs`	`int`	Count of that customer's open jobs	Closed jobs. Count only, not the list
`recipient_count`	`int`	People on To and Cc, excluding the monitored mailbox

The Sent folder is borrowed from the Email Assistant, Junior. It works before any customer system is connected, and keeps itself current.

sender_known versus sender_domain_known is the field most likely to be misread. A new office manager emailing about an existing job is sender_known: false, sender_domain_known: true, and treating that as an unknown sender would be wrong.

References in the text¶

Field	Type	Means	Excludes
`job_references`	`list[str]`	Strings matching the tenant's job-number pattern	Whether they resolve. That is `resolved_job_ids`
`resolved_job_ids`	`list[str]`	Of those, the ones matching a real job
`unresolved_references`	`list[str]`	Found in the text, matching no job. A typo, or a competitor's number
`invoice_references`	`list[str]`	Matching the tenant's invoice pattern
`reference_count`	`int`	Total distinct references found

Two or more job_references is the deterministic multi-intent signal, and it is more reliable than the model noticing a discourse marker like "also".

unresolved_references exists because the fact and the lookup are separate. "Quotes something that looks like a job number" comes from the text. "That number is real" is a lookup. When they disagree, that is worth surfacing rather than silently ignoring.

Dates¶

Field	Type	Means	Excludes
`dates_mentioned`	`list[DateMention]`	Every date found, with its raw text	What the date is for. That is an observation
`sent_at`	`datetime`	From the `Date` header	Receipt or processing time

class DateMention(BaseModel):
    date: date
    raw_text: str          # "next Thursday"
    resolved_against: date # sent_at, never processing time

Relative dates resolve against sent_at

"Next Thursday" means next Thursday from when they sent it. If an item sits in a queue overnight and resolves against processing time, the date silently shifts by a day.

Content shape¶

Field	Type	Means	Excludes
`has_attachments`	`bool`		Inline images and signature logos
`attachment_types`	`list[str]`	File extensions	Contents. Nothing is opened
`body_length`	`int`	Characters, after quoted history is stripped	The quoted chain
`quoted_history_present`	`bool`	Quoted text detected	Its reliability, which is low

Guardrail facts¶

Keyword and threshold matches, with per-tenant word lists in config. These exist so guardrails never depend on a model judgement (rules).

Field	Type	Means	Excludes
`mentions_legal_action`	`bool`	Solicitor, legal action, tribunal, dispute language	Frustration. That is `sentiment`
`mentions_safety`	`bool`	Injury, hazard, shock, fire, incident	Routine safety compliance paperwork
`mentions_insurer`	`bool`	Insurer, claim, assessor	Any judgement about urgency
`max_amount_mentioned`	`Decimal \\| None`	Largest currency amount in the text	Whether it is quoted, owed or disputed. That is an observation
`classification_failed`	`bool`	Set by code when the model returns `unclear` or the parse fails twice	Low confidence. That is not a failure

classification_failed is the deliberate bridge. unclear is a model output, and it is the one thing a guardrail genuinely needs. Rather than making an exception to facts-only, code sets a fact when classification fails.

Word lists are per tenant, because an electrician's safety vocabulary is not a plumber's.

These will miss things

A keyword list will not catch "I've spoken to my solicitor" unless someone listed it. That is an accepted trade: a fact-based miss is findable and fixable by anyone, whereas an observation-based miss means debugging a model.

The model can still flag concern through an observation, which raises priority and surfaces it to a human. It just is not the thing gating an action.

Observations, produced by the model¶

Judgement. These need an eval harness, and they can be wrong.

Field	Type	Means	Excludes
`category`	`Literal[...]`	The primary category. What the message is asking for. See the eleven categories	Urgency, which is `priority`. Who it is from, which is a fact. What to do, which is the action
`categories_present`	`list[str]`	Every category present, in the order they appear. Code picks the primary from this by `category_selection`	Any judgement about which matters most. That is policy, not observation
`secondary_categories`	`list[str]`	Whatever was not chosen as primary
`priority`	`Literal["low", "normal", "high", "urgent"]`	How urgent	A deadline. That is a date fact
`confidence`	`Literal["certain", "likely", "unsure"]`	The model's own estimate, in words	A percentage. Models report 0.9 or 0.95 for almost everything
`summary`	`str` max 280	One line, so a human scanning the queue need not open it	A reply, and any recommendation
`sentiment`	`Literal["neutral", "frustrated", "angry", "appreciative"]`	Tone	A judgement about the relationship
`date_contexts`	`list[str]`	What each mentioned date is for, aligned with `dates_mentioned`	The dates themselves, which are facts
`requests_callback`	`bool`	They asked to be phoned rather than emailed	A separate ask. It qualifies whatever they are asking about
`references_prior_promise`	`bool`	They refer to something we said we would do	Whether we actually said it. That is a lookup
`stated_urgency`	`Literal["none", "soon", "urgent"] \\| None`	Urgency they claimed, in their words	Our assessment, which is `priority`
`question_count`	`int`	Distinct questions asked	Categories. Three questions can be one ask
`is_automated`	`bool`	No-reply notification, delivery receipt, out-of-office	Marketing, which is a category
`mentions_money`	`MoneyMention \\| None`	Amounts, and whether quoted, owed or disputed	Resolving against real invoices

confidence is three words, not a number, borrowed from the Email Assistant, Junior. Asked for a percentage, models say 0.9 or 0.95 for almost everything, so a number carries little. Three words are easier for the model to use honestly and for a human to read.

It still never gates anything on its own in the Senior, where a human approves every draft. It is kept for spotting patterns: a run of unsure in one category says that category's definition is weak.

sentiment is not priority. An angry email about something trivial is still trivial, and a calm email about a safety issue is still urgent. They correlate and they are not the same, so rules can use both.

references_prior_promise is the highest-value attribute here. A customer chasing something we said we would do means we have already failed once, and that is invisible in category, sentiment and priority alike.

Categories versus attributes¶

Categories are about the work. Attributes are things true about the message.

Test
"Also price Coburg"	A different piece of work. Category
"Ring me"	The same work, different channel. Attribute
"You promised to call"	The same work, with history. Attribute
Angry tone	Attribute

The giveaway: an attribute can sit alongside any category. requests_callback makes sense on a quote request, a job status or a complaint. Two categories cannot co-exist without genuinely being two asks.

That discipline is what keeps the category list at eleven rather than thirty.

Enumerations¶

`category`¶

The eleven, being a vertical template for field services with per-client additions (ADR-0005). Full definitions in the overview.

`escalate_item.reason`¶

Removed. Escalation is no longer an action. See attention_reason below.

Approval `status`¶

The outcome a human chose. Four values, and wrong_classification is not among them because a correction is a separate field that may accompany any outcome.

Value	Means
`pending`	Awaiting a human. Only one per work item at a time
`approved`	Sent, or about to be
`regenerated`	Try again. `human_response_reason` is mandatory, and goes into the Draft prompt
`handled_by_human`	"Let me do it." They wrote it themselves
`superseded`	Replaced by a newer draft, whether requested or from staleness. Kept, never deleted

corrected_category is orthogonal. A human can say the read was wrong and choose any outcome: re-run with the correction, or handle it themselves. Both give us the label, which is the more valuable of the two things.

There is no rejected and no expired. "Discard" was replaced because every response carries information, and approvals do not expire (ageing and staleness handle the two problems it was conflating).

Masking tokens¶

One mapping per work item, shared by every model call on it, held in mask_map.

Token	Replaces
`[PERSON_n]`	A person's name
`[ORG_n]`	A company or organisation name
`[EMAIL_n]`	An email address
`[PHONE_n]`	A phone number
`[ADDRESS_n]`	A street address
`[ACCOUNT_n]`	An account, ABN, or reference number belonging to a person or business

Numbering is allocated on first sight and reused for the life of the item. If a name is [PERSON_1] at turn one and [PERSON_2] at turn three, the model sees two people in one conversation.

Tokens, never plausible fake names. A failed replacement then produces something obviously broken rather than something plausibly wrong that gets approved and sent.

What is masked and what is deliberately kept is OI-064, still open. Over-masking degrades the product: mask a site name and the model can no longer judge urgency or match a location.

Counters¶

Field	Type	Means	Excludes
`brain_calls`	`int`	Times the brain was invoked for this item. `max_brain_calls` halts a looping item	Model calls. A brain call may make zero, one or two
`model_calls`	`int`	Actual model invocations. The cost figure	Brain calls that hit the idempotency check and cost nothing
`tokens_in`, `tokens_out`	`int`	Cumulative across the item

Stages¶

Value	Reached when
`captured`	The item exists and the message row is written
`classified`	The model returned and parsed
`resolved`	Customer and job lookups finished, or were skipped
`decided`	Rules ran and an action was validated
`executed`	n8n's branch completed
`awaiting_human`	An approval record was written
`reconciled`	The pre-send check ran
`completed`	The exchange ended

Stage-level only. Sub-stage detail belongs in the tracing tool, not Postgres (stages).

Work item `status`¶

The outcome. What happened to the item.

Value	Means
`new`	Created, not yet stamped
`assessing`	The brain has it
`awaiting_approval`	A gated action is pending a human
`sent_by_us`	Approved and sent
`closed_by_human`	The human handled it without using our draft
`closed_manual`	The human replied themselves before we got there
`closed_no_action`	Nothing was needed

`attention_reason`¶

Nullable. Null means routine. Why this item needs care, kept separate from status because a flagged item can still be sent by us.

Value	Kind
null	Routine
`legal_risk`, `safety`, `complaint`, `high_value`, `relationship`, `never_automate`	Worked correctly
`ambiguous_job`, `multi_category`, `unclear_category`, `unresolved_reference`, `parse_failure`, `action_not_permitted`, `no_permitted_action`, `action_not_enabled`, `call_ceiling`	Fell short

The kind is a property of the value, not a stored field. Full table, including which permit a draft and which alert the Console, in the action catalogue.

Rule effects¶

Effect	Type	Means
`set_priority`	`Literal["low", "normal", "high", "urgent"]`	Overrides the model's estimate
`set_sentiment`	`Literal[...]`	Overrides the model's reading. Use sparingly, see rules
`set_category`	`Literal[...]`	Overrides the classification. Counted, because heavy use is a quality problem
`set_attention_reason`	`Literal[...]`	Flags the item for a human. Does not prevent a draft
`allow_actions`	`list[str]`	Narrows the permitted set. Never widens
`forbid_actions`	`list[str]`	Removes from the permitted set

There is no effect that chooses an action. Rules narrow, and the model picks from what remains.

Rule tiers¶

Tier	May reference	Owned by
`guardrail`	Facts only	Us. Locked
`vertical`	Facts and observations	Us
`tenant`	Facts and observations	The client, via the Console
`learned`	Facts and observations, and never widens	Later

Rules for adding an entry¶

Decide code or model first. If it can be computed deterministically, it must be, because those are testable and free.
Write the excludes column. If you cannot say what it excludes, the definition is not yet clear enough.
Prefer values over booleans. dates_mentioned with context beats has_deadline: true, because a boolean throws away the information a rule would have wanted.
Never let an observation duplicate a fact. If code can determine it, the model must not be asked.
Add it here in the same change that adds it to the schema or prompt.