Email triage: rules¶
Rules constrain. They never decide.
The model chooses an action from whatever the rules leave open, so a rule can narrow the space but never fills it. A rule that picks the action is code in a config table.
Where they run¶
In two places. Rules constrain and the model picks from what is left, so the rules that shape its options must run before it is asked.
| Tier | Runs at | May use |
|---|---|---|
| Guardrails | 1.2.1 Classify, before the model | Facts only |
| Vertical, tenant, learned | 1.2.3 Decide, after | Facts and observations |
The ordering: rules run at two points.
What a rule is¶
{fact, operator, value} → effect. Structured predicates over the published vocabulary. No expression language, no eval, no embedded scripting (ADR-0004).
{
"when": [
{ "fact": "mentions_legal_action", "op": "eq", "value": true }
],
"then": { "set_attention_reason": "legal_risk" }
}
Multiple conditions are AND. For OR, write two rules.
Two effects, not three¶
| Effect | Example |
|---|---|
| Set a field | priority: high, attention_reason: legal_risk |
| Narrow the action set | Forbid draft_reply, or leave only one option |
"Force a human to look at it" is not a separate effect. It is setting attention_reason, which flags the item without preventing a draft. Deciding is not the same as typing.
Which means there is exactly one place the model chooses, and rules only ever shrink the space it chooses from.
Narrowing to zero fails closed
If rules leave no permitted action, the item is flagged
attention_reason: no_permitted_action and reaches a human with no draft.
A contradiction between rules must never resolve as "do whatever".
Tiers, and what each may reference¶
Precedence is ADR-0005: guardrails, then vertical template, then tenant authored, then learned. A lower tier never widens what a higher one narrowed.
| Tier | May reference | Owned by |
|---|---|---|
| Guardrails | Facts only | Us. Locked |
| Vertical template | Facts, and observations for non-critical effects | Us |
| Tenant authored | Same | The client, via the Console |
| Learned | Same, and never widens | Later |
Why guardrails are facts only¶
Two reasons, and the second is the stronger one.
Mechanically, they have no choice. Guardrails run before the model, because they narrow the set of actions the prompt offers. A rule that runs before the model cannot key on the model's output.
And it is the right safety posture anyway. A guardrail keyed on category would be silently skipped whenever the model misclassifies, which is precisely the case where you most need it to hold.
| Keyed on facts | Keyed on a category or sentiment | |
|---|---|---|
| Model misclassifies | Guardrail still fires | Guardrail silently skipped |
| Auditable | Yes, the fact is checkable | Only as far as the model was right |
| Stable over time | Yes | Category definitions move |
"Anything mentioning legal action is flagged" fires on a fact, so it fires even if the message came back classified as marketing.
Why everything below may use observations¶
The classification, sentiment and summary have already been produced and paid for. Refusing to use them for priority or tone would be wasteful.
The tell for which tier a rule belongs to is what happens when it is wrong.
| Wrong | Cost |
|---|---|
| A misfired priority | Mild annoyance |
| A guardrail that fails to fire | The thing that ends up in a complaint |
Hard lines run on facts. Judgement calls may use judgement.
Guardrails stay a short list¶
Five or six, all keyword or threshold based, all locked. That is maintainable. Trying to encode all judgement as keywords is not, and it is why observations are permitted everywhere else.
The model's judgement still reaches a human. An observation raising priority or flagging sentiment is visible to the approver. It just is not the thing deciding whether something can be sent.
The guardrail set¶
Six, all locked, all facts. Fine-tuned during testing rather than settled now.
| Fires on | Flags | Narrows | |
|---|---|---|---|
| Legal | mentions_legal_action |
legal_risk |
Removes close_item |
| Safety | mentions_safety |
safety |
Removes close_item |
| High value | max_amount_mentioned > threshold |
high_value |
|
| Never automate | category in the tenant's never-automate list |
never_automate |
Removes every draft action |
| Classification failed | classification_failed |
unclear_category |
Removes close_item, create_job |
| New customer | sender_known false, sender_domain_known false, and has_unsubscribe_header false |
new_customer |
Removes close_item |
Guardrails route as well as flag¶
The flag is not only a marker. attention_reason decides who may approve: Owner for anything flagged, Approver for routine (3.3).
So a guardrail firing changes who the item goes to, without a separate routing mechanism.
The narrowing matters as much as the flag¶
A flag says "look carefully". Narrowing says "this cannot happen at all", and for several guardrails the second is the point.
close_item is removed by four of the six, because the shared failure is silence: an item nobody replies to and nobody notices. Flagging alone would not prevent it, since the model could still propose closing.
New customer¶
First contact from a stranger is where the reply matters most and where we know least.
The failure mode is not a bad reply, it is no reply. Someone emails for the first time, gets read as marketing, and closes. That is a lost customer nobody ever knew about, and it leaves no trace to investigate.
So it narrows as well as flags.
Unknown sender alone is not enough
The first version fired on any sender unknown by address and by domain. That includes nearly all marketing, since almost every promotion comes from someone the client has never written to. Owners would have drowned in flagged promotions, and a flag on everything is a flag on nothing.
The fix came from designing the Email Assistant, Junior: legitimate list mail carries an unsubscribe header, which Gmail and Yahoo have required of bulk senders since 2024. A stranger writing personally does not. So the guardrail fires on unknown, and no unsubscribe header: a person writing for the first time.
Where the Junior runs first, this guardrail fires far less, because promotions are already sorted out of Action before the Senior sees them. It still has to be right for a tenant running the Senior alone.
Never automate¶
The only guardrail that removes every draft action. A client who said a category must always be human did not ask for a starting point, and providing one ignores what they said (action catalogue).
It is also the only guardrail keyed on the tenant's own list rather than on message content.
High value is the weakest of the six¶
Worth knowing before it disappoints.
| Fires on | An amount in the email text |
| Catches | Invoice disputes, where the customer quotes a figure |
| Misses | Large quote requests, because nobody has quoted yet |
So it protects the money already in play rather than the money at stake. The expensive jobs are exactly the ones with no number in the email.
A better version would use the customer's history or the job value once resolved, which is an observation rather than a fact and therefore belongs in the vertical template rather than here. Worth revisiting after testing (OI-073).
Word lists are the real work¶
The keyword facts are only as good as their lists, and a list will miss things nobody thought of. Starting sets:
| Fact | Starting words |
|---|---|
mentions_legal_action |
solicitor, lawyer, legal action, legal advice, tribunal, VCAT, ombudsman, small claims, breach of contract, dispute, formal complaint |
mentions_safety |
injury, injured, hazard, shock, electrocut, fire, smoke, burning, incident, unsafe, dangerous, exposed wiring |
mentions_insurer |
insurer, insurance, claim number, assessor, underwriter |
Per tenant, because an electrician's safety vocabulary is not a plumber's, and they live in config rather than code.
These will miss things, and that is the accepted trade
A keyword list will not catch "I've had a word with someone about this" unless somebody listed it.
A fact-based miss is findable and fixable by anyone. An observation-based miss means debugging a model. That asymmetry is why guardrails are keyword-based despite being blunt.
The model can still flag concern through sentiment, which raises priority
and surfaces it to a human. It just is not the thing gating an action.
Expect to change these¶
Every threshold and word list here is a guess. Testing against a real inbox is what turns them into settings.
| Most likely to change | |
|---|---|
| The value threshold | No basis at all. A number to start from |
| Legal and safety word lists | Real emails will use words nobody listed |
Whether high_value earns its place |
It may fire on the wrong things entirely |
What should not change: guardrails keyed on facts only, and close_item removed wherever silence is the failure mode. Those are structural, and a test suggesting otherwise is probably telling you something else.
Facts the guardrails need¶
Keyword and threshold matches. Cheap, deterministic, testable, and per-tenant word lists where relevant.
| Fact | Type | Detects |
|---|---|---|
mentions_legal_action |
bool |
Solicitor, legal action, tribunal, dispute language |
mentions_safety |
bool |
Injury, hazard, shock, fire, incident |
mentions_insurer |
bool |
Insurer, claim, assessor |
max_amount_mentioned |
Decimal \| None |
Largest currency amount in the text |
classification_failed |
bool |
Set by code when the model returns unclear or the parse fails twice |
sender_known, sender_domain_known |
bool |
Already defined. Known by the customer record, or by the Sent folder: anyone the client has emailed in the last twelve months. Only when sender_authenticated |
has_unsubscribe_header |
bool |
A List-Unsubscribe header is present, so the email went to a list |
sender_authenticated |
bool |
Gmail's own check says the sender proved who they are. A forged "From" never counts as known |
The new-customer guardrail is all three: unknown by address, unknown by domain, and no unsubscribe header.
The Sent folder is a free source for sender_known, borrowed from the Junior. It works before any customer system is connected, and "we have written to them before" is about the strongest signal there is that an email matters. Free mail domains such as gmail.com are excluded from domain matching, or one past email to a Gmail address would make every Gmail sender known.
classification_failed deserves attention. unclear is a model output, and it is the one thing you genuinely want a guardrail on. Rather than making an exception to the facts-only rule, code sets a fact when classification fails. Same effect, discipline intact.
Word lists are per tenant, because an electrician's safety vocabulary is not a plumber's. They live in config, not code.
Worked example¶
Subject: Quote for switchboard upgrade Hi, we're after a price on upgrading the board at our Preston warehouse. Fairly urgent, our insurer is asking. Can someone call me?
By the time rules run:
{
"facts": {
"sender_known": true, "sender_domain_known": true,
"customer_open_jobs": 0, "job_references": [],
"mentions_insurer": true, "mentions_legal_action": false,
"mentions_safety": false, "max_amount_mentioned": null
},
"observations": { "category": "quote_request", "priority": "normal", "sentiment": "neutral" }
}
| Tier | Rule | Fires | Effect |
|---|---|---|---|
| Guardrail | mentions_legal_action → flag legal_risk |
No | |
| Guardrail | mentions_safety → flag safety |
No | |
| Guardrail | max_amount_mentioned > threshold → flag high_value |
No, no amount | |
| Template | mentions_insurer → high priority |
Yes | priority: high |
| Template | category = quote_request → forbid close_item |
Yes | Narrows |
Remaining: draft_reply, create_job. The model picks draft_reply, and attention_reason stays null, so it is a routine item.
Note the insurer rule overrode the model's priority: normal. Rules run after the model and win, which is why they are guardrails rather than suggestions.
Calibrating with a client¶
A client will say things like "your system keeps flagging Dave's emails as complaints, Dave is just blunt". Two very different problems wear those same clothes.
| They say | Actually means | Fix |
|---|---|---|
| "That is not a complaint" | The reading is wrong | Careful. See below |
| "We handle complaints ourselves, do not flag them" | The reading is fine, the response is wrong | A tenant rule changing the effect. Safe |
Correct the response, not the perception. Changing what happens is safe and reversible. Teaching the system that complaints are not complaints is neither.
What to add, and what to refuse¶
| Correction | Add a rule? |
|---|---|
| "Emails from this domain are always job requests" | Yes. A structural fact |
| "We handle complaints ourselves, do not flag them" | Yes. Correcting the response |
| "That is not a complaint, they always write like that" | No. See below |
| "The model keeps misreading urgency" | No rule. That is an eval and prompt problem |
Never suppress sentiment for a specific person
A rule that says \"ignore this sender's tone\" is a permanent blind spot. You can no longer tell the day that person is genuinely furious, and it is the one person most likely to matter.
The asymmetry decides it: flagging a blunt email costs someone thirty seconds. Missing a real complaint from a long-standing client costs the relationship.
The safe middle: suppress the flag, keep the sentiment. The item still records \"angry\", so it appears in reporting and a human can see the pattern, but nobody is interrupted. You keep the signal and lose the noise, rather than deleting the signal.
Two guards¶
Count the overrides. More than a handful for one tenant means a classification quality problem being papered over, and it belongs in health reporting.
Corrections feed the eval set. Every human correction of a category or sentiment is a labelled example, which is exactly what OI-060 needs. Client complaints become the training data you would otherwise have to manufacture.
One thing config cannot do¶
A tenant rule may not widen what a guardrail narrowed. "We do not want legal threats flagged" is not a config change, it is a conversation. The Console must refuse it rather than accepting it and quietly ignoring it.
Open¶
- OI-073: the guardrail facts do not exist yet, and the word lists need real emails to tune against.
high_valuein particular may not earn its place.