Custom Rules¶

Private beta

Email Security is in private beta. It is not generally available, and access is enabled per organization — if the ext-email-security extension is not in your catalog, this product is not turned on for you yet.

While it is in beta, expect the surface described here to move: commands, fields and event shapes may change between releases, and they may change in ways that are not backwards compatible. Pin a CLI version if you script against it, and re-read this page after upgrading.

Talk to us before relying on it in production.

Your own mail rules live in the dr-mail Hive. They are ordinary D&R detect blocks evaluated against the Message Data Model, and they compound with the managed pack in the same scoring pass — so a custom rule is evidence in the same verdict, not a parallel opinion.

A rule¶

# hive: dr-mail, record name: custom-vendor-bank-change
name: Payment-detail change from a first-contact sender
phase: pre_verdict
class: signal
weight: 70
confidence: 75
tags: [bec, finance]
attack_types: [bec]
fp_notes: >
  Fires on genuine new vendors during onboarding. Intended to compound with
  auth failures rather than to stand alone.
detect:
  op: and
  rules:
    - op: is
      path: enrichments/sender_profile/prevalence
      value: none
    - op: matches
      path: body/current_thread/text
      re: "(?i)(bank details|remittance|update our account)"

limacharlie hive set --hive-name dr-mail --key custom-vendor-bank-change \
  --input-file rule.yaml --enabled --oid $OID

Fields¶

Field	Required	Meaning
(record name)	✅	The record name is the rule id. It must start with `custom-`, which is what keeps your rules from ever colliding with a packaged one. It is also what an exclusion or a rule override names, which is why the id is the name rather than a field inside the body — a body field could be duplicated across two records
`phase`	✅	`pre_verdict` or `post_verdict` — see below
`detect`	✅	A standard D&R detect block over the MDM
`class`	—	`signal` (default), `detection` or `graymail`
`weight`	✅ for `signal` and `detection`	0–100. Must be 0 for `graymail`, because the graymail lane bypasses the score entirely and a weight there would never be read
`confidence`	—	0–100, default 100. An author who does not express a confidence means "when this fires, it is right"
`respond`	—	`post_verdict` only
`name`, `tags`, `attack_types`, `fp_notes`	—	Documentation and grouping. `fp_notes` is not required of your own rules — that discipline is ours, for the pack we ship

The two phases¶

Phase	Sees	May do
`pre_verdict`	The message and its enrichments, before the verdict exists	Contribute weighted evidence to the verdict. No `respond` block — there is no verdict yet to respond to, and a rule with one is refused
`post_verdict`	The whole message and its verdict	`respond` — dispatch a mail action, or raise a detection

What a `post_verdict` rule may respond with¶

Action
`extension request` naming `ext-email-security`	The way a rule reaches remediation. The typed action goes to the same executor every other action uses, which is where `alert_only` / `enforce` is decided
`report`	Raise a detection into the platform's detection stream

phase: post_verdict
class: signal
weight: 1
detect:
  op: and
  rules:
    - op: is
      path: verdict/verdict
      value: malicious
    - op: is
      path: mailbox/address
      value: cfo@corp.example
respond:
  - action: extension request
    extension name: ext-email-security
    extension action: quarantine_message
    extension request:
      msg_uuid: "{{ .msg_uuid }}"

Everything sensor-shaped — task, tag, isolate, seal, re-enroll, set variable — fails loudly in a mail rule with a message saying so. There is no sensor behind a message, and remediation goes through extension request.

This is the same machinery your automations compile to

A mailsec_policy/automations rule is compiled into exactly this shape: a post_verdict rule whose respond block is an extension request naming the action, bound to the message that matched. Policy is the easy path; a dr-mail rule is the escape hatch when your condition does not fit the match fields.

Matching one link, not any two links¶

A mail rule reads the message as JSON, and the paths it writes are the emitted event's own field names. Two constructs walk a list, and confusing them is the most common way a mail rule quietly matches the wrong thing.

`?` walks a list and compares values¶

? is a path segment. It stands for "every element", and the condition matches if any element satisfies it.

# Any link whose registrable domain is evil.example
op: is
path: links/?/href_url/domain/root
value: evil.example

Cheap, and right most of the time. But two conditions using ? can be satisfied by two different elements:

# WRONG if you meant "one link that is both"
op: and
rules:
  - op: is
    path: links/?/href_url/domain/root
    value: evil.example
  - op: is
    path: links/?/mismatched
    value: true

That fires on a message with a perfectly ordinary link to evil.example and a separate, unrelated link whose visible text disagrees with its destination. Nothing in it says "the same link" — and a phishing message that carries a tracking pixel and a footer link will satisfy pairs like this by accident.

`scope` re-roots a whole sub-rule onto one element¶

scope is an operator. It takes a path and a rule, and evaluates that whole sub-rule against each element in turn, with the element as the root — so every condition inside is about the same one.

# ONE link that both points at evil.example and lies about where it goes
op: scope
path: links
rule:
  op: and
  rules:
    - op: is
      path: href_url/domain/root
      value: evil.example
    - op: is
      path: mismatched
      value: true

Paths inside a scope are relative to the element — href_url/domain/root and mismatched, not links/?/href_url/domain/root. That is the other half of the trap: a rule that keeps the full path inside a scope block looks correct and matches nothing.

	`?`	`scope`
What it is	A segment in a `path`	An operator with `path` and `rule`
Correlates fields of one element	No	Yes
Paths inside	Full, from the message root	Relative to the element
Cost	One extraction	The sub-rule, once per element

`scope` is capped, and nesting is refused¶

Use ? unless you actually need the correlation, because scope is the one allowed operator whose cost the rule's own size does not describe: the element counts — links, attachments, headers, hops — come from the message, not from your rule.

At most two scope operators per rule.
A scope inside another scope is refused at save, not merely discouraged. Nesting multiplies: elements to the power of the depth.

Both refusals name the reason rather than reporting a generic validation error.

Validation¶

A dr-mail record is validated at write time by compiling it on the real engine, so a record that exists has already been proven to compile. Validate a candidate before you save it — the check calls the same function the Hive runs on save, so "valid here" means "savable there":

limacharlie mailsec rule validate --file rule.json --rule-id custom-vendor-bank-change --oid $OID

An invalid rule is a 200 carrying valid: false and the reason, not an error response: you asked whether the rule is valid and found out that it is not. The reason is the validator's own wording, because an author acts on the message and not on a status code.

Omitting --rule-id validates against a placeholder in the custom- namespace, so a rule you have not named yet does not fail on its name.

limacharlie hive validate --hive-name dr-mail --key <name> --input-file rule.yaml performs the same check through the generic Hive path.

Rules fail loudly, never quietly¶

A dr-mail record that cannot be decoded or converted fails the whole rule load for that organization rather than being skipped. That is the opposite of how a bad policy record is handled, and deliberately so: a dropped policy record costs one setting, while a dropped rule is a detection the organization believes exists and does not — silently reduced protection, which no report after the fact undoes.

Records are loaded in record-name order so the rule set is assembled identically on every pass, and a disabled record is honoured as your own off switch.

Backtesting¶

Before you enable a rule, find out what it would have matched.

limacharlie mailsec rule backtest --file rule.json --since "$(date -d '14 days ago' +%s)" \
  --oid $OID --output yaml

The response is deliberately honest about its own limits:

Field	Meaning
`coverage_note`	What was actually examined
`skipped_no_raw`	Messages whose raw copy had expired
`skipped_unparse`	Messages that could not be re-parsed
`truncated`	The run hit its bound
`precision`	`null`, not `0`, when nothing it matched has an analyst disposition yet

A precision figure whose denominator quietly shrank is a number that looks like a measurement and is not one — hence the skip counts. And 0 would read as "everything it matched was wrong" and would have you discard a good rule, so the absence of labels is reported as absence.

Backtests are bounded to the window this product retains rather than the full message history. Both rule validate and rule backtest are gated on mailsec.get: they reveal only messages you can already read, and a rule author should be able to check their work with the grant that lets them see what the rule would be matching.

Two kinds of rule cannot be backtested¶

Both are refused by name, and in neither case is the rule itself the problem: the backtest is what cannot be run, not the rule.

A rule using lookup. The lookup operator resolves one of your organization's own lookup Hive records, and the service that answers a backtest cannot reach them. The refusal names the resource it could not resolve, and says what to do instead: the rule is otherwise valid, so save it and it evaluates normally in the pipeline, where the lookup is resolved.

That is a real limitation, not a transient error to retry. The alternative would have been to report "0 messages matched" for a rule that in fact matches plenty, which is a claim about your mail that nothing looked at.

To size an IOC rule before enabling it, either backtest the same rule with the lookup clause removed — which tells you how much the rest of the logic narrows — or save it and watch it live, which is safe because a dr-mail rule contributes to a verdict and your automations are in alert_only until you say otherwise. See IOC & Reputation Feeds.

A post_verdict rule. It runs against the verdict a pass would compute, and a backtest replays a message rather than re-scoring it. Backtest the pre_verdict rules that produce the verdict instead.

Rules for `lookup` in a mail rule¶


Form	The resource must be `hive://lookup/<name>` — nothing else is accepted
Count	At most four `lookup` operators per rule. Each resolves a whole lookup record for your organization
Existence	Checked on save, not by `rule validate` — see below

rule validate does not check that the lookup exists

A lookup rule naming a record your organization does not have passes rule validate and then fails the save. That is the one place where "valid here means savable there" does not hold: the existence check needs to read your lookup records, and the validate call cannot.

The check itself is worth having, and the Hive does run it: a dangling hive://lookup/ reference is the most common authoring mistake, it would otherwise save cleanly and match nothing forever, and that reads as coverage. The refusal names the record and tells you to create it first.

So: write the lookup before you write the rule that names it, and treat a save failure after a clean validate as this, not as a mystery.

Tuning the managed pack¶

You do not need a custom rule to change a packaged one. Disable it, or replace its weight, for your organization:

policy_type: thresholds
rule_overrides:
  ms-link-unranked-domain:
    weight: 15
  ms-sender-first-contact:
    disabled: true

And to suppress a rule for a specific sender, domain or mailbox rather than everywhere, use an exclusion — which carries a reason and an optional expiry, so the hole in detection is reviewable.

Rules that act on emitted events¶

A dr-mail rule is one of two seats. The other is an ordinary D&R rule in dr-general matching the EMAIL_* events, which gets the platform's full response arsenal and can correlate mail with the rest of your telemetry. See Events & Automation.

Acting on a verdict¶

EMAIL_VERDICT carries every verdict decision — the rule pack's own at seq: 0, and each later override — so a rule that should fire whenever a message is judged malicious is written once, against one path:

# Detect
op: and
rules:
  - op: is
    path: routing/event_type
    value: EMAIL_VERDICT
  - op: is
    path: event/revision/verdict
    value: malicious

# Respond
- action: report
  name: email-verdict-malicious
- action: extension request
  extension name: ext-email-security
  extension action: quarantine_message
  extension request:
    msg_uuid: '{{ .event.msg_uuid }}'

That fires when the pack decides a message is malicious and when an analyst, the AI triage agent or a link detonation later decides so. Narrow it with the fields that distinguish them:

To match	Add
Only the rule pack's own decision	`path: event/revision/seq`, `value: 0`
Only overrides	`op: is greater than`, `path: event/revision/seq`, `value: 0`
Only what a human decided	`path: event/revision/mode`, `value: analyst`
Only a change to malicious	`path: event/revision/prior/verdict`, `op: is not`, `value: malicious`
A specific rule that fired	`op: is`, `path: event/revision/top_signals/?/rule_id`, `value: ms-link-credentials-in-url` — the `?` matches any element of the list (`seq 0` only; an override carries no signals)

Overrides go both ways

An override can also clear a verdict. A rule that quarantines on verdict: malicious will see the escalation, and a later benign revision does not undo the action it took — write the compensating rule if you want one.