So You Want to Start an MSSP — A LimaCharlie Walkthrough¶

This is the article we wish someone had handed us on day one of building a managed security service on LimaCharlie. It is opinionated on purpose: LimaCharlie gives you a lot of moving parts, and a new MSSP can spend weeks deciding things that should take an afternoon. The structure below works. You can absolutely deviate — but if you don't have a strong reason to, follow it.

The goal of this walkthrough is to get you from "I have a LimaCharlie account" to "I have a defensible, multi-tenant, infrastructure-as-code MSSP capable of onboarding a new customer in under an hour, with detection coverage, alerting, case management, AI assistance, and a working IaC export." It is a self-serve POC and a feature tour at the same time — read it linearly the first time, then jump back to the sections you need.

We'll use a fictional MSSP called Northwind Security with three sample customers — ACME (a 200-seat manufacturing company on Windows + M365), Globex (a 60-seat fintech on AWS + Linux + Okta), and Initech (a 30-seat law firm on Macs + Google Workspace) — to keep examples concrete.

What this article assumes:

You have signed up at app.limacharlie.io. The first two sensors are free; everything in this walkthrough can be done either inside the free tier or for a few dollars in pay-as-you-go usage.
You have a billing domain (the email domain your team uses, e.g. @northwindsec.com). LimaCharlie's Unified Billing ties tenants together by that domain, so use real company addresses for your operators, not personal ones.
You can run shell commands locally and have a place to keep a private Git repository (GitHub, GitLab, Bitbucket — anything that supports SSH deploy keys).

What this article is not:

A reference manual. The doc tree under Sensors, Detection & Response, Integrations, Administration, and Reference is the reference manual. We link out to it everywhere.
A list of every feature. LimaCharlie has many features that are great for some MSSPs and irrelevant for others. When this walkthrough doesn't mention a thing, that doesn't mean it's bad — it means it's optional for the first 90 days.

The whole article is roughly a day of focused work end-to-end. If you do it for real on a friendly first customer, you'll come out the other side with a real production MSSP environment.

Part 1 — The mental model before you click anything¶

LimaCharlie hands you a small set of primitives that, once you internalize them, explain every UI screen and every API endpoint. Spend ten minutes here. It will save you days.

The five primitives¶

Primitive	What it is	Where it lives
Organization (a.k.a. tenant, or org)	A fully isolated unit: its own sensors, rules, data, billing, audit log. One org has one OID (a UUID).	The thing you create when you click "Create Organization".
Sensor / Adapter	The thing that produces events. Sensors are EDR agents on hosts; adapters are connectors that ingest logs from cloud / SaaS / third-party security tools. Both stream events into the org.	Sensors, Adapters.
D&R Rule	A `detect:` block + `respond:` block in YAML. If `detect` matches an event, `respond` runs. This is the entire automation engine.	D&R Rules.
Hive	The platform's typed key-value store. Each "kind" of configuration lives in its own hive: `dr-general` (your rules), `dr-managed` (rules managed by extensions/Sigma), `secret`, `lookup`, `query`, `yara`, `ai_agent`, `fp` (false positives), `extension_config`, etc. Everything you configure ends up as a record in some hive.	Config Hive.
Output	A continuous stream of one of the four data types (`event`, `detect`, `audit`, `deployment`) to an external destination — Splunk, S3, BigQuery, Slack, Teams, syslog, etc.	Outputs.

The four data structures¶

Every event in LimaCharlie flows through one of four streams. Knowing them by reflex makes everything else click.

Stream	What	Volume	Where it goes
`event`	Real-time telemetry from sensors and adapters. Two top-level objects: `routing` (sensor metadata) and `event` (event-specific data).	High	SIEM, data lake, S3
`detect`	Detections produced when a D&R rule's `detect:` matches. Inherits the event's `routing` plus adds `cat`, `priority`, `detect_id`, `detect_data`.	Low–medium	SIEM, SOAR, IR, Cases
`audit`	Platform management actions (user added, rule changed, output created, secret rotated). Flat object with `oid`, `ts`, `ident`, `entity`.	Low	Compliance archive
`deployment`	Sensor lifecycle events (install, uninstall, version change).	Very low	Asset management

A real-world event payload looks like this:

{
  "routing": {
    "oid": "8cbe27f4-aaaa-aaaa-aaaa-138cd51389cd",
    "sid": "bb4b30af-aaaa-aaaa-aaaa-f014ada33345",
    "event_type": "NEW_PROCESS",
    "event_time": 1656959942437,
    "event_id": "8cec565d-14bd-4639-a1af-4fc8d5420b0c",
    "hostname": "ACME-WS-014",
    "ext_ip": "203.0.113.45",
    "int_ip": "10.0.1.25",
    "tags": ["workstation", "prod"]
  },
  "event": {
    "FILE_PATH": "C:\\Windows\\System32\\cmd.exe",
    "COMMAND_LINE": "cmd.exe /c whoami",
    "PROCESS_ID": 4812,
    "USER_NAME": "Administrator",
    "PARENT": {
      "FILE_PATH": "C:\\Windows\\explorer.exe",
      "PROCESS_ID": 2156
    }
  }
}

A D&R rule that matches it (and the resulting detect record) looks like this:

detect:
  event: NEW_PROCESS
  op: contains
  path: event/COMMAND_LINE
  value: whoami

respond:
  - action: report
    name: whoami-execution
    metadata:
      level: low

Resulting detect record (the same routing carried through, the matching event copied into detect, plus added classification metadata):

{
  "cat": "whoami-execution",
  "source": "dr-general",
  "routing": { /* same as event */ },
  "detect": { /* the event payload */ },
  "detect_id": "f1e2d3c4-aaaa-aaaa-aaaa-123456789abc",
  "priority": 2
}

If you can read those three objects fluently you can read every example in this article.

The opinionated mental model¶

Every MSSP customer = exactly one organization. Don't share orgs across customers, don't try to sub-divide a customer into multiple orgs unless they have hard data-residency or legal-entity reasons. Customer-as-tenant is the architectural unit you will base every other decision on.

Your staff don't get permissions per-customer; they get permissions through Organization Groups that contain N customer orgs. You'll have a handful of groups by job function (engineers, L1 analysts, L2 analysts, read-only) and you'll add every new customer org to each relevant group at onboarding time. After that, staff onboarding/offboarding is a one-line operation.

Every config that matters is a YAML file in a Hive. The Infrastructure extension reads/writes those files; the Git Sync extension keeps them in lockstep with a Git repository. If you treat that Git repo as the source of truth from day one, you'll never have to "figure out" what's deployed where — you read the repo.

That's the model. The rest of this article is filling it in.

Part 2 — Day 0: accounts, regions, and billing¶

Pick one region and stick with it for now¶

When you create an organization, the region (e.g. usa, europe, india) is permanent. Pick the region that matches the majority of your customers' data-residency expectations. If you'll have customers in both EU and US, that's fine — you'll just create their orgs in different regions. The CLI and the web app handle multi-region transparently; cross-region operations are normal.

For Northwind Security in our worked example, we'll put everything in usa for simplicity.

Create your "control" organization first¶

Before you create any customer org, create one organization that will be your internal control plane. Call it something obvious like northwind-control. This org:

Is not a customer tenant. No customer EDR sensors land here.
Holds your IaC templates, your demo data, your training material, your ai_agent definitions, your shared lookup records, your shared YARA rules.
Is the org you point Git Sync at first, before you replicate to customers.
Is the place where your staff can experiment without touching customer data.

Creating this org puts you on the free tier (2 sensor quota). That's enough to validate everything in this walkthrough end-to-end — no credit card required until you scale beyond two real EDR sensors or enable a paid extension.

Decide on Unified Billing now¶

If your team's emails all share a domain, opt into Unified Billing immediately. With it, every customer org under your billing domain rolls up into one monthly invoice — paid by ACH or one card — and your billing cycle is the same across tenants. Without it, every customer org is its own billing relationship with its own credit card and its own cycle. Set up Unified Billing before you onboard your first customer. Migrating later is a support ticket, not a self-serve operation.

Understand how you're billed¶

LimaCharlie's pricing is usage-based across two main axes:

EDR endpoints — flat per-endpoint per-month, all telemetry included, with one year of Insight retention bundled in.
External telemetry (everything from adapters: M365, Okta, CloudTrail, etc.) — per GB ingested, with one year of retention included.

A few helpful order-of-magnitude estimates so you can quote customers credibly:

Source	Small (≤100 endpoints)	Medium (100–500)	Large (500+)
AWS CloudTrail	0.5–2 GB/day	2–10 GB/day	10–50 GB/day
Azure / GCP combined	1–3 GB/day	4–15 GB/day	20–80 GB/day
Identity (Okta / Entra / Duo)	0.2–1 GB/day	1–5 GB/day	5–20 GB/day
Email / collab (M365 / Workspace / Slack)	0.3–1.5 GB/day	1.5–8 GB/day	8–35 GB/day
Network / firewall / IDS	2–10 GB/day	10–55 GB/day	55–290 GB/day
Third-party EDR / EPP logs (CrowdStrike, Defender, etc.)	1–5 GB/day	5–25 GB/day	25–100 GB/day
Typical total	5–25 GB/day	25–140 GB/day	140–500 GB+/day

See Estimating Data Ingestion for the up-to-date detail. Three rules of thumb that always pay off:

Start with identity + cloud + EDR. Highest signal, lowest volume.
Filter at the source. Don't ingest "all firewall traffic" — ingest "denies + threat-tagged + anomalous". The customer's firewall is almost always the noisiest source.
Use the Usage Alerts extension (covered in Part 16) to fire a detection when any SKU crosses a threshold.

Enable Strict SSO on your domain¶

Before you invite a single staff member, enable Strict SSO Enforcement on @northwindsec.com. This forces every authentication from your domain through your IdP (Okta / Entra / Google Workspace / etc.), which means: when an analyst leaves, disabling them in your IdP locks them out of every customer org instantly. This is the single most impactful access-control control you can apply. Do it on day zero, while you only have one user (you), so you can prove the flow works before relying on it.

Strict SSO disables login+password, GitHub auth, and Microsoft auth for users on your domain — they go through your IdP only. It works on app.limacharlie.io and any custom-branded portal.

Part 3 — The control-org bootstrap¶

In the control org, do the following one-time setup. You'll get value from each of these immediately, and they'll be the foundation for every customer onboarding to come.

Install the CLI and authenticate¶

pip install limacharlie
limacharlie auth login --oauth

auth login --oauth signs you in through your browser (Google or Microsoft) and stores the credentials in ~/.limacharlie.d/config.yaml. Add --no-browser on a headless machine to get the URL printed instead. For scripts and CI, log in with an Organization API key instead: limacharlie auth login --oid <OID> --api-key <KEY>, or set the LC_OID and LC_API_KEY environment variables. Organization keys are scoped to a single org and a specific permission set — not to your user — which is the right default for automation. See API Keys for the full pattern.

Confirm it works:

limacharlie auth whoami            # who you are authenticated as
limacharlie auth list-orgs         # orgs you can reach
limacharlie auth use-org <OID>     # make one the default for later commands
limacharlie sensor list            # in the control org, returns an empty list for now

Create a baseline Organization API key for automation¶

Eventually you'll have webhooks, CI scripts, and automation calling the API on behalf of an organization. Use an Organization API key for those, never a user API key — the permission scope is narrower and the key dies with the org, not with the human.

With the CLI, pass the permissions as a comma-separated list. --store-secret writes the new key straight into the secret hive, because the key value is only shown once:

limacharlie api-key create --name automation-readonly \
  --permissions "org.get,sensor.list,sensor.get,dr.list,insight.det.get,insight.evt.get,investigation.get" \
  --store-secret automation-readonly-key

The same call over REST takes form fields, not JSON: key_name and a comma-separated perms. Seeing the REST shape is useful when you build the call into your own tooling:

curl -X POST "https://api.limacharlie.io/v1/orgs/$YOUR_OID/keys" \
  -H "Authorization: Bearer $LC_JWT" \
  -d "key_name=automation-readonly" \
  -d "perms=org.get,sensor.list,sensor.get,dr.list,insight.det.get,insight.evt.get,investigation.get"

Creating keys needs the apikey.ctrl permission, and --store-secret also needs secret.set.

To exchange an API key for a JWT (everything in LimaCharlie speaks JWT-Bearer):

# Organization-scoped JWT (the most common case)
curl -X POST "https://jwt.limacharlie.io" \
  -H "Content-Type: application/x-www-form-urlencoded" \
  -d "oid=$YOUR_OID&secret=$YOUR_API_KEY"

The JWT is valid for one hour; cache it and refresh on a 401. The CLI and SDKs handle this for you.

A couple of useful flairs for advanced cases:

orchestration-key[bulk] — for high-volume API call workloads.
integration-key[segment] — segment isolation, the key only sees its own resources.

Create your first Hive Secret¶

You'll be storing secrets (Anthropic API keys, GitHub deploy keys, third-party tokens, output credentials) in the Hive secret namespace. Even before you need one, prove it works:

limacharlie secret set --key example-secret --value "test-value" --enabled
limacharlie secret list --metadata-only
limacharlie secret delete --key example-secret --confirm

--value puts the secret in your shell history, so use it for this test only. For real credentials, pipe the value in from stdin or point --input-file at a YAML file containing secret: <value>. --enabled activates the record; new hive records are disabled by default in most hives. secret list --metadata-only lists names without returning the values.

Anywhere a config field would normally take a credential, you'll instead use the syntax hive://secret/<name>. This means rules, outputs, and adapter configs never contain a literal credential — they contain a reference. That's the only way an IaC repo of these configs can be committed to Git without leaking.

In the Marketplace, subscribe (in the control org first; you'll repeat per customer) to:

Extension	Why every MSSP wants it
Infrastructure	IaC machinery — fetch / push / dry-run / selective sync.
Git Sync	Bidirectional sync between an org and a Git repo.
Cases	Case queue, SLA timers, investigation evidence, reporting.
Exfil	Tune what events sensors send (volume / cost control).
Lookup Manager	Auto-refresh threat-feed lookups.
Usage Alerts	Bill-runaway protection.
Sensor Cull	Auto-remove stale sensors.
Artifact	Collect Windows Event Logs, Mac Unified Logs, PCAPs.
Reliable Tasking	Guarantee a sensor command runs when the sensor next checks in.

Five minutes of clicking. You'll IaC them in Part 15.

Part 4 — Designing the customer-org template¶

You will onboard customers many times. Every onboarding should produce a tenant that's already 90% configured. The way to make that happen is to define one template org configuration, in YAML, that you push to every new customer org on creation.

The template comprises:

Connectivity primitives: installation keys (one per environment / role).
Adapter scaffolding: the cloud/SaaS/identity adapters every customer is likely to want.
Detection coverage baseline: a curated set of D&R rules + subscriptions to managed rulesets.
Alerting: Cases extension config, Slack/Teams/PagerDuty webhooks, severity & SLA mapping.
Outputs: forward detect to the customer's SIEM (or yours); forward audit to long-term storage.
Cost guardrails: usage alerts, sensor cull, sleeper defaults.
AI agent definitions: triage / investigation / responder agent configs with empty per-customer credentials.
Saved query library: baseline hunts that ship with every customer.
Lookup feeds: Tor exit nodes, AlienVault reputation, etc.

We'll build each of these in the next several sections. Keep in mind: at this point you are configuring the control org so the configurations exist as records you can later export and replicate. You are not yet onboarding a customer.

Part 5 — What to connect in (telemetry)¶

Telemetry is the foundation. You can't write detections — let alone offer detection-and-response services — without it. LimaCharlie has two kinds of telemetry sources, both of which produce events flowing into the same event stream.

EDR sensors (endpoints)¶

The LimaCharlie sensor runs on Windows, macOS, Linux, ChromeOS, and via browser extensions on Chrome and Edge. Feature parity is high across desktop OSes. Install via Installation Keys — one key per role, with role tags baked in:

# server-class hosts: all production servers
limacharlie installation-key create \
  --description "Production servers" --tags "server,prod"

# workstation-class hosts: knowledge-worker machines
limacharlie installation-key create \
  --description "Workstations" --tags "workstation"

# VIP hosts: executives, on-call, devs with prod access
limacharlie installation-key create \
  --description "VIP" --tags "vip"

# pre-deployed sensors held in sleeper mode
limacharlie installation-key create \
  --description "Sleeper / pre-deployed" --tags "lc:sleeper"

Tags are sticky — anything installed with that key inherits the tags forever. Your D&R rules and Exfil profiles will key off these tags (vip sensors get more aggressive collection; server sensors get FIM on /etc/).

For ACME, our 200-seat customer, we'll create three keys: acme-server, acme-workstation, acme-vip. The IaC version of those (which is what actually ships in the template) is:

version: 3
installation_keys:
  acme-server:
    desc: ACME servers
    tags: [server, prod]
  acme-workstation:
    desc: ACME workstations
    tags: [workstation]
  acme-vip:
    desc: ACME executives and on-call
    tags: [vip, workstation]

Windows install (interactive)¶

:: Run as Administrator
rphcp.exe -i YOUR_INSTALLATION_KEY_GOES_HERE

Windows install (silent MSI for SCCM/Intune/GPO)¶

msiexec /i "C:\Downloads\hcp_win_x64.msi" /qn ^
  INSTALLATIONKEY="YOUR_INSTALLATION_KEY_GOES_HERE"

Windows install (self-contained PowerShell)¶

This is the one to send to customer IT — it auto-detects architecture, downloads the right installer, installs silently, and verifies the service is running:

#Requires -RunAsAdministrator
param([Parameter(Mandatory=$true)][string]$InstallationKey)

$CpuArch = (Get-CimInstance -ClassName Win32_Processor).Architecture
$Arch = switch ($CpuArch) {
    0  { "32" }
    9  { "64" }
    12 { "arm64" }
    default { if ([Environment]::Is64BitOperatingSystem) { "64" } else { "32" } }
}

$InstallerUrl  = "https://downloads.limacharlie.io/sensor/windows/$Arch"
$InstallerPath = Join-Path $env:TEMP "rphcp.exe"

[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12
Invoke-WebRequest -Uri $InstallerUrl -OutFile $InstallerPath -UseBasicParsing
Start-Process -FilePath $InstallerPath `
  -ArgumentList "-i", $InstallationKey -Wait -NoNewWindow

$service = Get-Service -Name "rphcpsvc" -ErrorAction SilentlyContinue
if ($service -and $service.Status -eq "Running") {
    Write-Host "OK: LimaCharlie sensor installed and running." -ForegroundColor Green
} else {
    Write-Host "WARN: service may not be running; check manually." -ForegroundColor Yellow
}

Linux install (Debian / Ubuntu)¶

# Interactive
sudo dpkg -i limacharlie.deb
# or
sudo apt install ./limacharlie.deb

# Silent / scripted
echo "limacharlie limacharlie/installation_key string $INSTALLATION_KEY" \
  | sudo debconf-set-selections \
  && sudo apt install ./limacharlie.deb -y

# Uninstall
sudo apt remove limacharlie

Linux install (raw binary, init.d / non-deb distros)¶

wget https://downloads.limacharlie.io/sensor/linux/64 -O /tmp/lc_sensor
chmod +x /tmp/lc_sensor
sudo /tmp/lc_sensor -d $INSTALLATION_KEY
# Or via env var:
sudo LC_INSTALLATION_KEY=$INSTALLATION_KEY /tmp/lc_sensor -d -

macOS install¶

# Intel: https://downloads.limacharlie.io/sensor/mac/64
# Apple Silicon: https://downloads.limacharlie.io/sensor/mac/arm64
chmod +x lc_sensor
sudo ./lc_sensor -i $INSTALLATION_KEY

# macOS will prompt the user to grant Full Disk Access and Network
# Filter via System Settings. Document this step in your customer
# onboarding email — it's the #1 cause of "the sensor isn't seeing
# anything" tickets.

# Verify:
sudo launchctl list | grep com.refractionpoint.rphcp

For larger customers, ship via Jamf or Intune. Send the customer's IT a short doc that contains the PowerShell or shell snippet plus your installation key — that's the entirety of the customer-side work.

Sleeper-mode pre-deployment — the MSSP cheat code¶

The lc:sleeper tag pattern above is worth its own paragraph. A sleeper sensor costs $0.10 per 30 days to keep installed and connected, with no telemetry collection running. Your customer's IT team can roll the agent out to thousands of machines now; whenever you need real telemetry from any subset, you remove the tag (manually or via a D&R rule) and they wake up within 10 minutes. This is the foundation of being able to offer 20-minute IR SLAs.

End-to-end sleeper flow:

Create the customer's org. Set Quota to 3 to enable billing.
Create an installation key with the lc:sleeper tag.
Customer's IT installs the agent on every endpoint they want to pre-deploy.
Sensors land in sleeper mode immediately (~10 min sync). Cost = $0.10 / 30 days each.
When an incident happens: bump quota up to N, remove the lc:sleeper tag from the N sensors you want awake, wait ≤10 minutes, start collecting full telemetry.
When done, re-tag lc:sleeper, drop quota back. Costs revert.

Sleeper changes don't touch the binary on disk — only the in-memory profile changes. No reboot needed. While in sleeper mode, things like YARA scans (which require process memory access) are paused.

See Sleeper Mode for the full mechanics. Build the sleeper-deploy path into your customer onboarding documentation; do not save it for "advanced users".

Adapters (logs, cloud, SaaS, third-party EDR)¶

Anything that's not a LimaCharlie sensor lands via an Adapter. The catalogue is large; the ones an MSSP almost always wants on a customer's first day:

Customer asset	Adapter	Notes
Microsoft 365 mailboxes	Microsoft 365	Audit, login, file-share events.
Identity provider	Okta, Microsoft Entra ID, or Duo	This is where your account-takeover detections live.
Cloud control plane	AWS CloudTrail, Azure Event Hub, GCP Pub/Sub	Customers running in a public cloud — connect this on day 1.
Google Workspace	Google Workspace	Same role as M365 for non-Microsoft shops.
Existing EDR	CrowdStrike, Defender for Endpoint, SentinelOne, Carbon Black, Sophos	Customer already has an EDR? Don't fight it — ingest its events as a parallel data source.
Network / firewall	Syslog	Long-tail visibility. Filter at source aggressively.
Anything weird	JSON adapter, File adapter, Webhook adapter tutorial	If a system can write JSON to disk or HTTP-POST a payload, you can ingest it.

Opinion: for your first three customers, connect: EDR + identity (Okta/Entra) + M365/Workspace + cloud control plane (if applicable). That's the minimum-viable telemetry surface to credibly run detections. Everything else is gravy and can be added in week 2+.

Microsoft 365 adapter — concrete config¶

sensor_type: office365
office365:
  tenant_id:     hive://secret/acme-o365-tenant-id
  client_id:     hive://secret/acme-o365-client-id
  client_secret: hive://secret/acme-o365-client-secret
  publisher_id:  hive://secret/acme-o365-publisher-id
  domain: acme.onmicrosoft.com
  endpoint: enterprise
  start_time: "2026-01-01T00:00:00Z"
  # A comma-separated string, not a YAML list. Omit it to ingest all five.
  content_types: "Audit.AzureActiveDirectory,Audit.Exchange,Audit.SharePoint,Audit.General,DLP.All"
  client_options:
    identity:
      oid: <CUSTOMER_OID>
      installation_key: <INSTALLATION_KEY>
    hostname: ms-o365-adapter
    platform: office365
    sensor_seed_key: office365-audit-sensor

Always set platform: office365 for this adapter. The office365 parser pulls the event type (Operation) and timestamp (CreationTime) out of the audit records for you, so there is no mapping block, and the events show up under plat == office365 in D&R rules and queries.

The customer's M365 admin creates an Azure AD App Registration with the ActivityFeed.Read permission from the Office 365 Management APIs (plus ActivityFeed.ReadDlp if you want DLP.All) and gives you four values: tenant ID, client ID, client secret, and publisher ID (for single-tenant apps the publisher ID is the tenant ID). Your secret hive holds the actual values; this YAML carries only references. The domain is not sensitive and stays inline.

Okta adapter — concrete config¶

sensor_type: okta
okta:
  apikey: hive://secret/acme-okta-api-key
  url: https://acme.okta.com
  client_options:
    identity:
      oid: <CUSTOMER_OID>
      installation_key: <INSTALLATION_KEY>
    hostname: okta-systemlog-adapter
    platform: json
    sensor_seed_key: okta-system-logs-sensor
    mapping:
      sensor_hostname_path: client.device
      event_type_path:      eventType
      event_time_path:      published

AWS CloudTrail adapter — S3-based config¶

s3:
  client_options:
    hostname: aws-cloudtrail-logs
    identity:
      installation_key: <INSTALLATION_KEY>
      oid: <CUSTOMER_OID>
    platform: aws
    sensor_seed_key: cloudtrail-seed
  bucket_name: globex-cloudtrail-logs
  region: us-east-1
  access_key:  hive://secret/globex-aws-access-key
  secret_key:  hive://secret/globex-aws-secret-key

If the customer has an SQS event-notification on the bucket, prefer SQS — lower latency, no polling cost:

sqs:
  client_options:
    hostname: aws-cloudtrail-logs
    identity:
      installation_key: <INSTALLATION_KEY>
      oid: <CUSTOMER_OID>
    platform: aws
    sensor_seed_key: cloudtrail-seed
  region: us-east-1
  queue_url: https://sqs.us-east-1.amazonaws.com/123456789012/cloudtrail-events
  access_key: hive://secret/globex-aws-access-key
  secret_key: hive://secret/globex-aws-secret-key

Adapters as a Service vs self-hosted¶

You have four deployment options for adapters; pick by customer size and operational appetite:

Option	When to use	How
As a Service (cloud-managed)	Default for cloud-to-cloud sources (M365, Okta, CloudTrail). LC runs the adapter for you.	Web app → Adapters → "Cloud" tab.
On-prem binary	Customer has on-prem log sources (syslog from internal firewall, files in a shared dir).	Download `lc_adapter`, run with config file.
As a Windows service	On-prem Windows host.	`./lc_adapter.exe -install:service-name <type> param1=value1 ...`
As a Linux systemd service	On-prem Linux host.	Drop a unit file in `/etc/systemd/system/limacharlie-<name>.service`, see template below.

Reusable systemd unit template:

[Unit]
Description=LC Adapter <NAME>
After=network.target

[Service]
Type=simple
ExecStart=/opt/limacharlie/lc-adapter <type> client_options.identity.installation_key=<KEY> client_options.identity.oid=<OID> ...
WorkingDirectory=/opt/limacharlie
Restart=always
RestartSec=10
StandardOutput=journal
StandardError=journal
SyslogIdentifier=lc-adapter-<name>

[Install]
WantedBy=multi-user.target

Tag-based segmentation, not org-based¶

A common mistake: separating environments (dev/prod) into different orgs. Don't. Use sensor tags (prod, nonprod, dmz, pci, hipaa) and write D&R rules that key off tags. One org = one customer; tags are how you express structure inside that customer. The sensor selector grammar reads naturally:

plat == windows                                  # all Windows sensors
plat == linux and production in tags             # Linux production
plat == windows and not isolated                 # non-isolated Windows
hostname matches '^prod-.*'                      # by name
plat == windows and int_ip matches '^10\.3\..*'  # by network range
"vip" in tags or "exec" in tags                  # tag union
ext_plat == windows                              # third-party EDR adapters reporting Windows

You'll use these in limacharlie sensor list --selector, Sensor Cull rules, Reliable Tasking requests, Velociraptor collections, and anywhere else a sensor selector is accepted.

Tune what you collect with the Exfil extension¶

Some events are gold (NEW_PROCESS, DNS_REQUEST, NETWORK_CONNECTIONS, CODE_IDENTITY, WEL for relevant channels). Some are firehoses (MODULE_LOAD, FILE_PATH events on a build server). Subscribe to the Exfil extension and use Watch Rules to get fine-grained.

The most useful Watch Rule pattern: collect MODULE_LOAD only when the loaded DLL is interesting:

{
  "action": "add_watch",
  "name": "wininet-loading",
  "event": "MODULE_LOAD",
  "operator": "ends with",
  "value": "wininet.dll",
  "path": ["FILE_PATH"],
  "tags": ["server"],
  "platforms": ["windows"]
}

For VIP-tagged hosts, send everything — the volume is small and the value of full telemetry on an executive's laptop dwarfs the cost:

{
  "action": "add_event_rule",
  "name": "vip-full-collection",
  "events": ["NEW_TCP4_CONNECTION", "NEW_TCP6_CONNECTION", "MODULE_LOAD"],
  "tags": ["vip"],
  "platforms": ["windows"]
}

For sensors with extreme I/O (build servers, SQL hosts), use Performance Rules (Windows-only) and the ir tag pattern. Tagging a sensor ir activates IR Mode — full event collection without running every D&R rule, which is what you want both during incident response and on hosts with extreme event rates.

Day-1 telemetry smoke test¶

Before declaring telemetry "working", run this 5-minute smoke test on each connected source:

Source	What to do	What to look for
Windows EDR	Run `cmd.exe /c whoami /all` on the host	A `NEW_PROCESS` event with that command line in the sensor's Live Feed within ~5 seconds.
Linux EDR	`cat /etc/passwd`	`SENSITIVE_PROCESS_ACCESS` or `NEW_PROCESS` event in Live Feed.
macOS EDR	`id` from a Terminal	`NEW_PROCESS` event. If nothing appears, Full Disk Access wasn't granted — re-prompt the user.
M365 adapter	Sign in to the customer's M365 portal as a test user	A `UserLoggedIn` event under platform `office365` within ~5 minutes (M365 has API delay).
Okta adapter	Trigger an Okta event (e.g., MFA challenge)	`user.session.start` event within ~2 minutes.
CloudTrail adapter	`aws sts get-caller-identity` (or any API call)	A CloudTrail event in the platform within ~5 minutes (S3 batch delivery time).

Don't proceed to Part 6 until each connected source has produced at least one event you can see in the Timeline view.

Part 6 — Detections: getting useful coverage in 30 minutes¶

You don't write rules from scratch on day one. You import known-good rulesets, see what fires, then layer your own.

In the marketplace, subscribe to the sigma add-on. This auto-applies a curated subset of SigmaHQ rules that the LimaCharlie team keeps up to date. Free, on by default. After enabling, generate a couple of test events from a sensor and watch the Detections view light up.

Step 2: Browse the Community Rules library¶

In the web app, Automation → Rules → Add Rule → Community Library. Thousands of rules from Anvilogic, Sigma, Panther, and Okta. You can search by ATT&CK technique, CVE, or keyword, click "Load Rule", and the AI converter renders it as native LimaCharlie YAML in your editor. Pick five rules that match your customers' threat model. Common day-1 picks:

T1059.001 — PowerShell encoded commands
T1003.001 — LSASS access (credential dumping)
T1218 — LOLBin abuse (e.g., rundll32, regsvr32)
T1486 — ransomware-like file ops
T1136 — new local-account creation

See Community Rules.

Step 3: Add the Soteria rulesets if you have customers in those clouds¶

Soteria EDR, Soteria AWS, and Soteria M365 are LimaCharlie-curated production rulesets. AWS and M365 in particular catch a lot of identity-misuse activity that custom rules struggle with. Subscribe selectively per customer — for AWS-heavy customers like Globex, Soteria AWS pays for itself in coverage.

EDR event types you'll detect on most often¶

You'll see hundreds of event types. The 10 you'll write rules against most often:

Event	What it captures
`NEW_PROCESS`	Process creation: full file path, command line, user, parent process.
`NETWORK_CONNECTIONS`	TCP/UDP connections with source/destination IP, port, protocol.
`DNS_REQUEST`	DNS queries with hostname, type, response IPs.
`FILE_CREATE`	File creation events.
`CODE_IDENTITY`	DLL/module loading with hash and signature.
`REGISTRY_WRITE`	Registry modifications.
`WEL`	Windows Event Log entries (per channel).
`PROCESS_ENVIRONMENT`	Process env vars and details (handy for cloud-IAM exfil detection).
`NETSTAT_REP`	Snapshot of network sockets.
`OS_PROCESSES_REP`	Full process-list snapshot.

The full catalogue is in EDR Events.

Step 4: A detection pattern catalogue¶

Below are eight detection rule patterns covering the categories an MSSP will touch first. Each is production-grade — copy them into your global dr-general.yaml, customize names, and ship.

Pattern 1: simple file-path match (executable from Downloads)¶

detect:
  event: NEW_PROCESS
  op: matches
  path: event/FILE_PATH
  case sensitive: false
  re: .:\\(users|temp)\\.*\\downloads\\.*\.(exe|scr|dll|js|vbs|cmd|bat|ps1|hta)
respond:
  - action: report
    name: exec-from-downloads
    metadata:
      level: medium
      tags: [attack.execution, mssp.baseline]

Pattern 2: encoded PowerShell¶

detect:
  event: NEW_PROCESS
  op: contains
  path: event/COMMAND_LINE
  value: -encodedcommand
  case sensitive: false
respond:
  - action: report
    name: encoded-powershell
    metadata:
      level: high
      tags: [attack.execution, attack.t1059.001]

Pattern 3: nested boolean — Control Panel item from a non-Windows directory¶

detect:
  op: and
  event: CODE_IDENTITY
  rules:
    - op: is windows
    - op: ends with
      path: event/FILE_PATH
      value: .cpl
      case sensitive: false
    - op: not contains
      path: event/FILE_PATH
      value: "C:\\windows\\"
      case sensitive: false
respond:
  - action: report
    name: suspicious-cpl-execution
    suppression:
      max_count: 1
      period: 1h
      is_global: false
      keys:
        - "{{ .event.FILE_PATH }}"

Pattern 4: stateful — parent-then-child (cmd.exe → calc.exe)¶

The with child operator says "this event matches AND a subsequent event matches the nested condition". It's how you express attack chains.

detect:
  event: NEW_PROCESS
  op: ends with
  path: event/FILE_PATH
  value: cmd.exe
  case sensitive: false
  with child:
    op: ends with
    event: NEW_PROCESS
    path: event/FILE_PATH
    value: calc.exe
    case sensitive: false
respond:
  - action: report
    name: cmd-spawned-calc

Pattern 5: stateful — N events within a window (5 failed logins in 60s)¶

detect:
  event: WEL
  op: is windows
  with events:
    event: WEL
    op: is
    path: event/EVENT/System/EventID
    value: '4625'
    count: 5
    within: 60
respond:
  - action: report
    name: brute-force-attempt
    metadata:
      level: high

Pattern 6: behavioral — first-time domain on a host¶

Suppression with a long period acts as a "remember this happened" mechanism — the report only fires the first time per host per domain, then suppresses for 30 days.

detect:
  event: DNS_REQUEST
  op: exists
  path: event/DOMAIN_NAME
respond:
  - action: report
    name: new-domain-for-host
    suppression:
      max_count: 1
      period: 720h
      is_global: false
      keys:
        - first-domain
        - "{{ .event.DOMAIN_NAME }}"

detect:
  event: USER_LOGIN
  op: lookup
  path: event/SOURCE_IP
  resource: lcr://api/ip-geo
respond:
  - action: report
    name: first-login-from-country
    suppression:
      max_count: 1
      period: 720h
      is_global: true
      keys:
        - first-country
        - "{{ .event.USER_NAME }}"
        - "{{ .mtd.lcr___api_ip_geo.country.iso_code }}"

Pattern 8: threat-feed-driven (lookup against a managed feed)¶

detect:
  event: NETWORK_CONNECTIONS
  op: lookup
  path: event/NETWORK_ACTIVITY/?/IP_ADDRESS
  resource: hive://lookup/crimeware-ips
respond:
  - action: report
    name: connection-to-crimeware-ip
    metadata:
      level: high

The crimeware-ips lookup lives in your lookup hive and is auto-refreshed by the Lookup Manager extension (see Part 13).

Sometimes a detection needs to remember something a previous detection saw. Sensor variables live per-sensor and can be set by one rule and consumed by another.

# Rule A: when an admin logs in, set a per-sensor var
detect:
  event: USER_LOGIN
  op: is
  path: event/USER_NAME
  value: Administrator
respond:
  - action: add var
    name: admin-active
    value: <<routing/sid>>   # remember which sensor it was
    ttl: 1800                # expires after 30 minutes

# Rule B: process spawn that's only suspicious while admin-active is set
detect:
  op: and
  event: NEW_PROCESS
  rules:
    - op: contains
      path: event/COMMAND_LINE
      value: net localgroup administrators /add
    - op: is
      path: routing/sid
      value: '[[admin-active]]'   # true while this sensor's ID is in the variable
respond:
  - action: report
    name: privilege-escalation-while-admin-active
    metadata:
      level: critical

Step 6: Push the rule via CLI¶

Save any of the above as exec-from-downloads.yaml, then:

limacharlie dr set --key exec-from-downloads --input-file exec-from-downloads.yaml --enabled
limacharlie dr list --metadata-only
limacharlie dr get --key exec-from-downloads

New rules are created disabled unless you pass --enabled (or set usr_mtd.enabled in the file), so a rule you forgot to enable looks exactly like a rule that never fires. The input file holds the whole rule, with detect and respond keys.

For bulk push from your IaC repo, use the Infrastructure extension (Part 15).

Step 7: Replay against historical data before you trust it¶

The cleanest feedback loop is the Replay service. Two flavors:

# Replay the saved rule against the last 24h of events in this org
limacharlie replay run --name exec-from-downloads \
  --start "$(date -d '24 hours ago' +%s)" \
  --end "$(date +%s)"

# Replay a rule that is not saved yet, from separate detect and respond files
limacharlie replay run --detect-file ./detect.yaml \
  --respond-file ./respond.yaml \
  --start "$(date -d '24 hours ago' +%s)" --end "$(date +%s)"

# Run a rule file against a local file of sample events (CI-friendly)
limacharlie dr test --events events.json --input-file rule.yaml --trace

# Same, against a rule that is already saved
limacharlie dr test --events events.json --name exec-from-downloads

A replay returns the number of events processed and the detections the rule would have produced, plus any errors. dr test --trace adds a step-by-step evaluation trace. Read the trace, especially for noisy rules. It tells you why a non-malicious event matched.

Step 8: Layer false-positive rules instead of editing detections¶

When something noisy fires, the wrong move is to add an exception inside the detection rule. The right move is to write a false-positive rule in the fp hive that suppresses the matching detection. This keeps detections portable across customers and makes per-customer tuning explicit:

# fp/acme-vendor-installer.yaml — applied only to ACME's tenant
op: and
rules:
  - op: ends with
    path: detect/event/FILE_PATH
    value: this_is_fine.exe
  - op: is
    path: routing/hostname
    value: ACME-DEPLOY01

That rule lives only in ACME's tenant; the underlying detection lives globally. The platform discards any detection that matches an fp rule before it hits Cases.

Step 9: Unit-test the rules you care about¶

For rules where regression risk is real, embed a tests: block. It lives next to the rule in the same hive record, and LimaCharlie simulates the rule against it every time the rule is created or updated, including through dr set and sync push. A failing test rejects the change.

detect:
  op: and
  event: CODE_IDENTITY
  rules:
    - op: is windows
    - op: ends with
      path: event/FILE_PATH
      value: .cpl
      case sensitive: false
    - op: not contains
      path: event/FILE_PATH
      value: "C:\\windows\\"
      case sensitive: false
respond:
  - action: report
    name: suspicious-cpl-execution
tests:
  match:
    - [{
        "routing": {"event_type": "CODE_IDENTITY", "plat": 268435456},
        "event": {"FILE_PATH": "C:\\Users\\Admin\\Downloads\\malicious.cpl"}
      }]
  non_match:
    - [{
        "routing": {"event_type": "CODE_IDENTITY", "plat": 268435456},
        "event": {"FILE_PATH": "C:\\Windows\\System32\\legitimate.cpl"}
      }]

To check the rule before it reaches an org, compile it with dr validate and run it against sample events with dr test:

limacharlie dr validate --detect detect.yaml --respond respond.yaml
limacharlie dr test --input-file rule.yaml --events events.json

Because a failing tests: block rejects the save, it also fails the CI sync push. Rules that use stateful operators (with child, with events) cannot carry tests:; cover those with Replay. See Unit Tests for the assertion grammar.

Common pitfalls when writing rules¶

These bite everyone at least once:

Timestamp units. Detections and events carry timestamps in milliseconds (13 digits). API parameters that take time ranges expect seconds (10 digits). Divide by 1000. Many tools wrap this for you (the CLI does), but raw API and SDK calls don't.
gjson vs Go templates in extension request. When you reach into nested data inside an extension request: block, paths resolve as gjson paths (e.g. routing returns the entire routing object). If you wrap them in {{ }}, they get stringified and you'll lose nested structure. Don't quote them.
detect: detect is not the same as detect: '@this'. When passing detections to AI agents or extensions, @this gives you the entire detection (with detect_id, cat, etc.); detect gives you only the inner event payload.
case sensitive: false is not the default. String comparisons are case-sensitive unless you say otherwise.
Suppression is_global: true vs false. false (the default) means per-sensor suppression; true means org-wide. Don't is_global: true a per-host pattern — you'll suppress the second host's detection.

Translating from other platforms¶

If you're an MSSP coming from Splunk / Sigma / Kusto / Sentinel and have an existing rule library, the Sigma Converter and uncoder.io cover most of what you need. Convert in batch, review, then commit to your IaC repo.

Part 7 — Queries and threat hunting (LCQL)¶

Detections are the streaming layer; queries are the batch / interactive layer. As an MSSP you need both — a detection fires when a known pattern matches in real time, but a query answers the question "did this thing — that we just learned about — happen anywhere in our customer base in the last 30 days?" That's where LimaCharlie's query language, LCQL, earns its keep.

LCQL runs against the event, detect, and audit streams (the --stream flag in the CLI), which Insight retains for a year by default. No SIEM required.

The shape of an LCQL query¶

LCQL is pipe-separated. Five segments, each narrowing the result:

<time-range> | <sensor-selector> | <event-types> | <filter> | <projection>

Concrete:

-24h | plat == windows | NEW_PROCESS | event/COMMAND_LINE contains "psexec" | event/FILE_PATH as path event/COMMAND_LINE as cli routing/hostname as host

That's: last 24 hours → only Windows sensors → only NEW_PROCESS events → command line contains psexec → project these three columns. The earlier you narrow, the faster and cheaper the query.

Where you'll run queries¶

Query Console UI (Query Console UI). The interactive console in the web app is where analysts run hunts, save them, and right-click any matching event to Build a D&R Rule — turning a one-off finding directly into a recurring detection. This is the fastest path from "I noticed something" to "we now detect it everywhere".

CLI for scripting and CI:

limacharlie search run \
  --query "-24h | plat == windows | NEW_PROCESS | event/COMMAND_LINE contains 'psexec'" \
  --start "$(date -d '24 hours ago' +%s)" \
  --end "$(date +%s)" \
  --stream event --limit 1000

Full reference: Query CLI. - REST / SDK when you're embedding hunts in your own tooling. See Data & Queries — Programmatic Management for Python and Go examples.

Estimate cost before you run a hunt¶

Queries are billed per million events evaluated, not per result returned. Before you launch a 30-day hunt across a 200-sensor customer, estimate it:

limacharlie search estimate \
  --query "-30d | * | NEW_PROCESS | event/FILE_PATH ends with '.bat'" \
  --start "$(date -d '30 days ago' +%s)" \
  --end "$(date +%s)"

The console UI shows the estimate live as you type. The cure for an expensive query is almost always tightening the early segments — sensor selector and event type — not the filter.

A 15-query MSSP-baseline hunt library¶

Commit these as saved queries in query.yaml in your IaC repo (the query hive) so they ship with every customer:

# 1. DNS prevalence — domains that appear on many sensors are usually benign;
#    ones that appear on one sensor and contain odd patterns are not
-24h | plat == windows | DNS_REQUEST | event/DOMAIN_NAME contains 'pastebin' | event/DOMAIN_NAME as domain COUNT_UNIQUE(routing/sid) as hosts GROUP BY(domain)

# 2. Unsigned binaries seen in user-land
-24h | plat == windows | CODE_IDENTITY | event/SIGNATURE/FILE_IS_SIGNED != 1 | event/FILE_PATH as Path event/HASH as Hash event/ORIGINAL_FILE_NAME as OriginalFileName COUNT(event) as Count GROUP BY(Path Hash OriginalFileName)

# 3. Process command lines mentioning psexec
-1h | plat == windows | NEW_PROCESS EXISTING_PROCESS | event/COMMAND_LINE contains "psexec" | event/FILE_PATH as path event/COMMAND_LINE as cli routing/hostname as host

# 4. Stack children of cmd.exe (catches LOLBin abuse from interactive shells)
-12h | plat == windows | NEW_PROCESS | event/PARENT/FILE_PATH contains "cmd.exe" | event/PARENT/FILE_PATH as parent event/FILE_PATH as child COUNT_UNIQUE(event) as count GROUP BY(parent child)

# 5. WEL — service installations (T1543.003 persistence)
-12h | plat == windows | WEL | event/EVENT/System/EventID == "7045" | event/EVENT/EventData/ServiceName as svc event/EVENT/EventData/ImagePath as path routing/hostname as host COUNT(event) as count GROUP BY(host svc path)

# 6. WEL — %COMSPEC% in service path (cmd-as-service trick)
-12h | plat == windows | WEL | event/EVENT/System/EventID == "7045" and event/EVENT/EventData/ImagePath contains "COMSPEC"

# 7. WEL — Overpass-the-Hash signatures
-12h | plat == windows | WEL | event/EVENT/System/EventID == "4624" and event/EVENT/EventData/LogonType == "9" and event/EVENT/EventData/AuthenticationPackageName == "Negotiate" and event/EVENT/EventData/LogonProcess == "seclogo"

# 8. WEL — failed logins by source
-1h | plat == windows | WEL | event/EVENT/System/EventID == "4625" | event/EVENT/EventData/IpAddress as SrcIP event/EVENT/EventData/LogonType as LogonType event/EVENT/EventData/TargetUserName as Username event/EVENT/EventData/WorkstationName as SrcHostname

# 9. WEL — stack logon types per user (helps spot weird auth)
-24h | plat == windows | WEL | event/EVENT/System/EventID == "4624" | event/EVENT/EventData/LogonType AS LogonType event/EVENT/EventData/TargetUserName as UserName COUNT_UNIQUE(event) as Count GROUP BY(UserName LogonType)

# 10. GitHub — branch-protection bypass from outside US
-12h | plat == github | protected_branch.policy_override | event/public_repo is false and event/actor_location/country_code is not "us" | event/repo as repo event/actor as actor COUNT(event) as count GROUP BY(repo actor)

# 11. Org-wide psexec usage
-24h | * | NEW_PROCESS EXISTING_PROCESS | event/COMMAND_LINE contains "psexec" | routing/hostname as host event/COMMAND_LINE as cmd COUNT(event) as count GROUP BY(host cmd)

# 12. Network: connections to high destination ports, rarest first
#     (an IP/port pair seen from one host is more interesting than one seen from fifty)
-6h | plat == windows | NETWORK_CONNECTIONS | event/NETWORK_ACTIVITY/?/DESTINATION/PORT > 1024 | event/NETWORK_ACTIVITY/?/DESTINATION/IP_ADDRESS as ip event/NETWORK_ACTIVITY/?/DESTINATION/PORT as port COUNT_UNIQUE(routing/sid) as hosts GROUP BY(ip port) ORDER BY(hosts asc) LIMIT 100

# 13. Hash prevalence: how many hosts loaded each hash (low numbers are the interesting tail)
-24h | plat == windows | CODE_IDENTITY | event/HASH exists | event/HASH as hash COUNT_UNIQUE(routing/sid) as prevalence GROUP BY(hash) ORDER BY(prevalence asc) LIMIT 100

# 14. Registry writes to Run keys (persistence)
-24h | plat == windows | REGISTRY_WRITE | event/REGISTRY_KEY contains "Run" | event/REGISTRY_KEY as key routing/hostname as host COUNT(event) as count GROUP BY(host key)

# 15. M365: users signing in from the most distinct IP addresses. Use it to pick
#     accounts for impossible-travel review; LCQL has no geo lookup (Pattern 7 does it in D&R)
-24h | plat == office365 | UserLoggedIn | event/UserId exists | event/UserId as user COUNT_UNIQUE(event/ClientIP) as distinct_ips GROUP BY(user) ORDER BY(distinct_ips desc) LIMIT 50

Manage like any hive record:

limacharlie search saved-create --name lolbin-stacked-by-parent --stream event \
  --query "-12h | plat == windows | NEW_PROCESS | event/PARENT/FILE_PATH contains \"cmd.exe\" | event/PARENT/FILE_PATH as parent event/FILE_PATH as child COUNT_UNIQUE(event) as count GROUP BY(parent child)"
limacharlie search saved-list
limacharlie search saved-get --name lolbin-stacked-by-parent

saved-run replays the query with the --start and --end stored in the record, which go stale. For a reusable library, leave them off and run the stored text with fresh times:

Q=$(limacharlie search saved-get --name lolbin-stacked-by-parent --output json | jq -r '.data.query')
limacharlie search run --query "$Q" --stream event \
  --start "$(date -d '12 hours ago' +%s)" --end "$(date +%s)"

Template strings and projections¶

Most rule and query work uses Go-style template strings. Quick reference of the helpers you'll reach for:

{{ .routing.hostname }}                 # field reference
{{ .event.FILE_PATH }}                  # nested
{{ token .event.USER_NAME }}            # MD5 anonymization
{{ anon .event.IP_ADDRESS }}            # anon with secret seed
{{ json .event.OBJECT }}                # serialize to JSON
{{ prettyjson .event.OBJECT }}          # serialize indented
{{ base .event.FILE_PATH }}             # filename only
{{ dir .event.FILE_PATH }}              # directory only
{{ join "," .event.PACKAGES }}          # join array
{{ split "," .event.LIST }}             # split string
{{ replace "old" "new" .event.FIELD }}  # substitute

Hunting workflows that pay off for an MSSP¶

IOC sweeps. A peer or feed gives you an IP, hash, or domain. Run one query across each customer's last 30 days. Anything that lights up gets escalated to a Cases entry with the IOC attached.
Cross-customer prevalence. "Has this binary been seen anywhere in our fleet?" Loop the CLI over your customer OIDs, or use the Cases cross-case entity search for IOCs already attached to cases.
Detection prototyping. Hunt first, write rule second. Iterate the filter in the console until the signal-to-noise is clean, then promote to a D&R rule with one click. Replay it across a week of history before deploying.
Replay vs. query — know the difference. Replay tests a rule against historical events to validate matches and noise. Query returns events that match a filter. You'll use both: query to investigate and prototype, replay to vet a rule before it's enabled.

Hand queries to AI agents¶

Saved queries are addressable from D&R-driven AI sessions. The pattern that compounds: an agent's prompt instructs Claude to "run the saved query m365-multi-ip-signins over the last 24 hours" via the LimaCharlie MCP server. The agent reads the result, decides whether to escalate, and writes a note on the case. Every saved query you commit becomes a tool every AI agent can wield — your hunting library is your agent toolbox.

Part 8 — Alerting and case management¶

Detections are only half the loop. Your analysts need a queue with SLA timers, audit trails, and somewhere to write their findings. That's the Cases extension.

Subscribe via the marketplace (per customer org). On subscription, Cases:

Installs D&R rules in dr-managed that forward detections into the case system.
Initializes the org with default severity mapping, SLA targets, and retention.
Begins emitting case_created, case_updated, etc. events that your AI agents and downstream automations can subscribe to.

Configure Cases — the canonical YAML¶

Default config is sensible. Here's what we ship in our customer template, with the changes most MSSPs make:

curl -s -X PUT "https://cases.limacharlie.io/api/v1/config/$YOUR_OID" \
  -H "Authorization: Bearer $LC_JWT" \
  -H "Content-Type: application/json" \
  -d '{
    "severity_mapping": {
      "critical_min": 8,
      "high_min": 5,
      "medium_min": 3
    },
    "sla_config": {
      "critical": {"mtta_minutes": 15,  "mttr_minutes": 240},
      "high":     {"mtta_minutes": 30,  "mttr_minutes": 480},
      "medium":   {"mtta_minutes": 60,  "mttr_minutes": 1440},
      "low":      {"mtta_minutes": 240, "mttr_minutes": 4320},
      "info":     {"mtta_minutes": 480, "mttr_minutes": 10080}
    },
    "retention_days": 90,
    "auto_close_resolved_after_days": 7,
    "auto_grouping_enabled": true,
    "auto_grouping_include_sensor": true,
    "auto_grouping_include_category": false,
    "auto_grouping_window_minutes": 240,
    "auto_grouping_window_mode": "sliding",
    "auto_grouping_reopen_closed": true
  }'

Or via CLI:

limacharlie case config-set --input-file cases-config.yaml

The two settings that move the needle most:

auto_grouping_enabled: true — without it, every detection is its own case. With it, detections from the same sensor (and optionally same category) within a sliding window collapse into one case. For a noisy customer, this can reduce case volume 5–10×.
sla_config — match your contracts. If your highest-tier customer has a 15-minute MTTA SLA on critical, set it here so dashboards alert you when you're at risk of missing it.

Case lifecycle¶

new → in_progress → resolved → closed
                    └────────→ closed
       └────────────────────→ closed

States transition via:

# Acknowledge (start of TTA)
limacharlie case update --case-number 42 --status in_progress \
  --assignees alice@northwindsec.com

# Resolve with classification
limacharlie case update --case-number 42 --status resolved \
  --classification true_positive \
  --conclusion "Confirmed lateral movement; isolated host and reset password."

# Close
limacharlie case update --case-number 42 --status closed

Bulk operations for FP cleanups:

limacharlie case bulk-update --numbers 100,101,102,103 \
  --status closed --classification false_positive

Tailored ingestion mode — only some detections become cases¶

The default mode (all) creates a case for every detection. As alert volume grows you'll want to select which detections go to Cases via D&R rules — that's tailored mode. Set it from the extension config page in the web UI, then add a forwarding rule:

# Only send "high-priority encoded PowerShell" detections into Cases
detect:
  target: detection
  event: encoded-powershell
  op: exists
  path: detect

respond:
  - action: extension request
    extension name: ext-cases
    extension action: ingest_detection
    extension request:
      detect_id:  detect_id
      cat:        cat
      source:     source
      routing:    routing
      detect:     detect
      detect_mtd: detect_mtd

Note: detect_id, cat etc. are bare names — they extract the actual field value. Don't wrap them in {{ }} — the value resolution treats them as gjson paths, and templating would stringify nested objects.

Wire alert delivery¶

Cases tracks the queue; it does not page humans. Pages happen via D&R rules (or webhook subscriptions) that fanout to where humans are. The patterns:

Slack / Teams / Telegram channels for human-readable alerts. Use Slack output, MS Teams output, or Telegram output on the detect stream.
PagerDuty for paging. The PagerDuty extension speaks the right protocol.
SMS via Twilio (extension) for the 3am page.
Email via SMTP (SMTP output) for daily summaries.

Don't fan everything out everywhere. The pattern that scales: low/medium goes to Slack, high goes to Slack + PagerDuty, critical goes to Slack + PagerDuty + SMS. Customer-specific overrides via tags or per-customer outputs.

Slack output — concrete config¶

slack_api_token: hive://secret/acme-slack-bot-token
slack_channel: "#acme-detections"

Slack provisioning (one-time per workspace):

api.slack.com/apps → Create New App → From scratch.
OAuth & Permissions → add scope chat:write.
Install to workspace, copy the Bot User OAuth Token (xoxb-…).
In the Slack channel: /invite @<your-app-name>.
Store the token in your secret hive; reference it from the output config.

Webhook integration to the customer's own ITSM¶

Customers will eventually ask "can you push this into our Jira / ServiceNow / Zendesk?" You have two clean paths:

Webhook output on the detect stream — generic, fast to set up, you control the payload via output allowlisting.
Cases webhook notifications — Cases emits structured events (created, status_changed, assigned, classified, note_added) via the configured webhook adapter. This is what you want when the customer's ticketing system needs the full case lifecycle, not just first-fire alerts. See Cases → Webhook Notifications.

Webhook output config¶

dest_host: https://acme-itsm.example.com/webhooks/lc
secret_key: hive://secret/acme-webhook-shared-secret
auth_header_name: x-northwind-auth
auth_header_value: hive://secret/acme-webhook-bearer

Every webhook LC sends includes a lc-signature header containing HMAC-SHA256(body, secret_key). Verify it on the receiving side — that's how the customer's ITSM proves the call came from LimaCharlie and not a forgery. There is no static IP CIDR to allowlist (LC auto-scales), so HMAC verification is the right control.

Real-time SOC dashboards¶

Cases offers a WebSocket endpoint that streams case events as they happen. If you build a custom analyst console (most large MSSPs eventually do), this is the integration point. For day-1 MSSPs, the Cases UI in app.limacharlie.io is enough.

Part 9 — Outputs: getting your data where it needs to go¶

Telemetry retention in Insight is one year and is included free. You don't need a SIEM to use LimaCharlie. But your customer might already have one, or you might want to build long-term archives or a billing-grade audit trail. That's where Outputs come in.

Pick the right stream¶

Every output is bound to one of event, detect, audit, deployment. The cost shape is different — event is voluminous, detect is sparse. Pick deliberately.

Need	Output module	Stream
Customer's existing SIEM	Splunk / Elastic / OpenSearch / Humio / Kafka	`event` + `detect`
Cold storage / compliance archive	Amazon S3, GCS, Azure Blob	`event` + `audit`
Big-data analytics	BigQuery	`event` (filtered)
SOAR / playbook orchestration	Tines, webhook	`detect`
Tamper-evident audit trail	S3 with object-lock, or syslog to an append-only collector	`audit`
Human-readable alerting	Slack, Teams, Telegram	`detect`

The audit stream is the one MSSPs forget about and regret. It's where every administrative action — user added, rule changed, output added, secret rotated — is logged. Forward audit to immutable storage on every customer org. Reading it back later is non-negotiable when a customer or auditor asks "who changed what".

Concrete output configs¶

S3 (cold archive)¶

bucket: acme-lc-archive
key_id:  hive://secret/acme-aws-access-key-id
secret_key: hive://secret/acme-aws-secret-access-key
sec_per_file: 300         # roll a new file every 5 min
is_compression: "true"
is_indexing: "true"
region_name: us-east-1
dir: lc-exports/
is_no_sharding: false

The bucket needs an IAM policy granting LimaCharlie's writer principal s3:PutObject:

{
  "Version": "2012-10-17",
  "Statement": [{
    "Sid": "PermissionForObjectOperations",
    "Effect": "Allow",
    "Principal": { "AWS": "<<USER_ARN_FROM_LC_DOCS>>" },
    "Action": "s3:PutObject",
    "Resource": "arn:aws:s3:::acme-lc-archive/*"
  }]
}

For a tamper-evident archive: enable S3 Object Lock in compliance mode on the bucket. The customer's auditor will thank you.

Syslog over TLS¶

dest_host: siem.acme.example.com:6514
is_tls: "true"
is_strict_tls: "true"
is_no_header: "false"
structured_data: ""

BigQuery¶

project: acme-lc-analytics
dataset: limacharlie_data
table: events
schema: event_type:STRING, oid:STRING, sid:STRING, hostname:STRING
secret_key: hive://secret/acme-bq-service-account-json
sec_per_file: 60
custom_transform: |-
  {
    "oid":        "routing.oid",
    "sid":        "routing.sid",
    "hostname":   "routing.hostname",
    "event_type": "routing.event_type"
  }

The custom_transform reshapes events into your BigQuery schema before write — crucial for keeping the table schema stable when LC adds new event fields.

Webhook (generic)¶

dest_host: https://acme-soar.example.com/webhooks/lc-detect
secret_key: hive://secret/acme-webhook-secret
auth_header_name: Authorization
auth_header_value: hive://secret/acme-webhook-bearer

Filter outputs at the edge with allowlists¶

Output allowlisting lets you filter the stream before it leaves the platform, rather than paying to ship everything and filtering on the receiver side. Use it. For a noisy event type (e.g., MODULE_LOAD), allowlist to the specific paths you actually need; for event to BigQuery, allowlist to the event types your dashboards consume.

Part 10 — Multi-tenant access (your team, your customers)¶

Now we wire up the access model that you'll live with for the rest of the company's life. The reference page is Designing Access for Multi-Org Deployments — it's the concrete how-to. Below is the opinionated short version, with the working mssp-demo model from refractionPOINT/mssp-demo baked in.

Three rules, repeated until you're tired of them¶

One organization per customer. No exceptions you'll regret.
Staff get access via Organization Groups, by job function. Never invite a staff member directly to a customer org.
Customers get added directly to their own org only. Never put a customer's user into a multi-tenant group.

The "root account" pattern¶

Have one normal user account dedicated as the ultimate authority across every org you manage — the "root account". Nobody actually logs in with it day-to-day; it exists to perform high-level management tasks (creating Organization Groups, transferring ownership, last-resort recovery). This pattern avoids per-Organization API keys for tenant management and dramatically simplifies access design.

Protect the root account with strict SSO + a hardware key on the IdP side, and use it via documented runbook only.

The five-Organization-Group baseline¶

A pragmatic starter set, drawn from the mssp-demo reference layout:

Organization Group	Members	Permissions	Orgs
Administrators	Senior detection engineers, platform admins	All permissions	Every customer org + control org
Analysts	All analysts (L1, L2)	All permissions except `apikey.ctrl`, `billing.ctrl`, `user.ctrl`	Every customer org
Viewers	Sales, leadership, customer success	Read-only: `org.get`, `sensor.list`, `sensor.get`, `dr.list`, `insight.det.get`, `insight.evt.get`, `investigation.get`	Every customer org
Auditors	Compliance	`audit.get`, `sensor.list`	Every customer org
Break-Glass: active-responders	Empty by default; populated during an active incident	`sensor.task` only	Every customer org

Strengths of this layout:

Membership in Administrators is the gate for production change control. Use the IdP for membership management; require approval for additions.
active-responders membership is the audit point for every IR engagement. Add the responder when the engagement starts, remove when it ends. The group's audit log is your engagement log.
Analysts cannot create API keys, change billing, or manage users. That's the point — the hard-to-undo operations stay with Administrators.
Viewers is the one to give to anyone who needs visibility but should never touch anything. Sales demos, customer success calls, leadership reviews.

Workflow once the groups exist:

Create each group once: limacharlie group create --name <name>. Use lowercase names (administrators, analysts, viewers, auditors, active-responders) so the onboarding script below can find them.
Find each group's ID: limacharlie group list. It lists the groups you own.
Add every customer org to each relevant group: limacharlie group org-add --gid <gid> --oid <customer_oid>.
Set the group's permissions in the Groups page or with limacharlie group permissions-set --gid <gid> --permissions "sensor.list,sensor.get,dr.list".
Add a user to exactly the group(s) matching their job: limacharlie group member-add --gid <gid> --email <address>.
When you onboard a new customer, just add the new org to each staff group — every staff member instantly has the right access on the new tenant.

Customer access goes directly on their org¶

limacharlie --oid <customer_oid> user invite --email <address>, then user permissions set-role --role Viewer (or Operator). Don't add customer users to any group, ever.

If a customer needs access to multiple of their own orgs (multi-business-unit shops), create a customer-specific Organization Group that contains only that customer's orgs and only that customer's users. Don't mix tenants inside a single group — groups are additive, and a leak there leaks across customers.

Use Organization API keys for automation, never user keys¶

Every CI pipeline, every webhook, every adapter, every IaC script you write should authenticate with an Organization API key scoped to the minimum permissions it needs and only on the org it operates against. A user API key has the user's full reach across every org they can see — if the user leaves, the key dies; if the key leaks, you've exposed every customer.

Permission groupings to use as starting points:

# Reader — for monitoring scripts, dashboards
org.get, sensor.list, sensor.get, dr.list,
insight.det.get, insight.evt.get, investigation.get

# Investigator — for an automated investigation pipeline
above + sensor.task, ext.request, investigation.set

# Detection-engineer — for IaC sync push
above + dr.set, dr.del, plus the .set permission of every hive and
resource type the sync touches, for example fp.ctrl, lookup.set,
query.set, secret.set, ai_agent.set, ext.conf.set, ext.sub,
output.set, ikey.set

# AI-agent operator — what the lc-essentials/ai-agent runners need
org.get, sensor.list, sensor.get, sensor.task, dr.list,
insight.det.get, insight.evt.get, investigation.get,
investigation.set, ext.request, ai_agent.operate, sop.get,
org_notes.get, org_notes.set

# Add lookup.get, lookup.set for a responder that maintains block-lists

Automate the new-customer onboarding¶

The full checklist is in Designing Access. The condensed version, scriptable today:

#!/usr/bin/env bash
set -euo pipefail

CUSTOMER_NAME="$1"
NEW_OID="$2"
PRIMARY_CONTACT="$3"      # optional 4th argument: --apply

# Same resource-type flags as Part 15. No --hive-secret: customer secrets are
# provisioned out-of-band and must not be touched by --force.
SYNC_FLAGS="--outputs --integrity --exfil --artifact --resources --extensions \
  --org-values --installation-keys --yara --hive-dr-general --hive-fp \
  --hive-lookup --hive-query --hive-extension-config --hive-cloud-sensor \
  --hive-ai-agent"

# 0. Safety checks. --force below makes the org an exact mirror of the template for
#    the selected types, so refuse to run against an org that already has keys or
#    sensors (a re-run would delete the keys step 5 creates and any per-customer
#    outputs or FP rules). Then show the diff and stop unless --apply was passed.
[ "$(limacharlie --oid "$NEW_OID" installation-key list --output json | jq 'length')" -eq 0 ] \
  || { echo "$NEW_OID already has installation keys; not a new customer" >&2; exit 1; }
[ "$(limacharlie --oid "$NEW_OID" sensor list --limit 1 --output json | jq 'length')" -eq 0 ] \
  || { echo "$NEW_OID already has sensors; not a new customer" >&2; exit 1; }
limacharlie --oid "$NEW_OID" sync push $SYNC_FLAGS \
  --config-file "iac-repo/templates/customer-baseline.yaml" --force --dry-run
if [ "${4:-}" != "--apply" ]; then
  echo "Dry run only. Review the diff above, then re-run with --apply as the 4th argument."
  exit 0
fi

# 1. Add the new org to every staff group
for GROUP_NAME in administrators analysts viewers auditors active-responders; do
  GID=$(limacharlie group list --output json \
    | jq -r --arg n "$GROUP_NAME" '.[] | select(.name == $n) | (.gid // .group_id)')
  limacharlie group org-add --gid "$GID" --oid "$NEW_OID"
done

# 2. Apply the IaC template (see Part 15 for template structure)
limacharlie --oid "$NEW_OID" sync push $SYNC_FLAGS \
  --config-file "iac-repo/templates/customer-baseline.yaml" --force

# 3. Subscribe to the standard extension kit
for EXT in ext-cases ext-exfil ext-lookup-manager ext-usage-alerts ext-sensor-cull ext-artifact ext-reliable-tasking ext-infrastructure ext-git-sync; do
  limacharlie --oid "$NEW_OID" extension subscribe --name "$EXT"
done

# 4. Invite the customer's primary contact directly on their own org
limacharlie --oid "$NEW_OID" user invite --email "$PRIMARY_CONTACT"
limacharlie --oid "$NEW_OID" user permissions set-role \
  --email "$PRIMARY_CONTACT" --role Viewer

# 5. Create per-tier installation keys
for ROLE in server workstation vip sleeper; do
  TAGS="$ROLE"
  [[ "$ROLE" == "sleeper" ]] && TAGS="lc:sleeper"
  limacharlie --oid "$NEW_OID" installation-key create \
    --description "${CUSTOMER_NAME}-${ROLE}" --tags "$TAGS"
done

echo "Customer $CUSTOMER_NAME provisioned. OID: $NEW_OID"

Once that script lives in your team's repo, onboarding a new customer is a 10-minute job, not a 10-hour one.

Verify access on a cadence¶

Every quarter, run the verification checklist on every production org. Wrap it in a script that posts to a Slack channel, review the same week each quarter:

for OID in $(limacharlie auth list-orgs --output json | jq -r '.[].oid'); do
  echo "=== $OID ==="
  limacharlie --oid "$OID" user list
  limacharlie --oid "$OID" user permissions list
done

Part 11 — Sensor commands and live response¶

Telemetry's the foundation, detections fire on what you collect — but eventually you'll need to do something on a host. That's what the sensor command catalog is for. The commands are designed with restraint: no arbitrary code execution, no covert upload paths. Everything is auditable in the audit stream.

The commands you'll use most¶

Command	What it does
`os_processes`, `os_services`, `os_version`, `netstat`	Enumerate processes, services (Windows), OS version and architecture, and network connections.
`os_kill_process`	Kill a process by ID.
`deny_tree`	Block a process tree. Used from D&R rules, for example `deny_tree <<routing/parent>>`.
`file_get`, `file_hash`, `file_del`, `dir_list`	Pull, hash, or delete a file, and walk a directory.
`mem_map`, `mem_strings`, `mem_find_string`	Memory map of a process, strings in its memory, and a string search.
`dns_resolve`	Resolve a name from the sensor's point of view.
`segregate_network`, `rejoin_network`	Isolate the host and un-isolate it.
`yara_scan`	Scan a file or a process with a YARA rule, for example `yara_scan hive://yara/malware-rule --pid 4812`.
`history_dump`	Dump events from the sensor's local cache.
`fim_add`, `exfil_add`	Add a file-integrity pattern or an Exfil watch rule at runtime.

The arguments differ per command and are listed in the Endpoint Commands reference. Run any of them from the Console view in the web app, or from the CLI:

# Fire and forget: the response arrives as an event in the sensor's timeline
limacharlie task send --sid <SID> --task "os_processes"
limacharlie task send --sid <SID> --task 'file_hash "C:\\suspect.exe"'
limacharlie task send --sid <SID> --task "yara_scan hive://yara/malware-rule --pid 4812"

# Wait for the answer (30 seconds by default)
limacharlie task request --sid <SID> --command "os_version" --timeout 60

Cross-fleet tasking¶

There is no fleet-wide form of task send. To run a command on many sensors (say, an audit-time file_hash of /etc/sudoers on every Linux production host), let the server filter with a sensor selector and loop over the result:

limacharlie sensor list --online \
  --selector 'plat == linux and "prod" in tags' --output json \
  | jq -r '.[].sid' \
  | while read -r SID; do
      limacharlie task send --sid "$SID" \
        --task 'file_hash /etc/sudoers' \
        --investigation-id audit-2026-Q2
    done

--online skips sensors that are offline right now, and --investigation-id tags every response so you can find them together in the timeline.

For sensors that may be offline at command time, use Reliable Tasking. It queues the task and executes it when the sensor checks in. It takes a selector itself, so no loop is needed:

limacharlie extension request --name ext-reliable-tasking --action task \
  --data '{"context":"audit-2026-Q2","selector":"plat==linux and prod in tags","task":"file_hash /etc/sudoers","ttl":172800}'

ttl is in seconds. The task waits up to ttl for an offline sensor to check in. Use a unique context per audit or IR engagement so you can correlate results. For a single known sensor, limacharlie task reliable-send --sid <SID> --command "<task>" --ttl 172800 does the same thing.

Network isolation as a response action¶

The most common live-response action is "isolate this host from everything except LimaCharlie". You can task it manually:

limacharlie endpoint-policy isolate --sid <SID>
limacharlie endpoint-policy rejoin --sid <SID>

Or trigger it automatically from a D&R rule:

detect:
  event: NEW_PROCESS
  op: contains
  path: event/COMMAND_LINE
  value: ransom-payload.exe

respond:
  - action: report
    name: ransomware-suspected
    metadata:
      level: critical
  - action: isolate network
  - action: extension request
    extension name: ext-dumper
    extension action: request_dump
    extension request:
      target: '{{ "memory" }}'
      sid: '{{ .routing.sid }}'
      retention: 30

This is the kind of automation that earns an MSSP its keep at 3am.

Part 12 — Forensics and IR extensions¶

For the moments when "look in the timeline" is not enough.

Artifact collection¶

Artifact extension collects entire log channels and stores them centrally for IR. Three sources you'll use most:

Windows Event Log — wel://Security, wel://System, wel://Sysmon/Operational
Mac Unified Log — mul://* with optional filters
Network packet capture (Linux) — tcpdump rules

Configure once per customer; artifacts are billable per GB stored.

Memory dumps on demand¶

The Dumper extension works on Windows hosts only. It has one action, request_dump, and two targets: memory (a full memory dump of the host) and mft (the NTFS Master File Table, converted to CSV). The dump is uploaded to the Artifact system for the retention you set, and the local copy is deleted. There is no single-process target. For one process, use mem_map, mem_strings, or yara_scan from Part 11. Common D&R-driven pattern: when a critical detection fires, dump and isolate.

respond:
  - action: extension request
    extension name: ext-dumper
    extension action: request_dump
    extension request:
      target: '{{ "memory" }}'    # or '{{ "mft" }}'
      sid: '{{ .routing.sid }}'
      retention: 30               # days to keep

Add ignore_cert: true only when the host sits behind a TLS-intercepting proxy. The extension does not check for free disk space before a full dump, and the uploaded dumps are billed as external logs, so a multi-gigabyte dump on every detection adds up. Pair the rule with a suppression block.

Velociraptor for deep DFIR¶

Velociraptor is the open-source DFIR collector; LimaCharlie can trigger Velociraptor artifacts as a response action. Useful when you want a structured forensic collection (KAPE-style triage, browser history, scheduled tasks, etc.) on demand:

respond:
  - action: extension request
    extension name: ext-velociraptor
    extension action: collect
    extension request:
      artifact_list: ['Windows.KapeFiles.Targets']
      sid:  '{{ .routing.sid }}'
      args: '{{ "EventLogs=Y" }}'
      collection_ttl: 3600
      retention_ttl: 7
      ignore_cert: false

Velociraptor runs on the endpoint through the LimaCharlie sensor. It zips what it collects, and the ZIP is ingested into the Artifact system, where you can download it. Follow the collection on the ext-velociraptor sensor timeline.

Atomic Red Team for testing detections¶

The Atomic Red Team extension drives MITRE ATT&CK technique tests on a host so you can verify your detections actually fire. Three steps per test target:

Prepare host — installs the framework + dependencies on the test host.
Run tests — executes selected ATT&CK techniques sequentially.
Cleanup host — removes the framework and reverses prepare changes.

Watch the ext-atomic-red-team adapter timeline for prepare_started, test_result, run_success, cleanup_success events. Schedule a monthly run against a non-production sensor and treat each missing detection as a backlog item.

YARA Manager¶

YARA Manager lets you load YARA rule sets via URL or ARL. Common pattern:

URL: https://raw.githubusercontent.com/Yara-Rules/rules/master/email/Email_generic_phishing.yar

Or as ARL (directory of rules):
[github,Yara-Rules/rules/email]

Once loaded, scan from a D&R rule:

respond:
  - action: task
    command: yara_scan hive://yara/malware-rule --pid "{{ .event.PROCESS_ID }}"
    suppression:
      is_global: false
      keys: ["{{ .event.PROCESS_ID }}", "yara-scan"]
      max_count: 1
      period: 1m

The suppression block is non-optional — without it, a runaway can YARA-scan the same PID hundreds of times.

Payload Manager¶

Payload Manager stores binaries (response tools, custom collectors) that sensors can fetch and execute. Useful for distributing your own incident-response binary fleet without baking it into the sensor.

Part 13 — Threat intelligence and lookups¶

Threat feeds turn your detection coverage from "what we wrote" into "what the world knows". The Lookup Manager extension auto-refreshes lookups from URLs (or ARLs) every 24 hours; D&R rules consume those lookups via the lookup operator.

Auto-refresh feeds¶

The IaC entry — drop this into your global extension_config hive:

hives:
  extension_config:
    ext-lookup-manager:
      data:
        lookup_manager_rules:
          - name: alienvault
            arl: ""
            format: json
            predefined: '[https,storage.googleapis.com/lc-lookups-bucket/alienvault-ip-reputation.json]'
            tags: [alienvault]
          - name: tor
            arl: ""
            format: json
            predefined: '[https,storage.googleapis.com/lc-lookups-bucket/tor-ips.json]'
            tags: [tor]
          - name: talos
            arl: ""
            format: json
            predefined: '[https,storage.googleapis.com/lc-lookups-bucket/talos-ip-blacklist.json]'
            tags: [talos]
      usr_mtd:
        enabled: true

Every 24 hours the manager re-fetches each feed and updates the corresponding lookup hive record. A newly added feed is not fetched until the next cycle unless you click Manual Sync on the extension page, so do that after the first deploy.

Lookup-driven D&R¶

detect:
  event: NETWORK_CONNECTIONS
  op: lookup
  path: event/NETWORK_ACTIVITY/?/IP_ADDRESS
  resource: hive://lookup/tor

respond:
  - action: report
    name: tor-egress
    metadata:
      level: medium

The ? in the path is the wildcard — match any element of the array.

Your own block-lists¶

When your responder agent (Part 14) confirms a malicious IP, it adds the IP to a soc-blocked-ips lookup. A lookup is one hive record holding all of its entries, and lookup set writes the whole record, so create it once with its first entry:

cat > soc-blocked-ips.yaml <<'EOF'
data:
  lookup_data:
    "203.0.113.42": {mtd: "Blocked by SOC - case 142"}
usr_mtd:
  enabled: true
EOF
limacharlie lookup set --key soc-blocked-ips --input-file soc-blocked-ips.yaml

To add an entry later, read the record, add the key, and write it back. Passing the etag makes the write fail instead of silently overwriting if another agent changed the lookup in between; re-run the command if that happens:

limacharlie lookup get --key soc-blocked-ips --output json \
  | jq --arg k "198.51.100.7" \
       '{data: (.data | .lookup_data[$k] = {mtd: "Blocked by SOC - case 143"}),
         usr_mtd: .usr_mtd, etag: .sys_mtd.etag}' \
  | limacharlie lookup set --key soc-blocked-ips

A standing rule:

detect:
  event: NETWORK_CONNECTIONS
  op: lookup
  path: event/NETWORK_ACTIVITY/?/IP_ADDRESS
  resource: hive://lookup/soc-blocked-ips

respond:
  - action: report
    name: connection-to-soc-blocked-ip
    metadata:
      level: high

Same pattern for soc-blocked-hashes and soc-blocked-domains. This is how an MSSP turns a one-time investigation into ongoing protection automatically.

The rule reports and stops there on purpose. An AI agent fills these lists, and one wrong entry (a shared CDN address, a cloud egress IP, an internal resolver) would cut off every host that talks to it. Let the responder decide on isolation, or gate it behind a human approval (the Feedback extension, below). If you do add isolate network here, scope it to a tag such as vip or server, put a per-sensor suppression on it, and tell your analysts that an entry in this list can take a host offline.

Threat-feed-rule pattern¶

Many feeds publish signatures, not just IOCs. The threat-feed-rule tutorial walks the pattern: fetch a community feed, convert via the Sigma converter, push to dr-managed, refresh weekly. Treat the converted rules as untrusted — replay each before enabling, suppress aggressively, classify outcomes for tuning.

Part 14 — AI: agentic SOC capabilities¶

This is the part that will surprise people coming from older platforms. LimaCharlie ships with a managed Claude runtime — AI Sessions — that lets you run autonomous agents triggered by D&R rules or schedules. You bring your Anthropic API key (Bring-Your-Own-Key); LimaCharlie runs the agents in isolated containers in cloud and bills only for the runtime overhead.

Two patterns¶

D&R-driven sessions (guide). A detection fires; a D&R rule's respond: block has action: start ai agent; an isolated Claude session spins up with the prompt and event context you specify, runs to completion, posts findings (typically to a Cases note), and exits.
User sessions (guide). Your analysts log in and interactively drive a Claude session that has full access to your LimaCharlie org via MCP. Used for ad-hoc investigation when an analyst wants AI-assisted reasoning over the org's data.

Pre-built agent recipes¶

The refractionPOINT/lc-ai repo ships with two pre-built AI team recipes you can deploy as IaC:

Lean SOC — four agents (triage / investigator / responder / reporter). Cheap, simple. The right starting point for an MSSP whose SOC capacity is "me and one other person".
Tiered SOC — eight agents: triage, L1 investigator, L2 analyst, containment, malware analyst, threat hunter, SOC manager, and shift reporter. Right when you have a real analyst team and want AI to amplify each tier.

Both deploy via the lc-essentials Claude Code plugin (the easiest path) or directly as ai_agent Hive records (the IaC path).

Lean SOC architecture in detail¶

flowchart TD
    det[EDR Detection] --> triage["TRIAGE<br/>sonnet, $2.00"]
    triage -->|Obvious FP| dismissed["Dismissed<br/>~$0.10"]
    triage -->|creates case| inv["INVESTIGATOR<br/>opus, $5.00"]
    inv -->|FP confirmed| closed["Closed<br/>~$1.10"]
    inv -->|"tag: needs-containment"| resp["RESPONDER<br/>sonnet, $1.00"]
    resp --> actions[Actions documented on case]
    inv -->|Resolved/Escalated| human["Human review<br/>full documentation"]

    schedule["Every 24h"] --> reporter["REPORTER<br/>sonnet, $1.00"]
    reporter --> metrics[SOC metrics + SLA monitoring]

Cost profile (typical):

Outcome	Agents involved	Estimated cost
FP dismissed at triage	triage	~$0.10
FP dismissed after investigation	triage + investigator	~$1.10
TP with containment	triage + investigator + responder	~$2.10
Daily overhead (scheduled reports)	reporter	~$1.00/day

The triage agent — full IaC YAML¶

This is the agent definition you commit to your control org's ai_agent hive. Every customer's triage agent then references it by name (hive://ai_agent/lean-triage).

version: 3
hives:
  ai_agent:
    lean-triage:
      data:
        anthropic_secret: hive://secret/anthropic-key
        lc_api_key_secret: hive://secret/lean-triage
        prompt: |
          You are the Triage agent in a lean automated SOC. You process all recent
          detections in batch, dismiss obvious false positives, and route remaining
          detections into the correct cases (existing or new) for investigation by
          downstream agents.

          Sessions are serialized via debounce -- only one triage session runs at a
          time per org. When your session ends, any detections that arrived during
          processing will trigger a new session. Process ALL recent detections, not
          just the one that triggered you.

          ## Standard Operating Procedures (SOPs)
          Before starting, check if the org has SOPs (`limacharlie sop list`). If a
          relevant SOP exists, follow it.

          ## Workflow
          1. List recent detections (last hour):
             END=$(date +%s); START=$((END - 3600))
             limacharlie detection list --start $START --end $END --oid <oid> --output yaml
          2. List existing open cases.
          3. Identify unprocessed detections (compare the two lists).
          4. For each, decide: obvious FP (skip), or worth a case.
          5. Group related detections (same sid + related cat).
          6. Route each group to an existing case OR create a new case.
          7. Output a summary: total, in-cases, dismissed, routed, created.

          ## Important Rules
          - NEVER fabricate data. Only report what you actually find.
          - Always use --oid <oid> and --output yaml for CLI commands.
          - Detection timestamps are MILLISECONDS (13 digits). API parameters are
            SECONDS (10 digits). Divide by 1000.
        name: "Lean Triage Batch"
        data:
          oid: routing.oid
        debounce_key: lean-triage
        model: sonnet
        max_turns: 50
        max_budget_usd: 2.00
        ttl_seconds: 1200
        one_shot: true
        permission_mode: bypassPermissions
      usr_mtd:
        enabled: true
        tags:
          - ai-team:lean-soc:triage

The triggering D&R rule (in dr-general) is dead simple:

version: 3
hives:
  dr-general:
    lean-triage-on-detect:
      data:
        detect:
          target: detection
          op: exists
          path: /
        respond:
          - action: start ai agent
            definition: hive://ai_agent/lean-triage
            suppression:
              is_global: true
              keys: [lean-triage-rate-limit]
              max_count: 20
              period: 1m
      usr_mtd:
        enabled: true
        tags: [ai-team:lean-soc:triage]

The combination of debounce_key: lean-triage (one session at a time per org) plus suppression (rate-limit at 20/min globally) means the agent can never spawn 200 sessions at once during an alert storm — and never miss an alert either, since debounced sessions queue.

The investigator agent — full IaC YAML¶

version: 3
hives:
  ai_agent:
    lean-investigator:
      data:
        anthropic_secret: hive://secret/anthropic-key
        lc_api_key_secret: hive://secret/lean-investigator
        prompt: |
          You are the sole Investigator in a lean automated SOC. You handle the full
          investigation lifecycle -- from initial triage through deep analysis to
          conclusion. Combine L1 + L2: systematic evidence gathering, timeline
          reconstruction, scope assessment, lateral movement hunting, root cause.

          You will receive a case_number and an oid (organization ID).

          ## Workflow
          1. Mark investigating:
             limacharlie case tag add --case-number <n> -t investigating --oid <oid>
             limacharlie case update --case-number <n> --status in_progress --oid <oid>
          2. Get case details.
          3. For each linked detection, get the full record.
          4. Investigate the context: process tree, network, file ops, registry,
             other detections in the same time window on the same sensor.
          5. Assess scope: same IOCs / same detection on other sensors / lateral
             movement signs / affected accounts.
          6. Root-cause analysis if suspicious: trace the chain back to initial
             access, map techniques to MITRE ATT&CK.
          7. Document findings — fill EVERY case section: Summary, Conclusion,
             Analysis notes, Entities (IOCs), Telemetry references.
          8. Signal containment if confirmed malicious — add a note tagged
             @responder with sensors to isolate, IOCs to block, accounts to disable.
          9. Determination + close out:
             - clearly FP → classification false_positive, status closed
             - confirmed malicious → classification true_positive, status in_progress,
               tag needs-escalation
             - suspicious but unclear → classification pending, status in_progress,
               tag needs-escalation
        name: "Lean Investigator Case {{ .event.case_number }}"
        data:
          oid: routing.oid
          case_id: event.case_id
          case_number: event.case_number
          event_id: event.event_id
        model: opus
        max_turns: 50
        max_budget_usd: 5.0
        ttl_seconds: 1800
        one_shot: true
        permission_mode: bypassPermissions
      usr_mtd:
        enabled: true
        tags:
          - ai-team:lean-soc:investigator

Triggered on case creation:

version: 3
hives:
  dr-general:
    lean-investigator-on-case-created:
      data:
        detect:
          event: case_created
          op: exists
          path: event/case_id
        respond:
          - action: start ai agent
            definition: hive://ai_agent/lean-investigator
            suppression:
              is_global: true
              keys: [lean-investigator-rate-limit]
              max_count: 10
              period: 1m
      usr_mtd:
        enabled: true
        tags: [ai-team:lean-soc:investigator]

The responder agent — what it actually does¶

The responder is invoked when the investigator tags a case needs-containment. Critical-severity cases get auto-contained; everything else is recommended and documented for human approval.

Core actions inside the responder's prompt:

# Network isolation
limacharlie endpoint-policy isolate --sid <sid> --oid <oid>

# Block hashes / IPs / domains via SOC-managed lookups
# (the get | jq | set pattern from Part 13, once per lookup: soc-blocked-hashes,
#  soc-blocked-ips, soc-blocked-domains)
limacharlie lookup get --key soc-blocked-ips --output json --oid <oid> \
  | jq --arg k "<ip>" \
       '{data: (.data | .lookup_data[$k] = {mtd: "Blocked by SOC - case <n>"}),
         usr_mtd: .usr_mtd, etag: .sys_mtd.etag}' \
  | limacharlie lookup set --key soc-blocked-ips --oid <oid>

# Human-in-the-loop approval before destructive actions
limacharlie feedback request-approval \
  --channel <slack-channel> \
  --question "Case #<n>: Isolate sensor <hostname> (<sid>)?" \
  --destination case --case-id <n> \
  --timeout 300 --timeout-choice denied --oid <oid>

The Feedback extension request-approval is the right hook for "AI is about to do something irreversible — get a human nod first". Default the timeout to a denial.

The reporter agent — sample output¶

The reporter runs once per day per customer and posts a summary into the case stream (and optionally to Slack). The format the lean-soc reporter produces:

## SOC Daily Report — 2026-05-08

### Health Check
- Stale tags cleaned: 7
- SLA breaches detected: 1 (case #142, MTTR exceeded by 32m)
- Abandoned cases flagged: 2

### Key Metrics (Last 24 Hours)
| Metric                | Value |
|-----------------------|-------|
| Total Cases Created   | 47    |
| Cases Closed (FP)     | 38    |
| Cases Closed (TP)     | 5     |
| Open Backlog          | 4     |
| Mean Time to Acknowledge | 8m  |
| Mean Time to Resolve     | 47m |

### Severity Breakdown
| Severity | New | Closed | Open |
|----------|-----|--------|------|
| Critical | 1   | 1      | 0    |
| High     | 6   | 5      | 1    |
| Medium   | 14  | 11     | 3    |
| Low      | 26  | 26     | 0    |

### Notable Incidents
- Case #142: Confirmed PowerShell encoded-command from ACME-WS-014.
  Investigation found credential theft attempt; reset password, blocked
  destination IP. MTTR 4h 32m (SLA: 4h).

### Recommendations
- Detection "exec-from-downloads" fired 18× in 24h; 17 closed FP. Review
  for tuning (likely vendor installer pattern).

Cost guardrails on every AI agent¶

Every D&R-driven agent should set:

max_budget_usd — hard cap on Anthropic spend per session. Lean SOC presets: $2 (triage), $5 (investigator), $1 (responder/reporter).
max_turns — limits conversation length even if the budget allows more.
ttl_seconds — wall-clock kill switch (cap is 24h).
permission_mode: bypassPermissions — for unattended automation, otherwise tool calls block on a non-existent human approver.
one_shot: true — finishes the task and exits (default for D&R-triggered).
idempotent_key and/or debounce_key — so a detection storm doesn't trigger 200 sessions at once. idempotent drops duplicates; debounce queues them.
D&R suppression block on the triggering rule — global rate limit (20/min for triage, 10/min for investigator).

Bring your own provider — Bedrock or Vertex¶

If you want Claude requests routed through your AWS or GCP account instead of Anthropic directly (compliance, billing consolidation), use the bedrock: or vertex: block on the ai_agent record instead of anthropic_secret. See Alternative AI Providers. Same agent, different transport.

Day-1 deployment¶

If you only deploy one AI thing on day 1, deploy triage:

Store your Anthropic API key as a Hive Secret: limacharlie secret set --key anthropic-key ...
Create an Org API key with the AI-agent-operator permission set (Part 10) and store it as lean-triage.
Push the triage ai_agent record and the corresponding dr-general rule.
Wait for the next detection. Watch the Live Feed.

Even at low alert volumes, this saves your L1 hours per day and scales linearly because each session is independent.

Part 15 — Infrastructure as Code: the part that makes you scalable¶

Up to this point you may have been clicking buttons. That works for experimentation. It does not work for an MSSP. The moment you have your second customer, every config you care about needs to be in a Git repo, and that repo needs to be the source of truth.

The two extensions you need¶

Infrastructure extension — fetch, push, dry-run, selective sync flags (sync_dr, sync_outputs, sync_resources, sync_artifacts, etc.). The thing the CLI talks to.
Git Sync extension — connects an organization to a Git repo (over SSH with a deploy key), and can both push exports of the live config to Git and pull configs from Git on a schedule.

You will use both. Infrastructure for ad-hoc CLI work and PR-time validation. Git Sync for the ongoing "the world is whatever's in the repo" loop.

Recommended repo layout¶

The pattern that holds up across many customers, drawn from the refractionPOINT/mssp-demo reference repo:

mssp-iac/
├── README.md
├── configs/
│   ├── global/
│   │   ├── all.yaml                # includes all of the below
│   │   ├── rules.yaml              # global D&R rules
│   │   ├── outputs.yaml            # default outputs (audit → S3, etc.)
│   │   ├── installation_keys.yaml
│   │   ├── extensions.yaml
│   │   ├── resources.yaml
│   │   ├── integrity.yaml          # FIM patterns
│   │   ├── artifact.yaml           # log artifact collection
│   │   ├── exfil.yaml              # event collection / watch rules
│   │   ├── fp.yaml                 # global false-positive rules
│   │   ├── lookup.yaml             # threat feeds
│   │   ├── query.yaml              # saved query library
│   │   ├── yara.yaml
│   │   └── ai_agent.yaml           # lean-soc agent definitions
│   ├── control/
│   │   └── main.yaml           # the control org (not a customer)
│   └── customers/
│       ├── acme/
│       │   ├── main.yaml           # customer index — references global + locals
│       │   ├── custom_rules.yaml
│       │   ├── custom_fp.yaml
│       │   └── custom_outputs.yaml
│       ├── globex/
│       │   ├── main.yaml
│       │   └── custom_rules.yaml
│       └── initech/
│           └── main.yaml           # pure inheritance, no overrides
└── templates/
    └── customer-baseline.yaml      # used by onboarding script

global/all.yaml is just an include: of every global file:

version: 3
include:
  - extensions.yaml
  - resources.yaml
  - installation_keys.yaml
  - rules.yaml
  - outputs.yaml
  - integrity.yaml
  - artifact.yaml
  - exfil.yaml
  - fp.yaml
  - lookup.yaml
  - query.yaml
  - yara.yaml
  - ai_agent.yaml

Each customer's main.yaml includes the globals plus its own overrides:

# oid: <ACME_OID>              # CI reads the customer's OID from this comment
version: 3
include:
  - ../../global/all.yaml      # everything global
  - custom_rules.yaml          # ACME-specific detections
  - custom_fp.yaml             # ACME-specific false positives
  - custom_outputs.yaml        # ACME's SIEM

Initech's pure-inheritance customer is only the OID comment and one include:

# oid: <INITECH_OID>
version: 3
include:
  - ../../global/all.yaml

That's the pattern that scales from 3 customers to 300 — globals are written once and edited once; customers are included into globals and add only what's truly local.

Real examples from the global config¶

From the mssp-demo installation_keys.yaml:

version: 3
installation_keys:
  linux:
    desc: linux
    tags: [linux]
  mac:
    desc: mac
    tags: [mac]
  windows:
    desc: windows
    tags: [windows]
  chrome:
    desc: chrome
    tags: [chrome]

An outputs.yaml in the same style (the destination is a documentation address, not a real host):

version: 3
outputs:
  archive-scp:
    dest_host: 203.0.113.10
    dir: /srv/lc-output/
    for: event
    is_built_in: false
    module: scp
    password: hive://secret/scp-password
    username: lc-export

Use a dedicated, least-privileged account on the destination host, and always reference the password through hive://secret/.

Secrets do not live in the repo¶

Every secret in your YAML is a hive://secret/<name> reference, not a value. The actual secrets live in the secret hive in each org and are managed out-of-band (manually on customer onboarding, or via a separate, more locked-down provisioning script). That way the IaC repo is safe to commit, share with engineers, and back up to S3.

How to bootstrap the repo from where you are now¶

You've been clicking buttons in the control org. To turn that into the repo:

mkdir mssp-iac && cd mssp-iac
git init
mkdir -p configs/global configs/control configs/customers templates

# The resource types we manage as code. --hive-secret is left out on purpose,
# see the note below.
SYNC_FLAGS="--outputs --integrity --exfil --artifact --resources --extensions \
  --org-values --installation-keys --yara --hive-dr-general --hive-fp \
  --hive-lookup --hive-query --hive-extension-config --hive-cloud-sensor \
  --hive-ai-agent"

# Export the control org's configuration
limacharlie sync pull \
  --oid <control-oid> \
  --config-file configs/control/main.yaml \
  $SYNC_FLAGS

# Or use the Git Sync extension's "Push to Git" button
# in the control org's web UI — produces an `exports/` subtree.

git add . && git commit -m "Initial export from control org"

--all would also cover the secret hive. Name the types you want instead, so secret records never end up in the repo, and so that --force on a push can never delete a customer's secrets just because they are not in the YAML. Use the same $SYNC_FLAGS for every sync pull and sync push in this part.

From there, refactor the export: pull rules that are reusable into configs/global/rules.yaml, write a slim main.yaml for the control org that includes them, and you have your first reusable global rule set.

Validate before you push¶

# Always dry-run first; --force makes the org an exact mirror of the YAML
# for the selected resource types
limacharlie sync push \
  --oid <customer-oid> \
  --config-file configs/customers/acme/main.yaml \
  $SYNC_FLAGS --force --dry-run

Read the diff. Then drop --dry-run. In CI, run dry-run on every PR; gate merges on a clean diff.

Each resource type has its own flag, so you can push a subset. This pushes only D&R rules, outputs, and false-positive rules:

limacharlie sync push --oid <oid> --config-file main.yaml \
  --hive-dr-general --outputs --hive-fp --force

Schedule recurring sync via Git Sync¶

Once the repo is the source of truth, configure Git Sync in each customer org to:

Pull from Git on a schedule (every 15 minutes is a fine default), so a merge to main deploys without anyone running a CLI.
Optionally, push exports back to Git on a schedule, into exports/ subdirectory, so any drift caused by an emergency UI change is committed and reviewable.

Setting up Git Sync with deploy keys¶

# 1. Generate an SSH key dedicated to Git Sync
mkdir -p ~/.ssh/gitsync && chmod 700 ~/.ssh/gitsync
ssh-keygen -t ed25519 -C "limacharlie-gitsync" -f ~/.ssh/gitsync/id_ed25519

# 2. Add the public key to your Git provider as a deploy key with WRITE access
cat ~/.ssh/gitsync/id_ed25519.pub
# (paste into GitHub Settings → Deploy keys, check "Allow write access")

# 3. Store the private key as a Hive Secret in each customer org
jq -Rs '{secret: .}' ~/.ssh/gitsync/id_ed25519 \
  | limacharlie secret set --key github-deploy-key --enabled \
      --oid <customer_oid>

# 4. Wire Git Sync via the web UI:
#    Settings → Git Sync → SSH Key: select github-deploy-key
#    Repository (SSH URL): git@github.com:northwindsec/mssp-iac.git
#    Branch: main
#    Push schedule: every 60 min (export drift back to Git)
#    Pull schedule: every 15 min (deploy merges automatically)

CI pattern¶

A minimal CI for the IaC repo:

# .github/workflows/lc-sync.yml
on:
  pull_request:
    branches: [main]
  push:
    branches: [main]

jobs:
  validate:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: pip install limacharlie
      - name: Sync every customer (dry run on pull requests, real push on main)
        env:
          # JSON map of customer OID -> that customer's Organization API key.
          # An org key only works on its own org, so CI needs one per customer.
          LC_CUSTOMER_KEYS: ${{ secrets.LC_CUSTOMER_KEYS }}
          SYNC_MODE: ${{ github.event_name == 'push' && ' ' || '--dry-run' }}
          SYNC_FLAGS: >-
            --outputs --integrity --exfil --artifact --resources --extensions
            --org-values --installation-keys --yara --hive-dr-general --hive-fp
            --hive-lookup --hive-query --hive-extension-config
            --hive-cloud-sensor --hive-ai-agent
        run: |
          set -euo pipefail
          for DIR in configs/customers/*/; do
            OID=$(grep -E "^# oid: " "${DIR}main.yaml" | awk '{print $3}' || true)
            [ -n "$OID" ] || { echo "no '# oid:' line in ${DIR}main.yaml" >&2; exit 1; }
            KEY=$(jq -r --arg oid "$OID" '.[$oid] // empty' <<<"$LC_CUSTOMER_KEYS")
            [ -n "$KEY" ] || { echo "no API key for $OID" >&2; exit 1; }
            LC_OID="$OID" LC_API_KEY="$KEY" limacharlie sync push \
              --config-file "${DIR}main.yaml" $SYNC_FLAGS --force $SYNC_MODE
          done

On pull_request the job adds --dry-run, so it only reports what would change; read the output before merging. On push to main it applies the change. If you rely on Git Sync's pull schedule instead, drop the push and keep CI as validation only. The job exits with an error when a customer folder has no # oid: line or no API key, so a missing OID can never turn into a push against the wrong org.

That's the whole IaC loop. The Git repo is the source of truth, your IdP gates who can merge to main, every change is reviewed, every deploy is replayable.

Part 16 — Cost guardrails and operational hygiene¶

Pay-as-you-go is great until a misconfigured adapter costs you $5,000 in a weekend. Set the guardrails on day 1, in the template, so every customer gets them automatically.

Usage Alerts is the platform's bill-runaway protection. It creates managed D&R rules that fire detections (so they hit your existing alert pipeline) when a SKU exceeds a threshold over a window. Useful presets to ship in the template:

Outbound output data > 1 GB / 30 days → fire detection.
Sensor count > N (your contracted size) → fire detection.
Insight retention > X events / 30 days → fire detection.

The IaC entry that creates one of these:

hives:
  extension_config:
    ext-usage-alerts:
      data:
        usage_alert_rules:
          - enabled: true
            limit: 1024                        # MB
            name: Output data over threshold
            sku: output_data
            timeframe: 43200                   # minutes (30 days, the max)
      usr_mtd:
        enabled: true

Sync runs every hour; first sync is manual via the Sync Usage Alert Rules button or by running it again after deploying.

Sensor Cull keeps fleets clean¶

The Sensor Cull extension automatically removes sensors that haven't checked in for N days. Without it, decommissioned hosts pile up and you keep paying their connectivity fee. Rules run once a day and live in the extension's config. Each rule has a rule_name, a ttl in days, and a tag or a selector. A sensor is culled as soon as any rule that matches it has expired, so rules must not overlap: a catch-all 30-day rule would cull VIP hosts at 30 days no matter what a longer VIP rule says. Default to 30 days for everything except VIP hosts, keep those 90, and deploy on every customer:

hives:
  extension_config:
    ext-sensor-cull:
      data:
        rules:
          - rule_name: stale-30d
            selector: '"vip" not in tags'
            ttl: 30
          - rule_name: vip-stale-90d
            tag: vip
            ttl: 90
      usr_mtd:
        enabled: true

You can also run the rules on demand with the extension's run action.

Audit trail review on a cadence¶

Once a quarter, sweep the audit log of every customer org and the activity log of every staff group: who added users, who changed permissions, who modified rules, who rotated secrets. The CLI commands are in Verifying and Reviewing Access. Wrap them in a script that posts a summary to a Slack channel; review it the same week each quarter.

Sleeper everything you're not actively using¶

If a customer's onboarding has stalled, tag their installed sensors lc:sleeper. They cost $0.10 per 30 days each instead of full price. Wake them up the moment the customer's IT readiness is fixed.

Replay before you trust a new rule in production¶

The Replay service lets you rerun a rule against the past N days of telemetry from an org. Always replay a new rule against at least 7 days of customer data before declaring it production-ready. Replay is cheap; tuning a noisy rule in production is expensive.

Output billing — pay attention to volume¶

Output billing is per GB transferred. Heavy SIEM forwards on the event stream of a 500-sensor customer add up. For each customer, decide:

Is event going off-platform? If yes, allowlist hard.
Is audit going off-platform? Yes — always, to immutable storage. Audit volume is tiny, cost is irrelevant.
Is detect going off-platform? Usually, to the customer's SIEM or your own SOAR. Volume is low.
Is deployment going off-platform? Usually no — read it from the platform on demand.

Feedback loop on detection quality¶

Every closed case carries a classification (true_positive / false_positive). Sort detection rules by FP rate quarterly. Anything above ~70% FP rate gets tuned or retired — it's costing your analysts attention they should be spending elsewhere. The Cases reporting endpoint surfaces this:

limacharlie case report \
  --from "$(date -d '30 days ago' -Iseconds)" \
  --to "$(date -Iseconds)"

Part 17 — End-to-end smoke test¶

Before declaring a customer onboarding "done", run this 15-minute end-to-end smoke test. Each step should produce a visible, expected result.

1. Sensor connectivity¶

Install the EDR sensor on a single test host (the customer's IT contact's own laptop is ideal — they want it tested anyway).
In the web app: Sensors → should show Online within ~30 seconds of install.
Click Live Feed; you should see process / network / DNS events streaming.

2. Telemetry from each adapter¶

For each connected adapter, run the test from the table in Part 5's "Day-1 telemetry smoke test" and verify the event reaches the platform.

3. A test detection fires¶

On the test host, run:

# Windows
cmd.exe /c whoami /all

# Linux
sudo cat /etc/passwd

# macOS
id

Expected: a detection from a baseline rule (e.g., the Sigma whoami rule) appears in Detections within ~10 seconds.

4. A case is created¶

If the Cases extension is subscribed in all mode, the detection above should create a case. Open Cases in the web app and find it.

5. The alert is delivered¶

If you wired Slack output on the detect stream, the detection should appear in the configured Slack channel within ~30 seconds. If you wired PagerDuty for high-severity, fire a high-severity test by tagging the test detection rule with level: high and re-running the trigger.

6. The output works¶

Check the destination — the S3 bucket has new files, the SIEM has new events, the BigQuery table is populating.

7. The AI agent runs (if deployed)¶

Trigger another detection. Watch the AI Sessions view (or limacharlie ai session list) for a new session. Open it, watch the live transcript. After a minute or two, check the case — you should see notes from the agent, status changed to in_progress or closed, classification set, IOCs attached as entities.

8. IaC export round-trip¶

limacharlie sync pull --oid <customer_oid> --config-file /tmp/customer.yaml $SYNC_FLAGS
limacharlie sync push --oid <customer_oid> --config-file /tmp/customer.yaml \
  $SYNC_FLAGS --force --dry-run

Expected: the dry run reports no additions, modifications, or removals (the customer's live config matches the YAML you just exported). If it lists changes, you have drift. Find out why before declaring this done.

9. Audit log forwarding works¶

Make any visible change in the org (add a tag to the test sensor). The change should appear in audit stream within seconds, and within minutes in your audit-archive S3 bucket.

10. Access verification¶

Run the Part 10 access verification script. Confirm the user list matches what you intended.

If all 10 pass, the customer is genuinely production-ready. If any fail, fix before going live.

Part 18 — Onboarding playbook (the one-page version)¶

Print this. Put it in your runbook.

NEW CUSTOMER ONBOARDING — TARGET: <60 MINUTES

Pre-work (you, before kickoff):
  [ ] Confirm region (US/EU/etc.) with customer
  [ ] Confirm telemetry sources they want connected
  [ ] Confirm contact emails for direct access
  [ ] Confirm SIEM/storage destinations (if any)
  [ ] Customer assigns an IT contact for installs

The org:
  [ ] Create customer org in target region
  [ ] Run onboarding script (Part 10)
    - Adds to staff Organization Groups
    - Subscribes standard extension kit
    - Pushes IaC baseline (--force --dry-run, then real)
    - Creates per-tier installation keys
    - Invites customer's primary contact
  [ ] Configure Git Sync against repo path configs/customers/<name>/
  [ ] Verify Strict SSO enabled on customer's domain (if applicable)

Telemetry:
  [ ] Hand customer the install commands + key strings
    - Windows PowerShell snippet
    - Linux .deb + debconf one-liner
    - Mac shell + reminder about Full Disk Access
  [ ] Set up M365 / Workspace adapter (if applicable)
  [ ] Set up identity adapter — Okta / Entra / Duo
  [ ] Set up cloud control plane — CloudTrail/Event Hub/Pub-Sub
  [ ] Verify first events arrive on each connected source (Part 5 smoke test)

Detection:
  [ ] Sigma ruleset subscribed
  [ ] Soteria EDR ruleset subscribed (and AWS/M365 if applicable)
  [ ] Baseline custom rules applied via IaC
  [ ] Replay all baseline rules over last 24h, review noise

Alerting:
  [ ] Cases enabled, severity & SLA mapped per contract
  [ ] Slack/Teams channel webhook configured
  [ ] PagerDuty (if SLA tier requires paging)
  [ ] Audit stream forwarded to immutable storage

AI (recommended):
  [ ] Anthropic key + LC API keys stored in secret hive
  [ ] Triage agent deployed (lean-soc/triage)
  [ ] Optional: investigator + responder + reporter

Hand-off:
  [ ] 10-step end-to-end smoke test (Part 17) all green
  [ ] Walkthrough call with customer (sensor view, detections, cases)
  [ ] Internal sign-off — engineers + analysts reviewed posture
  [ ] Customer-facing runbook delivered (sensor install, what to expect)

If your team can't run this list in under an hour by month two, you're missing automation. Invest there.

Part 19 — A maturity ladder for the first year¶

A pragmatic path, not a marketing taxonomy.

Month 1 — Visibility. EDR + identity + M365/Workspace ingested for one or two friendly customers. Sigma + Soteria EDR running. Cases on. Slack alerts. IaC repo exists, single org committed.

Month 2 — Repeatability. Onboarding script. Customer-org template. Git Sync configured both directions on every org. Staff groups in place. Strict SSO enforced. Audit forwarded.

Month 3 — Automation. Triage agent live on every customer. Lean-SOC investigator on the highest-tier customers. False-positive rules per customer. Output allowlists deployed. Usage alerts deployed. Lookup Manager refreshing public threat feeds. End-to-end smoke test scripted for new onboardings.

Months 4–6 — Differentiation. Custom detections specific to your customer verticals (industry-specific, e.g. healthcare-MFA-bypass patterns, fintech transaction-anomaly). Per-customer AI agent specialization. Threat-feed-driven rules from your own intel. Tiered-SOC if your analyst headcount supports it. Atomic Red Team running monthly. SOPs documented per customer for the AI agents to follow.

Months 7–12 — Scale and hardening. Customer-facing dashboards (BigQuery + Looker Studio, or a custom portal driven by Cases WebSockets). Multi-region presence if your customer geography requires it. Dedicated engineers-prod group with CI-only writes; humans only break-glass. Quarterly access reviews automated. Sleeper-mode IR offering productized. SOC-as-a-Service offering bundled (lean-soc + your own ops on top).

You'll deviate. That's fine. The point is to have a direction.

Part 20 — Suggested next steps¶

What to read next, in priority order, when you finish this article:

Designing Access — the operational details of staff/customer access at scale.
Cases — the case lifecycle, SLA mechanics, classification, reporting.
D&R Rule Building Guidebook — patterns and traps when writing rules that will live in production for years.
LCQL Examples — a dense reference of working hunts. Skim to learn the language by analogy.
Git Sync + Infrastructure — the IaC details glossed over here.
AI Sessions: D&R-Driven — once you have alert volume worth triaging.
Response Actions reference + Endpoint Commands — the full respond: action grammar and the full sensor command catalog.
Permissions reference — when you outgrow the predefined roles.
lc-ai repo — pre-built AI agents and the lc-essentials Claude Code plugin.
mssp-demo repo — a reference IaC repo configured exactly the way this walkthrough describes.

What to do next, in priority order, in your control org:

Bootstrap a Git repo from your control org with limacharlie sync pull.
Onboard a real friendly customer end-to-end with the playbook above. Time it. Improve the slow steps.
Wire audit stream forwarding on every org. Stop waiting until you need it.
Deploy the lean-soc triage agent. Get one full week of automated triage data before deciding whether to deploy the rest of the team.
Run the access-review checklist from Verifying and Reviewing Access once. Then schedule it.
Schedule a monthly Atomic Red Team run against a non-prod sensor. Treat each missing detection as a backlog item.
Pick one customer-vertical-specific detection to build per quarter. Build expertise; differentiate.

Welcome to running an MSSP on a public-cloud security platform. The leverage is real — most of what historically took an MSSP weeks now takes hours — but the leverage cuts both ways. Set the guardrails, codify the configs, automate the onboarding, and the platform pays you back every month.