How do you measure an agentic SOC?
You measure an agentic SOC on four things: speed, accuracy, coverage and analyst time. Speed alone is misleading, because an agent that closes alerts quickly but wrongly makes the SOC worse. The useful question is whether agents help the team find and stop real threats faster without missing more of them.
Speed metrics
Speed metrics show how quickly the operation moves from signal to action. The standard set is often called MTTx:
- Mean time to detect (MTTD): from the start of malicious activity to the first alert.
- Mean time to acknowledge (MTTA): from alert to first human or agent action.
- Mean time to respond or contain (MTTR, MTTC): from alert to containment.
Agents usually move MTTA and MTTR the most, since they remove queue time. Report medians and the slowest 10 percent alongside averages, because a few long incidents can hide improvement.
Accuracy metrics
Accuracy metrics show whether agent decisions can be trusted.
- Verdict agreement: how often analysts agree with the agent's verdict on sampled cases.
- Missed true positives: real threats the agent closed as benign, found through sampling or later detection. This is the most important number on the page.
- Escalation precision: share of agent escalations that turn out to need human attention.
- Action reversals: how often automated actions had to be undone.
Coverage and capacity metrics
- Share of alerts handled by agents, by alert type and autonomy level.
- Share of alerts with full context when they reached a person.
- Detection coverage mapped to MITRE ATT&CK, since time saved on triage should turn into better detections.
- Analyst hours returned and what they were spent on: hunting, detection engineering, complex cases.
What to measure at each autonomy level
Level | Key metric | Evidence to move up |
|---|
1. Assisted triage | Verdict agreement on suggestions | High, stable agreement for the alert type |
2. Autonomous triage | Missed true positives in sampled closures | Very low miss rate over a sustained period |
3. Supervised response | Approval rate without changes | Approvals almost always granted unchanged |
4. Bounded autonomy | Action reversals and review findings | Reversals rare; guardrails held in real incidents |
5. Autonomous operations | Outcome metrics: MTTR, incident impact | Ongoing audit, not a one-time gate |
See agentic SOC maturity for what each level means.
Common measurement mistakes
- Counting closures as success. Closing more alerts is only good if they were benign.
- No baseline. Measure before turning agents on, or you cannot show what changed.
- Stopping sampling. Review a share of agent decisions permanently, not only during rollout.
- Averages only. Use medians and percentiles too.
Frequently asked questions
What is the single most important agentic SOC metric?
Missed true positives: real threats an agent closed as benign. Speed gains mean little if this number rises.
How large should review samples be?
Large enough to catch rare misses for each alert type. Many teams start with a high sampling rate during rollout and lower it as evidence builds, but never to zero.
How do we report agent impact to executives?
Pair one speed metric, such as median time to contain, with one accuracy metric, such as missed true positive rate, and one capacity metric, such as analyst hours redirected to hunting.
Anomali and Agentic SOC Metrics
Anomali records each agent action and its rationale, which gives teams the data they need to track agent accuracy and speed over time.
Request a demo to see how Anomali reports on agent performance.
Read next