Proving Coverage With an Agentic SOC
Have you realized yet what changes when you move from traditional SOC tools—where you build and own the content—to an agentic SOC?
Traditional detection content is portable. SIEM rules, signatures, YARA, regex, playbooks: even when platforms change, you still own the logic and can usually transform it for the next system.
With an agentic SOC, that starts to disappear.
The detection capability may live in a model, its instructions, the telemetry it can see, the tools it can call, and the way its reasoning is orchestrated. There may not be a static rule you can point to and say, “this is how we detect that.”
For now, I’m only talking about detection. Response is probably its own topic.
From Static Rules to Continuous Validation
When transitioning to an agentic SOC, the fundamental shift is from static, deterministic detection content to continuous evaluation.
Auditors and regulators do not actually require regex strings. They require evidence that controls work: proof of detection efficacy, operational boundaries, and decision trails.
So how do you prove coverage when your detection engine becomes probabilistic?
Around two years ago, I proposed following a test-driven-development workflow for security content:
Create the problem, then create the content that detects the problem.
My team even built a system around that idea: an automated, threat-intelligence-driven attack simulation program intended to drive continuously validated human- and automatically-generated security content.
If you have seen my other SOCFU posts, this is also how I test the agentic SOC. I plant known-bad behavior or configuration, hand the evidence to the agents blind, and measure what they actually find.
The more I work with agentic security, the more important that pattern looks.
What Coverage Might Look Like
Coverage in an agentic SOC is likely a function of four things:
Can it see it → Can it investigate it → Can it catch it → Can you prove what changed and when
-
Telemetry ingestion — Can it see it?
Does the agent have visibility into the necessary raw telemetry: EDR, network, mail, identity, cloud, or other relevant data sources? -
Tooling access — Can it investigate it?
Does the agent have the diagnostic tools it needs—API callers, threat-intelligence lookups, sandbox analyzers, configuration readers—to inspect what it sees? Telemetry plus tooling creates the prerequisites for coverage. It does not prove the coverage exists. -
Detection validation — Can it catch it?
Continuously run synthetic threat replays, seeded bad configurations, or controlled attack simulations and measure the pass rate. This is where the claimed capability becomes demonstrable. -
Change control — Can you prove what changed and when?
Keep the simulation content, seeded events, expected outcomes, and evaluation results version-controlled. If the model, prompt, orchestration, or platform changes, rerun the same tests.
That gives you something durable even when the underlying detection implementation is not. You can change models. You can change vendors. You can change orchestration layers. The validation corpus remains yours, and you can run it against whatever comes next.