Cutting L1 triage from 15 minutes to 30 seconds
In an earlier post I argued that AI will not replace L1 SOC analysts, and mentioned that first-pass triage dropped from about 15 minutes to about 30 seconds once alerts arrived pre-enriched. A lot of people asked the obvious follow-up: how, exactly? And what breaks?
This post is the long answer. It covers where the 15 minutes actually went, how the enrichment pipeline is put together, how we keep the model honest, and the specific ways it still gets things wrong. The tooling is deliberately described in generic terms. The pattern works with any SIEM, EDR, ticketing system and enterprise LLM you already have.
Where the 15 minutes went
When we timed L1 analysts on routine alerts, the investigation itself was rarely the slow part. The slow part was gathering. A typical alert meant:
- Opening the alert in the SIEM and reading the raw event.
- Pivoting to the EDR to see what the host and user were doing around that time.
- Checking every IP, domain and hash against two or three reputation services.
- Looking up the asset owner, business unit and criticality in the CMDB.
- Searching old tickets to see whether this user, host or rule had fired before.
- Writing up the findings in a ticket in a consistent format.
Each step takes a minute or two. None of them needs judgement. Together they add up to most of a quarter hour, repeated dozens of times a shift. That is the work we set out to remove.
How the pipeline works
The pipeline runs the moment an alert is created, before a person opens it. It has four stages.
The fourth stage is quiet but important: the analyst's verdict and any correction are written back to the ticket, so we can measure how often the suggestion was right.
Stage one: collect with code, not with the model
The most important design choice was to keep data collection deterministic. A SOAR platform or a small Python service runs a fixed set of lookups for each alert type: which queries to run in the SIEM, which EDR fields to pull, which reputation services to call, which CMDB attributes matter. The model does not decide what to look up for routine alerts. That keeps the evidence complete and repeatable, and it keeps cost and latency predictable.
For more open-ended questions we expose a small set of read-only tools to the model through the Model Context Protocol: search recent events for an entity, fetch a ticket, look up an asset. Every call is logged with the alert ID it was made for. Nothing the model can call changes state. Containment, blocking and account actions stay in playbooks that a person approves.
Stage two: a prompt built around evidence
The summary prompt is boring on purpose. It gives the model a role, the collected evidence as structured fields, and a fixed output format:
- What happened, in two or three sentences a manager could read.
- Timeline, built only from timestamps in the evidence.
- Entities, with their reputation and where that reputation came from.
- Suggested verdict (likely benign, needs investigation, likely malicious) with the specific evidence behind it.
- What is missing, meaning any lookup that failed or returned nothing.
Two instructions do most of the work. First, every claim must point at a field in the evidence. Second, if the evidence does not support a conclusion, the model must say "unknown" rather than guess. The "what is missing" section exists because a summary that silently skips a failed lookup looks more confident than it should.
Stage three: the analyst still decides
The analyst sees the summary at the top of the ticket, with links to the raw evidence underneath. Their job changes from gathering to checking. Does the timeline match the events? Does the verdict follow from the reasons? Is there anything the summary does not mention that bothers them?
That is where the 30 seconds comes from. It is the time to read and sanity-check a good summary on a routine alert, not the time to fully investigate a real incident. When something looks wrong, the analyst investigates as they always would, just with the context already in front of them.
What it gets wrong
This is the section I wish more vendor demos included. These are the failure modes we have actually seen.
- Confident summaries of thin evidence. When a lookup times out, an early version of the prompt would still write a tidy story. The "what is missing" section and the rule to say "unknown" fixed most of this.
- Over-trusting reputation. A clean reputation score is not proof of safety. New attacker infrastructure is often clean for its first few days. The model tended to treat "no detections" as "benign" until we told it explicitly that absence of reputation is not evidence of good behaviour.
- Timezone and ordering mistakes. Different tools log in different timezones and formats. The model occasionally produced a timeline in the wrong order. We now normalise every timestamp to UTC in code before the model sees it.
- Pattern-matching on the rule name. If a rule is called "Suspicious PowerShell", the summary leans suspicious regardless of the actual command. We now put the raw command line ahead of the rule name in the evidence.
- Hallucinated details. Rare, but real: a hostname or process that was not in the evidence. Requiring every claim to cite a field makes these easy to spot, and the analyst check catches the rest.
- Instructions hiding in the data. Email subjects, file names and user agents are attacker-controlled. Any of them can contain text that looks like an instruction to the model. That deserves its own post, but the short version is that the model never gets a tool that can act on such an instruction.
How we measure it
If you cannot show that automation helps, you are just moving the noise. We track a small set of numbers:
| Measure | Why it matters |
|---|---|
| Time to first decision | The headline number: how long an alert waits before a person makes a call. |
| Verdict agreement | How often the analyst agrees with the suggested verdict. Low agreement on a rule means the prompt or the evidence needs work. |
| Correction rate | How often the analyst edits the summary. A rising rate is an early warning. |
| Missed escalations | Alerts suggested as benign that turned out to matter. This is the one that must stay at zero. |
If you are building this
- Start with one high-volume, low-risk alert type. You will learn more from a thousand routine alerts than from ten complex ones.
- Collect in code, summarise with the model. Keep the model away from deciding what evidence exists.
- Make the model show its work. Every claim links to evidence. Unknown is an acceptable answer.
- Read-only tools only. Anything that changes state goes through a playbook with a human approval.
- Measure agreement, not just speed. Faster wrong answers are worse than slow right ones.
Frequently asked questions
What is AI-assisted alert triage? It is a pipeline that gathers the context an analyst needs for an alert, such as related events, reputation, asset owner and ticket history, and uses an LLM to summarise it before a person opens the alert. The analyst still makes the decision.
Does the LLM decide whether an alert is malicious? No. It suggests a verdict with the evidence behind it. An analyst reviews the summary and the raw evidence, then makes the call and owns the outcome.
Which alerts should you automate first? High-volume, low-risk alert types with a predictable investigation. They give you the most time back and the most data to measure accuracy before you move on to harder cases.
The point
The jump from 15 minutes to 30 seconds is not magic and it is not the model being smart. It is the result of moving the gathering work into code, giving the model a narrow job, and keeping a person on every decision. The model is the last mile of a pipeline that would be valuable even without it. That is also why, when it gets something wrong, the damage stays small.