Detection-as-code: treating detection rules like software

In a lot of SOCs, detection rules live only inside the SIEM. Someone writes a rule in the web console, saves it and moves on. Months later nobody knows who changed it, why a strange exclusion was added, or whether it still catches what it was built to catch. When it stops firing, nobody notices, because silence looks exactly like safety.

Detection-as-code is the fix. It means treating detection rules the way engineers treat software: stored in version control, reviewed before they ship, tested automatically and deployed by a pipeline. None of the ideas are new. Applying them to detections is what makes the difference.

What detection-as-code actually means

It comes down to four practices:

• Version control. Every rule is a file in a Git repository. The repository, not the SIEM console, is the source of truth.
• Peer review. Changes go through pull requests, so a second person reads the logic, the exclusions and the reasoning before anything ships.
• Automated testing. Each rule has test cases: sample events it must match and sample events it must not.
• Automated deployment. A CI pipeline validates the rules, runs the tests and pushes approved changes to the SIEM. Nobody edits production rules by hand.

What a rule looks like as code

The format matters less than the habit. Some teams store rules in their SIEM's native query language with a small metadata file next to each one. Others write rules in Sigma, an open, vendor-neutral format, and convert them for the SIEM in the pipeline. Here is a simplified example in a Sigma-style layout:

title: Encoded PowerShell launched by an Office application
id: 7c1d2a4e-0000-4000-8000-000000000001
status: stable
description: Office apps rarely start PowerShell with an encoded
  command. Common in malicious document payloads.
author: SOC detection engineering
references:
  - https://attack.mitre.org/techniques/T1059/001/
tags:
  - attack.execution
  - attack.t1059.001
logsource:
  category: process_creation
  product: windows
detection:
  parent:
    ParentImage|endswith:
      - '\winword.exe'
      - '\excel.exe'
      - '\outlook.exe'
  child:
    Image|endswith: '\powershell.exe'
    CommandLine|contains:
      - ' -enc '
      - ' -encodedcommand '
  condition: parent and child
falsepositives:
  - Rare legitimate add-ins; review and document any exclusion
level: high

Notice what comes along with the logic: an ID, a description of why the behaviour is suspicious, the ATT&CK technique it maps to, known false positives and a severity. That metadata is what lets a new analyst understand the rule at 3 a.m. without asking anyone.

Organising the repository

A simple structure works well and scales further than you would expect:

• rules/, grouped by log source or ATT&CK tactic, one file per rule.
• tests/, with a folder per rule holding sample events that should and should not match.
• exclusions/, kept separate from rule logic, each with an owner, a reason and an expiry date.
• pipeline/, with the scripts that validate, convert, test and deploy.

Keeping exclusions separate is a small decision with a big payoff. It makes every suppression visible in one place, and it lets you review them on a schedule instead of finding them buried inside rule logic years later.

Testing detections

Testing is where detection-as-code earns its keep. A rule test is simple: feed the rule a set of events and check the result. Each rule should have at least:

• True positive samples, real or simulated events of the attack behaviour. The rule must match all of them.
• True negative samples, normal activity that looks similar. The rule must match none of them.
• Regression cases, added every time a false positive is fixed, so the same noise never comes back.

Sample events can come from sanitised production logs, from lab detonations or from attack simulation frameworks. Strip anything sensitive before committing it. Hostnames, usernames and internal addresses do not belong in a test fixture.

Beyond unit tests, the pipeline can run broader checks: does the query parse? Does it reference fields that actually exist in the log source? Is it fast enough to run on schedule without timing out? Does it duplicate an existing rule?

The CI pipeline

The pipeline runs on every pull request and again on merge. A typical flow looks like this:

A useful addition is a volume check on recent production data. If a change would have produced hundreds of alerts yesterday, the reviewer should know that before approving it, not after the queue fills up.

What changes for the team

• Every rule has a history. When an alert looks wrong, you can see exactly who changed what and why.
• Rollback is one click. A bad deployment is reverted like any other code change.
• Review spreads knowledge. Junior analysts learn detection logic by reviewing it, and seniors catch broad exclusions before they ship.
• Coverage becomes measurable. With ATT&CK tags on every rule, you can map what you detect and where the gaps are.
• Silent failures surface. A test that fails after a log source changes its format tells you the rule is broken before an attacker does.

Getting started without boiling the ocean

You do not need to migrate every rule on day one. A gradual path works better:

• Export your existing custom rules into a repository as they are. That alone gives you history from today.
• Agree a metadata template and require it for new and changed rules.
• Add tests to your ten most important rules first, then to any rule you touch.
• Automate deployment last, once the team trusts the repository as the source of truth.
• Lock down direct editing in the SIEM console once the pipeline is reliable, with a documented break-glass process for emergencies.

Common objections

"It slows us down." A pull request adds minutes. Debugging an untracked rule change during an incident costs hours. In practice teams ship detections faster once deployment is automated, because nobody has to copy and paste between environments.

"Our analysts aren't developers." They do not need to be. Writing a YAML file and opening a pull request is a skill that takes an afternoon to learn, and it is one of the most useful things a junior analyst can pick up.

Frequently asked questions

Do I need Sigma to do detection-as-code? No. Sigma helps if you want rules that are portable between SIEMs, but storing rules in your SIEM's own query language with a metadata file works just as well. The practices matter more than the format.

What about rules that come from a vendor? Keep them out of your repository but track which ones you have enabled, and put any local exclusions you add to them under the same review process as your own rules.

How many tests does a rule need? Start with one positive and one negative sample. Add a regression case every time the rule produces a false positive you fix. Over time the tests grow with the rule's history.

The point

Detections are code whether you treat them that way or not. They have logic, dependencies and bugs. Version control, review, tests and a deployment pipeline will not write better detections for you, but they make sure the good ones stay good, the bad changes get caught and every rule can explain itself.