Incident response lessons from the field

Incident response playbooks are written in calm rooms. Incidents are not. Most of what I know about responding well I learned from the moments where something went slightly wrong: a step taken too early, a piece of evidence lost, a message sent to the wrong group, an assumption nobody checked.

These are some of those lessons. Every example is anonymised and generalised, with details changed or combined so that no organisation, person or case can be identified. The mistakes are real patterns, though, and I have seen most of them more than once.

1. Resetting the password is not containment

A user's account showed sign-ins from an unfamiliar location. The first response was to reset the password and close the ticket. A day later the same account was sending phishing emails internally. The attacker had never needed the password. They had a valid session token, and resetting the password did nothing to it.

What I do now: for any account compromise, revoke all active sessions and refresh tokens, review and remove any newly registered MFA methods, check for OAuth app consents granted during the window, and only then reset the password. I wrote more about why in MFA is not your last line of defense.

2. Check for persistence before declaring victory

In business email compromise cases the attacker's favourite trick is not the login. It is the mailbox rule: forward anything mentioning "invoice" to an external address, then move the original to an obscure folder so the user never sees the replies. Containment that removes the attacker's session but leaves the rule in place means the attacker keeps reading the mail.

What I do now: I keep a persistence checklist for each incident type. For email it includes inbox rules, forwarding settings, delegates, connected apps and newly created app passwords. For cloud accounts it includes new users, access keys, roles and trust policies. For endpoints it includes scheduled tasks, services, startup items and new local accounts.

3. Scope before you act, when you can

On one investigation, a compromised machine was isolated within minutes of the first alert. That felt like a win. But it also told the attacker we had noticed, and they quickly used a second foothold we had not found yet. The second foothold took far longer to locate than the first.

What I do now: if there is active harm, such as data leaving or ransomware spreading, contain immediately. If there is not, spend a short, time-boxed window scoping first: what else did this user, host or credential touch? Then contain everything at once. It is a judgement call each time, and the important part is making it on purpose rather than by reflex.

4. Preserve evidence before you clean up

A laptop with confirmed malware was reimaged the same afternoon to get the user back to work. It was the right outcome for the user. It was the wrong order for the investigation, because the memory and disk artefacts that would have shown how the malware arrived were gone.

What I do now: before any destructive remediation, capture what you will need: volatile data if the host is still running, the relevant files, and the EDR timeline. Most EDR platforms can collect this remotely in minutes. Then reimage.

5. Normalise time before building the timeline

One timeline put the data transfer before the login that made it possible. Nobody had made an error in reading the logs. One source logged in UTC, another in local time, and a third in the time zone of the server it ran on. The story looked impossible until the clocks were aligned.

What I do now: every timestamp goes into the timeline in UTC, with its original source and time zone noted. It is dull work and it prevents some of the most confusing hours in an investigation.

6. One log source is a rumour, two is evidence

In an insider-risk case, the first data suggested a departing employee had uploaded a large volume of files to personal cloud storage. It would have been easy to stop there. But a proxy log alone shows that bytes moved, not what they were or why. Correlating with endpoint file activity, the identity provider and the device record turned "probably" into a clear, defensible timeline, and it also ruled out an innocent explanation that the first source could not.

What I do now: any conclusion that could affect a person's job or a legal decision gets corroborated from at least two independent sources, and the report says which claims rest on which evidence.

7. Bring in HR and legal early on people cases

Investigations involving an employee are not only technical. Whether you can review certain data, how findings are shared, and who decides what happens next are questions for HR and legal. Bringing them in late means redoing work, or worse, having evidence questioned.

What I do now: as soon as an investigation points at an individual, the case follows the people-investigation process. Access to the case is limited, notes are factual and neutral, and opinions stay out of the report.

8. Write it down as you go

At the end of a long incident, nobody remembers exactly when a host was isolated, who approved a block, or which query found the second foothold. Reconstructing it afterwards is slow and error-prone.

What I do now: keep a running log in the ticket from the first minute: time, action, who, and why. It costs seconds per entry and saves hours when writing the report or handing over to the next shift.

9. Communicate in the right channel

Discussing an active incident in a chat channel the attacker might be able to read, from an account that might be compromised, is a classic mistake. So is telling too many people too early, which turns a contained incident into rumours.

What I do now: for serious incidents, move coordination to a separate, need-to-know channel, and assume any account involved in the incident may be watched until proven otherwise.

10. The lessons-learned meeting is not optional

The incidents that repeat are the ones nobody reviewed. A short, blameless review after each significant incident, covering what happened, what went well, what was slow and what changes, is where the real improvement comes from. The output should be a few specific actions with owners, such as a new detection, a logging fix or a playbook update, not a document nobody reads.

Frameworks like NIST SP 800-61 describe this as part of the incident lifecycle for a reason. Response that does not feed back into detection and prevention just waits for the next incident.

Frequently asked questions

What are the most common incident response mistakes? Resetting a password instead of revoking sessions, missing attacker persistence such as mailbox rules or new access keys, reimaging before preserving evidence, and building timelines from logs in mixed time zones.

Should you contain an incident immediately? If there is active harm, such as data leaving or ransomware spreading, yes. Otherwise a short, time-boxed scoping window first can stop the attacker falling back to a foothold you have not found yet.

What is a blameless post-incident review? A short meeting after an incident that looks at what happened, what went well and what was slow, without blaming individuals. It should end with a few specific actions, each with an owner.

The point

Most incident response mistakes are not about missing knowledge. They are about doing the right steps in the wrong order under pressure: acting before scoping, cleaning before preserving, resetting instead of revoking. The fix is preparation. Short checklists, a habit of writing things down and a blameless review afterwards will improve your next incident more than any new tool.