Incident Response

The Gap in Incident Response Tooling No One Talks About

Ryan Zahrai 6 min read
Abstract concept of a gap or missing connection in a workflow chain

If you map the incident response tooling market against what actually happens during an incident, a pattern becomes visible. The tools cluster at two ends of the timeline: detection and post-mortem analysis. The space in between, the actual execution of a response, is left largely to the on-call engineer and a Confluence page.

This is not an accident. It reflects how the market matured. Detection technology had to improve before response was even worth optimizing. Post-mortem tooling emerged to address the organizational learning problem after incidents. Execution was assumed to be a human problem that documentation could solve.

Documentation did not solve it. This post is about why, and what the gap looks like in practice.

The Three Tool Categories and What Each One Does

The incident response tooling stack for most teams in 2025 looks something like this.

Detection and monitoring: SIEM platforms like Splunk, detection rules in Datadog or Elastic, EDR agents on endpoints. These tools watch, correlate, and fire alerts when behavior matches a rule. They are largely passive. They observe and report. They do not act.

Alerting and on-call routing: PagerDuty, Opsgenie, and similar platforms receive the alert signal and route it to the right human at the right time. They manage escalation, acknowledge state, and incident duration. They are excellent at getting a human's attention. What they do not do is tell that human what to do, let alone do any of it.

Post-mortem and incident management: Tools like Jira for tracking, Confluence or Notion for runbook documentation, Blameless or PagerDuty Post-Mortem for structured analysis after the fact. These tools improve learning and tracking. They operate after the incident is contained.

What all three categories have in common: they do not touch the execution phase. The 10 to 25 minutes between an alert firing and a human engineer being ready to make diagnostic decisions. The phase where blast radius expands if it is not interrupted, and where response quality varies most based on who happens to be on-call that night.

Why Documentation Was the Default Answer

The field's answer to the execution gap has historically been documentation. Write detailed runbooks. Train engineers on them. Review and update them after post-mortems. This is good practice. We are not arguing against it. But runbook documentation has a fundamental limitation that is not addressable through better documentation: it requires a human to read, interpret, and execute it while under stress, at whatever hour the alert fired, with whatever cognitive state that engineer happens to be in.

Reading a runbook step that says "isolate the affected EC2 instance" and then executing that correctly requires the engineer to: remember or look up the isolation procedure, authenticate to the AWS console, identify the correct instance from the alert details, apply the correct security group or network ACL change, and verify the change took effect. Under normal conditions, an experienced engineer does this in 4 to 6 minutes. At 3am, having just woken up, potentially with an alert that is ambiguous about which instance, this takes longer and introduces more room for error.

Documentation is a forcing function for the human. It does not reduce the number of steps the human has to execute. It just makes sure they know what the steps are supposed to be.

The Execution Gap in Practice

Consider a common scenario: a Datadog alert fires for anomalous network traffic from an EC2 instance in a staging environment. The alert is categorized as high severity by the detection rule. The on-call engineer gets paged at 2:17am.

At 2:17am: engineer acknowledges the alert.
At 2:19am: engineer opens the runbook for EC2 network anomaly.
At 2:21am: engineer authenticates to AWS console (MFA required).
At 2:24am: engineer identifies the correct instance from the alert metadata.
At 2:26am: engineer applies isolation security group.
At 2:28am: engineer begins pulling CloudTrail logs to understand what happened before the alert.
At 2:38am: engineer has enough context to form a hypothesis about what occurred.

In this scenario, the nine minutes between 2:17am and 2:26am produced zero diagnostic value. They were pure execution of steps that had nothing to do with the security problem being investigated. And for those nine minutes, the compromised instance had active network connectivity.

We are not describing an exceptional scenario. This is what normal incident response looks like for lean teams with good tooling and good runbooks. The execution phase just takes time when a human is doing it.

What Sits in the Gap

The execution gap is not a small space. It is where most of the time-critical decisions in an incident happen, or fail to happen quickly enough. Containment, evidence collection, initial notification, timeline capture. All of these have clear procedural definitions. None of them require human judgment at the step level. All of them benefit enormously from being completed before the engineer begins active investigation.

The tooling market has not addressed this gap seriously for lean teams because the tooling that exists does address it for large teams, just in an expensive and complex way. SOAR platforms (security orchestration, automation and response) like Palo Alto XSOAR, Splunk SOAR, or Swimlane automate exactly this kind of response execution. They are also priced and architected for SOCs with dedicated playbook engineers and IT infrastructure capable of integrating with them.

A four-person security team at a 300-person company is not deploying Splunk SOAR. The cost, complexity, and maintenance overhead are all wrong for the scale. So they fall back to documentation and human execution, and the gap stays open.

The Design Question

Building tooling for the execution gap specifically, at a scale and price point that lean teams can actually use, requires making a set of design choices that differ from SOAR platforms.

The integration model has to be additive, not replace-everything. Teams already have PagerDuty and Datadog. The execution layer should receive from those tools and act on their alerts, not require replacing them.

The runbook model has to be opinionated enough to run without configuration expertise, but flexible enough to match real team procedures. YAML-based runbooks with defined action types (isolate, collect, notify) rather than a general-purpose workflow engine.

Audit and transparency matter at least as much as speed. An automated response that engineers cannot easily read and verify creates a different problem: teams stop trusting what the automation did, which undermines the value of the time savings.

These are the choices we made when designing Parachute. Not because SOAR platforms are wrong, but because they are the right choices for the problem as lean teams actually experience it.