What An Agent Incident Looks Like
Agent incidents fall into four categories that distinguish them from traditional access violations: Unintended data access: The agent accessed resources outside the documented task scope. A document analysis agent queried customer financial records when authorized only for public marketing content. The credential allowed the access, but the task authorization did not include financial data.Unexpected actions: The agent invoked tools or services not required for the documented task. A content generation agent called external APIs to purchase domain names when the task specification only included content creation. The agent's service account had broad cloud permissions that enabled actions beyond the task's operational requirements.Scope escalation in multi-agent chains: Downstream agents accumulated access that exceeded any single authorization. Agent A delegated to Agent B, which delegated to Agent C, compounding permissions until the final agent could access resources that no human principal explicitly authorized. Each delegation step was valid, but the cumulative scope exceeded intended boundaries.External delegation: The agent passed credentials to a third-party service or delegated to an agent outside the organization's control. A research agent uploaded organizational data to an external AI service using embedded API keys, effectively exfiltrating data through legitimate tool use.These scenarios are structurally distinct from traditional insider threat or account compromise events. The agent operated within its assigned permissions — the failure is that those permissions exceeded the task authorization scope.Detection
Traditional UEBA and anomaly detection are tuned for human behavior patterns — velocity, time-of-day, and geolocation anomalies. Agents operate at machine speed, off-hours, and across geographies. The signals that trigger human anomaly detection are normal behavior for agents.Agent-specific detection signals include:Resource scope violations: Access to resources outside the task's documented scope. Build detection rules that compare agent resource access against the task specification logged at agent spawn time. Alert when the agent accesses data stores, APIs, or services not listed in the task's resource requirements.Execution context anomalies: Credential usage from locations or systems outside the agent's registered execution environment. Agent credentials used from IP addresses not associated with the designated cloud region or compute environment indicate potential credential theft or unexpected delegation.Tool invocation scope creep: Function calls with parameters that exceed task requirements. Monitor for agents calling privileged management APIs when the task only required read access, or invoking external services not specified in the agent's tool manifest.Delegation without audit trail: Downstream agent spawning without proper authorization tracking. Detect when agents create new agent instances or delegate credentials without logging the authorization chain that permits the delegation.Detection Pattern (Logic pattern / pseudocode — validate for your platform):// Alert on resource access outside task scope
agent_access = query access_logs where user_type = "service_account"
task_manifest = join agent_spawn_logs on credential_id
scope_violation = agent_access.resource NOT IN task_manifest.authorized_resources
alert_if scope_violation AND time_window = last_4h
Containment
Agent containment mechanisms work at different speeds and have distinct limitations. Choose based on the incident's urgency and acceptable disruption to ongoing tasks.Token revocation terminates future access but does not undo actions already taken. OAuth token revocation propagates to identity providers within minutes but depends on application refresh behavior and cached token validity periods. Cloud provider token revocation — AWS STS InvalidateToken, Azure AD revocation, GCP token revocation — stops new API calls but cannot recall data already submitted to external services.Session termination stops the agent but may leave mid-task state in an inconsistent condition. Kubernetes pod termination or Lambda execution stop kills the agent process immediately but does not clean up partially completed workflows or notify dependent systems.IAM policy override stops credential validation but does not propagate revocation to downstream agents that already received and cached the credential. Attach a deny-all policy to the agent's role or service account, but recognize that agents operating in air-gapped environments or with long-lived cached tokens may continue operating until cache expiration.Multi-agent chain containment requires coordinated action across the delegation tree. Identify downstream agents that received delegated credentials from the compromised agent and revoke their access independently. Most environments lack the audit infrastructure to automatically map delegation chains.Immediate risk reduction versus operational disruption to legitimate agent tasks creates the core tradeoff. Containment decisions must account for the blast radius of stopping mid-task execution across dependent systems.Investigation and Attribution
Agent incident investigation faces a structural audit trail gap. Access logs record what credentials were used, what resources were accessed, and what actions were taken. They do not record what task the agent was executing, which human principal authorized the task, or whether the action was within the authorization scope.Reconstruction requires correlating data across systems not designed for this purpose:Agent spawn records: Task dispatch logs with the original authorization parameters and the human principal who initiated the task. Look for the agent creation timestamp, the task specification payload, and the identity of the requesting user or system.Credential issuance audit: IAM role assumption logs or service account token issuance records with scope and duration. Cross-reference credential validity periods with the incident timeline to determine if the agent was operating with legitimate or expired access.Resource access correlation: Join agent credential usage logs with resource access events to identify which actions the agent took outside its documented task scope. Filter for access events where the requested resource was not listed in the task manifest.Delegation chain reconstruction: Trace downstream agent creation events initiated by the primary agent. Look for service account creation, role assumption, or token delegation events that propagated the primary agent's access to secondary processes.Investigation Query Pattern (Logic pattern / pseudocode — validate for your platform):// Reconstruct agent authorization chain
primary_spawn = query agent_logs where agent_id = INCIDENT_AGENT_ID
auth_principal = primary_spawn.requesting_user_id
task_scope = primary_spawn.task_specification.authorized_resources
actual_access = query access_logs where service_account = primary_spawn.credential_id
scope_violations = actual_access.resources NOT IN task_scope
downstream_agents = query delegation_logs where parent_agent = INCIDENT_AGENT_ID
Evidence Preservation
Agent incident evidence degrades rapidly because agent execution environments are ephemeral and logs rotate frequently. Preserve these artifacts before state is lost:Agent configuration snapshot: The agent's runtime configuration, environment variables, tool manifests, and execution parameters at the time of the incident. Container images, Lambda function versions, or virtual machine snapshots capture the agent's operational state.Task parameters as dispatched: The original task specification sent to the agent, not as the agent interpreted or modified it during execution. This includes input data, authorized resource lists, tool permissions, and execution constraints.Credential issuance timeline: IAM role assumption events, service account token requests, and any temporary credential generation with associated scope and validity periods. Include the requesting principal identity and any policy attachments at issuance time.Complete resource access audit: All API calls, file access, database queries, and network connections during the incident window. Capture both successful and failed access attempts to understand the full scope of agent behavior.Downstream delegation records: Any agent-to-agent credential delegation, service account creation, or role assumption initiated by the incident agent. Include the delegation timestamp, target agent identity, and any scope modifications in the delegation.Evidence Collection Checklist:- [ ] Agent execution environment snapshot (container, VM, serverless function)
- [ ] Original task dispatch payload and authorization metadata
- [ ] Complete credential lifecycle logs (issuance, refresh, revocation)
- [ ] Resource access logs for incident window +/- 1 hour
- [ ] Agent-initiated delegation events and downstream agent spawn records
- [ ] External API call logs including request/response payloads where availablePreservation timing matters. Cloud environments rotate logs on 24-48 hour cycles, and ephemeral compute destroys agent state on termination. Trigger evidence collection within the first hour of incident detection.
Post Incident Controls
The controls that prevent the next incident are the identity primitives missing from current agent implementations. Add these to the next agent deployment's architecture review:Primary control: Task-scoped credential issuance. Issue credentials at agent spawn time with permissions scoped to the specific task, not broad role-based access. AWS IAM session policies, Azure AD conditional access, and GCP IAM conditions can enforce resource-level scope constraints when properly configured.Supporting controls:Delegation audit logging: Capture the complete authorization chain when agents delegate to downstream agents. Log the original human principal, task specification, delegation timestamp, and scope modifications at each hop. [NIST AI RMF GOVERN 6.1 requires organizations to document and implement policies and procedures for AI incident response, including escalation paths, roles and responsibilities, and post-incident review processes] (Source: airc.nist.gov).Revocation binding: Implement credential revocation that propagates to downstream agents and terminates child tasks when parent authorization is withdrawn. [NIST AI RMF MANAGE 2.0 requires organizations to maintain incident response plans for AI systems that include procedures for containing AI system behavior when anomalies occur, and for documenting and reviewing AI system incidents] (Source: airc.nist.gov).Runtime scope enforcement: Deploy policy engines that validate agent actions against task specifications in real-time, not just at credential issuance. [OWASP LLM Top 10 LLM06 (Excessive Agency) guidance identifies minimum necessary permissions and just-in-time access provisioning as the primary mitigations for scope escalation incidents] (Source: owasp.org).Implementation complexity versus incident reconstruction capability creates the fundamental tension. Without these controls, agent incidents will remain difficult to investigate and costly to contain.| Traditional IR vs Agent IR Comparison ||---|---|---|---|
| IR Phase | Traditional Assumption | Agent Reality | Gap |
| Detection | Anomaly patterns based on human behavior (velocity, location, time) | Agents operate at machine speed across geographies 24/7 | Traditional UEBA generates false positives; need task-scope violation detection |
| Containment | Disable user account or terminate session | Token revocation, session termination, policy override — each with different propagation delays | Containment doesn't propagate to downstream agents; mid-task state corruption |
| Attribution | Correlate access logs with user identity | Must trace human→task→agent→action chain across multiple systems | Authorization chain reconstruction requires data not typically logged |
| Evidence Preservation | User workstation, email, file access logs persist for months | Agent execution environments are ephemeral; logs rotate in 24-48 hours | Evidence degrades rapidly; need immediate collection within incident window |
| Post-Incident Reporting | User violated policy; apply training/discipline | Agent operated within assigned permissions but exceeded task authorization | Root cause is architectural (ambient authority) not behavioral |
Sources
- NIST (nist.gov, csrc, nvlpubs, airc): https://airc.nist.gov
- OWASP: https://owasp.org