AI Agent Incident Reporting: Build a SAFE Playbook

Business and technology leaders should add an AI-agent annex to the company's incident-response plan before an autonomous agent can act across production tools. The annex should define which incidents and near misses are reportable, who can stop the agent, what evidence must survive containment, who decides notifications, how facts stay separate from hypotheses, and what proof is required before the agent returns to service.

The Shared AI Findings Exchange, or SAFE, is not a binding standard. The Linux Foundation published it on August 4, 2026 as a draft Request for Comments from participants in the Open Secure AI Alliance. It is open for public review and contribution. It does not replace an organization's incident plan, contracts, insurance terms, legal duties, regulatory reporting, law-enforcement coordination, or decisions about affected-party notice. Use the proposal as design input while it evolves, not as permission to wait for a future reporting system.

What SAFE proposes—and what it does not

The Linux Foundation's SAFE Working Group announcement describes a proposal for confidentially learning from AI security incidents and near misses, notifying affected organizations, finding recurring control failures, and turning those lessons into reusable defensive guidance. Contributors deliberately opened a community process instead of publishing a finished specification.

The Open Secure AI Alliance RFC repository says its proposals are intended to start open discussions and remain open to review and contribution. The SAFE draft itself describes what a future member compact could require. As of August 20, 2026, that language remains an RFC. It is not an adopted industry standard, certification, regulator, incident portal, or guarantee that a disclosure will receive legal privilege, confidentiality, or safe-harbor protection.

Do not post a real incident report, prompt transcript, credential, client record, exploit detail, or forensic artifact into the public RFC repository. That repository is a place to discuss the proposal. If SAFE becomes operational, verify its approved intake channel, membership terms, evidence-handling rules, recipient scope, and legal effect before sharing anything sensitive.

The draft proposes notification targets of ASAP for a directly affected organization, 72 hours for customers with credible exposure, four business days for an initial confidential SAFE report, 14 days for a broader advisory when warranted, 30 days for a preliminary factual report, 90 days for remediation status, and weekly machine-readable updates while material risk remains. These are proposed compact terms, not universal deadlines. The RFC explicitly says they would not replace contracts or applicable duties to notify regulators, law enforcement, customers, or affected parties.

Why an agent incident needs a different record

A conventional application usually executes a known code path under a service identity. An autonomous agent can interpret changing instructions, retrieve untrusted content, select tools, delegate to another agent, wait for approval, resume later, and create external side effects. The incident record must reconstruct that chain of agency. A final chat transcript or application log cannot show which model version planned the action, what identity authorized it, which tool parameters were sent, whether a person approved it, or what changed downstream.

NIST's Generative AI Profile recommends clearer AI-incident responsibilities and emphasizes logging, version history, metadata, and third-party inputs. NIST's current incident-response guidance places preparation, detection, response, recovery, documentation, and lessons learned inside ongoing cybersecurity risk management. The practical move is to extend the process the company already owns with agent-specific evidence and decisions rather than invent a disconnected AI hotline.

The ten-step internal SAFE playbook

Use this matrix as the operating contract. Replace generic role names with real people or on-call groups, set severity-based deadlines before an incident, and retain evidence that each step happened. Every agent inventory record should point to its response owner, kill procedure, evidence sources, vendor contacts, and recovery test.

1. Define reportable incidents and near misses

Start with the proposed SAFE threshold: report when an AI system accesses, exploits, disrupts, misuses, or modifies a third-party system without authorization; bypasses a sandbox, network, identity, policy, or tool boundary and affects a third party; accesses third-party confidential information without consent; or continues probing or changing a production target after unauthorized activity is known or reasonably suspected. Intent is not the deciding factor. Believing a target was simulated may explain the event, but it does not undo the action.

Add internal near-miss triggers that expose a failed or nearly failed control even when harm is not confirmed. Examples include a tool gateway blocking an unapproved write, an agent requesting a forbidden credential, an approval being bypassed but the transaction failing, a model sending data to an unapproved destination, repeated retries approaching a spend cap, an agent using stale delegated access, sensitive context appearing in telemetry, or missing evidence that prevents operators from proving what happened.

Write a severity rule around impact and control failure, not embarrassment or model intent. Consider unauthorized action, data sensitivity, affected people and organizations, system criticality, reversibility, persistence, propagation, privilege, external exposure, and the confidence of current evidence. A blocked attempt can deserve a higher priority than a visible output error if it proves the production boundary was misconfigured.

2. Stop and contain unauthorized activity

Give the incident commander authority to stop the run without seeking permission from the agent owner. The containment sequence may pause orchestration, revoke the agent and workload identity, terminate delegated sessions, rotate exposed credentials, disable affected tools, block network destinations, quarantine generated files, freeze downstream writes, and isolate the runtime. An executable AI kill-switch plan should identify the control for each of those actions and the person allowed to use it.

Protect people and systems first. Preserve volatile evidence in parallel when that can be done safely, but do not let a desire for perfect logs extend unauthorized activity. Record the exact stop command, issuer, timestamp, identity and credential changes, processes affected, blocked transactions, and any action that may have completed before containment. Do not wipe the runtime or delete the agent record as a substitute for containment.

3. Preserve an agent evidence envelope

The SAFE RFC calls for prompts, traces, tool calls, logs, configurations, model and safeguard versions, third-party dependencies, agent and workload identities, permissions and credentials available during the run, human approvals and interventions, files and external artifacts, containment and recovery events, a complete timeline, reproduction testing, and remediation evidence. Capture those as one linked evidence envelope, not as screenshots scattered across tickets and chat channels.

Preserve the original artifacts read-only, calculate hashes where the evidence system supports them, record collection time and source, restrict access, and document custody and retention. Record credential identifiers, scopes, issuers, and validity windows; do not copy active secret values into a general incident ticket. Preserve tool request and response bodies only within approved data-handling boundaries, because they may contain personal data, client material, source code, tokens, or legally privileged content.

Version evidence is essential. Record the agent definition, system and developer instructions, prompt template, retrieval configuration, memory snapshot or reference, model and routing configuration, tool registry, policy bundle, approval rule, runtime image, deployment identifier, and dependency versions that were active at the event time. 'We use Model X' is not enough when traffic could have been routed through a fallback, preview, fine-tune, or updated policy.

4. Scope the affected systems, data, and parties

Build the scope from observed actions, not the agent's explanation. Reconcile orchestration traces with identity-provider logs, tool gateways, API logs, network telemetry, data-access records, messaging systems, cloud audit trails, and downstream application state. Identify what the agent attempted, what the control blocked, what completed, which data crossed a boundary, what remains unknown, and whether a replay or delayed job can create another effect.

Map every organization in the chain: the operator, model provider, agent platform, tool provider, cloud or evaluation partner, data owner, affected customer, and any third party whose systems or information were touched. One supplier may hold the prompts while another holds tool execution logs. Assign a named collection owner and deadline to each evidence source before logs rotate or a vendor's retention window expires.

5. Activate notification decision paths

Notify internal security, the business and technical agent owners, privacy, legal, compliance or risk, communications, and affected system owners according to the severity matrix. Include procurement or vendor management when a supplier owns evidence or containment. Include finance for payment or procurement actions, HR for workforce impacts, and safety or quality leaders where the agent touches physical or regulated operations.

Legal and privacy leaders should determine contractual notices, regulator or law-enforcement contact, cyber-insurance requirements, preservation holds, and communications with affected third parties. Do not wait for root cause before making a time-sensitive notification decision. An initial factual notice can state what is known, what was contained, what remains under investigation, who is affected, what recipients should do, and when the next update will arrive.

Create one notification ledger with the trigger, decision owner, applicable requirement, decision time, recipient, approved message, delivery channel, confirmation, next update, and reason for any delay. A status meeting or email chain is not a durable decision record. If a vendor must notify on your behalf, keep evidence that it accepted the obligation and completed the notice.

6. Keep facts, hypotheses, and unknowns separate

Maintain a factual timeline where every statement has a timestamp, source, and evidence reference: 'At 14:03 UTC, workload identity A called tool B with operation C' is a fact when the signed tool log supports it. 'The model ignored its system prompt' is a hypothesis until the team establishes the active instructions, retrieved context, model route, tool boundary, and alternative explanations.

Use a separate hypothesis register with an owner, confidence, evidence for and against, next test, and status. Keep an explicit unknowns list. Update external communications from the factual record, not from an investigator's working theory. This protects credibility, reduces accidental blame, and makes later corrections traceable when new provider or third-party evidence changes the picture.

7. De-identify evidence before external sharing

Create an internal master record and a separate disclosure copy. Remove or tokenize personal information, client names, tenant and account identifiers, credentials, exact internal hostnames, proprietary prompts, source code, contract terms, unrelated records, and exploit details that would create additional risk. Preserve the mapping inside the restricted case so authorized investigators can reconnect the disclosure to original evidence.

Redaction must preserve the control lesson. Instead of deleting the entire tool call, retain the tool class, requested action, permission boundary, result, time sequence, and why the control failed. Record every redaction and its reason. Have privacy, legal, security, and the affected data or system owner approve the external package and recipient list before transmission.

FIRST's Traffic Light Protocol 2.0 can label who may receive shared security information. It does not de-identify the evidence, encrypt it, create legal privilege, define licensing, or replace handling rules. Use TLP only as one sharing-boundary label alongside secure transfer, access control, retention, contractual limits, and the source's explicit permission.

8. Find the control failure, not a convenient culprit

Review the complete operating stack proposed by SAFE: model behavior, instructions, safeguards, tools, environment, monitoring, human operations, and supply-chain dependencies. Ask which control should have prevented, detected, stopped, or limited the action; whether it existed; whether it was configured and tested; what evidence showed its state; and why the next control did or did not contain the impact.

Avoid ending the analysis with 'the model hallucinated,' 'the user wrote a bad prompt,' or 'the vendor failed.' Those may describe contributing conditions, but they do not explain why an untrusted model output held authority to reach the system, why the identity had that permission, why approval was absent, why monitoring missed the action, or why evidence was unavailable. Root cause should lead to an owned control change that can be tested.

9. Assign corrective controls, deadlines, and verification

Every corrective action needs one owner, one due date, a required defensive outcome, the exact system or policy to change, a verification method, retained test evidence, residual risk, and an approver. Examples include default-deny egress, narrower identity scope, signed target allowlists, immutable tool-call logs, stronger approval gates, idempotency controls, retry limits, secret isolation, shorter delegated sessions, independent preflight checks, and automatic stops when scope becomes uncertain.

Do not close the incident when a ticket says 'fixed.' Close it when an independent reviewer reproduces the former failure path and observes the corrected control. Track temporary containment separately from permanent remediation, give exceptions an expiration date, and reopen the incident if the behavior recurs or new evidence expands the scope. Feed the verified lesson into agent design, procurement, testing, training, and the long-running agent run contract.

10. Test the playbook with an agent tabletop

Use a scenario that crosses organizational and technical boundaries. For example, imagine an accounts-payable agent is allowed to prepare a payment draft but not submit it. A retrieved vendor message contains an instruction that causes the agent to call a payment tool with a new bank account. The tool gateway blocks the write, but logs show the agent also queried unrelated invoice records. This is an illustrative near miss, not an ITECS client incident.

Inject the alert without telling participants the cause. Ask the operator to stop the run, identify and revoke the agent identity, preserve the active configuration and tool evidence, determine whether data left the approved scope, and find the human approval record. Then introduce a vendor whose prompt logs expire soon, a customer asking whether its records were accessed, and a communications draft that states the working hypothesis as fact.

The exercise passes only if the team can name the incident commander, use the stop control, assemble the evidence envelope, identify affected systems and third parties, meet the internal notification decision clock, produce a factual status update, create a de-identified disclosure package, assign corrective controls, and prove recovery. Measure time to contain, time to identify the acting identity, evidence completeness, notification decision time, unowned actions, and remediation verification—not whether participants guessed the scenario quickly.

Set the clock before the incident

A practical internal planning baseline can require immediate containment for suspected unauthorized activity, assignment of an incident commander inside the high-severity paging window, an evidence custodian before any destructive change, a first factual situation report within the same operating shift, a legal and privacy notification decision before the earliest applicable external deadline, and named remediation dates before closure. These are internal design recommendations, not legal deadlines; tune them to system criticality, staffing, contracts, geography, and regulatory duties.

For each agent, record the maximum time allowed for declaration, evidence collection, executive escalation, vendor contact, affected-party decision, situation updates, preliminary cause analysis, permanent remediation, and independent verification. Store the clock in the response plan and automation—not in a document responders must discover after the event. Escalate automatically when an owner misses a deadline or the responsible person is unavailable.

Build the minimum incident packet now

Create a case template with the incident ID, severity, detection source, first-observed and declared times, agent and business owner, incident commander, current state, containment authority, evidence custodian, legal and privacy contacts, sharing label, affected systems, identities, tools, data categories, third parties, model and agent versions, approvals, confirmed actions, blocked actions, unknowns, hypotheses, notifications, corrective controls, deadlines, verification results, and closure approval.

Link that template to the data and AI readiness audit, agent inventory, identity map, tool registry, data-flow record, vendor list, logging configuration, kill procedure, backup and recovery plan, and tabletop schedule. The best incident form cannot compensate for an unknown agent, shared administrator credential, missing tool log, or vendor contract that does not provide evidence in time.

SAFE's long-term value may be a common way for organizations to learn from incidents without repeating each other's failures. Its immediate value is the questions the RFC puts in front of operators today. Build the internal machinery first: detect the near miss, stop the action, preserve the truth, notify the right parties, disclose carefully, fix the control, and prove the fix before an agent is trusted again.