AI Security ยท SEC-05
AI in security incident response
Security incident management covers incidents involving AI, with playbooks, people who have the skills to handle them, and the ability to contain an affected AI service quickly.
Why it matters
An attack on or through an AI service can look different from a conventional incident and may need a model, tool or integration to be switched off fast. Preparing before an incident happens avoids slow, improvised responses. Non-security AI incidents are handled under MON-04.
Levels
| 0 Absent | Confirmed absent: the evidence shows nothing is in place. Not known is recorded as insufficient evidence, not 0. |
|---|---|
| 1 Initial | AI security incidents would be handled ad hoc, with no plan for AI-specific scenarios. |
| 2 Developing | The incident process mentions AI, but there are no AI playbooks and few responders know how AI services work. |
| 3 Defined | Security incident procedures include exercised AI playbooks, trained responders and a known containment route for each AI service, with disclosure and notification routes set. |
| 4 Managed | Response performance is measured, and lessons from exercises and real incidents are acted on. |
| 5 Optimising | Incident response for AI is improved from exercise results, real incidents and external threat information, and lessons are shared with peers. |
Level 3 criteria
Level 3 is met when every applicable criterion is met. Criteria marked with an overlay apply only to organisations with that characteristic (overlays).
- SEC-05.1 Security incident procedures include AI-specific scenarios, each with a playbook.
- SEC-05.2 Playbooks cover recovery from AI-specific attacks to a known good state, linked to continuity plans, and support for affected users and other parties.
- SEC-05.3 Each AI service has a known way to contain it.
- SEC-05.4 Responders are trained for the AI-specific scenarios.
- SEC-05.5 AI scenarios are exercised on a set cycle.
- SEC-05.6 Agreements with those the organisation provides AI to set out the incident support they and affected users will receive. PRV
- SEC-05.7 A published vulnerability disclosure route covers the organisation's AI services, and reports feed incident handling.
- SEC-05.8 Notification routes and deadlines for security breaches, including breaches in the AI supply chain, to regulators, customers and affected parties are set out as law and agreements require.
Assessment questions
- Does the security incident process cover incidents involving AI, and with what playbooks?
- Who would respond to an AI security incident, and have they been trained or exercised for it?
- How quickly could you contain or switch off an affected AI service?
- Incident procedures and playbooks covering AI scenarios
- Records of exercises or real incidents involving AI, with lessons and actions
- Recovery playbooks, the published vulnerability disclosure route and exercise records
How to reach Level 3
- Add AI-specific scenarios and playbooks to security incident procedures, including how to contain each AI service and recover it to a known good state, linked to continuity plans (OPM-09).
- Include support for affected users and other parties in the playbooks, and set it out in agreements where the organisation provides AI to others.
- Train responders for the AI scenarios and exercise them on a fixed cycle.
- Publish a vulnerability disclosure route covering AI services and feed reports into incident handling.
- Set out notification routes and deadlines for security breaches, including in the AI supply chain, to regulators, customers and affected parties as law and agreements require.
Common pitfalls
- Having no quick way to switch off a compromised AI service.
- Treating AI incidents as general IT incidents, so responders miss what makes them different.
Starter artefacts
- AI incident playbooks
- Containment method per AI service
- Vulnerability disclosure page
- Breach notification matrix
Ownership and sequence
- Accountable
- security lead
- Contributors
- delivery lead, business owner
- Depends on
- SEC-01
- Roadmap step
- 5
- Lifecycle
- deploy, operate
Cross-references
| AI Cyber Security Code of Practice | 1.1 partial; 5.3 direct; 6.4 direct; 6.5 direct; 10.3 direct; 12.1 partial |
|---|---|
| JSP 936 Part 1 | para 80 partial; para 195 partial |
| NIST AI 600-1 | R9 partial |
| NIST AI RMF 1.0 | GOVERN 4.3 partial; MEASURE 2.7 partial; MANAGE 2.3 partial; MANAGE 4.3 partial |
| UK Data and AI Ethics Framework | Privacy 10 direct; SAT 4.10 partial |
direct: this control at Level 3 meets the item. partial: it covers part of it.