Measure & Improve ยท MON-04
AI incidents and near misses
AI incidents and near misses that are not security incidents, such as wrong outputs, harm to people or misuse, are reported, investigated and learned from.
Why it matters
AI fails in ways that ordinary incident processes often do not recognise, such as confident but wrong answers or unfair treatment of some groups. Recording near misses as well as incidents shows patterns early, while the cost is still low.
Levels
| 0 Absent | Confirmed absent: the evidence shows nothing is in place. Not known is recorded as insufficient evidence, not 0. |
|---|---|
| 1 Initial | AI incidents are handled as they arise, if they are recognised at all; nothing is recorded. |
| 2 Developing | Some AI incidents are logged in general systems, but there is no agreed definition, investigation method or record of near misses. |
| 3 Defined | A defined process, linked to security incident handling, covers reporting, triage, investigation and resolution of non-security AI incidents and near misses that staff can recognise. |
| 4 Managed | Incident and near miss trends are analysed and reported, and actions are tracked to completion. |
| 5 Optimising | Lessons from incidents, including those reported by other organisations, routinely change designs, controls and training. |
Level 3 criteria
Level 3 is met when every applicable criterion is met. Criteria marked with an overlay apply only to organisations with that characteristic (overlays).
- MON-04.1 A defined process covers reporting, triage, investigation and resolution of non-security AI incidents and near misses.
- MON-04.2 A definition of an AI incident and a near miss is in place and staff understand it.
- MON-04.3 The process has a defined link to security incident handling.
Assessment questions
- What counts as an AI incident here, and how would staff report one?
- Are near misses recorded as well as incidents?
- Can you show an AI incident that led to a change?
- The AI incident process and definition
- The incident and near miss log, with investigations and actions
How to reach Level 3
- Write a plain definition of a non-security AI incident and near miss, with examples, and a process for reporting, triage, investigation and resolution that hands security cases to SEC-05.
- Investigate each incident and near miss to its cause, record the resolution, and pass serious incidents to MON-09 for notification and any suspension.
- Tell staff how to report, using the route for raising concerns (CUL-04), and keep a log of incidents and near misses.
Common pitfalls
- Recording only incidents that caused harm and ignoring near misses.
- Leaving AI incidents in the general IT queue, where wrong or unfair outputs are not recognised.
Starter artefacts
- AI incident definition and process
- Incident and near miss log
Cross-references
| AI Cyber Security Code of Practice | 10.3 partial |
|---|---|
| ISO/IEC 42001 | 10.2 partial |
| MITRE AI Maturity Model | P6.2 partial |
| NIST AI 600-1 | R2 partial; R3 partial; R11 partial |
| NIST AI RMF 1.0 | GOVERN 4.3 partial; GOVERN 6.2 partial; MEASURE 3.1 partial; MANAGE 2.3 partial; MANAGE 4.1 partial; MANAGE 4.3 partial |
| UK Data and AI Ethics Framework | Privacy 10 partial |
direct: this control at Level 3 meets the item. partial: it covers part of it.