Operating Model & Delivery ยท OPM-09
Continuity when AI fails
Services that depend on AI can continue safely when the AI is unavailable, degraded or producing wrong results, through planned fallbacks that have been tested.
Why it matters
AI services fail in ways other systems do not, including supplier outages, model changes and confident but wrong output. Services that rely on AI need a planned way to carry on, or to stop safely, so that a failure does not become harm to the people they serve.
Levels
| 0 Absent | Confirmed absent: the evidence shows nothing is in place. Not known is recorded as insufficient evidence, not 0. |
|---|---|
| 1 Initial | Dependence on AI is not considered in continuity planning; a failure would be handled when it happened. |
| 2 Developing | Some services have fallbacks, but they are informal, untested or do not cover degraded or wrong output. |
| 3 Defined | Every service that depends on AI has a tested fallback or safe stop, based on an impact analysis and part of continuity plans, and people are told when it is used. |
| 4 Managed | Recovery times are measured against targets, and failures in testing are fixed. |
| 5 Optimising | Continuity is improved from tests, incidents and supplier changes, and the degree of dependence on AI is set deliberately for each service. |
Level 3 criteria
Level 3 is met when every applicable criterion is met. Criteria marked with an overlay apply only to organisations with that characteristic (overlays).
- OPM-09.1 An impact analysis of AI unavailability exists for each service that depends on AI and sets its recovery objectives.
- OPM-09.2 Each such service has a documented fallback or safe stop, based on the impact analysis, covering unavailability, degradation and wrong output.
- OPM-09.3 Fallbacks and safe stops are included in continuity plans.
- OPM-09.4 Fallbacks are tested on a set cycle.
- OPM-09.5 Users and affected parties are told when a fallback or safe stop is in force.
Assessment questions
- Which services depend on AI, and what happens to each if the AI stops or gets things wrong?
- Are AI dependencies included in business continuity plans?
- When were the fallbacks last tested, and what did the test show?
- Impact analysis of AI unavailability with recovery objectives for each dependent service
- Continuity plans or service designs showing AI dependencies and fallbacks
- Fallback test records with results and actions
How to reach Level 3
- List every service that depends on AI, using the AI inventory (GOV-06).
- Analyse the impact of the AI being unavailable for each service and set recovery objectives from that analysis.
- Document a fallback or safe stop for each service that covers the AI being unavailable, degraded or producing wrong output, and add it to continuity plans.
- Prepare how users and affected parties will be told when a fallback or safe stop is in force.
- Test each fallback on a fixed cycle and record the results and actions.
Common pitfalls
- Planning only for outages and not for the AI giving confident but wrong answers.
- Assuming staff can return to the manual process when the skills or capacity to do so have gone.
Starter artefacts
- AI dependency impact analysis with recovery objectives
- Fallback and safe stop plan per service
- Fallback test record
Cross-references
| AI Cyber Security Code of Practice | 2.2 partial; 5.3 partial; 6.5 partial |
|---|---|
| EU AI Act | Art 26(5) partial; Art 15 partial |
| JSP 936 Part 1 | para 49e partial; para 78 partial; para 82 partial; para 91 partial; para 150 partial; para 171 partial; para 209 partial |
| MITRE AI Maturity Model | P6.3 partial |
| NIST AI 600-1 | R2 partial |
| NIST AI RMF 1.0 | GOVERN 6.2 direct; MEASURE 2.6 partial; MANAGE 2.3 partial; MANAGE 2.4 partial |
| UK Data and AI Ethics Framework | SAT 4.1 direct; SAT 4.10 direct; SAT 4.11 partial |
direct: this control at Level 3 meets the item. partial: it covers part of it.