Operating Model & Delivery ยท OPM-07
Service management for live AI
Every live AI service has defined support, change and problem management arrangements, including handling of changes to models and prompts.
Why it matters
Once live, AI needs the same support as any other service, plus extra care because its behaviour can change when models, prompts or data change. Without service management, problems go unreported and changes reach users untested.
Levels
| 0 Absent | Confirmed absent: the evidence shows nothing is in place. Not known is recorded as insufficient evidence, not 0. |
|---|---|
| 1 Initial | Live AI is supported informally by whoever built it. |
| 2 Developing | Some services are in the support process, but AI-specific changes such as model or prompt updates are not covered. |
| 3 Defined | Every live AI service has a support owner, a problem reporting route for users, and change and problem management that covers AI-specific changes. |
| 4 Managed | Service levels, problems and changes are measured and reported, and repeat problems lead to root cause fixes. |
| 5 Optimising | Service management practice for AI is refined from incident and change data and from user feedback. |
Level 3 criteria
Level 3 is met when every applicable criterion is met. Criteria marked with an overlay apply only to organisations with that characteristic (overlays).
- OPM-07.1 Every live AI service has a named support owner.
- OPM-07.2 Users of each live AI service have a route to report problems.
- OPM-07.3 Change management for each live AI service covers changes to the model, prompts, data and hosting platform.
- OPM-07.4 Problem management covers each live AI service, including problems arising from model, prompt, data and hosting platform changes.
Assessment questions
- If a user finds a problem with a live AI service, where do they report it?
- How are changes to the model or prompts behind a service approved and tested?
- Who supports each live AI service out of hours, if needed?
- Service catalogue or support records for live AI services
- Change records covering model or prompt updates
How to reach Level 3
- Name a support owner for every live AI service and add each service to the service catalogue.
- Give users a clear route to report problems with each service, using the existing service desk where there is one.
- Extend change management so that changes to models, prompts, data and the hosting platform are recorded, assessed and approved before they reach users.
- Bring AI problems into problem management so repeat faults are investigated, with release and rollback detail handled under TEC-05.
Common pitfalls
- Missing changes made by the supplier to the underlying model, which arrive without a change request.
- Leaving prompt edits outside change control because they look minor.
Starter artefacts
- Service catalogue entries for AI services
- Change request template covering model and prompt changes
Cross-references
| AI Cyber Security Code of Practice | 7.4 partial; 8.3 partial; 11.1 partial |
|---|---|
| Algorithmic Transparency Recording Standard | Tier 2 Tool specification partial |
| GDS AI Playbook | P5 partial |
| ISO/IEC 42001 | 8.1 partial; A.6.2.6 direct |
| JSP 936 Part 1 | para 119 partial; para 176 direct; para 181 direct |
| MITRE AI Maturity Model | P6.2 partial; P6.3 partial |
| NIST AI RMF 1.0 | MANAGE 2.2 partial; MANAGE 4.1 partial; MANAGE 4.2 partial |
| UK Data and AI Ethics Framework | Accountability 7 partial; SAT 3.10 direct; SAT 4.3 partial |
| UNESCO AI Maturity Framework | 4.1 partial; 4.3 partial |
direct: this control at Level 3 meets the item. partial: it covers part of it.