Operating Model & Delivery ยท OPM-06
Delivery standards for AI
AI work is delivered by multidisciplinary teams using user-centred methods to defined delivery standards.
Why it matters
AI that works technically can still fail in use if it does not fit how people work. Teams that bring together users, business, data, technical and risk knowledge, and that test with real users as they go, are more likely to build something people adopt.
Levels
| 0 Absent | Confirmed absent: the evidence shows nothing is in place. Not known is recorded as insufficient evidence, not 0. |
|---|---|
| 1 Initial | AI work is done by technical staff alone, with little contact with users. |
| 2 Developing | Some projects use multidisciplinary teams or user research, but there is no common standard. |
| 3 Defined | Documented delivery standards for multidisciplinary, user-centred AI delivery, with safety, security, privacy, accessibility and quality built in from design, are applied to all AI delivery. |
| 4 Managed | Delivery is checked against the standards at gates, and user adoption and satisfaction are measured and acted on. |
| 5 Optimising | Delivery standards are improved from user outcomes and lessons across the portfolio. |
Level 3 criteria
Level 3 is met when every applicable criterion is met. Criteria marked with an overlay apply only to organisations with that characteristic (overlays).
- OPM-06.1 Documented delivery standards for AI exist and are applied to all AI delivery.
- OPM-06.2 The standards require multidisciplinary teams, user research and iterative testing with users.
- OPM-06.3 The standards include AI-specific engineering practice and the design of how people and AI share work.
- OPM-06.4 Delivery teams include subject matter experts for the data and the work, involved in design and evaluation.
- OPM-06.5 The standards require safety, security and privacy by design, with privacy-protective default settings.
- OPM-06.6 Accuracy, robustness and reliability requirements are set at design for each use of AI.
- OPM-06.7 Diversity of perspective is considered for each team, including contracted third parties, and structured stakeholder input is added where it falls short.
- OPM-06.8 Services, documentation and notices are tested with users who have accessibility needs before go-live.
Assessment questions
- Who is in the team delivering a use of AI, and how are users involved?
- What delivery standards apply to AI work, and how is compliance checked?
- The delivery standards or playbook for AI work
- Team composition and user research records for a sample of projects
- Accessibility test records from before go-live
How to reach Level 3
- Write AI delivery standards requiring multidisciplinary teams that include subject matter experts for the data and the work, user research and iterative testing with users, and AI engineering practice.
- Include in the standards how people and AI share work, safety, security and privacy by design with protective defaults, and accuracy, robustness and reliability requirements set at design.
- Require each team to check its diversity of perspective, including contractors, and to seek structured stakeholder input where it falls short; research with people follows OPM-10.
- Require testing of services, documentation and notices with users who have accessibility needs before go-live, and apply the standards to all AI delivery.
Common pitfalls
- Bringing users in only at the end to accept something already built.
- Leaving accessibility testing until after launch.
Starter artefacts
- AI delivery standards or playbook
- Team composition checklist
Ownership and sequence
- Accountable
- delivery lead
- Contributors
- business owner, people lead
- Depends on
- OPM-01
- Roadmap step
- 3
- Lifecycle
- design, acquire_build, deploy
Cross-references
| EU AI Act | Art 50(1) partial; Art 11 partial; Art 15 partial; Art 17 partial |
|---|---|
| ISO/IEC 42001 | A.6.1.2 partial; A.6.1.3 direct; A.6.2.2 partial; A.6.2.3 partial |
| JSP 936 Part 1 | para 67 partial; para 119 direct; para 121 partial; para 122 partial; para 125 partial; para 137 partial |
| MITRE AI Maturity Model | P1.2 partial; P4.1 direct |
| NIST AI RMF 1.0 | GOVERN 3.1 partial; GOVERN 3.2 partial; GOVERN 4.1 partial; GOVERN 5.2 partial; MAP 1.2 partial; MAP 1.6 partial; MEASURE 4.1 partial |
| UK Data and AI Ethics Framework | Fairness 5 direct; Privacy 1 partial; Privacy 3 direct; Privacy 9 partial; Societal impact 2 partial; Safety 2 partial; SAT 1.2 direct; SAT 1.12 direct; SAT 1.13 partial; SAT 1.14 partial; SAT 2.1 direct; SAT 2.6 partial; SAT 2.9 partial; SAT 3.3 direct |
| UNESCO AI Maturity Framework | 6.5 direct |
direct: this control at Level 3 meets the item. partial: it covers part of it.