Operating Model & Delivery ยท OPM-10
Involving people in AI design and evaluation
Evaluations and research that involve people have their informed consent, ethics review and protection, use participants who represent those the service will serve, and for significant uses draw on the communities affected.
Why it matters
Testing AI with people gives the best evidence of how it will work in use, but it puts those people at some risk and is only useful if they reflect the people the service will serve. Engaging affected communities early on significant uses brings concerns to light while the design can still change.
Levels
| 0 Absent | Confirmed absent: the evidence shows nothing is in place. Not known is recorded as insufficient evidence, not 0. |
|---|---|
| 1 Initial | Research and evaluation with people happen informally, without consent records or ethics review. |
| 2 Developing | Some projects obtain consent or ethics review, but there is no common standard, participants are chosen for convenience, or affected communities are not engaged. |
| 3 Defined | Evaluations and research involving people have informed consent, ethics review, representative participants and protection of wellbeing and the right to withdraw, and significant uses engage affected communities at planning and design. |
| 4 Managed | Consent, ethics review and participant representation are checked at delivery gates, and community input and the response to it are reviewed for significant uses. |
| 5 Optimising | Research practice and community engagement are improved from participant feedback, outcomes and lessons across the portfolio. |
Level 3 criteria
Level 3 is met when every applicable criterion is met. Criteria marked with an overlay apply only to organisations with that characteristic (overlays).
- OPM-10.1 Evaluations and user research involving people have the informed consent of participants.
- OPM-10.2 Evaluations and user research involving people go through ethics review by the organisation's established route.
- OPM-10.3 Participants represent the people the service will serve.
- OPM-10.4 Research with participants protects their wellbeing and their right to withdraw.
- OPM-10.5 For significant uses, affected communities and experts who are not users are engaged at planning and design, and their input and the organisation's response are recorded.
Assessment questions
- How do evaluations and user research involving people get informed consent and ethics review?
- How do you make sure participants represent the people the service will serve, and that their wellbeing and right to withdraw are protected?
- For significant uses, which affected communities and outside experts were engaged at planning and design, and what was done with their input?
- Consent and ethics review records for evaluations involving people
- Participant recruitment records showing how representation was sought
- Engagement records for significant uses, with the input received and the organisation's response
How to reach Level 3
- Set rules for research with people: informed consent, and ethics review through the organisation's existing route.
- Recruit participants who represent the people the service will serve, and protect their wellbeing and their right to withdraw.
- For significant uses, engage affected communities and experts who are not users at planning and design, and record their input and the response.
- Build these rules into the delivery standards (OPM-06) so every team applies them.
Common pitfalls
- Treating ethics review and consent as optional for internal pilots that still involve people.
- Engaging communities only after the design is fixed, so their input cannot change it.
Starter artefacts
- Consent form and participant information template
- Stakeholder engagement record
Ownership and sequence
- Accountable
- business owner
- Contributors
- delivery lead, risk lead
- Depends on
- OPM-06
- Roadmap step
- 4
- Lifecycle
- strategy, design, acquire_build
Cross-references
| JSP 936 Part 1 | para 83 direct |
|---|---|
| NIST AI RMF 1.0 | MAP 1.2 partial; MEASURE 2.2 direct |
| UK Data and AI Ethics Framework | Societal impact 2 direct; Safety 1 partial; SAT 1.11 direct; SAT 1.14 direct; SAT 1.15 direct; SAT 2.9 partial |
direct: this control at Level 3 meets the item. partial: it covers part of it.