Technology & Platforms ยท TEC-06
Testing and evaluation before release
Before release, each AI system is tested for accuracy, robustness and behaviour against acceptance criteria agreed in advance, and released only if it meets them.
Why it matters
AI can perform well in a demonstration and badly on real cases, unusual inputs or the groups it serves least well. Testing against criteria set before results are seen gives the release decision an objective basis and a record to compare live performance with.
Levels
| 0 Absent | Confirmed absent: the evidence shows nothing is in place. Not known is recorded as insufficient evidence, not 0. |
|---|---|
| 1 Initial | Systems are tried informally before release; there are no acceptance criteria. |
| 2 Developing | Some testing is done, but criteria are set after the fact, test data is unrepresentative, or robustness and behaviour are not covered. |
| 3 Defined | Every release is tested against criteria and operating conditions set in advance, on representative data, with results signed off and the approach reviewed against live performance. |
| 4 Managed | Test coverage and results are reviewed across uses, releases that miss criteria are blocked or formally accepted, and test results are compared with live performance. |
| 5 Optimising | Test methods and criteria are refined from live performance and incidents, and evaluation is automated and repeated for every change. |
Level 3 criteria
Level 3 is met when every applicable criterion is met. Criteria marked with an overlay apply only to organisations with that characteristic (overlays).
- TEC-06.1 Acceptance criteria for accuracy, robustness and behaviour are agreed before testing starts.
- TEC-06.2 Intended operating conditions (where, when and on what inputs the AI can be relied on) are defined before testing and given to operators and risk owners.
- TEC-06.3 Limits on the system keep its use within the intended operating conditions.
- TEC-06.4 Testing uses representative data, including edge cases, kept independent of any data used to train or tune the model.
- TEC-06.5 Testing of generative AI covers harmful output. GEN
- TEC-06.6 Testing checks the explainability achieved against what was required and records its limits.
- TEC-06.7 Test results are recorded and signed off before every release.
- TEC-06.8 The testing approach and its metrics are reviewed on a set cycle against live performance and incidents.
Assessment questions
- What criteria must this system meet before it is released, and when were they agreed?
- Does the test data reflect the real cases and people the system will meet, including unusual ones?
- Who signs off test results, and what happens when a criterion is missed?
- Acceptance criteria and test plans for a sample of releases
- Test results and release sign-off records
- Review records for the testing approach, and harmful output tests for generative AI
- Defined operating conditions and their enforcement
How to reach Level 3
- Before testing, define the intended operating conditions and agree acceptance criteria for accuracy, robustness, behaviour and explainability, taken from the measures in MON-01.
- Test with representative data, including edge cases, kept independent of training data, and for generative AI include harmful output; security testing is SEC-06.
- Record and sign off results before every release, give operators and risk owners the operating conditions, and enforce them with limits.
- Review the testing approach and metrics on a fixed cycle against live performance and incidents.
Common pitfalls
- Setting acceptance criteria after seeing the results.
- Testing on data that overlaps the training data, which flatters accuracy.
- Describing operating conditions to operators but not enforcing them in the system.
Starter artefacts
- Test plan with acceptance criteria
- Operating conditions statement
- Test results and sign-off record
Ownership and sequence
- Accountable
- delivery lead
- Contributors
- business owner, data owner, risk lead
- Depends on
- MON-01
- Roadmap step
- 7
- Lifecycle
- acquire_build, deploy
Cross-references
| AI Cyber Security Code of Practice | 2.2 partial; 4.3 partial; 7.3 direct; 9.1 partial; 9.2 partial; 9.3 partial; 11.2 partial |
|---|---|
| Algorithmic Transparency Recording Standard | Tier 2 Model specification direct |
| EU AI Act | Art 9 partial; Art 15 partial |
| GDS AI Playbook | P4 partial; P5 partial |
| ISO/IEC 42001 | A.6.2.4 direct; A.9.4 partial |
| JSP 936 Part 1 | para 6 partial; para 36 partial; para 49c partial; para 49e direct; para 75 direct; para 76 direct; para 78 partial; para 122 partial; para 131 partial; para 144 partial; para 145 partial; para 158 direct; para 165 partial; para 170 direct; para 171 partial; para 172 direct; para 175 partial; para 176 partial; para 177 direct; para 190 partial; para 198 partial; para 209 partial |
| MITRE AI Maturity Model | P4.2 direct; P6.3 direct |
| NIST AI 600-1 | R2 direct; R3 direct; R6 partial; R8 partial; R11 partial |
| NIST AI RMF 1.0 | GOVERN 4.3 partial; MAP 2.3 partial; MAP 3.3 direct; MEASURE 1.1 partial; MEASURE 1.2 partial; MEASURE 2.1 partial; MEASURE 2.3 direct; MEASURE 2.5 direct; MEASURE 2.6 partial; MEASURE 2.7 partial; MEASURE 2.9 partial; MEASURE 2.13 direct |
| UK Data and AI Ethics Framework | Transparency 11 partial; Safety 7 direct; SAT 2.1 partial; SAT 2.5 partial; SAT 3.2 partial; SAT 3.5 partial |
| UNESCO AI Maturity Framework | 4.1 direct |
direct: this control at Level 3 meets the item. partial: it covers part of it.