Technology & Platforms ยท TEC-05
Managing models in live use
Models, prompts and their configuration are versioned and released under control, so the organisation knows what is running and can roll back to a known good version.
Why it matters
A small change to a model, prompt or setting can change behaviour as much as a code change, and suppliers may change models without notice. Version and release control means problems can be traced to a change and reversed quickly.
Levels
| 0 Absent | Confirmed absent: the evidence shows nothing is in place. Not known is recorded as insufficient evidence, not 0. |
|---|---|
| 1 Initial | Models and prompts are changed directly in live use; there is no record of what changed or when. |
| 2 Developing | Some versioning exists, for example for in-house models, but prompts, settings or supplier model versions are not controlled and rollback is uncertain. |
| 3 Defined | Models, prompts and configuration are versioned, released with approval and can be rolled back, with documentation for each version and advance notice of changes to users. |
| 4 Managed | Releases and rollbacks are recorded and reviewed, unplanned changes including supplier model updates are detected, and failed releases are investigated. |
| 5 Optimising | Release and rollback are largely automated and routinely exercised, and the process is refined from release outcomes. |
Level 3 criteria
Level 3 is met when every applicable criterion is met. Criteria marked with an overlay apply only to organisations with that characteristic (overlays).
- TEC-05.1 Models, prompts and configuration are under version control.
- TEC-05.2 Data sets used with each model are versioned alongside each model version.
- TEC-05.3 Changes are released through a controlled process with approval.
- TEC-05.4 Every live use of AI can be rolled back to a previous known good version.
- TEC-05.5 Documentation is kept for each version, suited to its users, operators and, where relevant, regulators, stating intended purpose and users, how it works, inputs and outputs, known limitations, and where it must not be used or relied on.
- TEC-05.6 Release approval includes checking the documentation against the released version.
- TEC-05.7 Users are told in advance, accessibly, of planned changes that affect them.
Assessment questions
- Which version of the model, prompts and configuration is running in live use right now?
- How is a change released, and who approves it?
- When did you last roll back, and how long did it take?
- Version history for models, prompts and configuration
- Release records with approvals
- A record of a rollback or rollback test
- Documentation for the current version, written for users, operators and regulators
- Use documentation stating purpose, users, limitations and where not to rely on it; advance notices of changes
How to reach Level 3
- Version models, prompts, configuration and the data sets used with each model version.
- Release changes through a controlled process with approval, and keep the previous known good version ready for rollback.
- Keep documentation for each version suited to users, operators and, where relevant, regulators, covering intended purpose and users, how it works, inputs and outputs, known limits and where it must not be used or relied on.
- Check the documentation against the released version as part of release approval.
- Tell affected users in advance, accessibly, about planned changes.
Common pitfalls
- Not tracking which version of a supplier's model is behind a service.
- Letting documentation drift from what is actually running.
Starter artefacts
- Version register for models, prompts and data sets
- Release approval record
- Per-version documentation template
- Rollback procedure
Ownership and sequence
- Accountable
- delivery lead
- Contributors
- business owner, security lead
- Depends on
- TEC-06
- Roadmap step
- 8
- Lifecycle
- deploy, operate, retire
Cross-references
| AI Cyber Security Code of Practice | 2.3 direct; 3.2 partial; 4.5 partial; 5.1 partial; 5.2 partial; 5.3 partial; 6.5 partial; 7.3 partial; 7.4 direct; 8.1 partial; 8.3 direct; 10.2 partial; 11.1 partial; 11.2 partial; 12.4 partial |
|---|---|
| Algorithmic Transparency Recording Standard | Tier 2 Description and rationale direct; Tier 2 Tool specification partial; Tier 2 Model specification partial |
| EU AI Act | Art 25 partial; Art 11 partial; Art 13 partial |
| GDS AI Playbook | P5 partial |
| ISO/IEC 42001 | A.6.2.3 partial; A.6.2.5 partial; A.6.2.6 partial; A.6.2.7 direct; A.8.2 direct |
| JSP 936 Part 1 | para 49c partial; para 61 partial; para 75 partial; para 162 partial; para 165 direct; para 166 partial; para 172 partial; para 175 partial; para 181 direct |
| MITRE AI Maturity Model | P1.2 direct; P6.1 partial; P6.3 partial; P6.4 partial |
| NIST AI RMF 1.0 | MAP 2.2 direct; MEASURE 2.5 partial; MEASURE 2.9 partial; MANAGE 1.4 partial; MANAGE 2.4 partial |
| UK Data and AI Ethics Framework | Transparency 1 partial; Transparency 8 partial; Transparency 9 direct; Accountability 6 partial; SAT 1.10 partial; SAT 2.3 direct; SAT 3.5 direct; SAT 4.1 partial; SAT 4.2 partial; SAT 4.4 partial |
| UNESCO AI Maturity Framework | 3.1 partial; 3.3 partial; 3.4 partial; 4.1 direct; 5.4 partial; 5.5 direct; 6.2 partial |
direct: this control at Level 3 meets the item. partial: it covers part of it.