Culture & Change ยท CUL-05
Learning from what did not work
Experiments and uses of AI that fail or are stopped are reviewed without blame, and the lessons are recorded and shared.
Why it matters
Many AI experiments should not continue, and stopping them is a sign the process works. If stopped work is hidden or treated as failure, teams repeat the same mistakes and become unwilling to try things.
Levels
| 0 Absent | Confirmed absent: the evidence shows nothing is in place. Not known is recorded as insufficient evidence, not 0. |
|---|---|
| 1 Initial | Stopped or failed work is quietly dropped; lessons stay with the people involved. |
| 2 Developing | Some teams review what went wrong, but findings are not recorded or shared beyond the team. |
| 3 Defined | Every stopped or failed AI experiment or use has a no-blame review, and its lessons are recorded in a shared place and passed to relevant teams. |
| 4 Managed | Lessons are checked for take-up, for example in new proposals and stage gates, and repeat mistakes are identified and followed up. |
| 5 Optimising | Lessons from inside and outside the organisation are used routinely to shape new proposals, and stopping work early is recognised as good practice. |
Level 3 criteria
Level 3 is met when every applicable criterion is met. Criteria marked with an overlay apply only to organisations with that characteristic (overlays).
- CUL-05.1 A no-blame review is held for every stopped or failed AI experiment or use.
- CUL-05.2 Lessons from each review are recorded in a shared place.
- CUL-05.3 Lessons are communicated to the relevant teams.
Assessment questions
- What happened to the last AI experiment that was stopped, and what was learned?
- Where are lessons recorded, and who reads them?
- How are teams who stop an experiment treated?
- Review records for stopped or failed AI work
- A lessons log or library, with evidence it is used in new proposals
How to reach Level 3
- Set a short no-blame review format, focused on what happened and what to do differently, not on who was at fault.
- Make the review a required step whenever an AI experiment or use is stopped or fails, linked to the stop decision (POR-07).
- Record lessons in one shared place that anyone proposing AI can search.
- Send each new lesson to the teams it is relevant to, for example through the community of practice (CUL-06).
Common pitfalls
- Reviewing only failures and not work that was stopped for good reasons.
- Writing long reports nobody reads instead of short, searchable lessons.
- Letting reviews turn into blame, which makes teams hide problems next time.
Starter artefacts
- No-blame review template
- Shared lessons log
Ownership and sequence
- Accountable
- ai governance lead
- Contributors
- delivery lead, business owner
- Depends on
- POR-06
- Roadmap step
- 5
- Lifecycle
- design, acquire_build, retire
Cross-references
| MITRE AI Maturity Model | P3.1 direct |
|---|---|
| NIST AI RMF 1.0 | GOVERN 4.1 partial |
| UNESCO AI Maturity Framework | 1.5 partial |
direct: this control at Level 3 meets the item. partial: it covers part of it.