Measure & Improve ยท MON-03
Benefits tracking
Whether the expected benefits of AI arrive is tracked and reported, for each use and across the portfolio, and shortfalls are acted on.
Why it matters
Benefits stated in a business case are forecasts, and many do not arrive in full or arrive later than planned. Tracking them shows which uses are worth continuing and keeps future business cases realistic.
Levels
| 0 Absent | Confirmed absent: the evidence shows nothing is in place. Not known is recorded as insufficient evidence, not 0. |
|---|---|
| 1 Initial | Benefits are not tracked after approval. |
| 2 Developing | Benefits are reviewed for some uses, usually once and informally, and not totalled across the portfolio. |
| 3 Defined | A named owner tracks each use's benefits against its business case on a set schedule, and results are reported across the portfolio. |
| 4 Managed | Benefit claims are checked against evidence, shortfalls trigger review of the use, and actual results are compared with forecasts. |
| 5 Optimising | Actual benefit data is used to calibrate new business cases and the organisation-level value case, and the tracking method is refined from experience. |
Level 3 criteria
Level 3 is met when every applicable criterion is met. Criteria marked with an overlay apply only to organisations with that characteristic (overlays).
- MON-03.1 Benefits for each use of AI are tracked against its business case on a set schedule.
- MON-03.2 Each use has a named owner for tracking its benefits.
- MON-03.3 Benefits are reported across the portfolio.
Assessment questions
- For a use of AI that has been live for some time, what benefits have actually been realised?
- Who tracks benefits, and how often?
- What happens when benefits fall short of the business case?
- Benefits tracking records for a sample of uses of AI
- Portfolio benefits report comparing forecast and actual results
How to reach Level 3
- Take the expected benefits from each business case (POR-05) and record them with measures and dates.
- Name an owner for benefits on each use and set a schedule for checking them.
- Track actual benefits against the business case on that schedule.
- Combine the results into a portfolio report that shows each use and the total.
Common pitfalls
- Checking benefits once soon after launch and never again.
- Counting time saved without asking what that time is now used for.
- Leaving benefits ownership with the delivery team, which moves on after go-live.
Starter artefacts
- Benefits register for each use
- Portfolio benefits report template
Cross-references
| ISO/IEC 42001 | 9.1 partial |
|---|---|
| MITRE AI Maturity Model | P6.1 partial |
| NIST AI RMF 1.0 | MAP 1.4 partial; MEASURE 4.3 partial; MANAGE 2.2 direct |
| UK Data and AI Ethics Framework | Fairness 6 partial; Societal impact 6 partial |
| UNESCO AI Maturity Framework | 1.4 direct |
direct: this control at Level 3 meets the item. partial: it covers part of it.