AACF
Risk, Ethics & Compliance ยท RSK-03

Impact assessment before deployment

Before a use of AI goes live, its likely effects on people and groups (including equality, privacy, rights and safety) and on society and the environment are assessed to a depth that matches its risk, and the findings are acted on.

Why it matters

Risk assessment looks at the risk to the organisation; impact assessment looks at what the use of AI could do to the people it touches. Some harms, such as a group being treated less favourably, only appear when someone asks that question directly and before the system is in use.

Levels
0 AbsentConfirmed absent: the evidence shows nothing is in place. Not known is recorded as insufficient evidence, not 0.
1 InitialEffects on people are not assessed before go-live, or only after a complaint.
2 DevelopingSome impact assessment happens, for example for data protection, but equality, rights and safety are not covered consistently, or findings are not acted on.
3 DefinedA proportionate impact assessment covering people, workforce, society and environment is signed off before each in-scope use goes live, kept current, and its mitigations tracked.
4 ManagedCompletion and quality of impact assessments are checked, assessments are revisited after significant change, and actual effects in live use are compared with what was predicted.
5 OptimisingImpact assessment shapes design from the start, and the method is improved from evidence of effects it missed.
Level 3 criteria

Level 3 is met when every applicable criterion is met. Criteria marked with an overlay apply only to organisations with that characteristic (overlays).

  1. RSK-03.1 Impact assessments, proportionate to risk, cover equality, privacy (including sensitive inferences about people), rights and safety.
  2. RSK-03.2 Impact assessments cover effects on the workforce: workload, working conditions, skills and progression.
  3. RSK-03.3 Impact assessments cover wider societal and environmental effects.
  4. RSK-03.4 An impact assessment is completed and signed off before each in-scope use of AI goes live.
  5. RSK-03.5 Each impact assessment is kept current while the use is live.
  6. RSK-03.6 Mitigations identified in impact assessments are tracked.
  7. RSK-03.7 For public-facing AI assistants, the impact assessment also covers over-reliance and people treating the AI as human.
Assessment questions
  • Which uses of AI need an impact assessment before going live, and who decides?
  • What does the assessment cover beyond data protection?
  • Can you show a case where an impact assessment changed the design or stopped a deployment?
Evidence an assessor would expect
  • The impact assessment template and guidance on proportionality
  • Completed and signed-off assessments for recent deployments
  • Records of mitigations and their completion
  • Impact assessment reviews during live use
How to reach Level 3
  1. Build an impact assessment template covering equality, privacy (including sensitive inferences about people), rights, safety, effects on the workforce (workload, working conditions, skills and progression) and wider societal and environmental effects.
  2. Add questions on over-reliance and people treating the AI as human for public-facing assistants.
  3. Scale the depth of assessment to risk and require sign-off before every in-scope use goes live.
  4. Track mitigations to completion and review the assessment during use and when the use changes.

Common pitfalls

  • Treating the impact assessment as a form to finish rather than findings to act on.
  • Leaving out effects on the organisation's own staff.

Starter artefacts

  • AI impact assessment template with proportionate depth
  • Mitigation tracker
Ownership and sequence
Accountable
business owner
Contributors
risk lead, data owner, ai governance lead
Depends on
RSK-02, DAT-05
Roadmap step
4
Lifecycle
design, acquire_build, deploy
Cross-references
Algorithmic Transparency Recording StandardTier 2 Operational data specification partial; Tier 2 Risks, mitigations and impact assessments direct
EU AI ActArt 26(9) partial; Art 27 partial
GDS AI PlaybookP2 partial
ISO/IEC 420016.1.4 direct; 8.4 direct; A.5.2 direct; A.5.3 partial; A.5.4 direct; A.5.5 direct
JSP 936 Part 1para 35 partial; para 49a direct; para 49d partial; para 52 direct; para 54 partial; para 70 direct; para 71 direct; para 86 partial; para 92 partial; para 141 partial; para 191 partial; para 192 partial; para 194 partial
MITRE AI Maturity ModelP1.3 direct; P5.2 partial; P6.1 partial; P6.4 partial
NIST AI 600-1R3 partial; R4 direct; R5 partial; R6 partial; R7 partial
NIST AI RMF 1.0GOVERN 4.2 direct; MAP 1.1 partial; MAP 1.6 partial; MAP 3.2 partial; MAP 5.1 direct; MEASURE 2.10 direct; MEASURE 2.12 partial
UK Data and AI Ethics FrameworkTransparency 2 partial; Transparency 7 partial; Fairness 2 partial; Fairness 4 direct; Fairness 7 direct; Privacy 3 direct; Privacy 8 direct; Environmental sustainability 3 partial; Societal impact 3 direct; Trade-offs 2 partial; Trade-offs 5 partial; SAT 1.5 partial; SAT 1.6 direct; SAT 1.13 direct; SAT 1.16 partial; SAT 2.9 direct; SAT 2.12 partial; SAT 3.3 partial; SAT 3.7 partial; SAT 3.8 direct; SAT 4.12 partial
UNESCO AI Maturity Framework5.2 direct; 5.3 partial

direct: this control at Level 3 meets the item. partial: it covers part of it.