AI Security ยท SEC-04
Preventing information leakage through AI
Sensitive information is protected from exposure through AI tools, prompts, training and outputs, with rules staff understand and technical measures that enforce them.
Why it matters
AI tools make it easy to paste sensitive material into a service that stores or learns from it, and outputs can reveal information the user was never entitled to see. Clear rules backed by technical measures protect information without forcing people away from approved tools.
Levels
| 0 Absent | Confirmed absent: the evidence shows nothing is in place. Not known is recorded as insufficient evidence, not 0. |
|---|---|
| 1 Initial | No specific rules; staff decide for themselves what information to put into AI tools. |
| 2 Developing | Guidance exists on what not to share with AI tools, but it is not enforced and does not cover outputs or training. |
| 3 Defined | Rules on which information may be used with which AI tools are communicated to staff and backed by technical measures, and tuned models inherit their data's protection. |
| 4 Managed | Leakage controls are monitored, attempted exposures are detected and followed up, and results are reported. |
| 5 Optimising | Controls are refined from monitoring and incident data so that protection improves without adding friction for legitimate use. |
Level 3 criteria
Level 3 is met when every applicable criterion is met. Criteria marked with an overlay apply only to organisations with that characteristic (overlays).
- SEC-04.1 Rules set out which classes of information may be used with which AI tools.
- SEC-04.2 The rules are communicated to all staff.
- SEC-04.3 Technical measures on approved tools support the rules.
- SEC-04.4 Models trained or tuned on protected information are handled at least at that protection level, taking account of aggregation. BLD
Assessment questions
- Which classes of information may be used with which AI tools, and how do staff know?
- What stops sensitive information leaking through prompts, stored conversations or outputs?
- How would you know if sensitive information had been put into an AI tool it should not have been?
- Rules or guidance linking information classification to AI tools
- Configuration or monitoring records for leakage prevention on approved tools
- Protection level records for models trained or tuned on protected information
How to reach Level 3
- Set rules for which classes of information may be used with which AI tools, linked to the approved tools list (TEC-01).
- Tell all staff the rules and include them in AI literacy learning (PPL-01).
- Configure technical measures on approved tools to enforce the rules, such as data loss prevention and supplier settings that stop retention or training on inputs.
- Handle any model trained or tuned on protected information at least at that protection level, taking account of what combined data could reveal.
Common pitfalls
- Relying on guidance alone with no technical measures behind it.
- Forgetting that a model can expose its training data through its outputs.
Starter artefacts
- Information and AI tool use rules
- Technical control settings record
Cross-references
| AI Cyber Security Code of Practice | 2.5 direct; 3.4 partial; 5.4 partial; 9.4 partial; 12.2 partial |
|---|---|
| Algorithmic Transparency Recording Standard | Tier 2 Operational data specification partial; Scope exemptions partial |
| GDS AI Playbook | P3 partial |
| JSP 936 Part 1 | para 64 partial; para 168 partial |
| MITRE AI Maturity Model | P4.2 partial; P5.2 direct |
| NIST AI 600-1 | R4 direct; R9 partial; R10 partial |
| NIST AI RMF 1.0 | MEASURE 2.10 partial |
| UK Data and AI Ethics Framework | Privacy 8 direct; Safety 4 direct |
| UNESCO AI Maturity Framework | 3.4 partial |
direct: this control at Level 3 meets the item. partial: it covers part of it.