Stop measuring AI literacy by course completion

The learning dashboard shows a completed course. The certificate has a name, a date, and a passing mark.

It does not show whether the person can tell when an AI-generated answer needs a source, whether customer data may enter the tool, what a confident error looks like in their domain, or when an output must be escalated instead of edited.

Completion is evidence that training was delivered. It is not evidence that someone can operate an AI-assisted workflow safely.

That distinction is becoming harder to ignore. Article 4 of the EU AI Act says providers and deployers should take measures, to their best extent, to ensure a sufficient level of AI literacy among staff and other people using AI on their behalf. The regulation explicitly connects sufficiency to people’s knowledge and experience, the context of use, and the people affected by the system Regulation (EU) 2024/1689.

The practical implication reaches beyond EU compliance: stop treating AI literacy as a course catalogue. Run it as a capability attached to real work.

The certificate proves the wrong thing

A generic course can establish a common vocabulary. People should understand that model output can be wrong, that apparently public tools may have different data and retention terms, and that accountability does not pass to the model.

The course becomes weak when it is also expected to prepare every role for every use.

A recruiter reviewing an AI-assisted candidate summary needs to understand source gaps, inappropriate inferences, fairness concerns, recordkeeping, and the boundary between administrative support and an employment decision. A software developer using a coding agent needs to inspect generated code, dependency choices, secret exposure, licenses, and tool permissions. A communications team needs rules for claims, confidential material, copyright, approval, and disclosure.

These people share a baseline. They do not share the same failure modes.

The European Commission’s AI literacy Q&A reflects that distinction. It says organisations should consider whether they provide or deploy AI, the risk of the systems involved, and what each target group needs to know. It also says a certificate is not required and that internal records of training or other guidance can be kept European Commission AI literacy Q&A.

Use that flexibility. A training transcript is easy to administer, but the useful question is narrower: can this person make the decisions their part of the workflow requires?

Build from decisions people make

Start with the approved AI workflows, not a list of fashionable AI topics.

For each workflow, write down the moments when the person must exercise judgment:

Workflow moment	Capability to practise	Evidence worth keeping
Choosing an input	Recognise restricted, personal, confidential, or irrelevant data before it enters the tool	A scenario response or supervised sample
Reviewing an output	Check sources, material omissions, domain errors, bias, and unsupported certainty	Review notes on a representative output
Using the result	Distinguish a draft, recommendation, and decision; preserve required human approval	Workflow observation or approval record
Handling failure	Stop, correct, report, or fall back when the system behaves unexpectedly	Incident exercise or escalation record
Changing the workflow	Notice when a new model, data source, permission, audience, or purpose changes the original approval	A change request or re-review decision

This is not a demand for an exam after every prompt. It is a design method for learning and development, the workflow owner, and risk functions. They can see which decisions recur across the company, which are role-specific, and which belong to a high-consequence process that needs deeper qualification.

NIST’s AI Risk Management Framework provides a useful operating frame. It calls for personnel and partners to receive AI risk training that enables their assigned duties, for human-AI roles and oversight responsibilities to be defined, and for operator proficiency and human oversight processes to be assessed and documented NIST AI RMF Core.

The emphasis is on duties and proficiency, not exposure to content.

Give everyone a baseline, then teach the work

A workable programme has a shared foundation and task-specific practice.

The shared foundation should be short enough to remain current. It can cover:

• which AI systems the organisation approves and where to find their terms
• the difference between generated material and verified work
• data classification, privacy, security, intellectual-property, and records boundaries
• how human accountability and review work
• how to report a failure, suspected harm, or unapproved use

Then add the smallest practice module that matches the workflow. Someone who drafts internal meeting notes with an approved assistant may need a data-boundary exercise and an output check. Someone supervising a system used in a consequential decision needs domain-specific failure cases, affected-person considerations, meaningful authority to intervene, and a tested fallback.

Managers need their own track. Their job is not only to use a tool. They decide whether employees have time to verify outputs, whether productivity targets make review unrealistic, whether people can report problems without penalty, and whether the workflow should pause.

Technical teams need more than model mechanics. If they connect an AI system to company data or tools, literacy includes access control, evaluation, logging, injection risks, failure containment, and the difference between a model response and permission to act.

This layered approach is not merely theoretical. The European Commission describes its internal programme as a common baseline with learning packages for generalists, managers, and developers, plus tool-specific and sector-specific material European Commission AI literacy practice. That is an example, not a compliance recipe: the Commission’s wider repository warns that inclusion does not amount to endorsement or a presumption of compliance Repository of AI literacy practices.

Observe capability without building worker surveillance

The best assessment resembles the work.

Ask a support agent to review a flawed draft built from fictional or properly de-identified data. Ask a manager what evidence they would need before expanding the workflow. Ask a developer to trace what happens when an agent receives an instruction embedded in an untrusted document. Ask a reviewer to identify the point where a suggestion becomes a decision.

The goal is not to catch people out. It is to expose gaps while the stakes are controlled.

Use proportionate evidence:

• a facilitated scenario for a new or sensitive workflow
• sampled quality review already performed as part of normal operations
• a tabletop failure exercise for people with stop or escalation duties
• a short attestation when the risk is low and the boundaries are simple
• a supervised practice run before someone receives broader access

Keep the minimum record needed to show what capability was addressed, for which role and workflow, when it was reviewed, and what follow-up was required. Avoid retaining raw prompts and outputs merely because they are available. They may contain employee, customer, source-code, commercial, or security-sensitive information.

Do not turn literacy data into a hidden performance-ranking system. People will conceal uncertainty if asking for help harms their evaluation. Aggregate programme gaps where possible, restrict access to individual records, set retention limits, and separate learning evidence from productivity monitoring.

The measurement rule from Measure the AI workflow, not how often people open the tool applies here too: collect evidence that answers an operating decision, not activity data that happens to be easy to count.

Make the workflow owner answer for sufficiency

Learning and development can maintain the baseline and help design practice. It cannot decide alone whether a person is prepared to supervise a particular workflow.

Put that decision with the workflow owner and the relevant domain reviewers.

A small capability record can carry:

Field	What it answers
Role and workflow	Who is expected to do what with which AI system?
Required judgments	Where must the person verify, approve, stop, or escalate?
Learning and practice	What preparation matched those judgments?
Evidence	What showed the person could perform them?
Limits	Which cases, data, permissions, or decisions remain out of scope?
Owner and review trigger	Who updates this when the workflow changes?

Do not create a permanent “AI literate” badge. Capability is contextual. A person may be prepared to use an approved assistant for internal drafting and unprepared to use the same model in hiring, customer advice, or automated action.

It also expires in practice even when the record has no formal expiry date. A new data source, model, vendor control, user group, output audience, or level of autonomy can change what people need to know. Attach literacy review to the same triggers described in Review AI workflows again when the workflow changes.

Keep the legal claim narrow

The EU position is live and should not be reduced to a slide saying “training required.”

The Commission’s current Q&A says Article 4 has applied since February 2, 2025, and that national market surveillance authorities will start supervising and enforcing Article 4 as of August 2, 2026. It also notes that the Commission proposed, through the Digital Omnibus, to shift the general Article 4 obligation toward Member States and the Commission promoting AI literacy. That proposal does not erase the current text by itself, and the Q&A says training duties connected to human oversight for deployers of high-risk systems remain European Commission AI literacy Q&A.

Do not use this article to classify a system, determine territorial scope, or declare compliance. Have qualified counsel track the final legislation, national enforcement, and sector-specific duties. Review the legal position again when the proposal changes, when guidance is updated, or when a workflow moves toward a high-risk or otherwise regulated use.

The operating model should survive that uncertainty. Role-specific preparation, meaningful oversight, data discipline, failure practice, and evidence of capability are useful even when a particular legal clause does not apply.

When training is the wrong intervention

Some failures are not literacy failures.

If the interface encourages people to paste restricted data, change the interface or access control. If reviewers cannot inspect a source, make provenance available or stop using the output for that purpose. If the production target leaves no time for review, change the target. If a model is unreliable for the task, replace it or return to a non-AI process.

Do not train people to compensate indefinitely for a bad system design.

Small organisations with a narrow, low-risk use may not need a learning platform, an AI officer, or a governance board. The Commission Q&A does not prescribe a specific governance structure for Article 4. A documented conversation, approved-use guidance, a realistic practice case, and a named owner may be enough for the actual risk.

High-consequence workflows need the opposite response. Training is one control among legal review, impact assessment, technical evaluation, access controls, monitoring, contestability, human authority, and a working fallback. A course completion must never be used to excuse missing safeguards.

The test is simple: if the certificate disappeared, could the organisation still show that the people operating and supervising the workflow know its boundaries, can recognise its important failures, and have the authority to act?

If not, the literacy programme has measured attendance where it needed to build capability.