Part 3: The Systems That Decide About People

The high-risk list in the AI Act beyond hiring (credit, education, public benefits), what an organisation using such a system must do, and the questions the law lets a buyer put to a vendor. Covers the December 2027 start date, the fundamental rights impact assessment and the right to an explanation. It explains the rules and does not give advice.

Tue, 10 Nov 2026 · 14 min

Made for you, by AI, from the sources below.

Listen to it in The AI Act, Explained for Your Workplace

Transcript

The script the two voices were given, with the sound cues taken out. A voice may say a word a little differently, so the audio is the episode.

Mia: Welcome to Walking Points. Picture someone who applies for a loan and is refused within seconds. They never see the software that made the call, and nobody can easily tell them where the decision was really made.

Adam: That sounds like it could happen to anyone. Is that what this episode is about?

Mia: It is. The topic is the systems that decide about people: high-risk AI beyond HR, deployer duties and vendor questions. This part was made in October 2026. It's the third and last part of our series on the AI Act, the EU law that sorts AI into risk tiers, and the high-risk tier carries the most weight. We'll walk through three ideas in order. First, the list of decisions the law treats as high-risk. Second, what an organisation that uses such a system has to do. Third, what to ask a vendor before signing.

Adam: And where did we leave off?

Mia: Last Tuesday we covered what already applies at work: the AI literacy duty, the banned practices, the transparency duties, the rules for general-purpose models, and the fines behind them. Nothing in that part has moved since. What has moved is the date for this part's topic, so let's start there.

Adam: Does any of the high-risk machinery apply yet?

Mia: Not yet. The original plan was that the rules for the high-risk list would apply from the second of August. Then the Digital Omnibus, the amending law we covered earlier in the series, was published on the twenty-fourth of July and entered into force on the twenty-seventh of July, according to the law firm Lewis Silkin. It pushed the stand-alone high-risk systems to the second of December 2027, and systems built into products under EU safety law to the second of August 2028.

Adam: So that's a preparation timeline, not a deadline that's already passed.

Mia: Right. About fourteen months to the date for the list. Now, the law gives high-risk two routes in. One is AI that's a safety component of a product already regulated in the EU, such as a medical device or a toy. The other is a fixed list in a part of the law called Annex III, which we introduced in Part 1. That list has eight areas, each a use that touches people's rights. The Commission can add to it over time, but today it's eight.

Adam: Most people hear high-risk AI and think of recruiting. Is that all of it?

Mia: It's one of the eight, and it's broader than people expect. Area four is employment and workers' management. It covers filtering applications, targeting job ads and evaluating candidates, and also decisions about promotion, termination, task allocation based on behaviour, and monitoring performance. But many organisations that never think of themselves as running HR AI use systems from the other areas.

Adam: Such as?

Mia: Area three is education and vocational training. That includes systems that decide who gets admitted to schools, universities and training, and systems that evaluate learning outcomes, including when the result steers what the learner sees next. It also covers software that monitors students during tests to catch prohibited behaviour, which is exam proctoring.

Adam: So an adaptive learning platform that scores someone and then picks their next lesson could be on the list.

Mia: Yes, it can fall under it. Then there's area five, access to essential private and public services and benefits, which reaches the widest. According to the wording on artificialintelligenceact.eu, it lists four uses. Public authorities deciding whether people are eligible for essential benefits and services, including healthcare, and whether to grant, reduce, revoke or reclaim them. Evaluating creditworthiness or setting a credit score, with one exclusion: systems that detect financial fraud are not on the list. Risk assessment and pricing in life and health insurance. And classifying emergency calls and dispatching emergency services, including emergency healthcare triage.

Adam: That's a long way from a recruiting tool.

Mia: It is. And if you work somewhere that buys software, you may already be using something from these areas without ever calling it high-risk AI. The other areas are narrower for a typical workplace. Biometrics, including emotion recognition where it's allowed at all, which we said in the previous part is banned at work and in schools. Critical infrastructure, like road traffic, water, gas, heating and electricity. Law enforcement, with uses like assessing the risk someone will reoffend. Migration and border control. And the administration of justice and democratic processes, including systems meant to influence an election.

Adam: Okay, but there's something I want to test. A lot of vendors say their tool only produces a score, and a human makes the actual decision. Does that get them out of the list?

Mia: Adam, that's exactly the argument a court has already looked at, before the AI Act applied. It's a case about a credit score, and it shows why a number handed to someone else can be the decision itself.

Adam: Then walk me through it.

Mia: It's called SCHUFA, case C-634/21, decided by the European Court of Justice on the seventh of December 2023. SCHUFA is a German credit reference agency that supplies scores to banks. Its argument was that it only prepared a number and the bank made the decision. The Court did not accept that. Where a bank relies heavily on the score to decide whether to grant a loan, producing the score is itself automated individual decision-making under the GDPR, the EU data protection law.

Adam: So the agency couldn't hide behind the bank.

Mia: Right. Commentary from the law firms A&O Shearman and Bird & Bird makes the same point. This is a GDPR ruling, not an AI Act one, and this is a briefing, not legal or financial advice. But it shows the worry the Act's list addresses: the person at the end of the chain can't see where the decision was really made.

Adam: That makes me wonder how any vendor ever escapes the list, if heavy reliance on a score counts as deciding.

Mia: There is an exit, and it's narrower than vendors might hope. Article 6(3) lets a provider, meaning the company that builds the system, conclude it is not high-risk if it poses no significant risk of harm to health, safety or fundamental rights, including by not materially influencing the outcome of a decision. The article gives four examples: a narrow procedural task, improving the result of a human activity already completed, detecting patterns without replacing a human assessment, or a purely preparatory task. Think of a tool that turns a scanned form into structured fields.

Adam: And a score the bank leans on?

Mia: That's hard to call merely preparatory, and the SCHUFA logic cuts the same way. There are also two hard limits. The exit never applies to profiling of people, so an Annex III system that profiles is always high-risk. And under Article 6(4), the provider has to write down its assessment before the system goes on the market, register it, and hand that paperwork to authorities on request. So a vendor can't just say it's exempt. It has to have it written down.

Adam: That brings us to the people buying these things.

Mia: Yes, the deployer, the organisation that uses a system under its own authority, as against the provider that builds it. If you work in an organisation that buys software, that deployer is the role you sit closest to. Article 26 sets out what a deployer of a high-risk system must do, and it has twelve paragraphs. First, use the system as the provider's instructions say. Second, put human oversight in the hands of people with the competence, training, authority and support to do it.

Adam: Is that where the AI literacy duty from last time gets its teeth?

Mia: Exactly. Oversight by someone with no training and no power to overrule is not oversight. Third, where the deployer controls the input data, make sure it's relevant and representative enough for the system's purpose. Fourth, monitor the system, and if it may present a risk, inform the provider and the market surveillance authority, the national body that polices products, and suspend use. And fifth, keep the logs the system generates, where the deployer controls them, for at least six months.

Adam: Does anyone on the receiving end get told anything?

Mia: Two duties are about them. An employer putting a high-risk system into use at work must inform the workers' representatives and the affected workers beforehand. And a deployer of an Annex III system that makes or helps make decisions about people must inform those people the system is being used. Public authorities must also check that the system is registered in the EU database.

Adam: Does every deployer have to do something extra beyond this?

Mia: Only some. Article 27 requires a fundamental rights impact assessment before first use, known as a FRIA. It applies to public bodies, to private entities providing public services, and to deployers of the credit-scoring and life and health insurance systems in area five. It describes the process the system is used in, who is likely to be affected, the specific risks of harm, the oversight measures, and what the deployer will do if those risks materialise. The result goes to the market surveillance authority. If a GDPR data protection assessment already covers part of it, the FRIA can cross-reference it, so it's not a duplicate.

Adam: And what if someone is on the receiving end of a decision and wants to know why?

Mia: That's Article 86. Anyone subject to a decision that a deployer took based on the output of an Annex III system, which has legal effects or similarly significantly affects them, can ask the deployer for clear and meaningful explanations of the role of the AI system in the decision and the main elements of the decision. It applies only where other EU law doesn't already give that right. In practice, an organisation has to be able to say, about a particular decision, what the system did and what the human did.

Adam: Mia, take a company that starts using a general-purpose assistant to rank job applicants. Is it just a customer, or something more?

Mia: Something more, and this is one rule that changes who carries all of this. Under Article 25, a deployer is treated as a provider, and takes on all the provider's duties, in three cases: it puts its own name or trademark on a high-risk system already on the market, it makes a substantial modification to one, or it changes the intended purpose of a system that wasn't high-risk so that it becomes high-risk. Your ranking example is the third. The company has turned a general tool into a high-risk system.

Adam: So the vendor's stated purpose really matters.

Mia: It does, and that leads straight to what to ask before signing. This is what the law forces a provider to put on paper, so it's also what a buyer is entitled to expect. If you ever sit across from a vendor, that's the list you can hold them to. Article 13 requires a high-risk system to come with instructions for use. Read it as a checklist.

Adam: What's on it?

Mia: Who the provider is and how to reach them when something goes wrong. The intended purpose in the vendor's own words, and whether the planned use is inside it. The accuracy levels, how they were measured, and the robustness and cybersecurity measures. Known circumstances that affect performance, including foreseeable misuse. How the system performs for particular groups of people. What input data it needs, and what's known about the data it was trained and tested on.

Adam: That's a lot of paper. What else?

Mia: Whether the system can explain its outputs. What human oversight the design includes, and what the overseer needs to be able to do. What computing resources it needs, how long it's expected to last, and how updates are handled. And what logging is built in, because that's what makes the six-month duty possible at all.

Adam: Is there anything beyond the Article 13 list?

Mia: A few practical questions the law points to. Has the provider classified the system as high-risk, or claimed the Article 6(3) exemption, and if exempt, is the assessment written down and the system registered? The instructions must also list changes planned at the conformity assessment, so a buyer can tell a planned update from a substantial modification. And does the contract split the work between provider and deployer clearly? Article 25 says contracts can allocate duties after a rebrand, so who does what should be written down.

Adam: Given the date is fourteen months off, why would anyone act now?

Mia: Because many of these systems are already on the market. And the Act asks of a deployer, in the end, that it can answer three questions about each one. Do we know what it decides or influences? Does a trained person with the authority to overrule it oversee it? And could we explain a particular decision to the person it affected?

Adam: Is there something a listener could actually try this week?

Mia: Here's one way to think about it. Take a single piece of software in your organisation that scores, ranks or screens people, and ask whether it decides or only influences. That's the first of the three questions the Act leaves a deployer able to answer, so it's a natural place to start.

Adam: What this part comes down to is that wherever software sits inside a decision about a person, an organisation should be able to say what it decides, who oversees it with real authority, and how it would explain one decision to the person it affected.

Mia: That was part three of three, the last part of our series on the AI Act, the EU law that sorts AI into risk tiers.

Heard something wrong? Tell us