EU Guidance Further Defines “High-Risk” AI Systems

The EU’s AI Act is an expansive piece of legislation. Serving as the EU’s AI omnibus law, the AI Act saves its most burdensome compliance obligations for “high-risk” AI systems. Last month, the European Commission issued draft guidelines on the “high-risk” classification, providing concrete examples of high-risk systems. These high-risk systems break down into two main categories.

1. Systems in which AI is used as a safety feature for certain listed products required to undergo third-party conformity assessments
2. Systems operating in a manner deemed high-risk by EU regulators.

While the AI systems that fall into the first category are relatively cut and dry, the second introduces some nuance. This point classifies a wide variety of AI systems as high-risk, but one of the most noteworthy risk categories is AI used in employment. A recent Faegre Drinker memo discusses the guidance and gives the following examples of high-risk employment applications:

• “Automated job-matching and ranking tool that scores/shortlists candidates
• Cross-platform candidate-sourcing tool that builds recruiter shortlists
• Platform ranking of self-employed service providers for customer display
• AI scoring of written/oral applicants’ answers to rank interview invitations
• Background-check system that produces composite “risk” scores
• Targeted job-ad engine deciding who sees vacancies
• Vision-assessment AI for pilot recruitment eligibility
• Apprenticeship-recruitment AI screening and short-listing applicants
• Shift-scheduler allocating work by behavioural signals (punctuality, ratings)
• Platform tutor-performance score triggering auto-suspension/deactivation
• Work-allocation engine (e.g., associates) assigning premium matters via metrics
• Dynamic pay-setting system adjusting remuneration by ratings/acceptance/time
• Civil-servant post-assignment AI using test scores and vacancies to finalise placements”

The memo notes that these high-risk compliance obligations apply to anyone deploying an AI tool where the output of that tool is used in the EU. Multinational companies should be careful. Using AI tools to recruit or assess European job candidates may subject companies to compliance obligations and liability under the EU AI Act.