Source: OJ L, 2024/1689, 12.7.2024

Article 14 Human oversight

Summary What does Article 14 of the AI Act say?

This article establishes the human oversight requirements for high-risk AI systems.

It sits within the broader set of requirements for high-risk AI systems under Chapter III and builds directly on those technical obligations by addressing the human element: ensuring that natural persons remain meaningfully in control during the operation of such systems.

The article covers both how oversight measures should be designed into systems by providers before market placement, and how deployers must be equipped to exercise that oversight in practice.

It also addresses a specific heightened standard for biometric identification systems listed in Annex III, requiring dual verification before any action is taken on the basis of the system's output.

Important points:

Providers must design human oversight into high-risk AI systems before placing them on the market, including the technical ability for operators to monitor, override, and halt the system.
Ensure that persons assigned oversight responsibilities are equipped to understand the system's limitations, detect anomalies, and guard against automation bias.
For high-risk AI systems used for remote biometric identification (Annex III, point 1(a)), no action may be taken by the deployer on the basis of the system's identification unless at least two competent natural persons have separately verified and confirmed it, with a carve-out where Union or national law deems this disproportionate in law enforcement, migration, border control, or asylum contexts.

Springlex's summary of the article, a reading aid, not a substitute for the legal text.

1. High-risk AI systems shall be designed and developed in such a way, including with appropriate human-machine interface tools, that they can be effectively overseen by natural persons during the period in which they are in use.
1. Human oversight shall aim to prevent or minimise the risks to health, safety or fundamental rights that may emerge when a high-risk AI system is used in accordance with its intended purpose or under conditions of reasonably foreseeable misuse, in particular where such risks persist despite the application of other requirements set out in this Section.
1. The oversight measures shall be commensurate with the risks, level of autonomy and context of use of the high-risk AI system, and shall be ensured through either one or both of the following types of measures:
  1. measures identified and built, when technically feasible, into the high-risk AI system by the provider before it is placed on the market or put into service;
  2. measures identified by the provider before placing the high-risk AI system on the market or putting it into service and that are appropriate to be implemented by the deployer.
1. For the purpose of implementing paragraphs 1, 2 and 3, the high-risk AI system shall be provided to the deployer in such a way that natural persons to whom human oversight is assigned are enabled, as appropriate and proportionate:
  1. to properly understand the relevant capacities and limitations of the high-risk AI system and be able to duly monitor its operation, including in view of detecting and addressing anomalies, dysfunctions and unexpected performance;
  2. to remain aware of the possible tendency of automatically relying or over-relying on the output produced by a high-risk AI system (automation bias), in particular for high-risk AI systems used to provide information or recommendations for decisions to be taken by natural persons;
  3. to correctly interpret the high-risk AI system’s output, taking into account, for example, the interpretation tools and methods available;
  4. to decide, in any particular situation, not to use the high-risk AI system or to otherwise disregard, override or reverse the output of the high-risk AI system;
  5. to intervene in the operation of the high-risk AI system or interrupt the system through a ‘stop’ button or a similar procedure that allows the system to come to a halt in a safe state.
1. For high-risk AI systems referred to in point 1(a) of Annex III, the measures referred to in paragraph 3 of this Article shall be such as to ensure that, in addition, no action or decision is taken by the deployer on the basis of the identification resulting from the system unless that identification has been separately verified and confirmed by at least two natural persons with the necessary competence, training and authority.
2. The requirement for a separate verification by at least two natural persons shall not apply to high-risk AI systems used for the purposes of law enforcement, migration, border control or asylum, where Union or national law considers the application of this requirement to be disproportionate.

Relevant recitals

Recital 66 Coverage of mandatory requirements

Requirements should apply to high-risk AI systems as regards risk management, the quality and relevance of data sets used, technical documentation and record-keeping, transparency and the provision of information to deployers, human oversight, and robustness, accuracy and cybersecurity. Those requirements are necessary to effectively mitigate the risks for health, safety and fundamental rights. As no other less trade restrictive measures are reasonably available those requirements are not unjustified restrictions to trade.

Recital 73 Human oversight

High-risk AI systems should be designed and developed in such a way that natural persons can oversee their functioning, ensure that they are used as intended and that their impacts are addressed over the system’s lifecycle. To that end, appropriate human oversight measures should be identified by the provider of the system before its placing on the market or putting into service. In particular, where appropriate, such measures should guarantee that the system is subject to in-built operational constraints that cannot be overridden by the system itself and is responsive to the human operator, and that the natural persons to whom human oversight has been assigned have the necessary competence, training and authority to carry out that role. It is also essential, as appropriate, to ensure that high-risk AI systems include mechanisms to guide and inform a natural person to whom human oversight has been assigned to make informed decisions if, when and how to intervene in order to avoid negative consequences or risks, or stop the system if it does not perform as intended. Considering the significant consequences for persons in the case of an incorrect match by certain biometric identification systems, it is appropriate to provide for an enhanced human oversight requirement for those systems so that no action or decision may be taken by the deployer on the basis of the identification resulting from the system unless this has been separately verified and confirmed by at least two natural persons. Those persons could be from one or more entities and include the person operating or using the system. This requirement should not pose unnecessary burden or delays and it could be sufficient that the separate verifications by the different persons are automatically recorded in the logs generated by the system. Given the specificities of the areas of law enforcement, migration, border control and asylum, this requirement should not apply where Union or national law considers the application of that requirement to be disproportionate.

Table of contents

Springlex and this text is meant purely as a documentation tool and has no legal effect. No liability is assumed for its content. The authentic version of this act is the one published in the Official Journal of the European Union.