Skip to content
RGPD

Article 15

Accuracy, robustness and cybersecurity

1.   High-risk AI systems shall be designed and developed in such a way that they achieve an appropriate level of accuracy, robustness, and cybersecurity, and that they perform consistently in those respects throughout their lifecycle.

2.   To address the technical aspects of how to measure the appropriate levels of accuracy and robustness set out in paragraph 1 and any other relevant performance metrics, the Commission shall, in cooperation with relevant stakeholders and organisations such as metrology and benchmarking authorities, encourage, as appropriate, the development of benchmarks and measurement methodologies.

3.   The levels of accuracy and the relevant accuracy metrics of high-risk AI systems shall be declared in the accompanying instructions of use.

4.   High-risk AI systems shall be as resilient as possible regarding errors, faults or inconsistencies that may occur within the system or the environment in which the system operates, in particular due to their interaction with natural persons or other systems. Technical and organisational measures shall be taken in this regard.

The robustness of high-risk AI systems may be achieved through technical redundancy solutions, which may include backup or fail-safe plans.

High-risk AI systems that continue to learn after being placed on the market or put into service shall be developed in such a way as to eliminate or reduce as far as possible the risk of possibly biased outputs influencing input for future operations (feedback loops), and as to ensure that any such feedback loops are duly addressed with appropriate mitigation measures.

5.   High-risk AI systems shall be resilient against attempts by unauthorised third parties to alter their use, outputs or performance by exploiting system vulnerabilities.

The technical solutions aiming to ensure the cybersecurity of high-risk AI systems shall be appropriate to the relevant circumstances and the risks.

The technical solutions to address AI specific vulnerabilities shall include, where appropriate, measures to prevent, detect, respond to, resolve and control for attacks trying to manipulate the training data set (data poisoning), or pre-trained components used in training (model poisoning), inputs designed to cause the AI model to make a mistake (adversarial examples or model evasion), confidentiality attacks or model flaws.

Common Questions

Frequently Asked Questions

What does it mean for AI systems to be accurate and robust?
Accuracy means the AI system makes correct decisions consistently, while robustness indicates it remains reliable even when errors occur or when operating conditions change; high-risk AI systems must meet these requirements to ensure safe and dependable performance throughout their entire lifecycle, including challenging or unexpected conditions.
How is cybersecurity ensured for high-risk AI systems?
High-risk AI systems must have cybersecurity measures built-in to protect against unauthorized access, modification, or interference; these safeguards can include ways to detect and prevent attacks, secure against data manipulation like data poisoning, and defend the model from malicious inputs designed intentionally to mislead or damage its performance.
What is a feedback loop in AI, and how should it be managed?
Feedback loops occur when an AI system uses its own outcomes as input for its future operations, potentially repeating errors or biases; high-risk systems that continue learning after deployment should have measures to reduce or remove biases and protect against negative feedback loops, thereby maintaining consistent fairness and effectiveness.
Who determines how accuracy and robustness of AI systems are measured?
The European Commission works with various experts, standard groups, and benchmarking organizations to develop clear guidelines, benchmarks, and measurement methods that specify exactly how to assess and report accuracy and robustness among high-risk AI systems, helping ensure transparency and quality.
← Article 14 Article 16 →