Activities of Samira RAFAELA related to 2021/0106(COD)
Plenary speeches (1)
Artificial Intelligence Act (debate)
Amendments (70)
Amendment 413 #
Proposal for a regulation
Recital 14
Recital 14
(14) In order to introduce a proportionate and effective set of binding rules for AI systems, a clearly defined risk- based approach should be followed. That approach should tailor the type and content of such rules to the intensity and scope of the risks that AI systems can generate for individuals and society, rather than depend on the type of technology. It is therefore necessary to prohibit certain artificial intelligence practices, to lay down requirements for high-risk AI systems and obligations for the relevant operators, and to lay down transparency obligations for certain AI systems.
Amendment 441 #
Proposal for a regulation
Recital 17 a (new)
Recital 17 a (new)
(17 a) AI systems used in law enforcement and criminal justice contexts based on predictive methods, profiling and risk assessment pose an unacceptable risk to fundamental rights and in particular to the right of non- discrimination, insofar as they contradict the fundamental right to be presumed innocent and are reflective of historical, systemic, institutional and societal discrimination and other discriminatory practices. These AI systems should therefore be prohibited;
Amendment 443 #
Proposal for a regulation
Recital 17 a (new)
Recital 17 a (new)
(17 a) AI systems used by law enforcement authorities or on their behalf to predict the probability of a natural person to offend or to reoffend, based on profiling and individual risk-assessment hold a particular risk of discrimination against certain persons or groups of persons, as they violate human dignity as well as the key legal principle of presumption of innocence. Such AI systems should therefore be prohibited.
Amendment 450 #
Proposal for a regulation
Recital 18
Recital 18
(18) The use of AI systems for ‘real- time’ remote biometric identification of natural persons in publicly accessible spaces for the purpose of law enforcement is considered particularly intrusive in the rights and freedoms of the concerned persons, to the extent that it may affect the private life of a large part of the population, evoke a feeling of constant surveillance and indirectly dissuade the exercise of the freedom of assembly and other fundamental rights. In addition, the immediacy of the impact and the limited opportunities for further checks or corrections in relation to the use of such systems operating in ‘real-time’ carry heightened risks for the rights and freedoms of the persons that are concerned by law enforcement activities. The use of those systems in publicly accessible places should therefore be prohibited.
Amendment 454 #
Proposal for a regulation
Recital 18
Recital 18
(18) The use of AI systems for ‘real- time’ remote biometric identification of natural persons in publicly accessible spaces for the purpose of law enforcement is considered particularly intrusive in the rights and freedoms of the concerned persons, to the extent that it may affect the private life of a large part of the population, evoke a feeling of constant surveillance and indirectly dissuade the exercise of the freedom of assembly and other fundamental rights. In addition, the immediacy of the impact and the limited opportunities for further checks or corrections in relation to the use of such systems operating in ‘real-time’ carry heightened risks for the rights and freedoms of the persons that are concerned by law enforcement activities.
Amendment 464 #
Proposal for a regulation
Recital 19
Recital 19
Amendment 465 #
Proposal for a regulation
Recital 19
Recital 19
Amendment 474 #
Proposal for a regulation
Recital 20
Recital 20
Amendment 477 #
Proposal for a regulation
Recital 20
Recital 20
Amendment 486 #
Proposal for a regulation
Recital 21
Recital 21
Amendment 487 #
Amendment 494 #
Proposal for a regulation
Recital 22
Recital 22
Amendment 495 #
Proposal for a regulation
Recital 22
Recital 22
Amendment 497 #
Proposal for a regulation
Recital 23
Recital 23
Amendment 498 #
Proposal for a regulation
Recital 23
Recital 23
Amendment 508 #
Proposal for a regulation
Recital 24
Recital 24
Amendment 511 #
Proposal for a regulation
Recital 24
Recital 24
(24) Any processing of biometric data and other personal data involved in the use of AI systems for biometric identification, other than in connection to the use of ‘real-time’ remote biometric identification systems in publicly accessible spaces for the purpose of law enforcement as regulated by this Regulation, including where those systems are used by competent authorities in publicly accessible spaces for other purposes than law enforcement, should continue to comply with all requirements resulting from Article 9(1) of Regulation (EU) 2016/679, Article 10(1) of Regulation (EU) 2018/1725 and Article 10 of Directive (EU) 2016/680, as applicable.
Amendment 515 #
Proposal for a regulation
Recital 24 a (new)
Recital 24 a (new)
Amendment 582 #
Proposal for a regulation
Recital 38
Recital 38
(38) Actions by law enforcement authorities involving certain uses of AI systems are characterised by a significant degree of power imbalance and may lead to surveillance, arrest or deprivation of a natural person’s liberty as well as other adverse impacts on fundamental rights guaranteed in the Charter. In particular, if the AI system is not trained with high quality data, does not meet adequate requirements in terms of its accuracy or robustness, or is not properly designed and tested before being put on the market or otherwise put into service, it may single out people in a discriminatory or otherwise incorrect or unjust manner. Furthermore, the exercise of important procedural fundamental rights, such as the right to an effective remedy and to a fair trial as well as the right of defence and the presumption of innocence, could be hampered, in particular, where such AI systems are not sufficiently transparent, explainable and documented. It is therefore appropriate to classify as high-risk a number of AI systems intended to be used in the law enforcement context where accuracy, reliability and transparency is particularly important to avoid adverse impacts, retain public trust and ensure accountability and effective redress. In view of the nature of the activities in question and the risks relating thereto, those high-risk AI systems should include in particular AI systems intended to be used by law enforcement authorities for individual risk assessments, polygraphs and similar tools or to detect the emotional state of natural person, to detect ‘deep fakes’, for the evaluation of the reliability of evidence in criminal proceedings, for predicting the occurrence or reoccurrence of an actual or potential criminal offence based on profiling of natural persons, or assessing personality traits and characteristics or past criminal behaviour of natural persons or groups, for profiling in the course of detection, investigation or prosecution of criminal offences, as well as for crime analytics regarding natural persons. AI systems specifically intended to be used for administrative proceedings by tax and customs authorities should not be considered high-risk AI systems used by law enforcement authorities for the purposes of prevention, detection, investigation and prosecution of criminal offences.
Amendment 592 #
Proposal for a regulation
Recital 39 a (new)
Recital 39 a (new)
(39 a) The use of AI systems in migration, asylum and border control management should in no circumstances be used by Member States or European Union institutions as a means to circumvent their international obligations under the Convention of 28 July 1951 relating to the Status of Refugees as amended by the Protocol of 31 January 1967, nor should they be used to in any way infringe on the principle of non- refoulement, or deny safe and effective legal avenues into the territory of the Union, including the right to international protection;
Amendment 595 #
Proposal for a regulation
Recital 39 a (new)
Recital 39 a (new)
(39 a) The use of AI systems in migration, asylum and border management should however not, at any point, be used by Member States or by the institutions or agencies of the Union to infringe on the principle of non- refoulement, the right to asylum or to circumvent international obligations under the Convention of 28 July 1951 relating to the Status of Refugees as amended by the Protocol of 31 January 1967.
Amendment 683 #
Proposal for a regulation
Recital 64
Recital 64
(64) Given the more extensive experience of professional pre-market certifiers in the field of product safety and the different nature of risks involved, it is appropriate to limit, at least in an initial phase of application of this Regulation, the scope of application of third-party conformity assessment for high-risk AI systems other than those related to products. Therefore, the conformity assessment of such systems should be carried out as a general rule by the provider under its own responsibility, with the only exception of AI systems intended to be used for the remote biometric identification of persons, for which and AI systems intended to be used to make inferences on the basis of biometric data that produce legal effects or affect the rights and freedoms of natural persons. For those types of AI systems the involvement of a notified body in the conformity assessment should be foreseen, to the extent they are not prohibited..
Amendment 939 #
Proposal for a regulation
Article 3 – paragraph 1 – point 3 a (new)
Article 3 – paragraph 1 – point 3 a (new)
(3 a) ‘risk’ means the combination of the probability of occurrence of a harm and the severity of that harm;
Amendment 940 #
Proposal for a regulation
Article 3 – paragraph 1 – point 3 b (new)
Article 3 – paragraph 1 – point 3 b (new)
(3 b) ‘significant harm‘ means a material harm to a person's life, health and safety or fundamental rights or entities or society at large whose severity is exceptional. The severity is in particular exceptional when the harm is hardly reversible, the outcome has a material adverse impact on health or safety of a person or the impacted person is dependent on the outcome;
Amendment 1181 #
(b) the placing on the market, putting into service or use of an AI system that exploits any of the vulnerabilities of an individual, including characteristics of such individual’s known or predicted personality or social or economic situation, a specific group of persons due to their age, physical or mental or disability, in order to materially distort the behaviour of a person pertaining to that group in a manner that causes or is likely to cause that person or another person physical or psychological harm;
Amendment 1223 #
Proposal for a regulation
Article 5 – paragraph 1 – point c a (new)
Article 5 – paragraph 1 – point c a (new)
(c a) the placing on the market, putting into service or use of an AI system for making individual risk assessments of natural persons in order to assess the risk of a natural person for offending or reoffending or for predicting the occurrence or reoccurrence of an actual or potential criminal offence based on profiling of a natural person or on assessing personality traits and characteristics or past criminal behaviour of natural persons or groups of natural persons;
Amendment 1234 #
Proposal for a regulation
Article 5 – paragraph 1 – point d – introductory part
Article 5 – paragraph 1 – point d – introductory part
(d) the use of ‘real-time’ remote biometric identification systems in publicly accessible spaces for the purpose of law enforcement, unless and in as far as such use is strictly necessary for one of the following objectives:.
Amendment 1239 #
Proposal for a regulation
Article 5 – paragraph 1 – point d – introductory part
Article 5 – paragraph 1 – point d – introductory part
(d) the use of ‘real-time’ remote biometric identification systems in publicly accessible spaces for the purpose of law enforcement, unless and in as far as such use is strictly necessary for one of the following objectives:;
Amendment 1250 #
Proposal for a regulation
Article 5 – paragraph 1 – point d – point i
Article 5 – paragraph 1 – point d – point i
Amendment 1254 #
Proposal for a regulation
Article 5 – paragraph 1 – point d – point i
Article 5 – paragraph 1 – point d – point i
Amendment 1260 #
Proposal for a regulation
Article 5 – paragraph 1 – point d – point ii
Article 5 – paragraph 1 – point d – point ii
Amendment 1261 #
Proposal for a regulation
Article 5 – paragraph 1 – point d – point ii
Article 5 – paragraph 1 – point d – point ii
Amendment 1269 #
Proposal for a regulation
Article 5 – paragraph 1 – point d – point iii
Article 5 – paragraph 1 – point d – point iii
Amendment 1274 #
Proposal for a regulation
Article 5 – paragraph 1 – point d – point iii
Article 5 – paragraph 1 – point d – point iii
Amendment 1286 #
Proposal for a regulation
Article 5 – paragraph 1 – point d a (new)
Article 5 – paragraph 1 – point d a (new)
(d a) the use of an AI system for the general monitoring, detection and interpretation of private content in interpersonal communication services, including all measures that would undermine end-to-end encryption..
Amendment 1290 #
Proposal for a regulation
Article 5 – paragraph 1 – point d a (new)
Article 5 – paragraph 1 – point d a (new)
(d a) The use of predictive, profiling and risk assessment AI systems in law enforcement and criminal justice;
Amendment 1292 #
Proposal for a regulation
Article 5 – paragraph 1 – point d b (new)
Article 5 – paragraph 1 – point d b (new)
(d b) The use of predictive, profiling and risk assessment AI system by or on behalf of competent authorities in migration, asylum or border control management, to profile an individual or assess a risk, including a security risk, a risk of irregular immigration, or a health risk, posed by a natural person who intends to enter or has entered the territory of a Member State, on the basis of personal or sensitive data, known or predicted, except for the sole purpose of identifying specific care and support needs;
Amendment 1303 #
Proposal for a regulation
Article 5 – paragraph 1 – point d c (new)
Article 5 – paragraph 1 – point d c (new)
(d c) the placing on the market, putting into service, or use of AI systems by law enforcement authorities or by competent authorities in migration, asylum and border control management, such as polygraphs and similar tools to detect deception, trustworthiness or related characteristics;
Amendment 1308 #
Proposal for a regulation
Article 5 – paragraph 1 – point d d (new)
Article 5 – paragraph 1 – point d d (new)
(d d) the use of AI systems by or on behalf of competent authorities in migration, asylum and border control management, to forecast or predict individual or collective movement for the purpose of, or in any way reasonably foreseeably leading to, the interdicting, curtailing or preventing migration or border crossings;
Amendment 1353 #
Proposal for a regulation
Article 5 – paragraph 2
Article 5 – paragraph 2
Amendment 1354 #
Proposal for a regulation
Article 5 – paragraph 2
Article 5 – paragraph 2
Amendment 1356 #
Proposal for a regulation
Article 5 – paragraph 2 – point a
Article 5 – paragraph 2 – point a
Amendment 1358 #
Proposal for a regulation
Article 5 – paragraph 2 – point b
Article 5 – paragraph 2 – point b
Amendment 1361 #
Proposal for a regulation
Article 5 – paragraph 2 – subparagraph 1
Article 5 – paragraph 2 – subparagraph 1
Amendment 1367 #
Proposal for a regulation
Article 5 – paragraph 3
Article 5 – paragraph 3
Amendment 1371 #
Proposal for a regulation
Article 5 – paragraph 3
Article 5 – paragraph 3
Amendment 1375 #
Proposal for a regulation
Article 5 – paragraph 3 – subparagraph 1
Article 5 – paragraph 3 – subparagraph 1
Amendment 1384 #
Proposal for a regulation
Article 5 – paragraph 4
Article 5 – paragraph 4
Amendment 1387 #
Proposal for a regulation
Article 5 – paragraph 4
Article 5 – paragraph 4
Amendment 1405 #
Proposal for a regulation
Article 5 a (new)
Article 5 a (new)
Article 5 a Amendments to Article 5 The Commission is empowered to adopt delegated acts in accordance with Article 73 to update the list of AI systems and practices prohibited under Article 5 of the present regulation, according to the latest development in technology and to the assessment of increased or newly emerged risks to fundamental rights.
Amendment 1437 #
Proposal for a regulation
Article 6 – paragraph 2
Article 6 – paragraph 2
2. In addition to the high-risk AI systems referred to in paragraph 1, AI systems referred to in Annex III shall also be considered high-risk in the meaning of this regulation, if they will be deployed in a critical area referred to in Annex III and an individual assessment of the specific application carried out in accordance with Art. 6a showed that a significant harm is likely to arise.
Amendment 1456 #
Proposal for a regulation
Article 6 a (new)
Article 6 a (new)
Amendment 1466 #
Proposal for a regulation
Article 7 – paragraph 1 – introductory part
Article 7 – paragraph 1 – introductory part
1. The Commission is empowered to adopt delegated acts in accordance with Article 73 to update the list in Annex III by adding high-risk AI systems where, after an adequate and transparent consultation process involving the relevant stakeholders, to update the list in Annex III by withdrawing areas from that list or by adding critical areas. For additions both of the following conditions arneed to be fulfilled:
Amendment 1468 #
Proposal for a regulation
Article 7 – paragraph 1 – introductory part
Article 7 – paragraph 1 – introductory part
1. The Commission is empowered to adopt delegated acts in accordance with Article 73 to update the list in Annex III by adding new area headings and high-risk AI systems where both of the following conditions are fulfilled:
Amendment 1476 #
Proposal for a regulation
Article 7 – paragraph 1 – point a
Article 7 – paragraph 1 – point a
(a) the AI systems are intended to be used in any of the areas listed in points 1 to 8 of Annex III or in the newly identified area headings;
Amendment 1503 #
Proposal for a regulation
Article 7 – paragraph 2 – point b a (new)
Article 7 – paragraph 2 – point b a (new)
(b a) the extent to which the AI system acts autonomously;
Amendment 1520 #
Proposal for a regulation
Article 7 – paragraph 2 – point e a (new)
Article 7 – paragraph 2 – point e a (new)
(e a) the potential misuse and malicious use of the AI system and of the technology underpinning it;
Amendment 1531 #
Proposal for a regulation
Article 7 – paragraph 2 – point g a (new)
Article 7 – paragraph 2 – point g a (new)
(g a) magnitude and likelihood of benefit of the deployment of the AI system for individuals, groups, or society at large;
Amendment 1538 #
Proposal for a regulation
Article 7 – paragraph 2 – point h – introductory part
Article 7 – paragraph 2 – point h – introductory part
(h) the extent to which existing Union legislation, in particular the GDPR, provides for:
Amendment 1909 #
Proposal for a regulation
Article 16 a (new)
Article 16 a (new)
Article 16 a Obligations of users of high-risk AI systems Users of high-risk AI systems shall conduct and publish a fundamental rights impact assessment, detailing specific information relating to the context of use of the high-risk AI system in question, including: (a) the affected persons, (b) intended purpose, (c) geographic and temporal scope, (d) assessment of the legality and fundamental rights impacts of the system, (e) compatibility with accessibility legislation, (f) potential direct and indirect impact on fundamental rights, (g) any specific risk of harm likely to impact marginalised persons or those at risk of discrimination, (h) the foreseeable impact of the use of the system on the environment, (i) any other negative impact on the public interest, (j) clear steps as to how the harms identified will be mitigated and how effective this mitigation is likely to be.
Amendment 2287 #
Proposal for a regulation
Title IV a (new)
Title IV a (new)
Rights of affected persons Article 52 a 1.Natural persons have the right not to be subject to non-compliant AI systems.The placing on the market, putting into service or use of non-compliant AI system gives rise to the right of the affected natural persons subject to such non-compliant AI systems to seek and receive redress. 2.Natural persons have the right to be informed about the use and functioning of AI systems they have been or may be exposed to, particularly in the case of high-risk and other regulated AI systems, according to Article 52. 3.Natural persons and public interest organisations have the right to lodge a complaint before the relevant national supervisory authorities against a producer or user of non-compliant AI systems where they consider that their rights or the rights of the natural persons they represent under the present regulation have been violated, and have the right receive effective remedy.
Amendment 2772 #
Proposal for a regulation
Article 68 a (new)
Article 68 a (new)
Amendment 2779 #
Proposal for a regulation
Article 68 b (new)
Article 68 b (new)
Article 68 b Right to an effective judicial remedy against a national supervisory authority 1. Without prejudice to any other administrative or non-judicial remedy, each natural or legal person shall have the right to an effective judicial remedy against a legally binding decision of a national supervisory authority concerning them. 2. Without prejudice to any other administrative or non-judicial remedy, each data subject shall have the right to a an effective judicial remedy where the national supervisory authority does not handle a complaint, does not inform the complainant on the progress or preliminary outcome of the complaint lodged within three months pursuant to Article 68a(3) or does not comply with its obligation to reach a final decision on the complaint within six months pursuant to Article 68a(4) or its obligations under Article 65. 3. Proceedings against a supervisory authority shall be brought before the courts of the Member State where the national supervisory authority is established.
Amendment 2986 #
Proposal for a regulation
Article 84 – paragraph 6
Article 84 – paragraph 6
6. In carrying out the evaluations and reviews referred to in paragraphs 1 to 4 the Commission shall take into account the positions and findings of the Board, of the European Parliament, of the Council, and of other relevant bodies or sources, including stakeholders, and in particular civil society.
Amendment 2993 #
Proposal for a regulation
Article 84 – paragraph 7
Article 84 – paragraph 7
7. The Commission shall, if necessary, submit appropriate proposals to amend this Regulation, in particular taking into account developments in technology and new potential or realised risks to fundamental rights, and in the light of the state of progress in the information society.
Amendment 3203 #
Proposal for a regulation
Annex III – paragraph 1 – point 7 – point b
Annex III – paragraph 1 – point 7 – point b
(b) AI systems intended to be used by competent public authorities or by third parties acting on their behalf to assess a risk, including but not limited to a security risk, a risk of irregular immigration, or a health risk, posed by a natural person who intends to enter or has entered into the territory of a Member State;
Amendment 3211 #
Proposal for a regulation
Annex III – paragraph 1 – point 7 – point d
Annex III – paragraph 1 – point 7 – point d
(d) AI systems intended to assist competent public authorities for the examination and assessment of the veracity of evidence and claims in relation tof applications for asylum, visa and residence permits and associated complaints with regard to the eligibility of the natural persons applying for a status.
Amendment 3220 #
Proposal for a regulation
Annex III – paragraph 1 – point 7 – point d a (new)
Annex III – paragraph 1 – point 7 – point d a (new)
(d a) AI systems intended to be used by or on behalf of competent authorities in migration, asylum and border control management for the forecasting or prediction of trends related to migration, movement and border crossings;
Amendment 3224 #
Proposal for a regulation
Annex III – paragraph 1 – point 7 – point d b (new)
Annex III – paragraph 1 – point 7 – point d b (new)
(d b) AI systems that are or may be used by or on behalf of competent authorities in law enforcement, migration, asylum and border control management for the biometric identification of natural persons;
Amendment 3226 #
Proposal for a regulation
Annex III – paragraph 1 – point 7 – point d c (new)
Annex III – paragraph 1 – point 7 – point d c (new)
(d c) AI systems intended to be used by or on behalf of competent authorities in migration, asylum and border control management to monitor, surveil or process data in the context of border management activities for the purpose of recognising or detecting objects and natural persons;