Bill Text: NY A10625 | 2023-2024 | General Assembly | Introduced
Bill Title: Requires the division of criminal justice services to formulate a protocol for the regulation of the use of artificial intelligence and facial recognition technology in criminal investigations; restricts the use of artificial intelligence-generated outputs in court.
Spectrum: Partisan Bill (Democrat 1-0)
Status: (Introduced) 2024-06-20 - referred to codes [A10625 Detail]
Download: New_York-2023-A10625-Introduced.html
STATE OF NEW YORK ________________________________________________________________________ 10625 IN ASSEMBLY June 20, 2024 ___________ Introduced by COMMITTEE ON RULES -- (at request of M. of A. Bichotte Hermelyn) -- read once and referred to the Committee on Codes AN ACT to amend the executive law, in relation to the regulation of the use of artificial intelligence and facial recognition technology in criminal investigations; and to amend the criminal procedure law, in relation to limiting the use of artificial intelligence-generated outputs in court The People of the State of New York, represented in Senate and Assem- bly, do enact as follows: 1 Section 1. Legislative intent. Artificial intelligence tools like 2 Cybercheck and facial recognition technologies have been used extensive- 3 ly in criminal investigations across the U.S., including nearly 8,000 4 cases in 40 states. However, their use has raised significant legal 5 challenges due to the lack of transparency in their methodologies and 6 concerns over accuracy and reliability. Defense attorneys have success- 7 fully argued in several cases that AI-generated evidence should be 8 excluded from court proceedings due to these issues. This act aims to 9 address these concerns by prohibiting the use of AI outputs, including 10 facial recognition, as evidence while allowing their use for investi- 11 gative purposes, thereby protecting the rights of defendants and main- 12 taining the integrity of the judicial process. 13 § 2. Section 837 of the executive law is amended by adding a new 14 subdivision 24 to read as follows: 15 24. (a) Promulgate a standardized and detailed written protocol for 16 the utilization of artificial intelligence by police agencies in the 17 investigation of criminal activity. The protocol shall address the 18 following topics: 19 (i) authorizing law enforcement agencies to utilize AI systems and FRT 20 for investigative purposes, including but not limited to data analysis, 21 pattern recognition, facial recognition, and predictive analytics; 22 (ii) authorizing law enforcement agencies to utilize AI systems and 23 FRT to assist in identifying potential suspects, uncovering evidence, 24 and generating investigative leads; EXPLANATION--Matter in italics (underscored) is new; matter in brackets [] is old law to be omitted. LBD15799-01-4A. 10625 2 1 (iii) requiring law enforcement agencies utilizing AI systems and FRT 2 to implement transparency measures, including maintaining records of 3 AI-generated outputs and the parameters used by the AI system. Such 4 records shall be subject to audit and review by an independent oversight 5 body designated by the department to ensure compliance with the proto- 6 col; 7 (iv) requiring law enforcement agencies utilizing AI systems and FRT 8 to undergo regular independent audits of FRT systems to assess their 9 accuracy and biases. The results of these audits shall be publicly 10 accessible and disclosed in any case where FRT evidence is used; and 11 (v) requiring that law enforcement officers utilizing AI systems and 12 FRT receive thorough training on the limitations and proper use of AI 13 systems and FRT, emphasizing the importance of understanding and miti- 14 gating biases and errors. 15 (b) For the purposes of this subdivision: 16 (i) "artificial intelligence" or "AI" means a machine-based system 17 that can, for a given set of human-defined objectives, make predictions, 18 recommendations, or decisions influencing real or virtual environments; 19 (ii) "AI-generated output" means any information, analysis, or deci- 20 sion produced by an AI system during a criminal investigation, including 21 but not limited to facial recognition, predictive policing models, and 22 other machine learning algorithms; and 23 (iii) "facial recognition technology" or "FRT" means a technology that 24 analyzes facial features and is capable of identifying or verifying a 25 person from a digital image or a video frame from a video source. 26 § 3. The criminal procedure law is amended by adding a new section 27 60.77 to read as follows: 28 § 60.77 Rules of evidence; use of artificial intelligence. 29 1. Artificial intelligence-generated outputs, including facial recog- 30 nition results, shall not be admissible as evidence in any criminal or 31 civil court proceeding. 32 2. No party in a criminal or civil case shall reference or introduce 33 artificial intelligence-generated outputs, including facial recognition 34 results, during questioning of witnesses, suspects, or defendants. 35 3. Artificial intelligence-generated outputs shall not form the basis 36 for any legal decision or judgment. 37 4. Defendants shall have the right to expert witnesses who can testify 38 about the reliability and limitations of artificial intelligence and 39 facial recognition technology systems used in the investigation of their 40 cases. Funding shall be available through the office of indigent legal 41 services to indigent defendants to access such experts. 42 5. All prosecutorial offices must disclose detailed information about 43 any artificial intelligence systems and facial recognition technology 44 used in investigations, including error rates, known biases, source 45 code, and algorithmic transparency, where possible. 46 § 4. This act shall take effect on the ninetieth day after it shall 47 have become a law.
