Bill Text: CA SB896 | 2023-2024 | Regular Session | Amended

California Senate Bill 896

Bill Title: Artificial Intelligence Accountability Act.

Spectrum: Moderate Partisan Bill (Democrat 4-1)

Status: (Introduced) 2024-04-29 - April 29 hearing: Placed on APPR suspense file. [SB896 Detail]

Download: California-2023-SB896-Amended.html

Amended IN Senate April 10, 2024

CALIFORNIA LEGISLATURE— 2023–2024 REGULAR SESSION

Senate Bill

No. 896

Introduced by Senator Dodd
(Coauthor: Senator Rubio)
(Coauthors: Assembly Members Low, Mathis, and Weber)

January 03, 2024

An act to add Chapter 5.9 (commencing with Section 11549.63) to Part 1 of Division 3 of Title 2 of the Government Code, relating to artificial intelligence.

LEGISLATIVE COUNSEL'S DIGEST

SB 896, as amended, Dodd. Artificial Intelligence Accountability Act.

Existing law requires the Secretary of Government Operations to develop a coordinated plan to, among other things, investigate the feasibility of, and obstacles to, developing standards and technologies for state departments to determine digital content provenance. For the purpose of informing that coordinated plan, existing law requires the secretary to evaluate, among other things, the impact of the proliferation of deepfakes, defined to mean audio or visual content that has been generated or manipulated by artificial intelligence that would falsely appear to be authentic or truthful and that features depictions of people appearing to say or do things they did not say or do without their consent, on state government, California-based businesses, and residents of the state.

This bill, the Artificial Intelligence Accountability Act, would, among other things, require the Government Operations Agency, the Department of Technology, and the Office of Data and Innovation to produce a State of California Benefits and Risk of Generative Artificial Intelligence Report that includes certain items, including an examination of the most significant, potentially beneficial uses for deployment of generative artificial intelligence tools by the state, and would require those entities to update the report, as prescribed. The bill would require, as often as is deemed appropriate by the Director of Emergency Services, the California Cybersecurity Integration Center, and the State Threat Assessment Center, those entities to perform a joint risk analysis of potential threats posed by the use of generative artificial intelligence to California’s critical energy infrastructure, including those that could lead to mass casualty events and environmental emergencies.

This bill would also require a state agency or department that utilizes generative artificial intelligence to directly communicate with a person, either through an online interface or telephonically, to clearly and in a conspicuous manner identify to that person that the person’s interaction with the state agency or department is being communicated through artificial intelligence.

This bill would also require an automated decisionmaking system, as defined, utilized by a state agency or department to be evaluated for risk potential before adoption, as specified.

Vote: MAJORITY Appropriation: NO Fiscal Committee: YES Local Program: NO

The people of the State of California do enact as follows:

SECTION 1.

This act shall be known as the Artificial Intelligence Accountability Act.

SEC. 2.

Chapter 5.9 (commencing with Section 11549.63) is added to Part 1 of Division 3 of Title 2 of the Government Code, to read:

CHAPTER 5.9. Artificial Intelligence Tools

11549.63.

The Legislature finds and declares all of the following:

(a) The Legislature recognizes the tremendous potential of artificial intelligence (AI) to improve the lives of its citizens and the functioning of government. However, the Legislature also recognizes that the use of AI must be guided by principles of fairness, transparency, privacy, and accountability to ensure that the rights and opportunities of all Californians are protected in the age of artificial intelligence.

(b) The Legislature further recognizes that generative artificial intelligence (GenAI) enables significant, beneficial uses through its unique capabilities, but GenAI raises novel risks compared to conventional AI across critical areas, including democratic and legal processes, biases and equity, public health and safety, and the economy, and requires measures to address insufficiently guarded governmental systems and unintended or emergent harmful effects from this technology. Additionally, because humans have explicit and implicit biases built into our society, GenAI has the capacity to amplify these biases as it learns from input data. Therefore, it is imperative to consider the implications on Californians of, among other categories, different regions, income, races, ethnicities, gender, ages, religions, abilities, and sexual orientation for all GenAI inputs, outputs, and products for both prioritizing implementations that may promote equity and guarding against bias and other negative impacts.

(c) No individual or group should be discriminated against on the basis of race, gender, age, religion, sexual orientation, or any other protected characteristic in the design, development, deployment, or use of AI systems. The unprecedented speed of innovation and deployment of GenAI technologies necessitates proactive guardrails to protect against potential risks or malicious uses, including, but not limited to, bioterrorism, cyberattacks, disinformation, deception, violation of privacy, and discrimination or bias.

(d) The Legislature affirms the importance of transparency in the use of AI systems. The public has the right to know when they are interacting with AI being used by the state and to have a clear and conspicuous identification of that interaction.

(e) The Legislature recognizes that the use of AI systems must be consistent with the protection of privacy and civil liberties and must be guided by a commitment to equity and social justice. It is the intent of the Legislature in enacting this legislation that all AI systems be designed and deployed in a manner that is consistent with state and federal laws and regulations regarding privacy and civil liberties and minimizes bias and promotes equitable outcomes for all Californians.

(f) This act, in addition to the 2022 White House Blueprint for an AI Bill of Rights, executive guidance from the governor, statutory or regulatory requirements, and evolving best practices should guide the decisionmaking of state agencies, departments, and subdivisions in the review, adoption, management, governance, and regulations of automated decisionmaking technologies. Further, state leadership on adopting these standards and best practices should encourage the private sector to adopt these practices and safeguards.

(g) Public colleges and universities should collaborate with the private sector and relevant state agencies to train students to meet the AI workforce development needs of the state, including providing instruction on AI and related ethical, privacy, and security considerations while advancing research on best practices. Further, there is the need for the state to recruit, retain, and train AI professionals in certain state jobs, and agencies should collaborate to facilitate a pipeline and infrastructure to accomplish that goal, including adopting appropriate job classifications and incentive programs.

(h) State agencies, departments, and boards should utilize their full range of authority to protect consumers, patients, passengers, and students from fraud, discrimination, and threats to privacy and to address other risks that may arise from the use of GenAI, including risks to financial stability. State agencies, departments, and boards should consider rulemaking and emphasize or clarify if existing regulations and guidance apply to GenAI or other automated decisionmaking systems. They should also clarify any responsibility of regulated entities to conduct due diligence with respect to third-party AI services they use and emphasize or clarify requirements and expectations related to the transparency of AI models and regulated entities’ ability to explain their use of AI models.

(i) The California Privacy Protection Agency should utilize its existing authority to promulgate automated decisionmaking technology regulations.

11549.64.

As used in this chapter:

(a) “Artificial intelligence” means an engineered or machine-based system that, for explicit or implicit objectives, infers, from the input it receives, how to generate outputs that can influence physical or virtual environments and that may operate with varying levels of autonomy.
~~(a)~~

(b) (1) “Automated decision system” means a computational process derived from machine learning, statistical modeling, data analytics, or artificial intelligence that issues simplified output, including a score, classification, or recommendation, that is used to assist or replace human discretionary decisionmaking and materially impacts natural persons.

(2) “Automated decision system” does not mean a spam email filter, firewall, antivirus software, identity and access management tool, calculator, database, dataset, or other compilation of data.

~~(b)~~

(c) “Generative artificial intelligence” means the class of artificial intelligence models that emulate the structure and characteristics of input data in order to generate derived synthetic content, including images, videos, audio, text, and other digital content.

~~(c)~~

(d) “High-risk automated decision system” means an automated decision system that is used to assist or replace human discretionary decisions that have a legal or similarly significant effect, including decisions that materially impact access to, or approval for, housing or accommodations, education, employment, credit, health care, and criminal justice.

~~(d)~~

(e) “Person” means a natural person.

~~(e)~~

(f) “Report” means the State of California Benefits and Risk of Generative Artificial Intelligence Report required by Section 11549.65.

11549.65.

(a) (1) The Government Operations Agency, the Department of Technology, and the Office of Data and Innovation shall produce a State of California Benefits and Risk of Generative Artificial Intelligence Report that includes all of the following:

(A) An examination of the most significant, potentially beneficial uses for deployment of generative artificial intelligence tools by the state.

(B) An explanation of the potential risks of the uses described in subparagraph (A) to individuals, communities, and government workers with a focus on high-risk uses, including the use of generative artificial intelligence to make a consequential decision affecting access to essential goods and services.

(C) An explanation of risks from bad actors and insufficiently guarded governmental systems, unintended or emergent effects, and potential risks toward democratic and legal processes, public health and safety, and the economy.

(2) The Government Operations Agency, the Department of Technology, and the Office of Data and Innovation shall update the report, as needed, to respond to significant developments and shall, as appropriate, consult with academia, industry experts, and organizations that represent state government employees.

(b) (1) (A) As often as is deemed appropriate by the Director of Emergency Services, the California Cybersecurity Integration Center, and the State Threat Assessment Center, those entities shall perform a joint risk analysis of potential threats posed by the use of generative artificial intelligence to California’s critical energy infrastructure, including those that could lead to mass casualty events and environmental emergencies.

(B) The entities described in subparagraph (A) shall develop, in consultation with appropriate external experts from academia and industry, a strategy to assess similar potential threats to other critical infrastructure.

(2) The analysis required by paragraph (1) shall be provided to the Governor, and, if appropriate, public recommendations shall be made reflecting changes to artificial intelligence technology, its applications, and risk management, including further private actions, administrative actions, and collaboration with the Legislature to guard against potential threats and vulnerabilities.

(c) (1) (A) The Government Operations Agency, the Department of General Services, the Department of Technology, and the California Cybersecurity Integration Center shall develop, maintain, and periodically evaluate and revise general guidelines for public sector procurement, uses, and required trainings for the use of generative artificial intelligence, including for high-risk scenarios, and including for consequential decisions affecting access to essential goods and services.

(B) The guidelines required by this paragraph shall build on guidance from the White House publication titled Blueprint for an Al Bill of Rights and the National Institute of Standards and Technology’s Al Risk Management ~~Framework~~ Framework, or succeeding document, and shall address safety, algorithmic discrimination, data privacy, and notice of when materials are generated by generative artificial intelligence.

(C) The Government Operations Agency shall engage and consult with organizations that represent state government employees and industry experts, including, but not limited to, trust and safety experts, academic researchers, and research institutions in developing the guidelines required by this paragraph.

(2) For purposes of the periodic evaluation and revision required by paragraph (1), the Government Operations Agency, the Department of General Services, the Department of Technology, and the California Cybersecurity Integration Center shall periodically evaluate any need to revise the guidelines and establish a consultative process by which to do so with academia, industry experts, and organizations that represent state government employees.

(d) (1) (A) The Government Operations Agency, the Department of Technology, and the Office of Data and Innovation shall develop, maintain, and periodically evaluate and revise guidelines for state agencies and departments to analyze the impact that adopting a generative artificial intelligence tool may have on vulnerable communities, including criteria to evaluate equitable outcomes in deployment and implementation of high-risk uses.

(B) The guidelines required by this paragraph shall inform whether and how a state agency or department deploys a particular generative artificial intelligence tool.

(e) The Government Operations Agency, the Department of General Services, and the Department of Technology shall update, as needed, the state’s project approval, procurement, and contract terms to incorporate analysis and feedback obtained pursuant to subdivisions (c) and (d).

(f) (1) To assist the Government Operations Agency and the Department of Technology in their efforts to perform any periodic review and update under this section, all state agencies and departments shall, as requested by the Government Operations Agency or the Department of Technology, conduct and submit an inventory of all current high-risk uses of generative artificial intelligence within the agency or department to the Department of Technology, which shall administer the inventory.

(2) A state agency or department shall appoint a senior level management personnel who will be responsible for maintaining, conducting, and reporting the results of the inventory described by paragraph (1) to the Department of Technology within 60 days of issuance of a request for an inventory pursuant to paragraph (1).

(g) Any state agency or department shall consider procurement and enterprise use opportunities in which generative artificial intelligence can improve the efficiency, effectiveness, accessibility, and equity of government operations consistent with the Government Operations Agency, the Department of General Services, and the Department of Technology’s guidelines for public sector generative artificial intelligence procurement.

(h) (1) The Department of Technology shall establish and maintain the infrastructure to conduct pilot projects of generative artificial intelligence projects, including Department of Technology-approved environments to test those pilot projects.

(2) An environment created pursuant to this subdivision shall be available to any state agency or department to help evaluate generative artificial intelligence tools and services, to further safe, ethical, and responsible implementations, and to inform decisions to use generative artificial intelligence consistent with state guidelines.

(i)(1)By July 1, ____, any state agency or department shall consider pilot projects of generative artificial intelligence applications, in consultation with organizations that represent state government employees and appropriate experts from academia and industry.
~~(2)A pilot project described by paragraph (1) shall measure both of the following:~~
(3) An environment created pursuant to this subdivision shall measure both of the following:

(A) How generative artificial intelligence can improve Californians’ experience with, and access to, government services.

(B) How generative artificial intelligence can support state employees in the performance of their duties in addition to any domain-specific impacts to be measured by the state agency or department.

~~(j)~~

(i) The Government Operations Agency, the Department of General Services, the Department of Technology, the Office of Data and Innovation, and the California Cybersecurity Integration Center shall engage with the Legislature and relevant stakeholders, including historically vulnerable and marginalized communities and organizations that represent state government employees, in the development and revision of any guidelines, criteria, reports, or training pursuant to this section.

~~(k)~~

(j) A state agency or department shall support the state government workforce and prepare for the next generation of skills needed to thrive in the generative artificial intelligence economy by complying with both of the following:

(1) The Government Operations Agency, the Department of Technology, and any other agencies deemed necessary shall make available trainings for state government worker use of state-approved generative artificial intelligence tools to achieve equitable outcomes and to ~~identity~~ identify and mitigate potential output inaccuracies, fabricated text, hallucinations, and biases of generative artificial intelligence, while enforcing public privacy and applicable state laws and policies. If appropriate, the Department of Technology and any other agency or department deemed necessary shall collaborate with organizations that represent state government employees and industry experts on developing and providing training.

(2) The Government Operations Agency, in consultation with appropriate state agencies and organizations that represent state government employees, shall establish criteria to evaluate the impact of generative artificial intelligence on the state government workforce and provide guidelines on how state agencies and departments can support state government employees to use these tools effectively and respond to these technological advancements.

~~(l)~~

(k) Legal counsel for any state agency or department shall consider any potential impact of generative artificial intelligence on regulatory issues under the respective agency’s or department’s authority and recommend necessary updates, if appropriate, as a result of this evolving technology.

11549.66.

(a) A state agency or department that utilizes generative artificial intelligence to directly communicate with a person, either through an online interface or telephonically, shall clearly and in a conspicuous manner identify to that person that the person’s interaction with the state agency or department is being communicated through artificial intelligence.

(b) A state agency or department that utilizes generative artificial intelligence to directly communicate with a person shall provide on the state agency’s or department’s internet website clear instructions, or a link to a web page with clear instructions, informing the person how to directly communicate with a person from the state agency or department.

11549.67.

(a) (1) An automated decisionmaking system utilized by a state agency or department shall be evaluated for risk potential before adoption.

(2) A ~~highrisk~~ high-risk automated decision system shall receive appropriate consultation, testing, risk identification, and mitigation consistent with this chapter and shall not be adopted or utilized without prior approval of the director of a state agency or department or that person’s designee before being adopted and utilized by a state agency or department.

(b) A ~~highrisk~~ high-risk automated decision system that is utilized by a state agency or department shall receive ongoing monitoring and clear organizational oversight.