SB 313: Department of Technology: Office of Artificial Intelligence: state agency public interface: use of AI.
- Session Year: 2023-2024
- House: Senate
- Latest Version Date: 2023-04-10
Current Status:
Failed
(2024-02-01: Returned to Secretary of Senate pursuant to Joint Rule 56.)
Introduced
In Committee
First Chamber
In Committee
Second Chamber
Enacted
Existing law establishes within the Government Operations Agency the Department of Technology, under the supervision of the Director of Technology, also known as the State Chief Information Officer. Existing law requires the director to, among other things, provide technology direction to agency and department chief information officers to ensure the integration of statewide technology initiatives, compliance with information technology policies and standards, and the promotion of the alignment and effective management of information technology services.
This bill would enact the California AI-ware Act, which would establish, within the Department of Technology, the Office of Artificial Intelligence, and would grant the office the powers and authorities power and authority necessary to guide the design, use, and deployment of automated systems by a state agency to ensure that all AI systems are designed and deployed in a manner that is consistent with state and federal laws and regulations regarding privacy and civil liberties and that minimizes bias and promotes equitable outcomes for all Californians. The bill would make related legislative findings and declarations.
Existing law imposes various requirements on state agencies concerning technological matters, including requiring every state agency that utilizes any method, device, identifier, or other database application on the internet to electronically collect personal information regarding any user to prominently display a prescribed notice at an initial point of communication with a potential user.
This bill would require any state agency that utilizes generative artificial intelligence to directly communicate with a natural person to provide notice to that person that the interaction with the state agency is being communicated through artificial intelligence, as specified. This bill would require the state agency to provide instructions, as specified, to inform the natural person how they can directly communicate with a natural person from the state agency.
The bill would make related legislative findings and declarations.
Existing law establishes within the Government Operations Agency the Department of Technology, under the supervision of the Director of Technology, also known as the State Chief Information Officer. Existing law requires the director to, among other things, provide technology direction to agency and department chief information officers to ensure the integration of statewide technology initiatives, compliance with information technology policies and standards, and the promotion of the alignment and effective management of information technology services.
This bill would establish, within the Department of Technology, the Office of Artificial Intelligence, and would grant the office the powers and authorities necessary to guide the design, use, and deployment of automated systems by a state agency to ensure that all AI systems are designed and deployed in a manner that is consistent with state and federal laws and regulations regarding privacy and civil liberties and that minimizes bias and promotes equitable outcomes for all Californians. The bill would make related legislative findings and declarations.
This chapter shall be known, and may be cited, as the California AI-ware Act.
11549.80.11549.81.In enacting this chapter, the Legislature finds and declares:
(a)The Legislature recognizes the tremendous potential of artificial intelligence (AI) to improve the lives of its citizens and the functioning of government. However, the Legislature also recognizes that the use of AI must be guided by principles of fairness, transparency, and accountability to ensure that the rights and opportunities of all Californians are protected in the age of artificial intelligence.
(b)The Legislature declares that no individual or group should be discriminated against on the basis of race, gender, age, religion, sexual orientation, or any other protected characteristic in the design, development, deployment, or use of AI systems.
(c)The Legislature affirms the importance of transparency in the use of AI systems. The public has the right to know when they are interacting with AI being used by the state, and to have a clear and conspicuous identification of that interaction.
(d)The Legislature recognizes that the use of AI systems must be consistent with the protection of privacy and civil liberties and must be guided by a commitment to equity and social justice. It is the intent of the Legislature in enacting this legislation that all AI systems be designed and deployed in a manner that is both of the following:
(1)Consistent with state and federal laws and regulations regarding privacy and civil liberties.
(2)Minimizes bias and promotes equitable outcomes for all Californians.
11549.81.11549.82.There is in state government, within the Department of Technology, the Office of Artificial Intelligence.
11549.82.11549.83.The office shall have the powers and authorities power and authority necessary to guide the design, use, or deployment of automated systems by a state agency to ensure that all AI systems are designed and deployed in a manner that is consistent with state and federal laws and regulations regarding privacy and civil liberties and that minimizes bias and promotes equitable outcomes for all Californians.
11549.84.(a)A state agency that utilizes generative artificial intelligence to directly communicate with a person, either through an online interface or telephonically, shall clearly and in a conspicuous manner identify to that person that their interaction with the state agency is being communicated through artificial intelligence.
(b)A state agency that utilizes generative artificial intelligence to directly communicate with a person shall provide on the agencys internet website clear instructions, or a link to a page with clear instructions, informing the person when and how the person can directly communicate with a person from the state agency.
(c)For purposes of this section, person means a natural person.
In enacting this chapter, the Legislature finds and declares:
(a)The Legislature recognizes the tremendous potential of artificial intelligence (AI) to improve the lives of its citizens and the functioning of government. However, the Legislature also recognizes that the use of AI must be guided by principles of fairness, transparency, and accountability to ensure that the rights and opportunities of all Californians are protected in the age of artificial intelligence.
(b)The Legislature declares that no individual or group should be discriminated against on the basis of race, gender, age, religion, sexual orientation, or any other protected characteristic in the design, development, deployment, or use of AI systems.
(c)The Legislature affirms the importance of transparency in the use of AI systems.
(d)The Legislature recognizes that the use of AI systems must be consistent with the protection of privacy and civil liberties and must be guided by a commitment to equity and social justice. It is the intent of the Legislature in enacting this legislation that all AI systems be designed and deployed in a manner that is both of the following:
(1)Consistent with state and federal laws and regulations regarding privacy and civil liberties.
(2)Minimizes bias and promotes equitable outcomes for all Californians.
11549.81.There is in state government, within the Department of Technology, the Office of Artificial Intelligence.
11549.82.The office shall have the powers and authorities necessary to guide the design, use, or deployment of automated systems by a state agency to ensure that all AI systems are designed and deployed in a manner that is consistent with state and federal laws and regulations regarding privacy and civil liberties and that minimizes bias and promotes equitable outcomes for all Californians.
|
Date
|
Status
|
|---|---|
| Feb 1, 2024 | Returned to Secretary of Senate pursuant to Joint Rule 56. |
| May 18, 2023 | May 18 hearing: Held in committee and under submission. |
| May 12, 2023 | Set for hearing May 18. |
| May 1, 2023 | May 1 hearing: Placed on APPR suspense file. |
| Apr 21, 2023 | Set for hearing May 1. |
| Apr 19, 2023 | From committee: Do pass and re-refer to Com. on APPR. (Ayes 11. Noes 0. Page 798.) (April 18). Re-referred to Com. on APPR. |
| Apr 11, 2023 | Set for hearing April 18. |
| Apr 11, 2023 | From committee: Do pass and re-refer to Com. on JUD. (Ayes 12. Noes 0. Page 656.) (April 11). Re-referred to Com. on JUD. |
| Apr 10, 2023 | From committee with author's amendments. Read second time and amended. Re-referred to Com. on G.O. |
| Mar 28, 2023 | Set for hearing April 11. |
| Feb 15, 2023 | Referred to Coms. on G.O. and JUD. |
| Feb 7, 2023 | From printer. May be acted upon on or after March 9. |
| Feb 6, 2023 | Introduced. Read first time. To Com. on RLS. for assignment. To print. |
|
Date
|
Description
|
Location
|
Votes
|
|
|---|---|---|---|---|
| May 1, 2023 | Placed on suspense file | Senate Standing Committee on Appropriations |
Passed |
|
| Apr 18, 2023 | Do pass, but first be re-referred to the Committee on [Appropriations] | Senate Standing Committee on Judiciary |
Passed |
|
| Apr 11, 2023 | Do pass, but first be re-referred to the Committee on [Judiciary] | Senate Standing Committee on Governmental Organization |
Passed |
This list shows positions taken on this bill by organizations on the date displayed, which is linked to the original source.
Positions by individuals are not included.
|
Organization
|
Position
|
Date
|
|---|---|---|
| Little Hoover Commission | ** Support |
SENATE COMMITTEE ON APPROPRIATIONS
Senator Anthony Portantino, Chair
2023 - 2024 Regular Session
SB 313 (Dodd) - Department of Technology: Office of Artificial Intelligence: state agency public interface: use of AI
Version: April 10, 2023 | Policy Vote: G.O. 12 - 0, JUD. 11 - 0 |
Urgency: No | Mandate: No |
Hearing Date: May 1, 2023 | Consultant: Janelle Miyashiro |
Bill Summary:
SB 313 establishes the Office of Artificial Intelligence within the California Department of Technology (CDT) and requires a state agency to disclose when it utilizes artificial intelligence (AI) to communicate with a person and provide instructions on how to communicate with a real person from the state agency, as specified.
Fiscal Impact:
The CDT estimates costs of $1.6 million and four positions in the first year and $2.2 million and eight positions in the second year and ongoing to establish the new office, develop AI strategy, and support the program (General Fund).
Unknown fiscal impact to various state agencies utilizing AI systems to provide the specified disclosure and update processes or directions on communicating with a real person from the agency. To the extent this workload is not absorbable within existing resources, there may be a fiscal impact to each impacted agency.
Background:
AI refers to the development of computer systems that can perform tasks typically requiring human intelligence. These tasks include learning, reasoning, problem-solving, perception, and natural language understanding. AI enables computers to learn from data and experiences, allowing them to handle complex tasks without explicit programming. AI has the potential to revolutionize numerous sectors, including healthcare, finance, and transportation, by improving efficiency, reducing costs, and enhancing decision-making capabilities.
Proposed Law:
- Establishes the Office of Artificial Intelligence within the CDT.
- Provides the office has the power and authority necessary to guide the design, use, or deployment of automated systems by a state agency to ensure that all AI systems are designed and deployed in a manner consistent with state and federal laws and regulations, as specified.
- Requires a state agency that utilizes generative AI to directly communicate with a person to:
- Clearly and conspicuously identify to that person that their interaction with the state agency is being communicated through AI.
- Provide clear instruction on its website of clear instructions, or a link to a page with clear instructions, informing the person when and how the person can directly communicate with a person from the state agency.
- Establishes the California AI-ware Act and states legislative findings and declarations.
Related Legislation:
SB 721 (Becker, 2023) creates the California Interagency Artificial Intelligence Working Group, and tasks the group with delivering a report to the Legislature regarding AI by January 1, 2025 and every two years thereafter, as specified. SB 721 is pending in this committee.
-- END --
Senator Thomas Umberg, Chair
2023-2024 Regular Session
Bill No: SB 313 (Author:Dodd)
Version: April 10, 2023
Hearing Date: April 18, 2023
Fiscal: Yes
Urgency: No
Consultant: CK
SUBJECT
Department of Technology: Office of Artificial Intelligence
DIGEST
This bill establishes the Office of Artificial Intelligence. The bill requires state agencies to disclose when they are using generative artificial intelligence to communicate with a person and to provide them an option to speak with a natural person at the agency.
EXECUTIVE SUMMARY
Owing to recent advances in processing power and the rise of big data, artificial intelligence’s (AI) capacity and the scope of its applications have expanded rapidly, impacting how we communicate, interact, entertain ourselves, travel, transact, and consume media. In ways we may not fully comprehend, AI empowers and encumbers us. It has been used to accelerate productivity, achieve efficiencies, liberate us from drudgery, write our college essay, help us understand and enjoy the world, connect with each other, and live longer, fuller lives. It has also been used to constrain personal autonomy, compromise privacy and security, foment social upheaval, exacerbate inequality, spread misinformation, and subvert democracy. For good or ill, its transformative potential seems boundless.
California does not regulate AI as such, but several laws address some of its most prominent applications, including autonomous vehicles, social media, bots, fake news, drones, privacy, and job displacement. These efforts offer creative approaches to novel problems, but the rapid proliferation and breadth of AI’s manifestations underscores the challenge of regulation in this field and the need for principled governance of AI.
This bill establishes the Office of Artificial Intelligence. The bill requires state agencies to disclose when they are using generative artificial intelligence to communicate with a person and to provide them an option to speak with a natural person at the agency. The bill is author-sponsored. There is no known support or opposition. It passed through the Senate Governmental Organization Committee on a vote of 12 to 0.
PROPOSED CHANGES TO THE LAW
Existing law:
- Establishes the California Department of Technology (CDT) within the Government Operations Agency, under the supervision of the Director of Technology (Director), also known as the State Chief Information Officer. (Gov. Code Sec. 11545(a).)
- Provides that the duties of the Director include:
- advising the Governor on the strategic management and direction of the state’s information technology (IT) resources;
- establishing and enforcing state IT strategic plans, policies, standards, and enterprise architecture, as specified;
- minimizing overlap, redundancy, and cost in state IT operations by promoting the efficient and effective use of information technology;
- providing technology direction to agency and department chief information officers to ensure the integration of statewide technology initiatives, compliance with IT policies and standards, and the promotion of the alignment and effective management of IT services;
- working to improve organizational maturity and capacity in the effective management of IT; and establishing performance management and improvement processes to ensure state IT systems and services are efficient and effective. (Gov. Code § 11545(b).)
This bill:
- Establishes the California AI-ware Act.
- Creates the Office of Artificial Intelligence within CDT. It provides the office with the power and authority necessary to guide the design, use, or deployment of automated systems by a state agency to ensure that all AI systems are designed and deployed in a manner that is consistent with state and federal laws and regulations regarding privacy and civil liberties and that minimizes bias and promotes equitable outcomes for all Californians.
- Requires state agencies that utilize generative AI to directly communicate with a person to clearly and conspicuously identify to that person that their interaction is being communicated through AI.
- Requires a state agency that so utilizes AI to provide clear instructions on their internet website, or a link to a page with such instructions, when and how a person can directly communicate with a natural person at the agency.
COMMENTS
1. The need for principled governance of AI
- What is AI?
Intelligence is an amorphous concept. Broadly, a thing can be considered intelligent if it processes information and solves problems in uncertain conditions. Artificial intelligence, then, is intelligence exhibited by the things we build. The term also refers to a field of study pioneered by Dartmouth professor John McCarthy in the 1950s, who described the endeavor of developing AI as “that of making a machine behave in ways that would be called intelligent if a human were so behaving.”[1]
In certain narrow fields, computers have far surpassed human capabilities. Some of the most dramatic milestones in AI’s advancement have come when computer programs have bested humans at games, prompting varied reactions in their human counterparts. When IBM’s Deep Blue defeated World Chess Champion Gary Kasparov in 1997 in their decisive match, Kasparov insisted that a human grand master must have been controlling the machine. Better sportsmanship was displayed by Jeopardy legend Ken Jennings upon realizing he was certain to lose to IBM’s Watson in 2011. On his Final Jeopardy question video screen, Jennings wrote: “I, for one, welcome our new computer overlords.” By contrast, when the world’s best Go player, a Chinese national, succumbed to Google DeepMind’s AlphaGo, the Chinese government blocked the live-stream.
This most recent illustration of AI’s ascendance is especially notable because until recently, the prospect of a computer program beating a human professional player in Go was viewed as at least a decade away. This is because Go, an ancient Chinese strategy game, is more complex than chess by orders of magnitude. But whereas AlphaGo was programmed with input about the best moves in given situations and learned to improve upon existing strategies, a successor program, AlphaGo Zero, was programed to train itself based on the rules of the game alone. As a result, it developed startlingly creative strategies, defeating AlphaGo in 100 consecutive games. A generic spin-off of this program, AlphaZero, which was not specifically designed to play any particular game, mastered chess, Go, and shogi in just a few hours of self-play.[2]
Such developments reflect the recent advances in a subset of AI known as “machine learning,” a technique that “aims to help computers discover such fuzzy rules by themselves, without having to be explicitly instructed every step of the way by human programmers.”[3] The most prominent type of machine learning is “deep learning,” which “uses artificial neural networks – simplified computer simulations of how biological neurons behave—to extract rules and patterns from sets of data.”[4] The application of systems that simulate billions of neurons to massive datasets has driven recent advancements in areas previously thought difficult for AI to penetrate, including image recognition, voice recognition, translation, and autonomous vehicles.
Developments in just the last year have shown how quickly the technology can advance. Most are likely familiar with the recent explosion of AI tools known as “generative AI,” such as ChatGPT, Midjourney, Stable Diffusion, and Bing Chat. The advantages these new tools have over previous iterations include scale and ease of use. They have integrated unfathomably large amounts of data from which to learn and function and have sophisticated natural language interfaces that are able to process user requests in plain language.[5]
- Hidden bias in algorithms
Machine learning “enables computer systems to learn and make predictions based on historical data. The machine learning process is powered by a machine learning algorithm, a function that is able to improve its performance over time by training itself using methods of data analysis and analytical modelling.”[6] Algorithms process enormous datasets and make decisions with speed and reliability that vastly exceed human capabilities. “They determine everything from what ads we see on the Internet, to whether we are flagged for increased security screening at the airport, to our medical diagnoses and credit scores. They lie behind two of the most powerful products of the digital information age: Google Search and Facebook’s Newsfeed.”[7] The most sophisticated algorithms need no supervision and use deep neural networks to “discover hidden patterns in data, typically those unrecognizable to, or difficult to discern by, humans.”[8] In addition to organizing vast troves of data, algorithms offer the possibility of eliminating human biases in areas such as hiring decisions, credit scores, and criminal sentencing.
However, an algorithm is only as good as the information it is analyzing. Flawed inputs will produce flawed outputs. And an algorithm may key on factors other than those intended by its designer. In one example, a software student was dismayed to learn his program that could reliably distinguish dogs from wolves had, in actuality, learned to recognize snow in the background of the pictures rather than the canine’s features.[9]
When we do not fully understand how an algorithm works, we are unable to determine which aspects of data it is focusing on.
And in many cases algorithms may inadvertently pick up human biases. In Weapons of Math Destruction: How Big Data Increases Inequality and Threaten Democracy, Cathy O’Neill shows that ultimately, a person shapes an algorithm, mediating the datasets gathered and deciding how to weigh them. These decisions are colored by our inherent biases and cultural predilections. Jacob Weisberg writes that “[c]orrelations reflected in historical data become invisibly entrenched in policy without programmers having ill intentions. Quantified information naturally points backward.”[10] Rather than eliminating bias, some algorithms reinforce it, cloaking discrimination with mathematical neutrality.
ProPublica explored this phenomenon in the field of criminal justice.[11] Some jurisdictions factor algorithm-driven risk assessments into criminal bail, sentencing, and parole decisions. In 2014, U.S. Attorney General Eric Holder warned that the risk scores might be injecting bias into the courts. “Although these measures were crafted with the best of intentions, I am concerned that they inadvertently undermine our efforts to ensure individualized and equal justice,” he stated, adding, “they may exacerbate unwarranted and unjust disparities that are already far too common in our criminal justice system and in our society.” After obtaining the risk scores assigned to more than 7,000 people arrested in Broward County, Florida, ProPublica found that the scores “proved remarkably unreliable in forecasting violent crime.”[12] ProPublica’s study validated Holder’s fears: “Black defendants were still 77 percent more likely to be pegged as at higher risk of committing a future violent crime and 45 percent more likely to be predicted to commit a future crime of any kind.”[13]
Wiesberg likens biased algorithms to a dog that barks at black people because it has absorbed its racist owner’s prejudices. He writes:
This kind of feedback loop helps to explain the “racist dog” phenomenon of ostensibly race-neutral criminal justice algorithms. If a correlation of dark skin and criminality is reflected in data based on patterns of racial profiling, then processing historical data will predict that blacks will commit more crimes, even if neither race nor a proxy for race is encoded as an input variable. The prediction brings more supervision, which supports the prediction.[14]
A lack of transparency reduces accountability, again underscoring the importance of oversight. “[A]lgorithms simply grind out their results, and it is up to humans to review and address how that data is presented to users, to ensure the proper context and application of that data.”[15]
As directed by the National Artificial Intelligence Initiative Act of 2020, the National Institute of Standards and Technology (NIST) developed the AI Risk Management Framework. Speaking to these issues, the framework asserts:
AI systems and the contexts in which they are deployed are frequently complex, making it difficult to detect and respond to failures when they occur. AI systems are inherently socio-technical in nature, meaning they are influenced by societal dynamics and human behavior. AI risks – and benefits – can emerge from the interplay of technical aspects combined with societal factors related to how a system is used, its interactions with other AI systems, who operates it, and the social context in which it is deployed.
These risks make AI a uniquely challenging technology to deploy and utilize both for organizations and within society. Without proper controls, AI systems can amplify, perpetuate, or exacerbate inequitable or undesirable outcomes for individuals and communities. With proper controls, AI systems can mitigate and manage inequitable outcomes.[16]
New York University School of Law Professor Sarah Valentine puts a finer point on it: “Helpful as algorithms may be, they inevitably target marginalized populations and exacerbate the social stratification and vast inequality that already exists in our society.”[17]
- Automation of jobs
The rapid progress in AI has transformed how we experience everyday life, including how we communicate, interact, entertain ourselves, travel, transact, and consume media. These developments constitute significant gains for society and promise more benefits in the near future.
But even the most beneficial forms of AI entail trade-offs, the most immediate and obvious of which is job reductions. For instance, a transition to autonomous vehicles in transportation industries, particularly trucking, promises to displace millions of jobs.[18] And automation’s decimation of the workforce is no longer limited to blue collar jobs. Algorithms that use machine learning, including generative AI, and image technology are disrupting white collar jobs, including in medicine, finance, and law.[19] AI expert Kai Fu Lee has stated that 40 percent of the world’s jobs could be automated within the next 15 years.[20] How long before robots begin analyzing legislation?
The social, economic, and political consequences of job reductions present a major challenge for policymakers.[21] The 2016 election was partly determined by a swelling population of disgruntled, economically dislocated individuals. And, as more and more humans do less and less, power and resources will continue to accumulate in the hands of fewer people, potentially exacerbating inequality and political instability. The future may be ruled not by AI, but by the governments or trillionaire oligarchs who control AI.[22] Russian President Vladimir Putin has even warned that “the one who becomes the leader in this sphere will be the ruler of the world.”[23]
- Security threats
AI poses profound security challenges. Some forms of AI are designed to be deployed maliciously, jeopardizing safety and privacy. A report by several AI research organizations describes these threats in chilling detail.[24] One form of malevolent AI are cyberattacks, which include computer viruses capable of disguising themselves, automated hacking programs, the malicious publication or distortion of private information, massive surveillance programs both private and governmental, and the use of speech synthesis and image manipulation for impersonation. A prominent example of this last category is the recently viral images created using Midjourney’s AI image generator, which included AI-generated deep fakes of former President Donald Trump being arrested and Pope Francis sporting stylish coats. Clearly these tools can be used to create widespread misinformation.
AI can also pose a threat to physical security. The Future of Life Institute, which argues that autonomous weapons—weapons capable of engaging targets without human intervention—may be “the third revolution in warfare, after gunpowder and nuclear arms.”[25] AI technology has reached the point where autonomous weapons, such as drones or even swarms of micro-drones, that hunt and kill people who meet predefined criteria, could feasibly be deployed in a matter of years. The development of such technology would almost inevitably precipitate a global arms race. Additionally, threats to physical safety include cyberattacks on physical systems, such as autonomous vehicles, airplanes, pacemakers, or the electrical grid. As AI grows more sophisticated, the magnitude of these threats grows.
Given these risks the Future of Life Institute recently published another open letter, this time calling for a pause on giant AI experiments:
Contemporary AI systems are now becoming human-competitive at general tasks, and we must ask ourselves: Should we let machines flood our information channels with propaganda and untruth? Should we automate away all the jobs, including the fulfilling ones? Should we develop nonhuman minds that might eventually outnumber, outsmart, obsolete and replace us? Should we risk loss of control of our civilization? Such decisions must not be delegated to unelected tech leaders. Powerful AI systems should be developed only once we are confident that their effects will be positive and their risks will be manageable. This confidence must be well justified and increase with the magnitude of a system's potential effects. OpenAI's recent statement regarding artificial general intelligence, states that "At some point, it may be important to get independent review before starting to train future systems, and for the most advanced efforts to agree to limit the rate of growth of compute used for creating new models." We agree. That point is now.
Therefore, we call on all AI labs to immediately pause for at least 6 months the training of AI systems more powerful than GPT-4. This pause should be public and verifiable, and include all key actors. If such a pause cannot be enacted quickly, governments should step in and institute a moratorium.[26]
2. Developing a strategy for dealing with AI: roadmaps and legislative efforts
- Blueprints for effective regulation and oversight
In January of 2017, AI researchers, economists, legal scholars, ethicists, and philosophers met in Asilomar, California to discuss principles for managing the responsible development of AI. The collaboration resulted in the Asilomar Principles. Aspirational rather than prescriptive, these 23 principles were intended to initiate and frame a dialogue by providing direction and guidance for policymakers, researchers, and developers. Its endorsers include 1,200 leading experts in the field of AI, including DeepMind founder Demis Hassabis and the late Stephen Hawking.
The Legislature subsequently adopted ACR 215 (Kiley, Ch. 206, Stats. 2018), which added the State of California to that list by endorsing the Asilomar Principles as guiding values for the development of artificial intelligence and related public policy. In broad strokes, those principles aim to do the following:
- Research issues: create beneficial AI; direct funding toward beneficial innovation; maintain constructive and healthy exchanges between AI researchers and policymakers; promote a culture of trust, cooperation, and transparency among researchers and developers of AI; and avoid corner-cutting on safety standards.
- Ethics and values: promote safety, failure transparency, judicial transparency, and responsible innovation; align human values with innovation; protect privacy and liberty; ensure that the benefits and prosperity created by AI are broadly shared; maintain human control over AI; develop AI that supports rather than subverts social and civil processes; and avoid an AI arms race.
- Longer-term issues: avoid assumptions regarding the capabilities of AI; give AI its due attention; and recognize that its risks are potentially catastrophic or existential.
More recently the Biden Administration has published its Blueprint for an AI Bill of Rights, which is a set of five principles and associated practices to help guide the
design, use, and deployment of AI to protect the rights of the American public:
- Safe and Effective Systems: You should be protected from unsafe or ineffective systems. Automated systems should be developed with consultation from diverse communities, stakeholders, and domain experts to identify concerns, risks, and potential impacts of the system.
- Algorithmic Discrimination Protections: Designers, developers, and deployers of automated systems should take proactive and continuous measures to protect individuals and communities from algorithmic discrimination and to use and design systems in an equitable way. This protection should include proactive equity assessments as part of the system design, use of representative data and protection against proxies for demographic features, ensuring accessibility for people with disabilities in design and development, pre-deployment and ongoing disparity testing and mitigation, and clear organizational oversight.
- Data Privacy: You should be protected from abusive data practices via built-in protections and you should have agency over how data about you is used. You should be protected from violations of privacy through design choices that ensure such protections are included by default, including ensuring that data collection conforms to reasonable expectations and that only data strictly necessary for the specific context is collected. Designers, developers, and deployers of automated systems should seek your permission and respect your decisions regarding collection, use, access, transfer, and deletion of your data in appropriate ways and to the greatest extent possible; where not possible, alternative privacy by design safeguards should be used. Systems should not employ user experience and design decisions that obfuscate user choice or burden users with defaults that are privacy invasive. Consent should only be used to justify collection of data in cases where it can be appropriately and meaningfully given. Any consent requests should be brief, be understandable in plain language, and give you agency over data collection and the specific context of use; current hard-to-understand notice-and-choice practices for broad uses of data should be changed. Enhanced protections and restrictions for data and inferences related to sensitive domains, including health, work, education, criminal justice, and finance, and for data pertaining to youth should put you first. In sensitive domains, your data and related inferences should only be used for necessary functions, and you should be protected by ethical review and use prohibitions. You and your communities should be free from unchecked surveillance; surveillance technologies should be subject to heightened oversight that includes at least pre-deployment assessment of their potential harms and scope limits to protect privacy and civil liberties. Continuous surveillance and monitoring should not be used in education, work, housing, or in other contexts where the use of such surveillance technologies is likely to limit rights, opportunities, or access. Whenever possible, you should have access to reporting that confirms your data decisions have been respected and provides an assessment of the potential impact of surveillance technologies on your rights, opportunities, or access.
- Notice and Explanation: You should know that an automated system is being used and understand how and why it contributes to outcomes that impact you. Designers, developers, and deployers of automated systems should provide generally accessible plain language documentation including clear descriptions of the overall system functioning and the role automation plays, notice that such systems are in use, the individual or organization responsible for the system, and explanations of outcomes that are clear, timely, and accessible. Such notice should be kept up-to-date and people impacted by the system should be notified of significant use case or key functionality changes. You should know how and why an outcome impacting you was determined by an automated system, including when the automated system is not the sole input determining the outcome.
- Human Alternatives, Consideration, and Fallback: You should be able to opt out from automated systems in favor of a human alternative, where appropriate. Appropriateness should be determined based on reasonable expectations in a given context and with a focus on ensuring broad accessibility and protecting the public from especially harmful impacts.[27]
- Legislative attempts
A host of bills in recent years have sought to take initial steps toward meaningful regulation of AI by creating advisory bodies to report to the Legislature.
AB 976 (Chau, 2019) would have established the Artificial Intelligence in State Government Services Commission and required the advisory commission to annually convene a public process to gather input on how artificial intelligence and data science could be used to improve state services. AB 459 (Kiley, 2019) would have required the Commission proposed by AB 976 to report to the Legislature on its recommended minimum standards for the use of AI in state government. AB 1576 (Calderon, 2019) would have provided for the creation of an advisory working group to report on specified issues relating to the use of AI by California-based businesses. SB 348 (Chang, 2019) would have required the Director to devise a strategic plan, as specified, to utilize AI technology to improve state services. Each of these bills died in the Senate or Assembly Appropriations Committees.
AB 594 (Salas, 2019) would have authorized the Director of CDT to designate a position with the department to evaluate the uses of AI in state government and to advise the Director on incorporating AI into state IT strategic plans, policies, standards and enterprise architecture, and requires CDT to adopt guidelines by January 1, 2021 to govern the use and implementation of AI technologies in state government functions, as specified. However, the bill was vetoed by Governor Gavin Newsom, who asserted that the legislation was not necessary to accomplish the goal of examining the impact of AI on work and our economy.
This bill similarly seeks to take a modest approach. It establishes the Office of Artificial Intelligence within CDT. It provides the office with the power and authority necessary to guide the design, use, or deployment of automated systems by a state agency to ensure that all AI systems are designed and deployed in a manner that is consistent with state and federal laws and regulations regarding privacy and civil liberties and that minimizes bias and promotes equitable outcomes for all Californians. This effectuates many of the principles laid out above.
The bill also specifically carries out several core principles of the Blueprint for an AI Bill of Rights, namely Notice and Explanation and Human Alternatives, Consideration, and Fallback. Where state agencies utilize generative AI to directly communicate with a person, the bill requires the agency to clearly and conspicuously identify to that person that their interaction is being communicated through AI. Agencies so deploying such technology are required to provide clear instructions, on their website or through a link thereupon, as to when and how a person can directly communicate with a natural person at the agency.
The author writes:
AI can do a lot and there are certainly benefits, but it can’t be trusted to regulate itself. Like any new industry we’ve seen, safeguards must be added as the risks emerge. Think of the auto industry. Over the years we’ve adopted speed limits, seatbelts and airbags. Now is the time for AI oversight to catch up, increasing consumer protections with the help of proactive, thoughtful policy.
SUPPORT
None known
OPPOSITION
None known
RELATED LEGISLATION
Pending Legislation:
SCR 17 (Dodd, 2023) affirms the California Legislature’s commitment to President Biden’s vision for a safe AI and the principles outlined in the “Blueprint for an AI Bill of Rights” and expresses the Legislature’s commitment to examining and implementing those principles in its legislation and policies related to the use and deployment of automated systems. SCR 17 is currently in the Assembly Privacy and Consumer Protection Committee.
SB 398 (Wahab, 2023) establishes the Government Services Advanced Technology Act, which requires the Department of Justice to develop and implement a comprehensive research plan to study the feasibility of using advanced technology to improve state and local government services. The bill must include an analysis of the potential benefits and risks of using AI to assist disaster victims in finding and applying for disaster relief funds and to assist individuals in determining their eligibility for various public benefits programs. The bill require the department, on or before January 1, 2026, to provide a report to the Legislature on the findings of its research. SB 398 is currently in the Senate Public Safety Committee.
SB 721 (Becker, 2023) creates, until January 1, 2030, the California Interagency AI Working Group made up of members with varied expertise to deliver a report to the Legislature regarding AI. The group is directed to specified tasks, including recommending a definition of AI and determining the relevant agencies to develop and oversee AI policy and implementation of that policy. SB 721 is currently in this Committee and is set to be heard the same day as this bill.
AB 331 (Bauer-Kahan, 2023) requires a deployer of an automated decision tool to perform an impact assessment for any automated decision tool the deployer uses and a developer of an automated decision tool to complete and document an assessment of any automated decision tool that it designs, codes, or produces. It requires deployers and developers to institute certain policies and provide certain disclosures. AB 331 is currently in the Assembly Appropriations Committee.
Prior Legislation:
SB 444 (Umberg, 2019) would have requested the Regents of the University of California to enact, no later than July 1, 2021, a resolution to authorize the university’s Berkeley and Irvine law schools, in consultation with the superior courts of the Counties of Alameda and Orange, respectively, to participate in a research program to develop AI or machine-learning solutions to address issues of access to justice faced by pro se indigent litigants who are residents of those counties. This bill died in the Assembly Higher Education Committee.
SB 348 (Chang, 2019) See Comment 2.
AB 459 (Kiley, 2019) See Comment 2.
AB 594 (Salas, 2019) See Comment 2.
AB 976 (Chau, 2019) See Comment 2.
AB 1576 (Calderon, 2019) See Comment 2.
ACR 215 (Kiley, Ch. 206, Stats. 2018) See Comment 2.
PRIOR VOTES:
Senate Governmental Organization Committee (12 Ayes , 0 Noes)
**************
Kaplan, Artificial Intelligence: What Everyone Needs to Know (2016) p. 1 (internal citations omitted). ↑
DeepMind’s AI became a superhuman chess player in a few hours, just for fun (Dec. 6, 2017) The Verge, https://www.theverge.com/2017/12/6/16741106/deepmind-ai-chess-alphazero-shogi-go. All internet citations are current as of April 7, 2023. ↑
How machine learning works (May 14, 2015) The Economist, https://www.economist.com/the-economist-explains/2015/05/13/how-machine-learning-works. ↑
Id. ↑
Written Statement of Aleksander Mądry for the hearing: Advances in AI: Are We Ready For a Tech Revolution (Mar. 8, 2023) Cybersecurity, Information Technology, and Government Innovation Subcommittee, https://oversight.house.gov/wp-content/uploads/2023/03/madry_written_statement100.pdf. ↑
Intro to AI for Policymakers: Understanding the Shift (Mar. 2018) Brookfield Institute, p. 4, http://brookfieldinstitute.ca/research-analysis/intro-to-ai-for-policymakers/. ↑
Jacob Weisberg, The Digital Poorhouse (June 7, 2018) The New York Review of Books, https://www.nybooks.com/articles/2018/06/07/algorithms-digital-poorhouse/. ↑
AI for Policymakers, supra, at p. 5. ↑
Wolfson, Husky or Wolf? Using a Black Box Learning Model to Avoid Adoption Errors (Aug. 24, 2017) UCI Applied Innovation, Currents, http://innovation.uci.edu/2017/08/husky-or-wolf-using-a-black-box-learning-model-to-avoid-adoption-errors/. ↑
The Digital Poorhouse, supra. ↑
Julia Angwin, et al., Machine Bias (May 23, 2016) ProPublica, https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing. ↑
Id. ↑
Id. ↑
The Digital Poorhouse, supra. ↑
Keith Kirkpatrick, Battling Algorithmic Bias (Oct. 2016) Communications of the ACM Vol. 59, No. 10, pp. 16-17, https://cacm.acm.org/magazines/2016/10/207759-battlingalgorithmic-bias/abstract. ↑
Artificial Intelligence Risk Management Framework (Jan. 2023) NIST, https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.100-1.pdf. ↑
Artificial Intelligence and Predictive Algorithms: Why Big Data Can Lead to Big Problems (2019) 46 Fordham Urb. L.J. 364, 365. ↑
Scott Santens, Self-Driving Trucks Are Going to Hit Us Like a Human-Driven Truck (May, 14, 2015) Medium, https://medium.com/basic-income/self-driving-trucks-are-going-to-hit-us-like-a-human-driven-truck-b8507d9c5961. ↑
Robert Gloy, How AI threatens white-collar jobs (Oct. 8, 2018) Technologist, https://www.technologist.eu/the-threat-to-white-collar-jobs/. ↑
Don Reisinger, AI Expert Says Automation Could Replace 40% of Jobs in 15 Years (Jan. 10, 2019) Fortune, http://fortune.com/2019/01/10/automation-replace-jobs/. ↑
AI also raises novel ethical questions. For instance, autonomous vehicles may calculate life-or-death risks differently than humans. In a variant of the classic thought experiment known as the “trolley problem,” we can expect scenarios in which an automated vehicle, in the split second before an unavoidable collision occurs, must decide among human lives. “Of course, it isn’t the car making the decision. The software engineers are making it, cosseted in their dim engineering warrens. They will play God.” Ethicists Will Solve the Robocar ‘Trolley Problem’ (May 28, 2017) Wired, https://www.wired.com/2017/05/autonomous-vehicles-trolley-problem/. ↑
Yuval Noah Harari, Why Technology Favors Tyranny (Oct. 2018) The Atlantic, https://www.theatlantic.com/magazine/archive/2018/10/yuval-noah-harari-technology-tyranny/568330/. ↑
Putin: Leader in artificial intelligence will rule world (Sep. 4, 2017) CNBC, https://www.cnbc.com/2017/09/04/putin-leader-in-artificial-intelligence-will-rule-world.html. ↑
See Miles Brundage, et al., The Malicious Use of Artificial Intelligence: Forecasting, Prevention, and Mitigation (Feb. 2018) pp. 23-30, https://img1.wsimg.com/blobby/go/3d82daa4-97fe-4096-9c6b-376b92c619de/downloads/1c6q2kc4v_50335.pdf. ↑
Future of Life Institute, Autonomous weapons: an open letter from AI & robotics researchers (Feb. 9, 2016) https://futureoflife.org/open-letter-autonomous-weapons/?cn-reloaded=1. ↑
Future of Life Institute, Pause Giant AI Experiments: An Open Letter (2023) https://futureoflife.org/open-letter/pause-giant-ai-experiments/. Signatories to the letter include Yoshua Bengio, Founder and Scientific Director at Mila, Turing Prize winner and professor at University of Montreal; Stuart Russell, Berkeley, Professor of Computer Science, director of the Center for Intelligent Systems, and co-author of the standard textbook “Artificial Intelligence: a Modern Approach"; Elon Musk, CEO of SpaceX, Tesla & Twitter; and Steve Wozniak, Co-founder, Apple. ↑
Blueprint For An AI Bill Of Rights (Oct. 2022) Office of Science and Technology Policy, https://www.whitehouse.gov/wp-content/uploads/2022/10/Blueprint-for-an-AI-Bill-of-Rights.pdf. ↑
SENATE COMMITTEE ON GOVERNMENTAL ORGANIZATION
Senator Bill Dodd
Chair
2023 - 2024 Regular
Bill No: SB Hearing Date: 4/11/2023
Author: | Dodd | ||
Version: | Amended | ||
Urgency: | Fiscal: | ||
Consultant: | |||
SUBJECT: . Intelligence: state agency public interface: use of AI.
DIGEST: This bill establishes the Office of Artificial Intelligence (AI) and enacts the California AI-ware Act, requiring any generative AI engaging in direct communication with a natural person on behalf of a state agency to clearly and prominently disclose to the individual that the interaction is being facilitated through an AI system.
ANALYSIS:
Existing law:
- Establishes, within the Government Operations Agency (GovOps), the California Department of Technology (CDT) to provide technology direction to agency and department chief information officers to ensure the integration of statewide technology initiatives, compliance with information technology policies and standards, and the promotion of the alignment and effective management of information technology (IT) services, as specified.
- Requires the Secretary of GovOps to evaluate, among other things, the impact the proliferation of deepfakes has and risks, including privacy risks, associated with the deployment of digital content forgery technologies and deepfakes on government, businesses, and residents of the state.
- Establishes the Office of Wildfire Technology Research and Development, within the Department of Forestry and Fire Protection (CalFIRE), to study, test, and advise regarding procurement of emerging technologies and tools in order to more effectively prevent and suppress wildfires within the state. The office serves as the central organizing hub for the state government’s identification of emerging wildfire technologies.
This bill:
- Establishes the Office of AI, within CDT, and provides the office with the powers and authorities necessary to guide the design, use, or deployment of automated systems by a state agency to ensure that all AI systems are designed and deployed in a manner that is consistent with state and federal laws and regulations regarding privacy and civil liberties and that minimizes bias and promotes equitable outcomes for all Californians.
- Requires any state agency that utilizes generative AI to directly communicate with a person, either through an online interface or telephonically, to clearly and in a conspicuous manner identify to that person that their interaction with the state agency is being communicated through AI.
- Requires any state agency that utilizes generative AI to directly communicate with a person to provide on the agency’s internet website clear instructions, or a link to a page with clear instructions, informing the person when and how they can directly communicate with a natural person from the state agency.
- Includes Legislative findings and declarations relating to the tremendous potential of AI and that the use of AI must be guided by principles of fairness, transparency, and accountability, among other things.
Background
Purpose of the Bill. According to the author’s office, “AI can do a lot and there are certainly benefits, but it can’t be trusted to regulate itself. Like any new technology we’ve seen, safeguards must be added as the risks emerge. Think of the auto industry. Over the years we’ve adopted speed limits, seatbelts, and airbags. Now is the time for AI oversight to catch up, increasing protections for people with the help of proactive, thoughtful policy.”
Further, the authors office notes that, “this bill not only establishes the Office of AI in state government but also requires state agencies to ensure transparency and human alternatives in their AI interactions with the public. This bill promotes transparency, equity, and fairness, all while maximizing the benefits AI can provide to our communities. By staying 'AI-ware,' we can create a future where AI systems are designed, deployed, and governed responsibly, prioritizing the well-being of all people.”
Artificial Intelligence. AI refers to the development of computer systems that can perform tasks typically requiring human intelligence. These tasks include learning, reasoning, problem-solving, perception, and natural language understanding. AI enables computers to learn from data and experiences, allowing them to handle complex tasks without explicit programming. AI has the potential to revolutionize numerous sectors, including healthcare, finance, and transportation, by improving efficiency, reducing costs, and enhancing decision-making capabilities.
One of the most advanced and well-known AI systems, GPT-4, was developed by OpenAI and released on Pi Day (3/14), 2023. GPT-4 has demonstrated remarkable capabilities compared to its predecessor, GPT-3.5, which had been released to the world just six months prior. For example, GPT-4 scored around the top 10% on a simulated bar exam, while GPT-3.5 scored around the bottom 10%.
GPT-4 also scored a five out of five on several Advanced Placement (AP) exams, including Art History, Biology, Environmental Science, Macro- and Microeconomics, and Psychology, showcasing a knowledge across a diverse range of subjects. GPT-4 is particularly impressive at reasoning, scoring in the 93rd percentile on the SAT Evidence-Based Reading & Writing exam and in the 99th percentile on the Graduate Record Examination (GRE) Verbal exam.
The remarkable advancements in AI, exemplified by GPT-4's impressive performance on various professional and academic benchmarks, showcase its potential for practical applications in diverse fields. From law and education to science and technology, AI's ability to understand complex tasks and generate nuanced responses could lead to transformative solutions and new insights across multiple disciplines.
There are different types of AI, such as rule-based systems to make decisions or carry out tasks, and machine learning systems, which learn from data and experience to improve their performance over time. Deep learning, a type of machine learning, utilizes neural networks that mimic the structure and function of the human brain. Neural networks are used in deep learning to process large amounts of data and learn to recognize patterns, make predictions, and perform other tasks with a high degree of accuracy.
AI is already being used in a wide range of applications, including virtual assistants, image and speech recognition, fraud detection, predictive analytics, and autonomous vehicles. AI excels at fraud detection and predictive analytics due to its ability to rapidly and accurately analyze large datasets, detect patterns and anomalies challenging for humans to discern, and learn from historical data to predict future events. By detecting potential fraudulent activity and predicting future outcomes, AI can help businesses and governments make more informed decisions, save time and resources, and prevent financial losses.
While AI has the potential to revolutionize government services and decision-making, it also presents several challenges and concerns. One major issue is the potential for bias in AI systems. AI algorithms are only as unbiased as the data they are trained on, and if that data is biased or incomplete, it can result in discriminatory outcomes. For example, if an AI system is used to assess job applicants, it may inadvertently discriminate against certain groups if the data used to train the system is biased towards certain demographics. This could perpetuate existing inequalities in the workplace. To minimize bias in AI systems, it is crucial to ensure that the data used to train these systems is representative of the target population.
Another concern is the potential for privacy violations. AI systems can collect and process vast amounts of personal data, and if not properly secured, that data could be vulnerable to misuse or hacking. This is particularly concerning in government contexts, where sensitive information such as criminal records, medical histories, and financial information is often collected and stored. Additionally, there is concern that AI could displace jobs in certain sectors, particularly those that rely heavily on repetitive or routine tasks. While AI has the potential to automate many tasks and increase efficiency, it could also lead to significant job losses and create new inequalities in the workforce.
Further, there is the risk of unintended consequences. AI systems are complex and difficult to fully understand, and it is possible that they could produce unexpected or unintended outcomes. This is particularly concerning in government contexts where decisions made by AI systems could have far-reaching consequences for individuals and communities. For example, if an AI system is used to determine eligibility for social services, errors or biases in the system could result in individuals being denied important benefits. As such, it is important that AI is developed and implemented in a responsible and ethical manner, with proper safeguards and oversight in place to mitigate these risks.
Global AI Competition. In September of 2017 Russian President Vladimir Putin warned about AI stating that, “the one who becomes the leader in this sphere will be the ruler of the world.” A recent report from the Australian Strategic Policy Institute (ASPI) found that the United States comes in second in the world in the majority of critical technology research areas examined, like AI, robotics, biotechnology, and quantum technology – and specifically in defense and space-related technologies. The report uses ASPI’s new “Critical Technology Tracker,” a tool that lets users track 44 technologies considered “foundational” for national security, economies, and more.
It is widely agreed that AI will play a significant role in shaping the future of technology, economics, and global politics. Countries that successfully develop and implement advanced AI technologies may gain a competitive edge over others in various fields, such as military, healthcare, and commerce. However, it's important to recognize that AI development is a global effort, and cooperation between nations will be essential to address the ethical, legal, and societal challenges that AI presents. While AI will likely be a key factor in determining global leadership, it's too simplistic to say that whoever controls AI will control the world.
The Future of Life Institute Open Letter. The Future of Life Institute (FLI) – a nonprofit organization founded to reduce global catastrophic and existential risks facing humanity, particularly existential risk from advanced AI – recently published an open letter titled “Pause Giant AI Experiments: An Open Letter.” The letter, released in March of 2023, highlights the rapid advancements in AI and the potential consequences of deploying powerful AI systems without adequate understanding and control.
The FLI writes that, “[c]ontemporary AI systems are now becoming human-competitive at general tasks, and we must ask ourselves: Should we let machines flood our information channels with propaganda and untruth? Should we automate away all the jobs, including the fulfilling ones? Should we develop nonhuman minds that might eventually outnumber, outsmart, obsolete and replace us? Should we risk loss of control of our civilization? Such decisions must not be delegated to unelected tech leaders.”
Signatories include Apple Co-Founder Steve Wozniak; SpaceX, Tesla, and Twitter CEO Elon Musk; Stability AI CEO Emad Mostaque; Executive Director of the Center for Humane Technology Tristan Harris; and Yoshua Bengio, Mila AI research institute founder. The letter calls for a six-month pause on AI experiments, stating that powerful AI systems should only be developed when we are confident in their positive effects and manageable risks.
AI in Government Services. AI is already being used in various ways within California government, including in the areas of transportation, healthcare, public safety, natural resource management, and emergency preparation and response. For instance, in 2020, the University of California, San Diego (UCSD) developed an AI-powered tool that can analyze chest X-rays to detect potential cases of COVID-19. This tool helped healthcare professionals quickly identify patients who needed immediate care, leading to more efficient allocation of medical resources during the pandemic.
In 2019, the number of traffic fatalities in the United States was 36,096 which is more than 11.17 deaths per 100,000 population. These numbers underscore the critical need to improve traffic, pedestrian, and bicyclist safety. The AI Pedestrian Traffic Safety System – tested on 12 hours of traffic videos captured from existing traffic cameras in the city of Los Angeles – demonstrated more than 98% accuracy in counting pedestrians in real time. This project occurred as part of LA’s public commitment to the global initiative Vision Zero, which endeavors to reduce severe injuries and deaths in roadway collisions. Los Angeles’s specific Vision Zero goal is to eliminate all traffic deaths and serious injuries by 2025. At full scale, the system can cover all 56,000 miles of LA streets and in real time.
After the 2018 Camp Fire in California, AI technology was used to analyze satellite imagery and assess the extent of the damage caused by the fire. This information helped emergency responders prioritize their efforts, allocate resources more efficiently, and speed up the recovery process for affected communities.
However, it is important to acknowledge that there are potential risks and negative consequences associated with AI. Bias, privacy violations, job displacement, and unintended consequences are some of the concerns associated with AI. These issues must be addressed and mitigated to ensure that the benefits of AI are maximized while minimizing any potential risks.
In 2016, a study by ProPublica revealed that a widely used AI-based risk assessment tool, COMPAS, was biased against African-American defendants in predicting their likelihood of reoffending. The algorithm was found to be more likely to falsely flag black defendants as high-risk and white defendants as low-risk, perpetuating existing racial biases in the criminal justice system.
Blueprint for an AI Bill of Rights. Among the great challenges posed to democracy today is the use of technology, data, and automated systems in ways that threaten the rights of the American public. In late 2022, the White House’s Office of Science and Technology Policy (OSTP) released its Blueprint for an AI Bill of Rights, which is a framework proposed by the OSTP to protect the American public in the age of artificial intelligence. It aims to address the challenges and threats posed by technology, data, and automated systems, while preserving civil rights and democratic values. The blueprint presents five key principles that should guide the design, use, and deployment of automated systems:
- Safe and Effective Systems: People should be protected from unsafe or ineffective systems. This involves consultation with diverse communities, pre-deployment testing, risk identification and mitigation, ongoing monitoring, and independent evaluation and reporting.
- Algorithmic Discrimination Protections: Individuals should not face discrimination by algorithms, and systems should be designed equitably. Proactive measures should be taken to prevent algorithmic discrimination, including proactive equity assessments, representative data usage, accessibility, pre-deployment disparity testing, and clear organizational oversight.
- Data Privacy: People should be protected from abusive data practices and have agency over how their data is used. This includes built-in privacy protections, informed consent, limited data collection, restrictions on sensitive data usage, and oversight for surveillance technologies.
- Notice and Explanation: Users should be aware that an automated system is being used and understand its impact on them. This involves providing plain language documentation, timely and accessible explanations of outcomes, and keeping users informed about significant changes.
- Human Alternatives, Consideration, and Fallback: Individuals should be able to opt out of automated systems when appropriate, and have access to human alternatives for remedy and decision-making. This includes timely human consideration, effective fallback processes, and tailored systems within sensitive domains.
These principles aim to ensure that AI and automated systems are used responsibly, protecting individual rights and maintaining democratic values.
Deepfakes. Deepfakes. The term “deepfake” refers to a specific kind of synthetic media where a person in an image or video is swapped with another person’s likeness. These deceptive life-like videos and recordings can effectively make it appear as though an individual said or did something that never actually took place.
Deepfakes take their name from the fact that they use deep learning technology to create fake videos. Deep learning technology is a kind of machine learning that applies neural net simulation to massive data sets. Artificial intelligence effectively learns what a particular face looks like at various angles in order to transpose the face onto another individual, similar to an extremely high quality mask that automatically adapts as the individual moves.
SB 1216 (Gonzalez, Chapter 885, Statutes of 2022) requires, upon appropriation by the Legislature, the Secretary of GovOps to evaluate, among other things, the impact the proliferation of deepfakes has on government, businesses, and residents of the state, as specified.
California AI-ware Act. The California AI-ware Act requires state agencies to clearly disclose when they are using AI systems in their interactions with the public, promoting transparency and fostering trust between the government and its citizens. This bill ensures that users have the option to seek human assistance, when available, if they feel the need to do so. By focusing on transparency and access to human alternatives, the California AI-ware Act aims to ensure that AI systems employed by state government align with the principles outlined in the Blueprint for an AI Bill of Rights, ultimately protecting the rights and interests of Californians who interact with these systems.
With AI technology becoming more sophisticated and capable of emulating human-like interactions, it is important for individuals to be aware of whether they are communicating with an AI or a human representative. This awareness allows people to make informed decisions and evaluate the information they receive during these interactions. By requiring state agencies to clearly identify when AI systems are being used, the act promotes transparency and sets the stage for building trust between the government and the public.
There are numerous potential use cases for state government applications. For instance, AI systems could be used to streamline and automate customer service inquiries for various departments, such as the Department of Motor Vehicles or the Employment Development Department. By handling routine inquiries, AI systems can help reduce wait times and improve the efficiency of these departments, ultimately benefiting the public. However, the California AI-ware Act ensures that users are aware of their interaction with AI systems and provides them with the option to seek human assistance if needed.
California Office of AI. This bill establishes the Office of AI as the centralized authority within the CDT to oversee and guide the design, use, and deployment of AI systems by state agencies. This part of the bill seeks to ensure that AI systems used by the government align with principles of fairness, transparency, and accountability, and uphold the rights and opportunities of all Californians.
The Office of AI is intended to serve as a regulatory body that ensures AI systems are developed and deployed in accordance with state and federal laws and regulations regarding privacy and civil liberties. Furthermore, the office is tasked with minimizing bias and promoting equitable outcomes for all Californians. This bill shares similar goals and principles to the Blueprint for an AI Bill of Rights, particularly in areas such as Algorithmic Discrimination Protections, Data Privacy, and Notice and Explanation.
In terms of future use case scenarios, the Office of AI can play a crucial role in various aspects of government operations involving AI. For instance, the office can provide guidance and oversight for AI systems used in areas such as criminal justice, healthcare, education, and environmental management. By ensuring that these AI systems are designed and deployed ethically, the office can help prevent biased decision-making, protect privacy, and promote transparency.
Another potential use case for the Office of AI involves setting standards and best practices for AI development and deployment across different state agencies. By establishing a common framework for responsible AI use, the office can help create a cohesive approach to AI implementation within the state government. This can lead to more effective and efficient use of AI technology, benefiting both the government and the citizens it serves.
Additionally, the Office of AI can serve as a hub for collaboration between state agencies, AI developers, and other stakeholders. By fostering dialogue and cooperation, the office can help ensure that AI systems are designed and deployed with a thorough understanding of the potential risks and benefits, as well as the unique needs and concerns of the communities they serve.
The creation of the Office of AI as part of the AI-ware Act seeks to establish a centralized authority to guide and oversee the design, use, and deployment of AI systems by state agencies. The office aims to promote principles of fairness, transparency, and accountability, which are aligned with the values outlined in the Blueprint for an AI Bill of Rights. By focusing on ethical AI development and deployment, the Office of AI can help ensure that AI technology is used in ways that benefit all Californians and uphold their rights and opportunities.
Prior/Related Legislation
SB 398 (Wahab, 2023) would require DOJ to develop and implement a comprehensive research plan to study the feasibility of using advanced technology to improve state and local government services, as specified. (Pending in the Senate Public Safety Committee)
SB 721 (Becker, 2023) establishes the California Interagency AI Working Group consisting of 11 members with expertise in computer science, AI, technology, workforce development, and data privacy to report to the Legislature regarding the implications of AI by January 1, 2025, and every two years thereafter. (Pending in the Senate Governmental Organization Committee)
SCR 17 (Dodd, 2023) affirms the California Legislature’s commitment to President Biden’s vision for a safe AI and the principles outlined in the “Blueprint for an AI Bill of Rights” and expresses the Legislature’s commitment to examining and implementing those principles in its legislation and policies related to the use and deployment of automated systems. (Pending in the Assembly Privacy and Consumer Protection Committee)
AB 302 (Ward, 2023) requires CDT to conduct an inventory of all high-risk automated decision systems being used in state agencies, as specified. (Pending in the Assembly Appropriations Committee)
AB 331 (Bauer-Kahan, 2023) requires a deployer and a developer of an automated decision tool to perform an impact assessment for any such tool the deployer uses that includes, among other things, a statement of the purpose of the automated decision tool and its intended benefits, as specified. (Pending in the Assembly Privacy and Consumer Protection Committee)
AB 642 (Ting, 2023) sets minimum standards for use of facial recognition technology by law enforcement, including requiring law enforcement agencies to have a written policy, as specified. (Pending in the Assembly Public Safety Committee)
AB 768 (Boerner Horvath, 2023) establishes the Office of Broadband and Digital Equity to serve as a centralized state office for broadband and digital activities within the state and to develop and implement policies to promote broadband service and to protect consumers from unfair business practices relating to broadband services, as specified. (Pending in the Assembly Communications and Conveyances Committee)
SB 1216 (Gonzalez, Chapter 885, Statutes of 2022) requires, upon appropriation by the Legislature, the Secretary of GovOps to evaluate, among other things, the impact the proliferation of deepfakes has on government, businesses, and residents of the state, as specified.
SB 109 (Dodd, Chapter, Statutes of 2021) established the Office of Wildfire Technology Research and Development and the corresponding Advisory Board to conduct research and testing on emerging technologies to prevent, predict, and fight wildfires.
FISCAL EFFECT: Appropriation: No Fiscal Com.: Yes Local: No
SUPPORT:
Little Hoover Commission
OPPOSITION:
None received
ARGUMENTS IN SUPPORT: The Little Hoover Commission writes in support of the bill that, “[i]n its 2018 report, Artificial Intelligence: A Roadmap for California, the Commission recommended the State appoint an AI special advisor that would create a strategic plan to incorporate AI within the state government and oversee the safe and transparent deployment of AI. The Commission also emphasized the need for an AI policy making agenda that would protect core values – autonomy, responsibility, privacy, transparency, and accountability. SB 313 would advance the goals of the Commission’s recommendations. For this reason, the Little Hoover Commission supports this bill.”
DUAL REFERRAL: Senate Governmental Organization Committee & Senate Judiciary Committee
Discussed in Hearing
Senate Standing Committee on Judiciary
Senate Standing Committee on Governmental Organization
Bill Author
Bill Co-Author(s):