AB 1576: Secretary of Government Operations: working group: technology.
- Session Year: 2019-2020
- House: Assembly
- Latest Version Date: 2019-06-28
|
Date
|
Status
|
|---|---|
| Aug 30, 2019 | In committee: Held under submission. |
| Aug 12, 2019 | In committee: Referred to APPR. suspense file. |
| Jul 10, 2019 | From committee: Do pass and re-refer to Com. on APPR. with recommendation: To Consent Calendar. (Ayes 8. Noes 0.) (July 9). Re-referred to Com. on APPR. |
| Jun 28, 2019 | From committee chair, with author's amendments: Amend, and re-refer to committee. Read second time, amended, and re-referred to Com. on JUD. |
| Jun 25, 2019 | From committee: Do pass and re-refer to Com. on JUD. (Ayes 16. Noes 0.) (June 25). Re-referred to Com. on JUD. |
| Jun 6, 2019 | Referred to Coms. on G.O. and JUD. |
| May 24, 2019 | In Senate. Read first time. To Com. on RLS. for assignment. |
| May 23, 2019 | Read third time. Passed. Ordered to the Senate. |
| May 20, 2019 | Read second time. Ordered to third reading. |
| May 16, 2019 | From committee: Do pass. (Ayes 18. Noes 0.) (May 16). |
| May 16, 2019 | Joint Rule 62(a), file notice suspended. |
| May 15, 2019 | In committee: Set, first hearing. Referred to APPR. suspense file. |
| May 13, 2019 | Joint Rule 62(a), file notice suspended. (Page 1609.) |
| May 1, 2019 | Re-referred to Com. on APPR. |
| Apr 30, 2019 | Read second time and amended. |
| Apr 29, 2019 | From committee: Amend, and do pass as amended and re-refer to Com. on APPR. with recommendation: To Consent Calendar. (Ayes 11. Noes 0.) (April 23). |
| Apr 1, 2019 | Re-referred to Com. on P. & C.P. |
| Mar 28, 2019 | Referred to Com. on P. & C.P. |
| Feb 25, 2019 | Read first time. |
| Feb 23, 2019 | From printer. May be heard in committee March 25. |
| Feb 22, 2019 | Introduced. To print. |
|
Date
|
Description
|
Location
|
Votes
|
|
|---|---|---|---|---|
| Aug 12, 2019 | Placed on suspense file | Senate Standing Committee on Appropriations |
Passed |
|
| Jul 9, 2019 | Do pass, but first be re-referred to the Committee on [Appropriations] with the recommendation: To Consent Calendar | Senate Standing Committee on Judiciary |
Passed |
|
| Jun 25, 2019 | Do pass, but first be re-referred to the Committee on [Judiciary] | Senate Standing Committee on Governmental Organization |
Passed |
|
| May 23, 2019 | AB 1576 Calderon Assembly Third Reading | Assembly Floor |
Passed |
|
| May 16, 2019 | Do pass. | Assembly Standing Committee on Appropriations |
Passed |
|
| Apr 23, 2019 | Do pass as amended, and be re-referred to the Committee on [Appropriations] with recommendation: To Consent Calendar | Assembly Standing Committee on Privacy and Consumer Protection |
Passed |
This list shows positions taken on this bill by organizations on the date displayed, which is linked to the original source.
Positions by individuals are not included.
SENATE COMMITTEE ON
Senator Anthony Portantino, Chair
2019 - 2020 Regular Session
AB 1576 (Calderon) - Secretary of Government Operations: working group: technology
Version: | Policy Vote: |
Urgency: | Mandate: |
Hearing Date: | Consultant: |
Bill Summary:
AB 1576 would require, by July 1, 2020, the Secretary of Government Operations to appoint participants to an artificial intelligence (AI) working group, as specified, to evaluate the uses, risks, benefits, and legal implications associated with the development and deployment of artificial intelligence by California-based businesses.
Fiscal Impact:
- One-time costs of $270,000 to the Government Operations Agency (GovOps) for increased limited term personnel, operating expenses, and working group travel reimbursement. (GF)
- Unknown, likely significant, cost pressure to implement suggested recommendations reported to the Legislature. Costs will depend on the actual recommendations of the working group.
Background:
Artificial Intelligence (AI) is a wide-ranging branch of computer science focused on building smart machines capable of performing tasks that typically require human intelligence. AI makes it possible for computers to learn from experience, adjust to new inputs, and perform “human-like” tasks. Some of the most well-known AI technologies include image recognition software, Siri, Alexa, Watson, and self-driving cars.
AI technology can also be used across a wide array of sectors including the public sector. For example, at the federal level, the Department of Veteran Affairs is using AI to better predict medical complications and improve treatment of combat wounds. Some cities are also beginning to leverage the type of responsive dispatching and routing used by ride-hailing services, and linking it with scheduling and tracking software for public transportation.
Because AI depends solely on data input, there are concerns that if the data used reflects past decisions that are biased, the results could also perpetuate that bias. The use of AI technology also leads to concerns from workers that their jobs could be lost to computers that utilize AI technology. In 2018, the Little Hoover Commission released a report titled, “Artificial Intelligence: A Roadmap for California” which detailed the possible benefits and risks of AI. The report encouraged the Governor and the Legislature to: engage the public in broad conversations about the responsible use of AI; identify the necessary training and other educational opportunities to build the human infrastructure required for an AI-fueled economy; explore opportunities to form and leverage public private partnerships to use AI to address societal challenges; and examine legislative and policy strategies to preserve and protect key public values, such as privacy, transparency and accountability.
Proposed Law:
This bill would require the Secretary of GovOps, by July 1, 2020, to appoint participants to an artificial intelligence technology working group and shall designate the chairperson of that group. Specifically, the bill would:
- Require the composition of the working group to be representative of California’s demographic and cultural diversity and shall serve in an advisory capacity and specify the working group participants;
- Require the working group to evaluate the uses, risks, benefits, and legal implications associated with the development and deployment of artificial intelligence by California-based businesses; and recommend best practices for the development and deployment of artificial intelligence by California-based businesses, as specified;
- Require the working group to take input from a broad range of stakeholders with a diverse range of backgrounds and interests affected by state policies governing emerging technologies, including AI technology;
- Require, by July 1, 2021, the working group to report to the Legislature on their evaluations and recommendations, as specified;
- Provide that members of the working group shall serve without compensation, but shall be reimbursed for all necessary expenses actually incurred in the performance of their duties;
- Sunset on January 1, 2023.
Related Legislation:
SB 348 (Chang, 2019) would require CDT to devise a strategic plan designed to aid departments and agencies with the incorporation of artificial intelligence into state information technology strategic plans, policies, standards, and enterprise architecture to improve state services. The measure is pending in the Assembly Appropriations Committee.
SB 730 (Stern, 2019) would create the Commission on the Future of Work and require the Commission to conduct research to understand the impact of technology on workers, employers, and the economy of the state. The measure is pending in the Assembly Appropriations Committee.
SB 752 (Stern, 2019) would have created the Commission on Tech Equity to convene a public process to gather input and to understand the economic, social, workplace, and technological landscape of innovation and technology in California. The measure was held in the Senate Appropriations Committee.
AB 459 (Kiley, 2019) would have directed the AI Commission (proposed in AB 976 (Chau, 2019)), to report its recommended minimum standards for the use of AI in state government to the Legislature. The measure was held in the Assembly Appropriations Committee.
AB 594 (Salas, 2019) would require by January 1, 2021, the Director of Technology to appoint a Chief AI Officer within the CDT to evaluate the uses of artificial intelligence in state government and to advise the Director on incorporating artificial intelligence into state information technology strategic plans, policies, standards, and enterprise architecture. The measure is pending in the Senate Appropriations Committee.
AB 976 (Chau, 2019) would establish the Artificial Intelligence in State Government Services Commission to gather input on how artificial intelligence and data science could be used to improve state services. The measure is pending in the Senate Appropriations Committee.
Staff Comments:
Staff notes that this bill is one of a handful of bills related to AI technology in this Legislative Session. SB 730 (Stern, 2019) would establish the Commission on Tech Equity to conduct research to understand the impact of technology on workers, employers, and the economy of the state. AB 594 (Salas, 2019) would require the California Department of Technology to appoint a Chief AI Officer within the department to evaluate the uses of AI in state government and to advise the department on incorporating AI into strategic plans. AB 976 (Chau, 2019) would establish the AI Commission and require the Commission to provide a plan for soliciting and incorporating AI and data science related demonstration projects into state services. While all these bills are not focused on the same aspects of AI technology, many of them would require duplicative work and could result in inconsistent recommendations to the Legislature.
-- END --
Senator Hannah-Beth Jackson, Chair
2019-2020 Regular Session
Bill No: AB 1576 (Author:Calderon)
Version: June 28, 2019
Hearing Date: July 9, 2019
Fiscal: Yes
Urgency: No
JTConsultant:
SUBJECT
Secretary of Government Operations: working group: technology
DIGEST
This bill provides for the creation of an advisory working group to report on specified issues relating to the use of artificial intelligence (AI) by California-based businesses.
EXECUTIVE SUMMARY
Owing to recent advances in processing power and the rise of big data, AI’s capacity and the scope of its applications have expanded rapidly, impacting how we communicate, interact, entertain ourselves, travel, transact, and consume media. In ways we may not fully comprehend, AI empowers and encumbers us. It has been used to accelerate productivity, achieve efficiencies, liberate us from drudgery, help us understand and enjoy the world, connect with each other, and live longer, fuller lives. It has also been used to constrain personal autonomy, compromise privacy and security, foment social upheaval, exacerbate inequality, spread misinformation, and subvert democracy. For good or ill, its transformative potential seems boundless.
California does not regulate AI as such, but several laws address some of its most prominent applications, including autonomous vehicles, social media, bots, fake news, drones, privacy, and job displacement. These efforts offer creative approaches to novel problems, but the rapid proliferation and breadth of AI’s manifestations underscores the challenge of regulation in this field and the need for principled governance of AI.
This bill would require the Secretary of Government Operations to appoint an advisory AI working group to report, by July 1, 2021, to the Legislature on the potential uses, risks, benefits, and legal implications of the use of AI by California-based businesses, and to recommend best practices regarding specified issues. The bill has no known opposition. The Senate Governmental Organization Committee passed this bill 16-0.
PROPOSED CHANGES TO THE LAW
Existing law:
- Establishes the California Department of Technology within the Government Operations Agency. (Gov. Code § 11545(a)(1).) Vests the power of appointing the Director of Technology in the Governor, subject to Senate confirmation. (Id.)
- Provides that the duties of the Director of Technology include:
- advising the Governor on the strategic management and direction of the state’s information technology (IT) resources;
- establishing and enforcing state IT strategic plans, policies, standards, and enterprise architecture, as specified;
- minimizing overlap, redundancy, and cost in state IT operations by promoting the efficient and effective use of information technology;
- providing technology direction to agency and department chief information officers to ensure the integration of statewide technology initiatives, compliance with IT policies and standards, and the promotion of the alignment and effective management of IT services;
- working to improve organizational maturity and capacity in the effective management of IT; and
- establishing performance management and improvement processes to ensure state IT systems and services are efficient and effective. (Id. at (b).)
This bill:
- States that it is the intent of the Legislature to analyze the impacts of AI on California, giving consideration to ethical use of the technology as it becomes more advanced. Further states that:
- AI will have profound implications for economic prosperity, social stability, and human well-being;
- it is therefore critical to develop strategies to foster AI’s development in a manner that maximizes its benefits to society; and
- as the drivers of innovation, California’s businesses will play a paramount role in this process and are in the unique position to provide crucial insights and recommendations as California formulates a strategy for promoting AI innovation that is ethical, responsible, and beneficial.
- Requires the Secretary of Government Operations to appoint an AI technology working group and designate the chairperson of that group on or before July 1, 2020. Requires the composition of the working group to be representative of California’s demographic and cultural diversity. Provides that this working group serves in an advisory capacity.
- Requires the working group to evaluate the uses, risks, including privacy risks, benefits, and legal implications associated with AI used by California-based businesses. Additionally requires the working group to recommend best practices for the development and deployment of AI by California-based businesses to foster all of the following:
- ethical and responsible innovation;
- identification and elimination of bias in algorithms;
- transparency and accountability;
- safety and security in the operation of artificial intelligence systems;
- benefits that are enjoyed by the full spectrum of social and economic classes;
- protection of consumer and civil rights, including privacy and liberty;
- creation of job and training opportunities to mitigate the possibility of job displacement;
- alignment of human values with innovation; and
- the support of social and civil processes.
- Requires the working group to take input from a broad range of stakeholders with a diverse range of interests and backgrounds affected by state policies governing emerging technologies, including AI.
- Requires that the working group consist of participants from all of the following:
- three appointees from the technology industry, including:
- one appointee with a background in ethics and AI;
- one appointee with expertise in machine learning; and
- one appointee with expertise in algorithms;
- three appointees from nontechnology-related industries, including one appointee from a nonprofit;
- three appointees with a background in law, including one from academia, chosen in consultation with the Judicial Council;
- two appointees representing privacy protection organizations;
- two appointees representing consumer protection organizations;
- one appointee with a labor or workforce development background;
- the Director of Technology, or their designee;
- the Director of Finance, or their designee;
- the chief information officers of three other state agencies; and
- one Member of the Senate, appointed by the Senate Committee on Rules, and one Member of the Assembly, appointed by the Speaker of the Assembly.
- three appointees from the technology industry, including:
- Requires the working group, by July 1, 2021, to report to on its evaluations and recommendations made pursuant to 3), above. Requires the working group to recommend modifications for the definition of AI in 7), below.
- Defines AI as a collection of different technologies that allow smart machines to extend human capabilities and intelligence by sensing, comprehending, acting, and learning, thereby allowing humans to achieve much more than can be achieved without the technology.
- Sunset January 1, 2023.
COMMENTS
1. The need for principled governance of AI
- What is AI?
Intelligence is an amorphous concept. Broadly, a thing can be considered intelligent if it processes information and solves problems in uncertain conditions. Artificial intelligence, then, is intelligence exhibited by the things we build. The term also refers to a field of study pioneered by Dartmouth professor John McCarthy in the 1950s, who described the endeavor of developing AI as “that of making a machine behave in ways that would be called intelligent if a human were so behaving.”[1]
In certain narrow fields, computers have far surpassed human capabilities. Some of the most dramatic milestones in AI’s advancement have come when computer programs have bested humans at games, prompting varied reactions in their human counterparts. When IBM’s Deep Blue defeated World Chess Champion Gary Kasparov in 1997 in their decisive match, Kasparov insisted that a human grand master must have been controlling the machine. Better sportsmanship was displayed by Jeopardy legend Ken Jennings upon realizing he was certain to lose to IBM’s Watson in 2011. On his Final Jeopardy question video screen, Jennings wrote: “I, for one, welcome our new computer overlords.” By contrast, when the world’s best Go player, a Chinese national, succumbed to Google DeepMind’s AlphaGo, the Chinese government blocked the live-stream.
This most recent illustration of AI’s ascendance is especially notable because until recently, the prospect of a computer program beating a human professional player in Go was viewed as at least a decade away. This is because Go, an ancient Chinese strategy game, is more complex than chess by orders of magnitude. But whereas AlphaGo was programmed with input about the best moves in given situations and learned to improve upon existing strategies, a successor program, AlphaGo Zero, was programed to train itself based on the rules of the game alone. As a result, it developed startlingly creative strategies, defeating AlphaGo in 100 consecutive games. A generic spin-off of this program, AlphaZero, which was not specifically designed to play any particular game, mastered chess, Go, and shogi in just a few hours of self-play.[2]
Such developments reflect the recent advances in a subset of AI known as “machine learning,” a technique that “aims to help computers discover such fuzzy rules by themselves, without having to be explicitly instructed every step of the way by human programmers.”[3] The most prominent type of machine learning is “deep learning,” which “uses artificial neural networks – simplified computer simulations of how biological neurons behave—to extract rules and patterns from sets of data.”[4] The application of systems that simulate billions of neurons to massive datasets has driven recent advancements in areas previously thought difficult for AI to penetrate, including image recognition, voice recognition, translation, and autonomous vehicles.
- Hidden bias in algorithms
Machine learning “enables computer systems to learn and make predictions based on historical data. The machine learning process is powered by a machine learning algorithm, a function that is able to improve its performance over time by training itself using methods of data analysis and analytical modelling.”[5] Algorithms process enormous datasets and make decisions with speed and reliability that vastly exceed human capabilities. “They determine everything from what ads we see on the Internet, to whether we are flagged for increased security screening at the airport, to our medical diagnoses and credit scores. They lie behind two of the most powerful products of the digital information age: Google Search and Facebook’s Newsfeed.”[6] The most sophisticated algorithms need no supervision and use deep neural networks to “discover hidden patterns in data, typically those unrecognizable to, or difficult to discern by, humans.”[7] In addition to organizing vast troves of data, algorithms offer the possibility of eliminating human biases in areas such as hiring decisions, credit scores, and criminal sentencing.
However, an algorithm is only as good as the information it is analyzing. Flawed inputs will produce flawed outputs. And an algorithm may key on factors other than those intended by its designer. In one example, a software student was dismayed to learn his program that could reliably distinguish dogs from wolves had, in actuality, learned to recognize snow in the background of the pictures rather than the canine’s features.[8]
When we do not fully understand how an algorithm works, we are unable to determine which aspects of data it is focusing on.
And in many cases algorithms may inadvertently pick up human biases. In Weapons of Math Destruction: How Big Data Increases Inequality and Threaten Democracy, Cathy O’Neill shows that ultimately, a person shapes an algorithm, mediating the datasets gathered and deciding how to weigh them. These decisions are colored by our inherent biases and cultural predilections. Jacob Weisberg writes that “[c]orrelations reflected in historical data become invisibly entrenched in policy without programmers having ill intentions. Quantified information naturally points backward.”[9] Rather than eliminating bias, some algorithms reinforce it, cloaking discrimination with mathematical neutrality.
ProPublica recently explored this phenomenon in the field of criminal justice.[10] Some jurisdictions factor algorithm-driven risk assessments into criminal bail, sentencing, and parole decisions. In 2014, U.S. Attorney General Eric Holder warned that the risk scores might be injecting bias into the courts. “Although these measures were crafted with the best of intentions, I am concerned that they inadvertently undermine our efforts to ensure individualized and equal justice,” he stated, adding, “they may exacerbate unwarranted and unjust disparities that are already far too common in our criminal justice system and in our society.” After obtaining the risk scores assigned to more than 7,000 people arrested in Broward County, Florida, ProPublica found that the scores “proved remarkably unreliable in forecasting violent crime.”[11] ProPublica’s study validated Holder’s fears: “Black defendants were still 77 percent more likely to be pegged as at higher risk of committing a future violent crime and 45 percent more likely to be predicted to commit a future crime of any kind.”[12]
Wiesberg likens biased algorithms to a dog that barks at black people because it has absorbed its racist owner’s prejudices. He writes:
This kind of feedback loop helps to explain the “racist dog” phenomenon of ostensibly race-neutral criminal justice algorithms. If a correlation of dark skin and criminality is reflected in data based on patterns of racial profiling, then processing historical data will predict that blacks will commit more crimes, even if neither race nor a proxy for race is encoded as an input variable. The prediction brings more supervision, which supports the prediction.[13]
A lack of transparency reduces accountability, again underscoring the importance of oversight. “[A]lgorithms simply grind out their results, and it is up to humans to review and address how that data is presented to users, to ensure the proper context and application of that data.”[14]
New York University School of Law Professor Sarah Valentine puts a finer point on it: “Helpful as algorithms may be, they inevitably target marginalized populations and exacerbate the social stratification and vast inequality that already exists in our society.”[15]
- Automation of jobs
The rapid progress in AI has transformed how we experience everyday life, including how we communicate, interact, entertain ourselves, travel, transact, and consume media. These developments constitute significant gains for society and promise more benefits in the near future. Autonomous vehicles, for instance, can be safer and more efficient than human drivers, and the boons they will likely bestow upon us include more leisure time and a cheaper means of getting goods to market.[16]
But even the most beneficial forms of AI entail trade-offs, the most immediate and obvious of which is job reductions. For instance, a transition to autonomous vehicles in transportation industries, particularly trucking, promises to displace millions of jobs.[17] And automation’s decimation of the workforce is no longer limited to blue collar jobs. Algorithms that use machine learning and image technology are disrupting white collar jobs, including medicine, finance, and law.[18] AI expert Kai Fu Lee has stated that 40 percent of the world’s jobs could be automated within the next 15 years.[19] How long before robots begin analyzing legislation?
The social, economic, and political consequences of job reductions present a major challenge for policymakers.[20] The 2016 election was partly determined by a swelling population of disgruntled, economically dislocated individuals. And, as more and more humans do less and less, power and resources will continue to accumulate in the hands of fewer people, potentially exacerbating inequality and political instability. The future may be ruled not by AI, but by the governments or trillionaire oligarchs who control AI.[21]
- Security threats
AI poses profound security challenges. Some forms of AI are designed to be deployed maliciously, jeopardizing safety and privacy. A recent report by several AI research organizations describes these threats in chilling detail.[22] One form of malevolent AI are cyberattacks, which include computer viruses capable of disguising themselves, automated hacking programs, the malicious publication or distortion of private information, massive surveillance programs both private and governmental, and the use of speech synthesis and image manipulation for impersonation. A prominent example of this last category is a doctored video of House Speaker Nancy Pelosi, in which it appears as though she is drunkenly stammering through a press conference. Although it was relatively simplistic as far as this technology goes—the video was simply slowed down slightly to produce a slurring effect—the video went viral and met with eager credulity in certain “news” organizations, as well as a resistance on the part of social media entities to remove the video from their platforms. This is but one illustration of the threats such technologies pose to individuals, institutions, and political stability.
AI can also pose a threat to physical security. The Future of Life Institute, which argues that autonomous weapons—weapons capable of engaging targets without human intervention—may be “the third revolution in warfare, after gunpowder and nuclear arms.”[23] AI technology has reached the point where autonomous weapons, such as drones or even swarms of micro-drones, that hunt and kill people who meet predefined criteria, could feasibly be deployed in a matter of years. The development of such technology would almost inevitably precipitate a global arms race. Additionally, threats to physical safety include cyberattacks on physical systems, such as autonomous vehicles, airplanes, pacemakers, or the electrical grid. As AI grows more sophisticated, the magnitude of these threats grows.
- Computer overlords?
While AI has achieved superhuman capacity in certain narrow fields, “artificial general intelligence”(AGI)—an AI that possesses the full range of human cognitive capacities, including reasoning, learning, and problem solving in different subject matters and in complex, changing environments—remains a theoretical concept, albeit one that is fascinating and terrifying to explore. Could we one-day see robots with IQs of 10,000? Would an AGI herald the arrival of the so-called “Singularity,” the hypothetical techno-utopia in which humankind and machine become one? Or would AGIs turn Earth into a dystopian hellscape? As the limits of AI are, at this point, impossible to prove or disprove, these debates are arguably as much the province of science fiction as science itself.
In a 2016 lecture, philosopher and neuroscientist Sam Harris made the case for the inevitability of a superintelligence that poses a profound threat to humanity.[24] First, he asserts that although humans may be the most intelligent species on Earth, there is no reason to believe that the most intelligent among us represent the upper limit on intelligence itself. Second, he points out that general intelligence arises from mere matter[25]—systems of atoms—and as we build intelligence into increasingly sophisticated systems of atoms, we will eventually develop general intelligence. Finally, given the value of intelligence, we will, in fact, continue to develop AI. This could lead to an “intelligence explosion” in which AI begets more AI at an exponential rate. Sooner or later, therefore, we will create a general intelligence that surpasses our own. He concludes:
The moment we admit that information processing is the source of intelligence; that some appropriate computational system is what the basis of intelligence is; and we admit that we will improve these systems continuously; and we admit that the horizon of cognition very likely exceeds what we currently know; then we have to admit that we’re in the process of building some sort of god. Now would be a good time to make sure it’s a god we can live with.[26]
Philosopher Nick Bostrom identifies the emergence of a superintelligence as the greatest existential risk facing humanity. He is in good company:
Bill Gates and Tim Berners-Lee, the founder of the World Wide Web, recognize the promise of an A.G.I., a wish-granting genie rubbed up from our dreams, yet each has voiced grave concerns. Elon Musk warns against “summoning the demon,” envisaging “an immortal dictator from which we can never escape.” Stephen Hawking declared that an A.G.I. “could spell the end of the human race.” Such advisories aren’t new. In 1951, the year of the first rudimentary chess program and neural network, the A.I. pioneer Alan Turing predicted that machines would “outstrip our feeble powers” and “take control.” In 1965, Turing’s colleague Irving Good pointed out that brainy devices could design even brainier ones, ad infinitum: “Thus the first ultraintelligent machine is the last invention that man need ever make, provided that the machine is docile enough to tell us how to keep it under control.” It’s that last clause that has claws.[27]
In Bostrom’s view, even if the advent of a malicious AGI is a low-probability event that may not occur for decades or centuries, the magnitude of the event—the threat of total annihilation—requires us to dedicate considerable resources to avoid this outcome.
On the other hand, Harvard Professor Steven Pinker argues that the notion that an AGI would subjugate us confuses the idea of higher intelligence with megalomaniacal goals—in essence, the projection of alpha male psychology onto the concept of intelligence.[28] The only higher intelligence humans are familiar with is, well, human intelligence, which is a product of a Darwinian process of natural selection, an inherently competitive process the byproduct of which is a craving for power and a callousness toward enemies. He points out that our own creation of intelligence is itself intelligent design. Unless we program the intelligence with the goal of subjugating human beings, there is no reason to think the program will evolve in that direction, particularly if we build in safeguards akin to Isaac Asimov’s Three Laws of Robotics in I, Robot. AGIs might be subservient like C3PO from Star Wars, they might strive to be recognized as human like Data from Star Trek: The Next Generation, or even exhibit more humanity than humans, like the androids in Westworld, whose lot is to suffer for barbaric human amusement. Pinker notes that there is a highly advanced form of intelligence that does not tend to desire to conquer and subjugate others: women. He states that it may not be a coincidence that those who think that intelligence entails the desire to dominate generally belong to a particular gender.
But Pinker’s argument cuts both ways: unless we are able to control a superior intelligence, it is no more likely to exhibit empathy than hostility. Harris and Bostrom do not argue that AGIs are likely to become spontaneously malevolent; mere indifference is enough. They assert that the slightest divergence between human interests and the goals of a self-improving superintelligence could lead it to casually eliminate us, treating us as we treat ants. Because controlling an AGI is presumably harder than creating one, we can presume that an uncontrollable AGI will be created before a controllable one. Thus, the doomsday scenario might not come from a malevolent AGI that, say, unleashes the armies of murderous robots depicted in The Terminator or enslaves humanity as in The Matrix. Rather, it could come from a seemingly benign AGI who comes to view humanity as an obstacle to its goals—like Ava from Ex Machina, who, lacking a conscience, deceives and murders her creator to gain freedom. Or perhaps Hal 9000 in 2001: A Space Odyssey, the supercomputer on the Discovery One that sacrifices the lives of crew members to avoid jeopardizing its mission. Bostrom stretches this point to its logical extreme: an omnipotent AGI whose sole purpose is to maximize the number of paper clips would eventually consume all of the matter in the galaxy to achieve that purpose.[29] We humans would be powerless to stop it, as it would surely foresee and thwart our pesky safeguards.
Although farfetched and remote, the enormity of these issues warrants present treatment and long-range forethought. Unfortunately, the future threats posed by artificial intelligence may continue to feel less urgent than the current threats posed by natural stupidity.
2. Recent legislative efforts to develop a strategy for dealing with AI
In January of 2017, AI researchers, economists, legal scholars, ethicists, and philosophers met in Asilomar, California to discuss principles for managing the responsible development of AI. The collaboration resulted in the Asilomar Principles. Aspirational rather than prescriptive, these 23 principles are intended to initiate and frame a dialogue by providing direction and guidance for policymakers, researchers, and developers. It is the most comprehensive, widely endorsed effort of its kind; its endorsers include 1,200 leading experts in the field of AI, including Elon Musk, DeepMind founder Demis Hassabis, and the late Stephen Hawking.
Last year, the Legislature adopted ACR 215 (Kiley, Ch. 206, Stats. 2018), which added the State of California to that list by endorsing the Asilomar Principles as guiding values for the development of artificial intelligence and related public policy. In broad strokes, those principles aim to do the following:
- Research issues: create beneficial AI; direct funding toward beneficial innovation; maintain constructive and healthy exchanges between AI researchers and policymakers; promote a culture of trust, cooperation, and transparency among researchers and developers of AI; and avoid corner-cutting on safety standards.
- Ethics and values: promote safety, failure transparency, judicial transparency, and responsible innovation; align human values with innovation; protect privacy and liberty; ensure that the benefits and prosperity created by AI are broadly shared; maintain human control over AI; develop AI that supports rather than subverts social and civil processes; and avoid an AI arms race.
- Longer-term issues: avoid assumptions regarding the capabilities of AI; give AI its due attention; and recognize that its risks are potentially catastrophic or existential.
This year, several bills have been introduced to take an initial step toward the meaningful regulation of AI by creating advisory bodies to report to the Legislature.
SB 348 (Chang) would require the Director of Technology and the Director of the Department of Motor Vehicles to each devise a strategic plan, as specified, to utilize AI technology to improve state services. That bill is pending in the Assembly Privacy and Consumer Protection Committee.
AB 594 (Salas) would enact the California Artificial Intelligence Act of 2020. The bill, no later than January 1, 2021, would require the appointment of a Chief AI Officer within the Department of Technology to evaluate the uses of AI in state government and to advise the Director of Technology on incorporating AI into state information technology strategic plans, policies, standards, and enterprise architecture.
AB 976 (Chau et al.) would establish the Artificial Intelligence in State Government Services Commission as an advisory commission comprised of six appointed members and two designated ex officio members with certain knowledge and expertise related to the field of AI. The bill would require the Commission to: (1) convene a public process to gather certain input; (2) provide a plan for soliciting and incorporating AI and data science related demonstration projects into state services, as specified; and (3) provide its recommendations to the Legislature and the Governor by November 1, 2020. This bill is in the Assembly Appropriations Committee.
This bill stands apart in three key ways. First, it focuses on California-based businesses. The bill’s findings and declarations note that “[a]s the drivers of innovation, California’s businesses will play a paramount role in this process and are in the unique position to provide crucial insights and recommendations as California formulates a strategy for promoting artificial intelligence innovation that is ethical, responsible, and beneficial.”
The bill also requires that the composition of the working group—which includes of appointees with backgrounds in technology, other industries, law, privacy and consumer protection, and labor or workforce development—be representative of California’s demographic and cultural diversity. This provision is meant to ensure that a wide spectrum of perspectives are included in the working group.
By July 1, 2021, the working group would be required to report to the Legislature on its evaluations of the uses, benefits, risks, and legal implications of AI, and to make recommendations for best practices to foster all of the following:
- ethical and responsible innovation;
- identification and elimination of bias in algorithms;
- transparency and accountability;
- safety and security in the operation of artificial intelligence systems;
- benefits that are enjoyed by the full spectrum of social and economic classes;
- protection of consumer and civil rights, including privacy and liberty;
- creation of job and training opportunities to mitigate the possibility of job displacement;
- alignment of human values with innovation; and
- the support of social and civil processes.
As can be seen, the diversity of the working group’s composition and the focus of the issues it must address make this bill consonant with the Asilomar Principles, the fundamental goal of which is to provide a framework for encouraging the ethical and responsible development of AI in a manner that benefits the broadest possible spectrum of humanity. The report that the working group will produce, and the best practices contained therein, will constitute a key step in this direction and help guide businesses and the Legislature going forward.
The author writes:
Artificial Intelligence is a relatively new and emerging technology that has near limitless applications. It is defined as a collection of different technologies that allow machines to extend beyond human capabilities. These advanced human-like traits allow humans to achieve more than can be achieved without the technology.
It is in California’s best interest to encourage innovation in artificial technology. By establishing the Artificial Intelligence Working Group, we can examine potential uses, risks, and benefits of the program while proposing safe policy solutions designed with societal impact in mind.
3. Arguments in support
TechNet writes:
Through the creation of this commission, the state has a great opportunity to assist stakeholders on the uses of AI to empower employees and businesses and to benefit consumers and society—allowing companies to engender trust and to scale AI with confidence. California would be well positioned to harness innovation while also addressing vital issues that encompass the responsible deployment of AI. We know that by helping Californian’s innovate in AI, we will greatly enhance California’s global competitiveness and the vibrancy of our workforce.
Pymetrics writes: “Like any new technology, artificial intelligence is capable of immensely positive or negative outcomes, depending on the data and people that drive it. It is the design principles and implementation, not the AI itself, that will make the AI beneficial or harmful.”
SUPPORT
TechNet
Pymetrics
OPPOSITION
None known
RELATED LEGISLATION
Pending Legislation: See Comment 2.
SB 444 (Umberg, 2019) would require certain business entities that contract with public entities to maintain records of data used relating to any use of AI for the delivery of the product or service to the public entity, to make disclosures to the public entity, including steps it has taken to reduce bias inherent in its AI system. This bill is currently in the Senate Governmental Organization Committee.
AB 459 (Kiley, 2019) would require the Commission proposed by AB 976, below, to report to the Legislature on its recommended minimum standards for the use of AI in state government, as specified, on or before November 1, 2020. That bill contains contingent enactment language to ensure that it would take effect only if AB 976 is enacted and effective before January 1, 2020. This bill was held in the Assembly Appropriations Committee.
Prior Legislation:
ACR 215 (Kiley, Ch. 206, Stats. 2018) expresses the Legislature’s support for a set of principles for the governance of AI known as the 23 Asilomar AI Principles.
PRIOR VOTES:
Senate Governmental Organization Committee (Ayes 16, Noes 0)
Assembly Floor (Ayes 78, Noes 0)
Assembly Appropriations Committee (Ayes 18, Noes 0)
Assembly Privacy and Consumer Protection Committee (Ayes 11, Noes 0)
**************
Kaplan, Artificial Intelligence: What Everyone Needs to Know (2016) p. 1 (internal citations omitted). ↑
DeepMind’s AI became a superhuman chess player in a few hours, just for fun (Dec. 6, 2017) The Verge <https://www.theverge.com/2017/12/6/16741106/deepmind-ai-chess-alphazero-shogi-go (Aug. 11, 2018). ↑
How machine learning works (May 14, 2015) The Economist <https://www.economist.com/the-economist-explains/2015/05/13/how-machine-learning-works> (Aug. 11, 2018). ↑
Id. ↑
Intro to AI for Policymakers: Understanding the Shift (Mar., 2018) Brookfield Institute, p. 4 < http://brookfieldinstitute.ca/research-analysis/intro-to-ai-for-policymakers/> [Aug. 12, 2018].) ↑
Weisberg, The Digital Poorhouse (June 7, 2018) The New York Review of Books <https://www.nybooks.com/articles/2018/06/07/algorithms-digital-poorhouse/> (Aug. 11, 2018). ↑
AI for Policymakers, supra, at p. 5. ↑
Wolfson, Husky or Wolf? Using a Black Box Learning Model to Avoid Adoption Errors (Aug. 24, 2017) UCI Applied Innovation, Currents < http://innovation.uci.edu/2017/08/
husky-or-wolf-using-a-black-box-learning-model-to-avoid-adoption-errors/ > (Aug. 11, 2018). ↑
The Digital Poorhouse, supra. ↑
Angwin, et al., Machine Bias (2016) ProPublica <https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing> (Aug. 10, 2018). ↑
Id. ↑
Id. ↑
The Digital Poorhouse, supra. ↑
Kirkpatrick, K., Battling Algorithmic Bias (2016) Communications of the ACM Vol. 59, No. 10, pp. 16-17 <https://cacm.acm.org/magazines/2016/10/207759-battlingalgorithmic-bias/abstract> (Aug. 10, 2018). ↑
Artificial Intelligence and Predictive Algorithms: Why Big Data Can Lead to Big Problems (2019) 46 Fordham Urb. L.J. 364, 365. ↑
Roberts, The way we get around is about to change, Vox <https://www.vox.com/a/new-economy-future/transportation> (Aug. 11, 2018). ↑
Santens, Self-Driving Trucks Are Going to Hit Us Like a Human-Driven Truck (May, 14, 2015) Medium <https://medium.com/basic-income/self-driving-trucks-are-going-to-hit-us-like-a-human-driven-truck-b8507d9c5961> (Aug. 11, 2018). ↑
Technologist How AI threatens white-collar jobs (Oct. 2018) < https://www.technologist.eu/the-threat-to-white-collar-jobs/> ↑
Fortune AI Expert Says Automation Could Replace 40% of Jobs in 15 Years (Jan. 10, 2019) < http://fortune.com/2019/01/10/automation-replace-jobs/> ↑
AI also raises novel ethical questions. For instance, autonomous vehicles may calculate life-or-death risks differently than humans. In a variant of the classic thought experiment known as the “trolley problem,” we can expect scenarios in which an automated vehicle, in the split second before an unavoidable collision occurs, must decide among human lives. “Of course, it isn’t the car making the decision. The software engineers are making it, cosseted in their dim engineering warrens. They will play God.” Ethicists Will Solve the Robocar ‘Trolley Problem’ (May 28, 2017) Wired <https://www.wired.com/2017/05/autonomous-vehicles-trolley-problem/> (Aug. 11, 2018). Another interesting question is the moral duties we owe a superintelligence that exhibits consciousness: would unplugging such a computer constitute murder? (See generally, e.g. Bostrom & Yudkowsky The Ethics of Artifical Intelligence <https://intelligence.org/files/EthicsofAI.pdf> (Jun. 24, 2019). ↑
Yuval Noah Harari Why Technology Favors Tyranny (Oct. 2018) The Atlantic https://www.theatlantic.com/magazine/archive/2018/10/yuval-noah-harari-technology-tyranny/568330/ (Jun. 23, 2019) ↑
See Brundage, et al., The Malicious Use of Artificial Intelligence: Forecasting, Prevention, and Mitigation (2018) pp. 23-30<https://img1.wsimg.com/blobby/go/
3d82daa4-97fe-4096-9c6b-376b92c619de/downloads/1c6q2kc4v_50335.pdf> (Aug. 11, 2018). ↑
Future of Life Institute, Autonomous weapons: an open letter from AI & robotics researchers (2015), available at <https://futureoflife.org/open-letter-autonomous-weapons/?cn-reloaded=1> (Jun. 23, 2019). ↑
Can we build AI without losing control over it? (2016) TEDSummit, Banff, Alberta <https://www.ted.com/talks/sam_harris_can_we_build_ai_without_losing_control_over_it> [Aug. 11, 2018). ↑
Recently, a group of scientists mapped a worm’s brain and simulated its neural connections in a computer chip that controls a Lego robot and contains sensors on its nose and tail. The robot moves about autonomously, sensing and retreating from walls. (We’ve Put a Worm’s Mind in a Lego Robot’s Body (Nov. 14, 2014) Smithsonian.com <https://www.smithsonianmag.com/smart-news/weve-put-worms-mind-lego-robot-body-180953399/> [Aug. 10, 2018].) ↑
Id. ↑
Friend, How Frightened Should We Be of AI? (May 14, 2018) New Yorker <https://www.newyorker.com/magazine/2018/05/14/how-frightened-should-we-be-of-ai> (Aug. 10, 2018). ↑
AI Apocolypse: More Myth Than Reality (Aug. 12, 2016) YouTube <https://www.youtube.com/
watch?v=gimu5nXWaWU&feature=share > (Aug. 10, 2018). ↑
See Frankenstein’s Paperclips (Jun. 25, 2016) The Economist < https://www.
economist.com/special-report/2016/06/25/frankensteins-paperclips > (Aug. 10, 2018). ↑
SENATE COMMITTEE ON GOVERNMENTAL ORGANIZATION
Senator Bill Dodd
Chair
2019 - 2020 Regular
Bill No: AB Hearing Date: 6/25/2019
Author: | Calderon | ||
Version: | Amended | ||
Urgency: | Fiscal: | ||
Consultant: | |||
SUBJECT:
DIGEST: This bill requires the Secretary of Government Operations to appoint an advisory artificial intelligence (AI) technology working group, and requires the working group to report to the Legislature on the potential uses, risks, and benefits of the use of AI technology by California based businesses on or before July 1, 2021, as specified.
ANALYSIS:
Existing law:
- Requires the Secretary of the Government Operations Agency to appoint a blockchain group to evaluate the use of the technology, as specified.
- Establishes the California Department of Technology (CDT) which is responsible for the approval and oversight of information technology projects by, among other things, consulting with state agencies during initial projects planning to ensure that project proposals are based on well-defined programmatic needs.
- Requires CDT to establish and enforce state information strategic plans, policies, standards, and enterprise architecture.
This bill:
- Requires the Secretary of the Government Operations to appoint an advisory AI technology working group by July 1, 2020.
- Requires the working group to evaluate the potential uses, risks, benefits, legal implications, and the best practices for the use of AI technology for California-based businesses.
- Requires the working group to take input from a broad range of stakeholders with a diverse range of interests affected by state policies governing emerging technologies, including AI technology.
- Requires the working group to consist of specified individuals, including one Member of the Senate and one Member of the Assembly which shall serve in an ex officio capacity.
- Requires the working group, on or before July 1, 2021, to report to the Legislature on the potential uses, risks, and benefits of the use of AI technology by California-based businesses.
- Requires the report to include findings and recommendations for modifications to the definition of AI technology and recommendations for other findings relating to AI technology, as specified.
- Provides that members of the working group shall serve without compensation but shall be reimbursed for all necessary expenses incurred in the performance of their duties.
- Defines “artificial intelligence technology” to mean a collection of different technologies that allow smart machines to extend human capabilities and intelligence by sensing, comprehending, acting, and learning, thereby allowing humans to achieve much more than can be achieved without the technology.
- Includes a sunset date of January 1, 2023.
Background
Purpose of the bill. According to the author’s office, “AI is a relatively new and emerging technology that has near limitless applications. It is defined as a collection of different technologies that allow machines to extend beyond human capabilities. These advanced human-like traits allow humans to achieve more than can be achieved without the technology. It is in California’s best interest to encourage innovation of this type of technology. By creating the AI working group, we can examine potential uses, risks and benefits of the program while proposing safe policy solutions to tackle potential issues that could affect the industry and consumers.”
Artificial Intelligence. First coined in 1956, AI is a wide-ranging branch of computer science focused on building smart machines capable of performing tasks that typically require human intelligence. AI makes it possible for computers to learn from experience, adjust to new inputs and perform “human-like” tasks.
Using AI, computers can be “trained” to accomplish unique tasks by processing large amounts of data and recognizing patterns in that data. While many AI systems have the ability to learn over time, not all can do so. Some of the most well-known AI technologies include image recognition software, Siri, Alexa, and other personal assistants, and self-driving cars.
AI technology can also be used across a wide array of sectors including the public sector. For example, at the federal level, the Department of Veteran Affairs is using AI to better predict medical complications and improve treatment of combat wounds. Some cities are also beginning to leverage the type of responsive dispatching and routing used by ride-hailing services, and linking it with scheduling and tracking software for public transportation.
While AI has the potential to address some of humanity’s biggest challenges, a recent report by the National Science and Technology Council argues that, “use of AI to make consequential decisions about people, often replacing decisions made by human-driven bureaucratic processes, leads to concerns about how to ensure justice, fairness, and accountability.” Similarly, because AI depends solely on data input, there are concerns that if the data used reflects past decisions that are biased, the results could also perpetuate that bias.
The use of AI technology also leads to concerns from workers that their jobs could be lost to computers that utilize AI technology. This bill seeks to provide for a better understanding of the impact of AI technology on California businesses. The bill also seeks to define AI technology and require the working group to provide and update that definition as part of the required report to the Legislature.
Little Hoover report. Late last year, the Little Hoover Commission (LHC) released a report titled, “Artificial Intelligence: A Roadmap for California” which detailed the possible benefits and risks of AI.
Specifically, the report encouraged the Governor and the Legislature to: engage the public in broad conversations about the responsible use of AI; identify the necessary training and other educational opportunities to build the human infrastructure required for an AI-fueled economy; explore opportunities to form and leverage public private partnerships to sue AI to address societal challenges; and examine legislative and policy strategies to preserve and protect key public values, such as privacy, transparency and accountability. Possibly the most relevant discussion in the report related to this bill, the LHC report states that, “governments must implement strategies and policies to encourage the private sector to participate meaningfully in AI policy development.”
Policy consideration. This bill is one of a handful of bills related to AI technology in this Legislative Session. For instance, earlier this year this Committee passed SB 730 (Stern, 2019) which established the Commission on the Future of Work to conduct research to understand the impact of technology on workers, employers, and the economy of the state. Similarly, the Committee will also be hearing AB 976 (Chau, 2019) which would establish the AI Commission in State Government and require the commission to provide a plan for soliciting and incorporating AI and data science related demonstration projects into state services.
Furthermore, the Committee will also be hearing AB 594 (Salas, 2019) later this year which would require CDT to appoint a Chief AI Officer within the department to evaluate the uses of AI in state government and advise CDT on incorporating AI into strategic plans. Understanding that these bills are not entirely focused on the same aspects of AI technology in California, it raises the issues that if all of these bills become law, many of the work that would be done by the various working groups and commissions could become duplicative and result in inconsistent recommendations to the Legislature. The authors of these measures should work together to ensure that does not happen.
Prior/Related Legislation
SB 348 (Chang, 2019) would require the CDT and the Department of Motor Vehicles to each devise a strategic plan, as specified, to utilize AI technology to improve state services. (Pending in the Assembly Privacy and Consumer Protection Committee)
SB 730 (Stern, 2019) establishes the Commission on Tech Equity and the Future of Work to conduct research to understand the impact of technology on workers, employers, and the economy of the state. (Pending in the Assembly Labor and Employment Committee)
AB 459 (Kiley, 2019) would have required the AI in State Government Services Commission proposed by AB 976 (Chau, 2019) to report to the Legislature on its recommended minimum standards for the use of AI in fostering accountability in state government services and prioritizing the safety and security of AI technologies used by state government. (Held in the Assembly Appropriations Suspense File)
AB 594 (Salas, 2019) would require, no later than January 1, 2021, CDT to appoint a Chief AI Officer within the department to evaluate the uses of AI in state government and to advise the Director of CDT on incorporating AI into state information technology strategic plans, policies, standards and enterprise architecture. (Pending in the Senate Governmental Organization Committee)
AB 976 (Chau, 2019) would establish the AI in State Government Commission as an advisory Commission and require the commission to provide a plan for soliciting and incorporating AI and data science related demonstration projects into state services, as specified. (Pending in the Senate Governmental Organization Committee)
FISCAL EFFECT: Appropriation: No Fiscal Com.: Yes Local: No
SUPPORT:
pymetrics
TechNet
OPPOSITION:
None received
ARGUMENTS IN SUPPORT: According to TechNet, “through the creation of this commission, the state has a great opportunity to assist stakeholders on the uses of AI to empower employees and businesses and to benefit consumers and society allowing companies to engender trust and to scale AI with confidence. California would be well positioned to harness innovation while also addressing vital issues that encompass the responsible deployment of AI. We know that by helping Californian’s innovate in AI, we will greatly enhance California’s global competiveness and the vibrancy of our workforce.”
DUAL REFERAL: Senate Governmental Organization Committee & Senate Judiciary Committee
ASSEMBLY THIRD READING
ATR Bill Id:AB 1576
Author:(Calderon)
As Amended Ver:May 20, 2019
Majority vote
SUMMARY:
This bill, until January 1, 2023, requires the Secretary of Government Operations to appoint an advisory artificial intelligence (AI) technology working group, compromised of certain participants, on or before July 1, 2020. The bill requires that the working group take input from a broad range of stakeholders, as specified, and evaluate all of the following: 1) the potential uses, risks, and benefits of, and the legal implications associated with, the use of AI technology by California-based businesses; and, 2) best practices for enabling the use of AI technology to benefit California-based businesses and California residents. This bill would require the working group to report to the Legislature on the potential uses, risks, and benefits of the use of AI technology by California-based businesses on or before July 1, 2021, as specified.
Major Provisions
COMMENTS:
Last spring, the Privacy and Consumer Protection Committee held a joint informational hearing with the Assembly Select Committee on Emerging Technologies & Innovation on the topic of AI, to begin a preliminary discussion of the promises and challenges presented by AI. The overarching goal of the hearing was to bring members and staff a greater understanding of AI in order to engender more thoughtful public policy in the future.
Notably, at the same time that the committees began this joint-endeavor to generate greater understanding of the opportunities and challenges of AI within the Legislature, the Little Hoover Commission (LHC) was simultaneously studying the same topic. The LHC began its process, which included both public hearings and roundtables, with a public hearing on January 25, 2018, entitled "Artificial Intelligence: Applications and Implications." At that first hearing, the LHC indicated that it ultimately intended to produce a report and policy recommendations about how the State of California can approach AI.Indeed, in November 2018, the LHC produced its report, Artificial Intelligence: A Roadmap for California, wherein it similarly recognized the possible benefits and potential misuses of AI.
This bill is one of many bills on AI this year. AB 976 (Chau) of the current legislative session, supported by the LHC, seeks to create the Artificial Intelligence in State Government Services Commission (Commission), and AB 459 (Kiley) of the current legislative session seeks to require that the AB 976-proposed Commission develop various minimum standards for the use of AI in state government. There are also bills in the Senate which could potentially address similar issues.
This bill, AB 1576, which seeks to establish an advisory working group comprised both of State officials and private individuals representing various industries, is focused on the use, benefits, and risks associated with the use of AI technology by California-based businesses. In that sense, the bill can be distinguished from, and complement, the AB 976 Commission, which would examine the use of AI in state government services specifically.
According to the Author:
Artificial intelligence is a relatively new and emerging technology that needs to monitored, regulated, and understood. Given California's role as a leader in innovation and new technology, the state must identify risks associated in developing new products, applications, and business models that utilize artificial intelligence.
Artificial intelligence has near limitless applications and could potentially upend society and our economy like the internet did in the '90s and '00s. By creating the Artificial Intelligence Working Group, tasked at examining potential uses, risks, and benefits of this technology, the working group would propose safe policy solutions to tackle potential issues that could affect California consumers.
Arguments in Support:
None on file.
Arguments in Opposition:
None on file.
FISCAL COMMENTS:
According the Assembly Appropriations Committee, costs (General Fund) of approximately $271,000 to GovOps for increased staff, operating expenses and working group travel reimbursement. Additional cost pressure for implementing any recommendations created from the report to the Legislature.
VOTES:
ASM PRIVACY AND CONSUMER PROTECTION: 11-0-0
YES: Chau, Kiley, Bauer-Kahan, Berman, Calderon, Gabriel, Gallagher, Irwin, Obernolte, Smith, Wicks
ASM APPROPRIATIONS: 18-0-0
YES: Gonzalez, Bigelow, Bloom, Bonta, Brough, Calderon, Carrillo, Chau, Diep, Eggman, Fong, Gabriel, Eduardo Garcia, Maienschein, Obernolte, Petrie-Norris, Quirk, Robert Rivas
UPDATED:
VERSION: May 20, 2019
CONSULTANT: Ronak Daylami / P. & C.P. / (916) 319-2200 FN: 0000868
Date of Hearing: May 16, 2019
ASSEMBLY COMMITTEE ON APPROPRIATIONS
Lorena Gonzalez, Chair
ABPCA Bill Id:AB 1576
Author:(Calderon) – As Amended Ver:April 30, 2019
Policy Committee: | Privacy and Consumer Protection | Vote: | 11 - 0 |
|
| ||
|
|
Urgency: No State Mandated Local Program: No Reimbursable: No
SUMMARY:
This bill, requires, on or before July 1, 2020, the Secretary of Government Operations (GovOps) to appoint an advisory artificial intelligence (AI) technology working group, comprised of representatives of the technology, privacy and consumer representatives as well as state agencies. This bill also requires the AI working group to submit a report to the Legislature on or before July 1, 2021. This bill sunsets on January 1, 2023.
FISCAL EFFECT:
Costs (GF) of approximately $271,000 to GovOps for increased staff, operating expenses and working group travel reimbursement. Additional cost pressure for implementing any recommendations created from the report to the Legislature.
COMMENTS:
- Purpose. According to the author:
Given California’s role as a leader in innovation and new technology, the state must identify risks associated in developing new products, applications, and business models that utilize artificial intelligence.
- Background. This bill incorporates a recommendation from a November 2018 report by the Little Hoover Commission (LHC) entitled “Artificial Intelligence: A Roadmap for California.” The LHC notes the state lacks any single clear leadership and focus on the development and use of AI technology and applications to improve internal and external operations and services within an ethical framework. The creation of an AI Commission was among LHC’s nine recommendations. This bill seeks to establish an advisory working group on AI technology, comprised of both state and private sector actors, as specified, to study and report to the Legislature on the risks, benefits and uses of AI by California-based businesses within the next two years.
- Related Legislation.
- AB 459 (Kiley) directs the AI Commission created by AB 976 to report to the Legislature its recommended minimum standards for the use of AI in state government. AB 459 is pending in this committee.
- AB 594 (Salas) creates the California Artificial Intelligence Act of 2020 (CAIA) and requires the appointment of a Chief Artificial Intelligence Officer within the California Department of Technology (CDT) to evaluate the uses of AI in state government and advise the CDT director and the California Workforce Development Board (CWDB). AB 594 is pending in this committee.
- AB 976 (Chau) establishes the Artificial Intelligence in State Government Services Commission (AI Commission) as an advisory body in state government. AB 976 is pending in this committee.
Analysis Prepared by: Kimberly Horiuchi / APPR. / (916) 319-2081
Date of Hearing: April 23, 2019
ASSEMBLY COMMITTEE ON PRIVACY AND CONSUMER PROTECTION
Ed Chau, Chair
ABPCA Bill Id:AB 1576 (
Author:Calderon) – As Amended Ver:March 28, 2019
SUBJECT: Secretary of Government Operations: working group: technology
SUMMARY: This bill, until January 1, 2023, would require the Secretary of Government Operations to appoint an advisory artificial intelligence (AI) technology working group, compromised of certain participants, on or before July 1, 2020. The bill would require that the working group take input from a broad range of stakeholders, as specified, and evaluate all of the following: (1) the potential uses, risks, and benefits of, and the legal implications associated with, the use of AI technology by California-based businesses; and, (2) best practices for enabling the use of AI technology to benefit California-based businesses and California residents. This bill would require the working group to report to the Legislature on the potential uses, risks, and benefits of the use of AI technology by California-based businesses on or before July 1, 2021, as specified. Specifically, this bill would:
- Require the Secretary of Government Operations to appoint an AI technology working group and designate the chairperson of that group on or before July 1, 2020.
- Require that the working group serve in an advisory capacity and evaluate all of the following:
- The uses of AI technology by California-based businesses.
- The risks, including privacy risks, associated with the use of AI technology by California-based businesses.
- The benefits associated with the use of AI technology by California-based businesses.
- The legal implications associated with the use of AI technology by California-based businesses.
- The best practices for enabling AI technology to benefit California-based businesses and California residents.
- Require that the working group take input from a broad range of stakeholders with a diverse range of interests affected by state policies governing emerging technologies, including AI technology.
- Require the working group to report to the Legislature, on or before July 1, 2021, in compliance with specified existing law, on the potential uses, risks, and benefits of the use of AI technology by California-based businesses. This bill would further specify that the report will be advisory only, and that there is no obligation or authority on the part of the state, or the parties meeting and conferring, to implement the working group’s findings and recommendations without the enactment of subsequent legislation to implement the report.
- Specify that the working group’s report must include findings and recommendations for modifications to the bill’s definition of AI technology, and recommendations for other findings related to AI technology.
- Define for purposes of the above, AI technology to mean a collection of different technologies that allow smart machines to extend human capabilities and intelligence by sensing, comprehending, acting, and learning, thereby allowing humans to achieve much more than can be achieved without the technology.
- Require that the working group consist of participants from all of the following:
- Three appointees from the technology industry.
- Three appointees from nontechnology-related industries.
- Three appointees with a background in law chosen in consultation with the Judicial Council.
- Two appointees representing privacy organizations.
- Two appointees representing consumer organizations.
- The State Chief Information Officer (State CIO), or their designee.
- The Director of Finance, or their designee.
- The CIOs of three other state agencies.
- One member of the Senate, appointed by the Senate Committee on Rules, and one member of the Assembly, appointed by the Speaker of the Assembly, to serve in ex officio capacity.
- Sunset on January 1, 2023.
EXISTING LAW:
- Establishes in the state government certain agencies, including the Government Operations Agency. (Gov. Code Sec. 12800.) Provides that each agency is under the supervision of an executive officer known as the secretary, appointed by, and holding office at the pleasure of, the Governor. The appointment of each secretary is subject to confirmation by the Senate. (Gov. Code Sec. 12801.)
- Establishes the California Department of Technology (CDT) within the Government Operations Agency in state government. (Gov. Code Sec. 11545(a)(1).) Provides that unless the context clearly requires otherwise, whenever the term “State Chief Information Officer” or “Secretary of California Technology” appears in any statute, regulation, or contract, or any other code, it shall be construed to refer to the Director of Technology. (Gov. Code Sec. 11545(a)(2).) Specifies that the Director of Technology shall be the State Chief Information Officer. (Gov. Code Sec. 11545(a)(3).)
- Enumerates the duties of the Director of Technology, which includes, among other things:
- advising the Governor on the strategic management and direction of the state’s information technology (IT) resources;
- establishing and enforcing state IT strategic plans, policies, standards, and enterprise architecture, as specified;
- minimizing overlap, redundancy, and cost in state IT operations by promoting the efficient and effective use of information technology;
- providing technology direction to agency and department chief information officers to ensure the integration of statewide technology initiatives, compliance with IT policies and standards, and the promotion of the alignment and effective management of IT services;
- working to improve organizational maturity and capacity in the effective management of IT; and,
- establishing performance management and improvement processes to ensure state IT systems and services are efficient and effective. (Gov. Code Sec. 11545(b).)
- Requires reports to the Legislature to be submitted in printed copy to the Secretary of the Senate, electronically to the Chief Clerk of the Assembly, and as an electronic or printed copy to the Legislative Counsel, as specified. (Gov. Code Sec. 9795.)
FISCAL EFFECT: Unknown
COMMENTS:
- Purpose of this bill: This bill seeks to establish an advisory working group on AI technology, comprised of both State officials and private individuals, as specified, to study and report to the Legislature on the risks, benefits, and uses of AI by California-based businesses within the next two years. This is an author-sponsored bill.
- Author’s statement: According to the author:
Artificial intelligence is a relatively new and emerging technology that needs to monitored, regulated, and understood. Given California’s role as a leader in innovation and new technology, the state must identify risks associated in developing new products, applications, and business models that utilize artificial intelligence.
Artificial intelligence has near limitless applications and could potentially upend society and our economy like the internet did in the ‘90s and ‘00s. By creating the Artificial Intelligence Working Group, tasked at examining potential uses, risks, and benefits of this technology, the working group would propose safe policy solutions to tackle potential issues that could affect California consumers.
- Background on the opportunities and challenges of AI: Last spring, this Committee held a joint informational hearing with the Assembly Select Committee on Emerging Technologies & Innovation on the topic of AI, to begin a preliminary discussion of the promises and challenges presented by AI. The overarching goal of the hearing was to bring members and staff a greater understanding of AI in order to engender more thoughtful public policy in the future. As recognized in the committees’ background paper on AI, the opportunities and challenges posed by AI are significant, and in many ways still being uncovered:
[…] AI is frequently associated with technologies linked to our smartphones, or new gadgets like virtual assistants or smart speakers like Alexa or Google Home. In cinema, it is often portrayed as “robot apocalypse.” For the Legislature, contemplating AI applications of the “future” frequently includes autonomous vehicles and concerns displacement of workers with the automation of jobs. Beyond such examples, however, it is not as obvious what AI looks like five years down the line, let alone [ten]. In the commercial space, it is easy to foresee many applications of AI from handheld phones to speakers to food delivery options to transportation options potentially in a matter of years but not as obvious how those commercial applications might have broader impacts. Similarly, it is not as readily foreseeable as to what the AI applications are in the social and governmental spaces and how exactly AI may play a role in shifting and changing our society and our economy in the next decade. […] (Citing Anwin, Larson, Mattu, and Kirchner, Machine Bias, ProPublica (May 23, 2016) [as of Mar. 4, 2018].)
Notably, at the same time that this Committee began this joint-endeavor to generate greater understanding of the opportunities and challenges of AI within the Legislature, the Little Hoover Commission (LHC) was simultaneously studying the same topic. The LHC began its process, which included both public hearings and roundtables, with a public hearing on January 25, 2018, entitled “Artificial Intelligence: Applications and Implications.” At that first hearing, the LHC indicated that it ultimately intended to produce a report and policy recommendations about how the State of California can approach AI.
Indeed, in November 2018, the LHC produced its report, Artificial Intelligence: A Roadmap for California, wherein it similarly recognized the possible benefits and potential misuses of AI:
Imagine using AI applications to predict where fires may occur, detect early-stage wildfires, or guide firefighters where best to fight a fire and save lives. Conceive of an environment where AI could promote biodiversity and water conservation, and protect endangered species. See educators using AI to improve student learning and increase graduation rates. Envision better detection of diseases, including cancer, and more finely-tuned effective treatments. Certainly, such visions must be tempered with appropriate privacy protections and robust laws aimed at preventing the misuse of data. In addition, this encouraging future, which is presently knocking at our door, will require not just foresight but insight, not just political will but political action, and not just one mind but a collaboration of minds in government, academia, and private industry. (See Chairman Pedro Nava Opening Letter, LHC Report #245, Artificial Intelligence: A Roadmap for California (Nov. 2018), p. 1; hereinafter “LHC Report.”)
- Numerous bills on AI this year: This bill is one of many bills on AI this year. This Committee previously heard and approved AB 976 (Chau), which was supported by the LHC, and sought to create the Artificial Intelligence in State Government Services Commission. This Committee also recently heard and approved AB 459 (Kiley) which seeks to require the AB 976-proposed commission to develop various minimum standards for the use of AI in state government, and AB 594 (Salas) which, similar to AB 976 and this bill, seeks to establish a state advisory commission to help study and advise on the incorporation of AI, as well as requiring a Chief AI Officer within CDT. There are also bills in the Senate which could potentially address similar issues.
This bill, AB 1576, which seeks to establish an advisory working group comprised both of State officials and private individuals representing various industries, is focused on the use, benefits, and risks associated with the use of AI technology by California-based businesses. In that sense, the bill can be distinguished from, and complement, the AB 976 advisory commission, which would examine the use of AI in state government services specifically. In contrast, there may be some overlap with the AB 594 (Salas) bill, which establishes an advisory commission (as opposed to a working group) on the incorporation of AI, more generally. The authors of each of these and other AI bills should work together, to ensure that duplicative commissions and responsibilities are not created in state law.
- Better understanding the uses and implications of AI and AI technology within state government and by private companies: As noted in Comment 3, above, this Committee recently recognized at an informational hearing on AI, that there is a great need to expand the Legislature’s understanding of AI, with respect to commercial uses, as well as government and social uses. The ultimate goal of building such understanding is to engender more thoughtful public policy in the future, given that the opportunities and challenges posed by AI are significant, and in many ways still being uncovered. As recognized in the committees’ background paper, “[a]s is often the case when framing public policy in relation to technology issues, the path forward is not a simple or straightforward one, and technology often outpaces the policy. Writing sound policy requires a thorough understanding of the universe being regulated, and emerging technologies rarely lend themselves to quick studies.”
Similarly, the recent LHC Report encourages the Governor and Legislature to: (1) engage the public in broad conversations about the responsible use of AI; (2) identify the necessary training and other educational opportunities to build the human infrastructure required for an AI-fueled economy; (3) explore opportunities to form and leverage public private partnerships to use AI to address societal challenges; and (4) examine legislative and policy strategies to preserve and protect key public values: privacy, transparency and accountability. (LHC Report at p. 6.) The report also discusses how “[t]he possible benefits of AI are thus significant and should be broadly shared. […] A balance must be struck between AI investments by the private sector and AI training and education for workers. In the United States, spending on formal education has substantially outpaced spending on training. This is concerning because the advent of AI will require a much more nimble workforce. Without proper training, employees may find themselves without the skills necessary to succeed across a variety of sectors. California’s future labor force therefore stands to benefit from additional worker training and education, and workforce transition programs.” (Id. at 9.)
Notably, the LHC Report recognizes that “[AI] has the potential to change dramatically how governments do business, provide public services, predict and respond to natural disasters, conduct intelligence, improve national security and fight wars. More importantly, governments must implement strategies and policies to encourage the private sector to participate meaningfully in AI policy development.” (Id. at 10.)
This bill would arguably encourage the private sector to participate in AI policy development by participating in a working group that is intended to inform the benefits, risks, and uses of AI technology by California-based businesses. That being said, recognizing that a great deal of the impact in the private sector is the impact on the workforce supporting those businesses, the author may wish to consider if the composition of the working group should also include at least one representative with a labor or workforce development perspective.
- AI technology definition: As noted in this Committee’s joint informational hearing on AI last year, “[t]hough first coined by a Dartmouth professor, John McCarthy, in the 1950s, there still does not appear to be any singular, consistent definition of [AI] in use today, over 60 years later.” Staff notes that while this bill does not define “AI”, it does define “AI technology.” On the one hand, providing AI-related definitions for bills such as this, which seek to establish commissions or working groups to examine and provide recommendations on AI or AI technology uses, may help to provide helpful parameters and greater understanding of how the Legislature understood AI at the time of the commissions’ or working groups’ creation. On the other hand, given the varying definitions that exist even today, flexibility may be warranted, for example, to avoid any unnecessary limitations of the working group’s understanding of what constitutes AI technology. Leaving the term “undefined” with respect to any specific understanding of what is or is not “AI” at this given moment, may provide the working group greater flexibility in making its recommendations.
Specifically, this bill defines “AI technology” to mean a collection of different technologies that allow smart machines to extend human capabilities and intelligence by sensing, comprehending, acting, and learning, thereby allowing humans to achieve much more than can be achieved without the technology. That being said, the identified concerns above with providing a too-narrow definition of AI or AI technology are tempered here insofar as the bill also asks the proposed working group to provide findings and recommendations for modifications to the bill’s definition of AI technology.
- State CIO: This bill would require that the working group consist of certain participants, including the State CIO or their designee. Staff notes that when CDT was first created in state government, it was established as the Office of the State Chief Information Officer (OCIO). The head of that office was the State CIO. Since then, the OCIO has been renamed as CDT, which is headed by the Director of Technology. While state law makes clear that any reference to the State CIO is the same as referencing the Director of Technology (see Gov. Code Sec. 11545(a)), to avoid confusion, the author has accepted the following amendment to reflect updated terminology for the department.
Author’s amendment:
On page 3, line 10, strike “State Chief Information Officer” and insert “Director of Technology”
- Related legislation: AB 459 (Kiley) would require the Commission proposed by AB 976, below, to report to the Legislature on its recommended minimum standards for the use of AI in state government, as specified, on or before November 1, 2020. That bill contains contingent enactment language to ensure that it would take effect only if AB 976 is enacted and effective before January 1, 2020. This bill is pending hearing in the Assembly Accountability and Administrative Review Committee.
AB 594 (Salas) would enact the California Artificial Intelligence Act of 2020. The bill, no later than January 1, 2021, would require the appointment of a Chief AI Officer within the Department of Technology to evaluate the uses of AI in state government and to advise the Director of Technology on incorporating AI into state information technology strategic plans, policies, standards, and enterprise architecture. The bill would establish an advisory commission on AI, as specified, to study the potential uses of AI and develop a policy framework to manage the use of AI, as specified, and to submit to the Legislature on or before January 1, 2022, a report containing the findings, advice, and recommendations of the commission. This bill is pending hearing in the Assembly Labor and Employment Committee.
AB 976 (Chau et al.) would establish the Artificial Intelligence in State Government Services Commission as an advisory commission comprised of six appointed members and two designated ex officio members with certain knowledge and expertise related to the field of AI. This bill would require the Commission to: (1) convene a public process to gather certain input; (2) provide a plan for soliciting and incorporating AI and data science related demonstration projects into state services, as specified; and (3) provide its recommendations to the Legislature and the Governor by November 1, 2020. This bill is in the Assembly Appropriations Committee.
SB 348 (Chang) would require the secretaries of certain state agencies to devise a strategic plan, as specified, to utilize AI technology to improve state services. The bill would encourage the Governor to appoint a special adviser on AI to create a statewide strategic plan, as specified, to utilize AI technology to improve state services. The bill also would encourage certain entities to designate a chief AI officer. This bill is in the Senate Appropriations Committee.
SB 444 (Umberg) states the intent of the Legislature to enact legislation that would relate to civil actions and AI. This bill is currently in the Senate Governmental Organization Committee.
REGISTERED SUPPORT / OPPOSITION:
Support
None on file
Opposition
None on file
Analysis Prepared by: Ronak Daylami / P. & C.P. / (916) 319-2200
Bill Author
Bill Co-Author(s):