Donate

No summary is available yet.

Date	Status
Jan 21, 2020	Consideration of Governor's veto stricken from file.
Oct 2, 2019	Vetoed by Governor.
Sep 12, 2019	Enrolled and presented to the Governor at 3:30 p.m.
Sep 9, 2019	Senate amendments concurred in. To Engrossing and Enrolling. (Ayes 79. Noes 0.).
Sep 6, 2019	In Assembly. Concurrence in Senate amendments pending. May be considered on or after September 8 pursuant to Assembly Rule 77.
Sep 6, 2019	Read third time. Passed. Ordered to the Assembly.
Sep 3, 2019	Read second time. Ordered to third reading.
Aug 30, 2019	From committee: Amend, and do pass as amended. (Ayes 7. Noes 0.) (August 30).
Aug 12, 2019	In committee: Referred to APPR. suspense file.
Jul 9, 2019	From committee: Do pass and re-refer to Com. on APPR. (Ayes 15. Noes 0.) (July 9). Re-referred to Com. on APPR.
Jun 6, 2019	Referred to Com. on G.O.
May 24, 2019	In Senate. Read first time. To Com. on RLS. for assignment.
May 23, 2019	Read third time. Passed. Ordered to the Senate.
May 20, 2019	Read second time. Ordered to third reading.
May 16, 2019	From committee: Amend, and do pass as amended. (Ayes 13. Noes 0.) (May 16).
May 8, 2019	In committee: Set, first hearing. Referred to APPR. suspense file.
Apr 25, 2019	From committee: Do pass and re-refer to Com. on APPR. with recommendation: To Consent Calendar. (Ayes 7. Noes 0.) (April 24). Re-referred to Com. on APPR.
Apr 3, 2019	From committee: Do pass and re-refer to Com. on L. & E. (Ayes 11. Noes 0.) (April 2). Re-referred to Com. on L. & E.
Mar 20, 2019	Re-referred to Com. on P. & C.P.
Mar 19, 2019	From committee chair, with author's amendments: Amend, and re-refer to Com. on P. & C.P. Read second time and amended.
Mar 18, 2019	Referred to Coms. on P. & C.P. and L. & E.
Feb 15, 2019	From printer. May be heard in committee March 17.
Feb 14, 2019	Read first time. To print.

Date	Description	Location	Votes
Sep 9, 2019	AB 594 Salas Concurrence in Senate Amendments	Assembly Floor	78 0 0 Passed
Sep 6, 2019	Assembly 3rd Reading AB594 Salas By Umberg	Senate Floor	38 0 1 Passed
Aug 30, 2019	Do pass as amended	Senate Standing Committee on Appropriations	7 0 0 Passed
Aug 12, 2019	Placed on suspense file	Senate Standing Committee on Appropriations	7 0 0 Passed
Jul 9, 2019	Do pass, but first be re-referred to the Committee on [Appropriations]	Senate Standing Committee on Governmental Organization	15 0 1 Passed
May 23, 2019	AB 594 Salas Assembly Third Reading	Assembly Floor	77 0 2 Passed
May 16, 2019	Do pass as amended.	Assembly Standing Committee on Appropriations	13 0 5 Passed
Apr 24, 2019	Do pass and be re-referred to the Committee on [Appropriations] with recommendation: To Consent Calendar	Assembly Standing Committee on Labor and Employment	7 0 0 Passed
Apr 2, 2019	Do pass and be re-referred to the Committee on [Labor and Employment]	Assembly Standing Committee on Privacy and Consumer Protection	11 0 0 Passed

Key

Pro

Against

Did Not Vote

GOVERNOR'S VETO

GV Bill Id:AB 594

Author:(Salas)

As Enrolled Ver:September 10, 2019

2/3 vote

SUMMARY:

Authorizes the director of the California Department of Technology (CDT) to designate a position within the department to evaluate the uses of artificial intelligence (AI) in state government and to advise the Director of Technology on incorporating AI into state information technology (IT) strategic plans, policies, standards and enterprise architecture, and requires CDT to adopt guidelines by January 1, 2021 to govern the use and implementation of AI technologies in state government functions, as specified.

Replace a requirement for CDT to appoint a Chief AI Officer to evaluate the uses of AI in state government and to advise the Director, as specified, with authorization to designate a position within CDT with those same functions, instead.
Require the Director, by January 1, 2021 and in consultation with relevant stakeholders and in consideration of the recommendations of the Future of Work Commission, to adopt guidelines to govern the use and implementation of AI technologies in state government functions. These are to include guidelines on the ethical use of AI to benefit the public good; identifying projects and project components to increase efficiency; incorporating new and emerging technologies; and utilizing AI to benefit workers.

Governor's Veto Message:

This bill would authorize the Director of Technology to designate a position to evaluate the uses of artificial intelligence in state government and require the adoption of related guidelines by 2021.

New technologies will transform and enhance our public services and improve job quality for public servants. The Department of Technology is currently responsible for evaluating and leveraging new technologies such as artificial intelligence and providing enterprise-wide solutions focused on improving the state's digital service delivery to California residents. The newly established Future of Work Commission is broadly examining the impact of artificial intelligence on work and our economy. Legislation is not necessary to accomplish these goals.

COMMENTS:

Purpose of this bill: This bill seeks to encourage a dedicated position within the CDT to evaluate the uses of AI in state government and to advise CDT's Director on incorporating AI into state IT strategic plans, policies, standards and enterprise architecture. Ultimately, this bill seeks to require the adoption of guidelines to govern the use and implementation of AI technologies in state government functions. This is an author-sponsored bill.
Little Hoover Commission's 2018 report on AI: After holding a series of public hearings and roundtables on the applications and implications of AI, in November 2018, the Little Hoover Commission (LHC) issued a report, Artificial Intelligence: A Roadmap for California, with policy recommendations on how the State of California can approach AI, recognizing both the possible benefits and potential misuses of AI.

The LHC Report encourages the Governor and Legislature to: a) engage the public in broad conversations about the responsible use of AI, b) identify the necessary training and other educational opportunities to build the human infrastructure required for an AI-fueled economy, c) explore opportunities to form and leverage public private partnerships to use AI to address societal challenges; and, d) examine legislative and policy strategies to preserve and protect key public values: privacy, transparency and accountability (Id. at p. 6). Notably, the LHC Report warns that "California state government is underprepared to take the lead [in AI]. It lacks any single clear leadership and focus on the development and use of AI technology and applications to improve internal and external operations and services within an ethical framework" (Id. at 15).

Legislative efforts this year on AI: This bill is one of many bills introduced on the topic of AI this year. Other bills included AB 976 (Chau, et al.) of the current legislative session, which was supported by the LHC and sought to create the Artificial Intelligence in State Government Services Commission; AB 459 (Kiley) of the current legislative session, which sought to require the AB 976-proposed commission to develop various minimum standards for the use of AI in state government; and AB 1576 (Calderon) of the current legislative session, which sought to establish an AI working group, as specified, to evaluate the uses, risks, benefits, and legal implications associated with the development and deployment of AI by California-based businesses. Those bills were held under submission by the Senate Appropriations Committee. A Senate Bill, SB 348 (Chang) of the current legislative session, would have required CDT's Director to devise a strategic plan designed to aid departments and agencies with the incorporation of AI into state IT strategic plans, policies, standards, and enterprise architecture to improve state services; and also would have encouraged the Governor to appoint a special adviser on AI to create a statewide strategic plan, as specified, to utilize AI to improve state services. SB 348 remains in the Assembly Appropriations Committee.

According to the Author:

AI is already playing an increasing role in jobs throughout California. In many situations, a human-computer team can be more useful [than] either one by themselves. For example, a recent radiology study asked a human pathologist and an AI to determine if a photographed cell contained cancer or not. The AI had a 7.5% error rate and the human had a 3.5% error rate, but when the human and AI worked together the error rate dropped dramatically to 0.5%, an 85% reduction. […] As AI technology evolves, massive changes will come to California's workforce. [Fifty-one] percent of time spent in US occupations is highly susceptible to replacement by AI. The areas likely to be hit the hardest are data collection, data processing, and predictable physical work. […]. However, there is opportunity for growth in other sectors of the economy to make up for jobs lost to automation and AI. […]

Most California [s]tate [a]gencies, 80%, do not currently use AI, but almost a quarter say they intend to use AI in the next 10 years. Between chatbots, automatic processing, and profiling potential parolees, [s]tate [a]gencies are slowly introducing AI to the State workplace. All these new systems, however, are occurring independent of one another. There is not currently an overall plan for how the State should use AI, nor are there frameworks on how to prepare workers and students for the coming changes to the economy. […]

Arguments in Support:

Arguments in Opposition:

FISCAL COMMENTS:

According to the Senate Appropriations Committee, likely minor costs to CDT to develop guidelines.

VOTES:

ASM PRIVACY AND CONSUMER PROTECTION: 11-0-0

YES: Chau, Kiley, Bauer-Kahan, Berman, Calderon, Gabriel, Gallagher, Irwin, Obernolte, Smith, Wicks

ASM LABOR AND EMPLOYMENT: 7-0-0

YES: Kalra, Flora, Carrillo, Diep, Gonzalez, Jones-Sawyer, Luz Rivas

ASM APPROPRIATIONS: 13-0-5

YES: Gonzalez, Bloom, Bonta, Calderon, Carrillo, Chau, Eggman, Gabriel, Eduardo Garcia, Maienschein, Petrie-Norris, Quirk, Robert Rivas

ABS, ABST OR NV: Bigelow, Brough, Diep, Fong, Obernolte

ASSEMBLY FLOOR: 78-0-2

YES: Aguiar-Curry, Arambula, Bauer-Kahan, Berman, Bigelow, Bloom, Boerner Horvath, Bonta, Brough, Burke, Calderon, Carrillo, Cervantes, Chau, Chen, Chiu, Choi, Chu, Cooper, Cunningham, Dahle, Daly, Diep, Eggman, Flora, Fong, Frazier, Friedman, Gabriel, Gallagher, Cristina Garcia, Gipson, Gloria, Gonzalez, Gray, Grayson, Holden, Irwin, Jones-Sawyer, Kalra, Kamlager-Dove, Kiley, Lackey, Levine, Limón, Low, Maienschein, Mathis, Mayes, McCarty, Medina, Melendez, Mullin, Muratsuchi, Nazarian, O'Donnell, Obernolte, Patterson, Petrie-Norris, Quirk, Quirk-Silva, Ramos, Reyes, Luz Rivas, Robert Rivas, Rodriguez, Blanca Rubio, Salas, Santiago, Smith, Mark Stone, Ting, Voepel, Waldron, Weber, Wicks, Wood, Rendon

ABS, ABST OR NV: Cooley, Eduardo Garcia

SENATE FLOOR: 39-0-1

YES: Allen, Archuleta, Atkins, Bates, Beall, Borgeas, Bradford, Caballero, Chang, Dahle, Dodd, Durazo, Galgiani, Glazer, Lena Gonzalez, Grove, Hertzberg, Hill, Hueso, Hurtado, Jackson, Leyva, McGuire, Mitchell, Monning, Moorlach, Morrell, Nielsen, Pan, Portantino, Roth, Rubio, Skinner, Stern, Stone, Umberg, Wieckowski, Wiener, Wilk

ABS, ABST OR NV: Jones

ASSEMBLY FLOOR: 79-0-0

YES: Aguiar-Curry, Arambula, Bauer-Kahan, Berman, Bigelow, Bloom, Boerner Horvath, Bonta, Brough, Burke, Calderon, Carrillo, Cervantes, Chau, Chen, Chiu, Choi, Chu, Cooley, Cooper, Cunningham, Daly, Diep, Eggman, Flora, Fong, Frazier, Friedman, Gabriel, Gallagher, Cristina Garcia, Eduardo Garcia, Gipson, Gloria, Gonzalez, Gray, Grayson, Holden, Irwin, Jones-Sawyer, Kalra, Kamlager-Dove, Kiley, Lackey, Levine, Limón, Low, Maienschein, Mathis, Mayes, McCarty, Medina, Melendez, Mullin, Muratsuchi, Nazarian, Obernolte, O'Donnell, Patterson, Petrie-Norris, Quirk, Quirk-Silva, Ramos, Reyes, Luz Rivas, Robert Rivas, Rodriguez, Blanca Rubio, Salas, Santiago, Smith, Mark Stone, Ting, Voepel, Waldron, Weber, Wicks, Wood, Rendon

UPDATED:

VERSION: September 10, 2019

CONSULTANT: Ronak Daylami / P. & C.P. / (916) 319-2200 FN: 0002494

CONCURRENCE IN SENATE AMENDMENTS

CSA1 Bill Id:AB 594

Author:(Salas)

As Amended Ver:August 30, 2019

Majority vote

SUMMARY:

The Senate Amendments:

Replace a requirement for CDT to appoint a Chief AI Officer to evaluate the uses of AI in state government and to advise the Director, as specified, with authorization to designate a position within CDT with those same functions, instead.
Require the Director, by January 1, 2021 and in consultation with relevant stakeholders and in consideration of the recommendations of the Future of Work Commission, to adopt guidelines to govern the use and implementation of AI technologies in state government functions. These are to include guidelines on: the ethical use of AI to benefit the public good; identifying projects and project components to increase efficiency; incorporating new and emerging technologies; and utilizing AI to benefit workers.

COMMENTS:

Purpose of this bill: This bill seeks to encourage a dedicated position within the CDT to evaluate the uses of AI in state government and to advise CDT's Director on incorporating AI into state IT strategic plans, policies, standards and enterprise architecture. Ultimately, this bill seeks to require the adoption of guidelines to govern the use and implementation of AI technologies in state government functions. This is an author-sponsored bill.
Little Hoover Commission's 2018 report on AI: After holding a series of public hearings and roundtables on the applications and implications of AI, in November 2018, the Little Hoover Commission (LHC) issued a report with policy recommendations on how the State of California can approach AI: Artificial Intelligence: A Roadmap for California. The report recognized the possible benefits and potential misuses of AI:

Imagine using AI applications to predict where fires may occur, detect early-stage wildfires, or guide firefighters where best to fight a fire and save lives. Conceive of an environment where AI could promote biodiversity and water conservation, and protect endangered species. See educators using AI to improve student learning and increase graduation rates. Envision better detection of diseases, including cancer, and more finely-tuned effective treatments. Certainly, such visions must be tempered with appropriate privacy protections and robust laws aimed at preventing the misuse of data. In addition, this encouraging future, which is presently knocking at our door, will require not just foresight but insight, not just political will but political action, and not just one mind but a collaboration of minds in government, academia, and private industry (See Chairman Pedro Nava Opening Letter, LHC Report #245, Artificial Intelligence: A Roadmap for California (Nov. 2018), p. 1; hereinafter "LHC Report").

The LHC Report encourages the Governor and Legislature to: 1) engage the public in broad conversations about the responsible use of AI, 2) identify the necessary training and other educational opportunities to build the human infrastructure required for an AI-fueled economy, 3) explore opportunities to form and leverage public private partnerships to use AI to address societal challenges; and, 4) examine legislative and policy strategies to preserve and protect key public values: privacy, transparency and accountability (Id. at p. 6). Notably, the LHC Report warns that "California state government is underprepared to take the lead [in AI]. It lacks any single clear leadership and focus on the development and use of AI technology and applications to improve internal and external operations and services within an ethical framework" (Id. at 15).

Legislative efforts this year on AI: This bill is one of many bills introduced on the topic of AI this year. Other bills included AB 976 (Chau, et al.) of the current legislative session, which was supported by the LHC and sought to create the Artificial Intelligence in State Government Services Commission; AB 459 (Kiley) of the current legislative session, which sought to require the AB 976-proposed commission to develop various minimum standards for the use of AI in state government; and AB 1576 (Calderon) of the current legislative session, which sought to establish an AI working group, as specified, to evaluate the uses, risks, benefits, and legal implications associated with the development and deployment of AI by California-based businesses. Those bills were held under submission by the Senate Appropriations Committee. A Senate Bill, SB 348 (Chang) of the current legislative session, would have required CDT's Director to devise a strategic plan designed to aid departments and agencies with the incorporation of AI into state IT strategic plans, policies, standards, and enterprise architecture to improve state services; and also would have encouraged the Governor to appoint a special adviser on AI to create a statewide strategic plan, as specified, to utilize AI to improve state services. SB 348 remains in the Assembly Appropriations Committee.

According to the Author:

As AI technology evolves, massive changes will come to California's workforce. [Fifty-one] percent of time spent in US occupations is highly susceptible to replacement by AI. The areas likely to be hit the hardest are data collection, data processing, and predictable physical work. […]. However, there is opportunity for growth in other sectors of the economy to make up for jobs lost to automation and AI. […]

Arguments in Support:

None on file.

Arguments in Opposition:

None on file.

FISCAL COMMENTS:

According to the Senate Appropriations Committee, likely minor costs to CDT to develop guidelines.

VOTES:

ASM PRIVACY AND CONSUMER PROTECTION: 11-0-0

YES: Chau, Kiley, Bauer-Kahan, Berman, Calderon, Gabriel, Gallagher, Irwin, Obernolte, Smith, Wicks

ASM LABOR AND EMPLOYMENT: 7-0-0

YES: Kalra, Flora, Carrillo, Diep, Gonzalez, Jones-Sawyer, Luz Rivas

ASM APPROPRIATIONS: 13-0-5

YES: Gonzalez, Bloom, Bonta, Calderon, Carrillo, Chau, Eggman, Gabriel, Eduardo Garcia, Maienschein, Petrie-Norris, Quirk, Robert Rivas

ABS, ABST OR NV: Bigelow, Brough, Diep, Fong, Obernolte

ASSEMBLY FLOOR: 78-0-2

YES: Aguiar-Curry, Arambula, Bauer-Kahan, Berman, Bigelow, Bloom, Boerner Horvath, Bonta, Brough, Burke, Calderon, Carrillo, Cervantes, Chau, Chen, Chiu, Choi, Chu, Cooper, Cunningham, Dahle, Daly, Diep, Eggman, Flora, Fong, Frazier, Friedman, Gabriel, Gallagher, Cristina Garcia, Gipson, Gloria, Gonzalez, Gray, Grayson, Holden, Irwin, Jones-Sawyer, Kalra, Kamlager-Dove, Kiley, Lackey, Levine, Limón, Low, Maienschein, Mathis, Mayes, McCarty, Medina, Melendez, Mullin, Muratsuchi, Nazarian, O'Donnell, Obernolte, Patterson, Petrie‑Norris, Quirk, Quirk-Silva, Ramos, Reyes, Luz Rivas, Robert Rivas, Rodriguez, Blanca Rubio, Salas, Santiago, Smith, Mark Stone, Ting, Voepel, Waldron, Weber, Wicks, Wood, Rendon

ABS, ABST OR NV: Cooley, Eduardo Garcia

SENATE FLOOR: 39-0-1

ABS, ABST OR NV: Jones

UPDATED:

VERSION: August 30, 2019

CONSULTANT: Ronak Daylami / P. & C.P. / (916) 319-2200 FN: 0002178

SENATE RULES COMMITTEE

Office of Senate Floor Analyses

(916) 651-1520 Fax: (916) 327-4478

AB 594

THIRD READING

Bill No: AB 594

Author: Salas (D)

Amended: 8/30/19 in Senate

Vote: 21

SENATE GOVERNMENTAL ORG. COMMITTEE: 15-0, 7/9/19

AYES: Dodd, Wilk, Allen, Archuleta, Bradford, Chang, Galgiani, Glazer, Hill, Hueso, Jones, Nielsen, Portantino, Rubio, Wiener

NO VOTE RECORDED: Borgeas

SENATE APPROPRIATIONS COMMITTEE: 7-0, 8/30/19

AYES: Portantino, Bates, Bradford, Durazo, Hill, Jones, Wieckowski

ASSEMBLY FLOOR: 78-0, 5/23/19 - See last page for vote

SUBJECT: Artificial intelligence

SOURCE: Author

DIGEST: This bill authorizes the Director of the California Department of Technology (CDT) to designate a position within the department to evaluate the uses of artificial intelligence (AI) in state government. This bill also requires CDT to adopt guidelines by January 1, 2021, to govern the use and implementation of AI technologies in state government functions.

ANALYSIS:

Existing law:

Establishes CDT which is responsible for the approval and oversight of information technology projects by, among other things, consulting with state agencies during initial projects planning to ensure that project proposals are based on well-defined programmatic needs.
Requires CDT to establish and enforce state information strategic plans, policies, standards, and enterprise architecture.
Establishes the California Workforce Development Board (CWDB) and makes the board responsible for assisting the Governor in the development, oversight, and continuous improvement of California’s workforce investment system and the alignment of the education and workforce investment systems to the needs of the 21^st century economy and workforce.
Requires the CWDB to assist the Governor in, among other things, the review of statewide policies, statewide programs, and of recommendations on actions that should be taken by the state to align workforce, education, training, and employment funding programs in the state in a manner that supports a comprehensive and streamlined workforce development system in the state, as specified.
Establishes, under Executive Order N-11-19, the Future of Work Commission whose primary mission is to study, understand, analyze, and make recommendations regarding, among other things, the impact on technology on work, workers, employers, jobs, and society.

This bill:

Provides that the Director CDT may designate a position within CDT to evaluate the uses of AI in state government and to advise the director on incorporating AI into state information technology strategic plans, policies, standards, and enterprise architecture.
Requires CDT, by January 1, 2021, in consultation with relevant stakeholders and in consideration of the recommendations of the Future of Work Commission, to adopt guidelines to govern the use and implementation of AI technologies in state government functions.
Requires the guidelines to include standards for all of the following:
1. The ethical use of AI to benefit the public good.
2. Identifying projects and project components to increase efficiency.
3. Incorporating new and emerging technologies.
4. Utilizing AI to benefit workers.

Background

Artificial Intelligence. First coined in 1956, AI is a wide-ranging branch of computer science focused on building smart machines capable of performing tasks that typically require human intelligence. AI makes it possible for computers to learn from experience, adjust to new inputs and perform “human-like” tasks.

Using AI, computers can be “trained” to accomplish unique tasks by processing large amounts of data and recognizing patterns in that data. While many AI systems have the ability to learn over time, not all can do so. Some of the most well-known AI technologies include image recognition software, Siri, Alexa, and other personal assistants, and self-driving cars.

AI technology can also be used across a wide array of sectors including the public sector. For example, at the federal level, the Department of Veteran Affairs is using AI to better predict medical complications and improve treatment of combat wounds. Some cities are also beginning to leverage the type of responsive dispatching and routing used by ride-hailing services, and linking it with scheduling and tracking software for public transportation.

While AI has the potential to address some of humanity’s biggest challenges, a recent report by the National Science and Technology Council argues that, “use of AI to make consequential decisions about people, often replacing decisions made by human-driven bureaucratic processes, leads to concerns about how to ensure justice, fairness, and accountability.” Similarly, because AI depends solely on data input, there are concerns that if the data used reflects past decisions that are biased, the results could also perpetuate that bias.

The use of AI technology also leads to concerns from workers that their jobs could be lost to computers that utilize AI technology. The author’s office argues that, “this bill would help ensure that AI is used to improve government services, avoid potential pitfalls, and prepare and train workers for the future.”

Little Hoover report. Late last year, the Little Hoover Commission (LHC) released a report titled, “Artificial Intelligence: A Roadmap for California” which detailed the possible benefits and risks of AI.

Specifically, the report encouraged the Governor and the Legislature to: engage the public in broad conversations about the responsible use of AI; identify the necessary training and other educational opportunities to build the human infrastructure required for an AI-fueled economy; explore opportunities to form and leverage public private partnerships to use AI to address societal challenges; and examine legislative and policy strategies to preserve and protect key public values, such as privacy, transparency and accountability.

Related/Prior Legislation

SB 348 (Chang, 2019) requires the CDT and the Department of Motor Vehicles to each devise a strategic plan, as specified, to utilize AI technology to improve state services. (Never heard in the Assembly Appropriations Committee)

SB 730 (Stern, 2019) establishes the Commission on Tech Equity and the Future of Work to conduct research to understand the impact of technology on workers, employers, and the economy of the state. (Pending in the Assembly Appropriations Committee)

AB 459 (Kiley, 2019) would have required the AI in State Government Services Commission proposed by AB 976 (Chau, 2019) to report to the Legislature on its recommended minimum standards for the use of AI in fostering accountability in state government services and prioritizing the safety and security of AI technologies used by state government. (Held in the Senate Appropriations Committee Suspense File)

AB 976 (Chau, 2019) would have established the AI in State Government Commission as an advisory Commission and required the commission to provide a plan for soliciting and incorporating AI and data science related demonstration projects into state services, as specified. (Held in the Senate Appropriations Committee Suspense File)

AB 1576 (Calderon, 2019) would have required the Secretary of Government Operations to appoint an advisory AI technology working group, and required the working group to report to the Legislature on the potential uses, risks, and benefits of the use of AI technology by California based businesses on or before July 1, 2021, as specified. (Held in the Senate Appropriations Committee Suspense File)

FISCAL EFFECT: Appropriation: No Fiscal Com.: Yes Local: No

According to the Senate Appropriations Committee, likely minor costs to CDT to develop guidelines.

SUPPORT: (Verified 8/30/19)

Little Hoover Commission

OPPOSITION: (Verified 8/30/19)

None received

ARGUMENTS IN SUPPORT: According to the Little Hoover Commission, “AB 594 would advance the goals of the Commission’s recommendations and help CDT focus on the strategic developments and use of AI technology for the benefit of Californians.”

ASSEMBLY FLOOR: 78-0, 5/23/19

AYES: Aguiar-Curry, Arambula, Bauer-Kahan, Berman, Bigelow, Bloom, Boerner Horvath, Bonta, Brough, Burke, Calderon, Carrillo, Cervantes, Chau, Chen, Chiu, Choi, Chu, Cooper, Cunningham, Dahle, Daly, Diep, Eggman, Flora, Fong, Frazier, Friedman, Gabriel, Gallagher, Cristina Garcia, Gipson, Gloria, Gonzalez, Gray, Grayson, Holden, Irwin, Jones-Sawyer, Kalra, Kamlager-Dove, Kiley, Lackey, Levine, Limón, Low, Maienschein, Mathis, Mayes, McCarty, Medina, Melendez, Mullin, Muratsuchi, Nazarian, Obernolte, O'Donnell, Patterson, Petrie-Norris, Quirk, Quirk-Silva, Ramos, Reyes, Luz Rivas, Robert Rivas, Rodriguez, Blanca Rubio, Salas, Santiago, Smith, Mark Stone, Ting, Voepel, Waldron, Weber, Wicks, Wood, Rendon

NO VOTE RECORDED: Cooley, Eduardo Garcia

Prepared by: Felipe Lopez / G.O. / (916) 651-1530

9/3/19 11:00:46

**** END ****

SENATE COMMITTEE ON

Senator Anthony Portantino, Chair

2019 - 2020 Regular Session

AB 594 (Salas) - Artificial intelligence


Version:	Policy Vote:
Urgency:	Mandate:
Hearing Date:	Consultant:

Bill Summary:

AB 594 would require by January 1, 2021, the Director of Technology to appoint a Chief Artificial Intelligence (AI) Officer within the Department of Technology (CDT) to evaluate the uses of artificial intelligence in state government and to advise the Director on incorporating artificial intelligence into state information technology strategic plans, policies, standards, and enterprise architecture.

Fiscal Impact:

Ongoing costs to CDT of approximately $332,000 per year in salary and benefits to the AI Officer and support staff. (GF)

Background:

Artificial Intelligence (AI) is a wide-ranging branch of computer science focused on building smart machines capable of performing tasks that typically require human intelligence. AI makes it possible for computers to learn from experience, adjust to new inputs, and perform “human-like” tasks. Some of the most well-known AI technologies include image recognition software, Siri, Alexa, Watson, and self-driving cars.

Because AI depends solely on data input, there are concerns that if the data used reflects past decisions that are biased, the results could also perpetuate that bias. The use of AI technology also leads to concerns from workers that their jobs could be lost to computers that utilize AI technology. In 2018, the Little Hoover Commission released a report titled, “Artificial Intelligence: A Roadmap for California” which detailed the possible benefits and risks of AI. The report encouraged the Governor and the Legislature to: engage the public in broad conversations about the responsible use of AI; identify the necessary training and other educational opportunities to build the human infrastructure required for an AI-fueled economy; explore opportunities to form and leverage public private partnerships to use AI to address societal challenges; and examine legislative and policy strategies to preserve and protect key public values, such as privacy, transparency and accountability.

Proposed Law:

This bill would require by January 1, 2021, the Director of Technology to appoint a Chief Artificial Intelligence Officer within CDT to evaluate the uses of artificial intelligence in state government and to advise the Director on incorporating artificial intelligence into state information technology strategic plans, policies, standards, and enterprise architecture. The AI Officer would be required to advise the California Workforce Development Board in its review of statewide policies and programs related to workforce, education, training, and employment for the purpose of preparing and retraining workers whose jobs may be displaced or transformed by artificial intelligence.

Related Legislation:

SB 348 (Chang, 2019) would require CDT to devise a strategic plan designed to aid departments and agencies with the incorporation of artificial intelligence into state information technology strategic plans, policies, standards, and enterprise architecture to improve state services. The measure is pending in the Assembly Appropriations Committee.

SB 730 (Stern, 2019) would create the Commission on the Future of Work and require the Commission to conduct research to understand the impact of technology on workers, employers, and the economy of the state. The measure is pending in the Assembly Appropriations Committee.

SB 752 (Stern, 2019) would have created the Commission on Tech Equity to convene a public process to gather input and to understand the economic, social, workplace, and technological landscape of innovation and technology in California. The measure was held in the Senate Appropriations Committee.

AB 459 (Kiley, 2019) would have directed the AI Commission (proposed in AB 976 (Chau, 2019)), to report its recommended minimum standards for the use of AI in state government to the Legislature. The measure was held in the Assembly Appropriations Committee.

AB 976 (Chau, 2019) would establish the Artificial Intelligence in State Government Services Commission to gather input on how artificial intelligence and data science could be used to improve state services. The measure is pending in the Senate Appropriations Committee.

AB 1576 would require the Secretary of Government Operations to appoint an AI technology working group and require the working group to report to the Legislature on the potential uses, risks, and benefits on the use of AI technology by California base businesses. The measure is pending in the Senate Appropriations Committee.

Staff Comments:

Staff notes that this bill is one of a handful of bills related to AI technology in this Legislative Session. SB 730 (Stern, 2019) would establish the Commission on Tech Equity to conduct research to understand the impact of technology on workers, employers, and the economy of the state. AB 1576 (Calderon, 2019) would require the Secretary of Government Operations to appoint an advisory AI technology working group. AB 976 (Chau, 2019) would establish the AI Commission and require the Commission to provide a plan for soliciting and incorporating AI and data science related demonstration projects into state services. While all these bills are not focused on the same aspects of AI technology, many of them would require duplicative work and could result in inconsistent recommendations to the Legislature.

-- END --

SENATE COMMITTEE ON GOVERNMENTAL ORGANIZATION

Senator Bill Dodd

Chair

2019 - 2020 Regular

Bill No: 594 Hearing Date: 7/9/2019

Author:
Version:	5/16/2019 Amended
Urgency:	No	Fiscal:	Yes
Consultant:	Felipe Lopez

SUBJECT: Artificial intelligence

DIGEST: This bill requires the Director of the California Department of Technology (CDT), no later than January 1, 2021, to appoint a Chief Artificial Intelligence (AI) Officer to evaluate the uses of AI in state government and to advise the Director of CDT on incorporating AI into state information technology strategic plans, policies, standards, and enterprise architecture. Additionally, the bill requires the AI Officer to advise the California Workforce Development Board (CWDB) in its review of statewide policies and programs related to workforce, education, training, and employment, as specified.

ANALYSIS:

Existing law:

Establishes CDT which is responsible for the approval and oversight of information technology projects by, among other things, consulting with state agencies during initial projects planning to ensure that project proposals are based on well-defined programmatic needs.
Requires CDT to establish and enforce state information strategic plans, policies, standards, and enterprise architecture.
Establishes the CWDB and makes the board responsible for assisting the Governor in the development, oversight, and continuous improvement of California’s workforce investment system and the alignment of the education and workforce investment systems to the needs of the 21^st century economy and workforce.
Requires the CWDB to assist the Governor in, among other things, the review of statewide policies, statewide programs, and of recommendations on actions that should be taken by the state to align workforce, education, training, and employment funding programs in the state in a manner that supports a comprehensive and streamlined workforce development system in the state, as specified.

This bill:

Requires the Director of CDT, no later than January 1, 2021, to appoint an AI Officer to evaluate the uses of AI in state government and to advise the director on incorporating AI into state information technology strategic plans, policies, standards, and enterprise architecture.
Requires the AI Officer to advise the CWDB in its review of statewide policies and programs related to workforce, education, training, and employment for the purpose of preparing and retraining workers whose jobs may be displaced or transformed by AI.

Background

Purpose of the bill. According to the author’s office, “with the accelerated development and deployment of AI and its increased role in government functions, it is critical that there is a designated expert at the state level to oversee and evaluate the use of AI to ensure that its being used to improve government functions and services. Furthermore, analyses of the transformative nature of emerging technologies that utilize AI all tend to alert or allude to the significant impact to the workforce, both in terms of job displacement and new opportunities. This bill would direct the AI Officer to work with and advise the CWDB to help prepare and retrain workers whose jobs may be displaced or transformed by AI. This bill would help ensure that AI is used to improve government services, avoid potential pitfalls, and prepare and train workers for the future.”

California Workforce Development Board. The CWDB was established by Executive Order in response to the mandate of the federal Workforce Investment Act of 1998. The CWDB assists the Governor in setting and guiding policy in the area of workforce development. All members of the CWDB are appointed by the Governor and represent the many facets of workforce development – business, labor, public education, higher education, economic development, youth activities, employment and training, as well as the Legislature.

Current law requires the CWDB to promote the development of a well-educated and highly skilled 21^st century workforce. The CWDB is also responsible for developing, implementing, and modifying the State Plan. The State Plan serves as the comprehensive framework and coordinated plan for the aligned investment of all federal and state workforce training and employment services funding streams and programs. This bill would require the AI officer to advise the CWDB in its review of statewide policies and programs related to workforce, education, training and employment in the State Plan.

Policy consideration. This bill is one of a handful of bills related to AI technology in this Legislative Session. For instance, earlier this year this Committee passed SB 730 (Stern, 2019) which established the Commission on the Future of Work to conduct research to understand the impact of technology on workers, employers, and the economy of the state. Similarly, the Committee also passed AB 976 (Chau, 2019) which would establish the AI Commission in State Government and require the commission to provide a plan for soliciting and incorporating AI and data science related demonstration projects into state services.

Furthermore, the Committee also passed AB 1576 (Calderon, 2019) which would require the Secretary of Government Operations to appoint an AI technology working group and require the working group to report to the Legislature on the potential uses, risks, and benefits on the sure of AI technology by California based businesses. Understanding that these bills are not entirely focused on the same aspects of AI technology in California, it raises the issues that if all of these bills become law, many of the work that would be done by the various working groups and commissions could become duplicative and result in inconsistent recommendations to the Legislature. The authors of these measures should work together to ensure that does not happen.

Prior/Related Legislation

SB 348 (Chang, 2019) would require the CDT and the Department of Motor Vehicles to each devise a strategic plan, as specified, to utilize AI technology to improve state services. (Pending in the Assembly Accountability and Administrative Review Committee)

AB 976 (Chau, 2019) would establish the AI in State Government Commission as an advisory Commission and require the commission to provide a plan for soliciting and incorporating AI and data science related demonstration projects into state services, as specified. (Pending in the Senate Appropriations Committee)

AB 1576 (Calderon, 2019) would require the Secretary of Government Operations to appoint an advisory AI technology working group, and requires the working group to report to the Legislature on the potential uses, risks, and benefits of the use of AI technology by California based businesses on or before July 1, 2021, as specified. (Pending in the Senate Appropriations Committee)

FISCAL EFFECT: Appropriation: Fiscal Com.: Local:

SUPPORT:

Little Hoover Commission

OPPOSITION:

None received

ASSEMBLY THIRD READING

ATR Bill Id:AB 594

Author:(Salas)

As Amended Ver:May 16, 2019

Majority vote

SUMMARY:

This bill requires, no later than January 1, 2021, the director of the Department of Technology to appoint a Chief Artificial Intelligence (AI) Officer within the department to evaluate the uses of AI in state government and to advise the Director of Technology on incorporating AI into state information technology (IT) strategic plans, policies, standards and enterprise architecture.

Major Provisions

COMMENTS:

Last spring, the Privacy and Consumer Protection Committee held a joint informational hearing with the Assembly Select Committee on Emerging Technologies & Innovation on the topic of AI, to begin a preliminary discussion of the promises and challenges presented by AI. The overarching goal of the hearing was to bring members and staff a greater understanding of AI in order to engender more thoughtful public policy in the future.

Notably, at the same time that the committees began this joint-endeavor to generate greater understanding of the opportunities and challenges of AI within the Legislature, the Little Hoover Commission (LHC) was simultaneously studying the same topic. The LHC began its process, which included both public hearings and roundtables, with a public hearing on January 25, 2018, entitled "Artificial Intelligence: Applications and Implications." At that first hearing, the LHC indicated that it ultimately intended to produce a report and policy recommendations about how the State of California can approach AI. Indeed, in November 2018, the LHC produced its report, Artificial Intelligence: A Roadmap for California, wherein it similarly recognized the possible benefits and potential misuses of AI.

This bill is one of many bills on AI this year. AB 976 (Chau), supported by the LHC, seeks to create the Artificial Intelligence in State Government Services Commission (Commission), and AB 459 (Kiley) seeks to require that the AB 976-proposed Commission develop various minimum standards for the use of AI in state government. There are also bills in the Senate which could potentially address similar issues. Similarly, AB 1576 (Calderon) establishes an advisory working group comprised both of State officials and private individuals representing various industries, and is focused on the use, benefits, and risks associated with the use of AI technology by California-based businesses.

This bill, AB 594, requires the director of the Department of Technology to appoint a Chief AI Officer within the department to advise the Director of Technology on incorporating AI into state IT strategic plans, policies, standards and enterprise architecture. In that sense, the bill can be distinguished from, and complement, the AB 976 Commission, which would examine the use of AI in state government services specifically and appears consistent with the statutory responsibilities of the Director to advise the Governor on the strategic management and direction of the state's IT resources, to provide technology direction to agency and department chief information officers to ensure the integration of statewide technology initiatives, and to establish and enforce state IT strategic plans, policies, standards, and enterprise architecture. (See Government Code Section 11545(b).)

According to the Author:

As AI technology evolves, massive changes will come to California's workforce. [Fifty-one] percent of time spent in US occupations is highly susceptible to replacement by AI. The areas likely to be hit the hardest are data collection, data processing, and predictable physical work. A 2017 study by McKinsey and Company predicted a loss of 39 million jobs and displacement of 23 percent of the workforce in the U.S. by 2030 due to increased automation.

However, there is opportunity for growth in other sectors of the economy to make up for jobs lost to automation and AI. An aging population and new green technologies could create demand for more doctors, nurses, manufacturing workers, construction workers, etc. There will also likely be an expansion in the technology sector, as the need for those who understand and can manage AI systems increases.

All these new systems, however, are occurring independent of one another. There is not currently an overall plan for how the State should use AI, nor are there frameworks on how to prepare workers and students for the coming changes to the economy. This bill and the Chief Artificial Intelligence Officer it creates would develop such a plan, and help guide California through this new area of opportunity.

Arguments in Support:

None on file.

Arguments in Opposition:

None on file.

FISCAL COMMENTS:

Possible one-time start-up costs and annual cost pressures in excess of $150,000 annually. Actual costs will depend on the extent of travel and expenses incurred by the members of the AI Commission, as well possible administrative and staff costs to operate the commission, and costs associated with drafting and disseminating recommendation to the Legislature and the governor. For example, the Commission created by this bill appears similar in structure and duties to the Little Hoover Commission (LHC). The LHC operating budget for fiscal year 2017-2018 was approximately $1.2 million dollars.
Costs (General Fund) of approximately $332,000 to CDT in salary and benefits for the Chief Artificial Intelligence Officer with the CDT.

VOTES:

ASM PRIVACY AND CONSUMER PROTECTION: 11-0-0

YES: Chau, Kiley, Bauer-Kahan, Berman, Calderon, Gabriel, Gallagher, Irwin, Obernolte, Smith, Wicks

ASM LABOR AND EMPLOYMENT: 7-0-0

YES: Kalra, Flora, Carrillo, Diep, Gonzalez, Jones-Sawyer, Luz Rivas

ASM APPROPRIATIONS: 13-0-5

YES: Gonzalez, Bloom, Bonta, Calderon, Carrillo, Chau, Eggman, Gabriel, Eduardo Garcia, Maienschein, Petrie-Norris, Quirk, Robert Rivas

ABS, ABST OR NV: Bigelow, Brough, Diep, Fong, Obernolte

UPDATED:

VERSION: May 16, 2019

CONSULTANT: Ronak Daylami / P. & C.P. / (916) 319-2200 FN: 0000853

Date of Hearing: May 8, 2019

ASSEMBLY COMMITTEE ON APPROPRIATIONS

Lorena Gonzalez, Chair

ABPCA Bill Id:AB 594

Author:(Salas) – As Amended Ver:March 19, 2019

Policy Committee:	Privacy and Consumer Protection	Vote:	11 - 0
	Labor and Employment		7 - 0

Urgency: No State Mandated Local Program: No Reimbursable: No

SUMMARY:

This bill creates the California Artificial Intelligence Act of 2020 (CAIA) and requires the appointment of a Chief Artificial Intelligence Officer within the California Department of Technology (CDT) to evaluate the uses of artificial intelligence (AI) in state government and advise the CDT director and the California Workforce Development Board (CWDB).

This bill also creates an advisory Commission on Artificial Intelligence (Commission) to study the potential uses of AI and develop a policy framework to manage the use of AI and requires the Commission to provide an advisory report to the Legislature on or before January 1, 2022.

FISCAL EFFECT:

Possible one-time start-up costs and annual cost pressures in excess of $150,000 annually. Actual costs will depend on the extent of travel and expenses incurred by the members of the AI Commission, as well possible administrative and staff costs to operate the commission, and costs associated with drafting and disseminating recommendation to the Legislature and the governor. For example, the Commission created by this bill appears similar in structure and duties to the Little Hoover Commission (LHC). The LHC operating budget for fiscal year 2017-2018 was approximately $1.2 million dollars.
Costs (GF) of approximately $332,000 to CDT in salary and benefits for the Chief Artificial Intelligence Officer with the CDT.

COMMENTS:

Purpose and Background. According to the author:

AI has proven itself incredibly useful in managing large systems. For instance, AI-enabled smart traffic management applications reduce wait times, energy use, and emissions by as much as 25 percent. There are already fleets of autonomous sailboats and watercraft that patrol the oceans collecting data on changes to the ecosystems, a much cheaper and safer option than having crewed vessels.

Background. In November 2018, the Little Hoover Commission issued a report entitled “Artificial Intelligence: A Roadmap for California.” The report notes the state lacks any single clear leadership and focus on the development and use of AI technology and applications to improve internal and external operations and services within an ethical framework. The creation of an AI Commission was among its nine recommendations for action to the governor and Legislature.
Related Legislation.
AB 459 (Kiley) directs the AI Commission created by this bill to report its recommended minimum standards for the use of AI in state government to the Legislature. AB 459 is pending in this committee.
AB 976 (Chau) establishes the Artificial Intelligence in State Government Services Commission as an advisory body in state government. AB 976 is pending in this committee.

Analysis Prepared by: Kimberly Horiuchi / APPR. / (916) 319-2081

Date of Hearing: April 24, 2019

ASSEMBLY COMMITTEE ON LABOR AND EMPLOYMENT

Ash Kalra, Chair

ABPCA Bill Id:AB 594 (

Author:Salas) – As Amended Ver:March 19, 2019

SUBJECT: Artificial intelligence

SUMMARY: Establishes the California Artificial Intelligence Act of 2020 and requires the appointment of a Chief Artificial Intelligence Officer within the California Department of Technology (CDT) to evaluate the uses of artificial intelligence (AI) in state government and advise the CDT director and the California Workforce Development Board.

Establishes an advisory Commission on Artificial Intelligence (Commission) to study the potential uses of AI and develop a policy framework to manage the use of AI. The Commission would be required to provide an advisory report to the Legislature on or before January 1, 2022.

Specifically, this bill:

Enacts the California Artificial Intelligence Act of 2020 (Act) and requires, no later than January 1, 2021, that the director of CDT appoint a Chief AI Officer within CDT to evaluate the uses of AI in state government and to advise the CDT director on incorporating AI into state information technology strategic plans, policies, standards, and enterprise architecture.
Requires that the Chief AI Officer advise the California Workforce Development Board in its review of statewide policies and programs related to workforce, education, training, and employment for the purpose of preparing and retraining workers whose jobs may be displaced or transformed by AI.
Establishes an advisory Commission comprised of eight members, including the Chief AI Officer, the Secretary of Labor, and six appointed members, which would include representatives from the private industry, higher education, and organized labor, as specified.
Requires the Commission to study the potential uses of AI and develop a policy framework to manage the use of AI as specified, and to submit to the Legislature on or before January 1, 2022, a report containing the findings, advice, and recommendations of the Commission. Requires that the framework include advice and recommended strategies to do the following:
1. Establish an ethical framework informed by the 23 Asilomar AI Principles and ensure the ethical use of artificial intelligence.
2. Maximize the standard of living and minimize suffering for all humans through the use of AI.
3. Encourage state agencies to create and implement a strategic plan to utilize AI in the execution of state functions.
4. Educate and train students and workers in the application of AI and to prepare them for the jobs of the future.
5. Minimize any potential negative impact of AI on the labor market.
Specifies that the report, above, will be advisory only and that there shall be no authority or obligation on the part of the state, or the parties meeting and conferring, to implement the findings of the Commission without the enactment of subsequent legislation to implement the report.
Defines various terms for purposes of this Act, including, among other things:
1. “23 Asilomar AI Principles” to refer to 23 principles of beneficial AI formulated at the conference organized by the Future of Life Institute in 2017, as specified.

EXISTING LAW:

Establishes the CDT within the Government Operations Agency in state government. (Gov. Code Sec. 11545(a)(1).)
Establishes the duties of the Director of Technology, which includes, among other things overseeing the CDT and advising the Governor on matters relating to information technology services and resources. (Gov. Code Sec. 11545(b).)
Requires reports to the Legislature to be submitted in printed copy to the Secretary of the Senate, electronically to the Chief Clerk of the Assembly, and as an electronic or printed copy to the Legislative Counsel, as specified. (Gov. Code Sec. 9795.)
Provides, in relevant part, that a bill that, as introduced or amended in either house of the Legislature, would require a state agency to submit a report on any subject to either house of the Legislature generally, a committee or office of either house of the Legislature, or the Legislative Counsel Bureau, shall include a provision that repeals the reporting requirement, or makes the requirement inoperative, no later than a date four years following the date upon which the bill, as enacted, becomes operative or four years after the due date of any report required every four or more years. (Gov. Code Sec. 10231.5(a).)

FISCAL EFFECT: Unknown

COMMENTS: According to the author, “As AI technology evolves, massive changes will come to California’s workforce. Fifty-one percent of time spent in US occupations is highly susceptible to replacement by AI. The areas likely to be hit the hardest are data collection, data processing, and predictable physical work. A 2017 study by McKinsey and Company predicted a loss of 39 million jobs and displacement of 23 percent of the workforce in the U.S. by 2030 due to increased automation.

Most California state agencies, 80 percent, do not currently use AI, but almost a quarter say they intend to use AI in the next 10 years. Between chatbots, automatic processing, and profiling potential parolees, state agencies are slowly introducing AI to the State workplace.”

Finally, the author states, “All these new systems, however, are occurring independent of one another. There is not currently an overall plan for how the state should use AI, nor are there frameworks on how to prepare workers and students for the coming changes to the economy. The need for public involvement and dedicated investment, interest and oversight at the state level – most notably in a state like California that is home to the biggest and most advanced technology companies in the world [e.g. Apple, Alphabet Inc., Facebook, Intel, etc.] that are heavily involved in the development and implementation of AI – it is absolutely critical and in the ultimate interest of the well-being of our population that we dedicate resources, time and expertise to examine and analyze how we might best use this technology and implement it in the future.”

What is Artificial Intelligence?

While there is no one definition as to what AI is, it could be referred to as “machines that respond to stimulation consistent with traditional responses from humans, given the human capacity for contemplation, judgment, and intention.”^[1]

In its background paper “Identifying the Promises and Challenges of Artificial Intelligence” for the joint informational hearing of the Assembly Committee on Privacy and Consumer Protection and the Assembly Select Committee on Emerging Technologies and Innovation, AI is, in part, described in the following way:

AI could very well exacerbate problems, as much as it could solve them. On the one hand, there is an incredible ability for AI to create a global paradigm shift that may propel society into an automation age and propose unique solutions to some of the world’s greatest problems. On the other hand, AI comes with its own risks, including bias or loss of privacy, among other things. Thus far, academics and technology companies have led the discussion on the future of AI. Given the potential challenges of AI to society, as well as the benefits, government arguably needs to become more actively involved in understanding AI, and the future of AI, both in terms of technology and public policy.

Impact of AI on Labor and the Future of Work

A recent article by the Proceedings of the National Academy of Sciences of the United States of America concluded that:

AI has the potential to reshape skill demands, career opportunities, and the distribution of workers among industries and occupations in the United States and in other developed and developing countries. However, researchers and policy makers are underequipped to forecast the labor trends resulting from specific cognitive technologies, such as AI.^[2]

Recently, in response to a growing awareness of AI, California’s Little Hoover Commission published its report “Artificial Intelligence: A Roadmap for California.” The purpose of the report is to provide:

“The new Governor and Legislature with an AI policymaking agenda that revolves around public engagement, building a human infrastructure, attacking pressing social needs, and protecting core values—autonomy, responsibility, privacy, transparency and accountability. In addition, the report offers lawmakers a high level plan that defines strategic AI objectives and major steps to achieve those objectives.”

This report, in addition to others, has helped to define the nature and complexity of AI, its impact on government, the economy, privacy issues, and, in many respects, all aspects of day-to-day living.

Committee Comments

Recognizing that AI could significantly disrupt the nature of work as we know it, the author may wish to consider adding language to the bill to focus on AI’s impact on the future of work and emphasize the steps the state should take to address the short-term and long-term impacts of AI on its workforce.

Furthermore, recognizing the infancy of our understanding of AI and its implications, the author may wish to consider removing references to the “23 Asilomar AI Principles.” Due to the evolving nature of AI technology, limiting this bill to the “23 Asilomar AI Principles” may unnecessarily restrict the scope of the work of the Commission. The author could replace this language with more general language to provide greater flexibility regarding the questions and considerations to be explored by the Commission.

REGISTERED SUPPORT / OPPOSITION:

Support

None on file.

Opposition

None on file.

Analysis Prepared by: Megan Lane / L. & E. /

Brookings Institution. Report – What is Artificial Intelligence? Darrel M. West. October 4, 2018 Website: https://www.brookings.edu/research/what-is-artificial-intelligence/ ↑
Proceedings of the National Academy of Sciences of the United States of America, “Toward understanding the impact of artificial intelligence on labor,” March 25, 2019 ↑

Date of Hearing: April 2, 2019

ASSEMBLY COMMITTEE ON PRIVACY AND CONSUMER PROTECTION

Ed Chau, Chair

ABPCA Bill Id:AB 594 (

Author:Salas) – As Amended Ver:March 19, 2019

SUBJECT: Artificial intelligence

SUMMARY: This bill would establish the California Artificial Intelligence Act of 2020 and provide for the appointment of a Chief Artificial Intelligence Officer within the California Department of Technology (CDT) to evaluate the uses of artificial intelligence (AI) in state government and advise the CDT director and the California Workforce Development Board, as specified. It would also establish an advisory Commission on Artificial Intelligence (Commission), as specified, to study the potential uses of AI and develop a policy framework to manage the use of AI. The Commission would be required to provide an advisory report to the Legislature on or before January 1, 2022, as specified. Specifically, this bill would:

Enact the California Artificial Intelligence Act of 2020 (Act) and require, no later than January 1, 2021, that the director of CDT appoint a Chief AI Officer within CDT to evaluate the uses of AI in state government and to advise the CDT director on incorporating AI into state information technology strategic plans, policies, standards, and enterprise architecture.
Require that the Chief AI Officer advise the California Workforce Development Board in its review of statewide policies and programs related to workforce, education, training, and employment for the purpose of preparing and retraining workers whose jobs may be displaced or transformed by AI.
Establish an advisory Commission comprised of eight members, including the Chief AI Officer, the Secretary of Labor, and six appointed members, which would include representatives from the private industry, higher education, and organized labor, as specified.
Require the Commission to study the potential uses of AI and develop a policy framework to manage the use of AI as specified, and to submit to the Legislature on or before January 1, 2022, a report containing the findings, advice, and recommendations of the Commission. This bill would further require that the framework include advice and recommended strategies to do the following:

Establish an ethical framework informed by the 23 Asilomar AI Principles and ensure the ethical use of artificial intelligence.
Maximize the standard of living and minimize suffering for all humans through the use of artificial intelligence.
Encourage state agencies to create and implement a strategic plan to utilize artificial intelligence in the execution of state functions.
Educate and train students and workers in the application of artificial intelligence and to prepare them for the jobs of the future.
Minimize any potential negative impact of artificial intelligence on the labor market.

Specify that the report, above, will be advisory only and that there shall be no authority or obligation on the part of the state, or the parties meeting and conferring, to implement the findings of the commission without the enactment of subsequent legislation to implement the report.
Define various terms for purposes of this Act, including, among other things:

“23 Asilomar AI Principles” to refer to 23 principles of beneficial AI formulated at the conference organized by the Future of Life Institute in 2017, as specified, which include, among other things:
- The goal of AI research should be to create not undirected intelligence, but beneficial intelligence.
- There should be constructive and healthy exchange between AI researchers and policymakers.
- If an AI system causes harm, it should be possible to ascertain why.
- AI systems should be designed and operated so as to be compatible with ideals of human dignity, rights, freedoms, and cultural diversity.
- The application of AI to personal data must not unreasonably curtail people’s real or perceived liberty.
- The economic prosperity created by AI should be shared broadly, to benefit all of humanity.

EXISTING LAW:

Establishes the CDT within the Government Operations Agency in state government. (Gov. Code Sec. 11545(a)(1).)
Enumerates the duties of the Director of Technology, which includes, among other things:

advising the Governor on the strategic management and direction of the state’s information technology (IT) resources;
establishing and enforcing state IT strategic plans, policies, standards, and enterprise architecture, as specified;
minimizing overlap, redundancy, and cost in state IT operations by promoting the efficient and effective use of information technology;
providing technology direction to agency and department chief information officers to ensure the integration of statewide technology initiatives, compliance with information technology policies and standards, and the promotion of the alignment and effective management of information technology services;
working to improve organizational maturity and capacity in the effective management of IT; and,
establishing performance management and improvement processes to ensure state IT systems and services are efficient and effective. (Gov. Code Sec. 11545(b).)

Requires reports to the Legislature to be submitted in printed copy to the Secretary of the Senate, electronically to the Chief Clerk of the Assembly, and as an electronic or printed copy to the Legislative Counsel, as specified. (Gov. Code Sec. 9795.)
Provides, in relevant part, that a bill that, as introduced or amended in either house of the Legislature, would require a state agency to submit a report on any subject to either house of the Legislature generally, a committee or office of either house of the Legislature, or the Legislative Counsel Bureau shall include a provision that repeals the reporting requirement, or makes the requirement inoperative, no later than a date four years following the date upon which the bill, as enacted, becomes operative or four years after the due date of any report required every four or more years. (Gov. Code Sec. 10231.5(a).)

FISCAL EFFECT: Unknown

COMMENTS:

Purpose of this bill: This bill seeks to establish an advisory Commission on AI and a Chief AI Officer in state government to collectively help study and advise on the incorporation of AI and, particularly, in relation to state IT and state workforce related issues. This is an author-sponsored bill.
Author’s statement: According to the author:

As AI technology evolves, massive changes will come to California’s workforce. [Fifty-one] percent of time spent in US occupations is highly susceptible to replacement by AI. The areas likely to be hit the hardest are data collection, data processing, and predictable physical work. A 2017 study by McKinsey and Company predicted a loss of 39 million jobs and displacement of 23 percent of the workforce in the U.S. by 2030 due to increased automation.

Most California [s]tate [a]gencies, 80 percent, do not currently use AI, but almost a quarter say they intend to use AI in the next 10 years. Between chatbots, automatic processing, and profiling potential parolees, [s]tate [a]gencies are slowly introducing AI to the State workplace.

All these new systems, however, are occurring independent of one another. There is not currently an overall plan for how the State should use AI, nor are there frameworks on how to prepare workers and students for the coming changes to the economy. AB 594 and the Chief Artificial Intelligence Officer it creates would develop such a plan, and help guide California through this new area of opportunity.

Background on the opportunities and challenges of AI: Last spring, this Committee held a joint informational hearing with the Assembly Select Committee on Emerging Technologies & Innovation on the topic of AI, to begin a preliminary discussion of the promises and challenges presented by AI. The overarching goal of the hearing was to bring members and staff a greater understanding of AI in order to engender more thoughtful public policy in the future. As recognized in the committees’ background paper on AI, the opportunities and challenges posed by AI are significant, and in many ways still being uncovered:

[…] AI is frequently associated with technologies linked to our smartphones, or new gadgets like virtual assistants or smart speakers like Alexa or Google Home. In cinema, it is often portrayed as “robot apocalypse.” For the Legislature, contemplating AI applications of the “future” frequently includes autonomous vehicles and concerns displacement of workers with the automation of jobs. Beyond such examples, however, it is not as obvious what AI looks like five years down the line, let alone [ten]. […]

For example, for many people, AI is not immediately associated with social justice. However, at the University of Southern California, the Center for Artificial Intelligence in Society (CAIS) has brought researchers together from around the world to focus on how computer science can be used to solve social problems. Indeed, from the CAIS’ perspective, AI can be used to improve society and fight social injustice. Their current projects include: AI for Cybersecurity; HIV prevention among homeless youth; Wildlife Conservation with drones; AI for Wildlife Conservation in Africa; Predictive modeling of tobacco use and prevention among abused children; Predictive models of vulnerability and housing prioritization for youth and families; Gang violence prevention using game theory; Social network-based substance abuse prevention for homeless youth; Predictive modeling for early identification of suicidal thinking among active duty service members; Network-based suicide prevention for college students; AI for public safety and security using game theory; and others.

At the same time, while AI may present unique solutions to social problems or even governmental ones, as indicated above, it may very well exacerbate others if not done with adequate safeguards in place. For example, governmental entities may turn to AI for useful applications in everything from enhancing delivery of services to better addressing public safety concerns. Consider how some states’ courts have sought to apply AI to conduct risk assessments (i.e. assessments of how likely a defendant is to commit future crimes) through the use of seemingly neutral algorithms. Already, concerns have been raised about how these algorithms may in fact reinforce or aggravate biases. (Citing Anwin, Larson, Mattu, and Kirchner, Machine Bias, ProPublica (May 23, 2016) [as of Mar. 4, 2018].)

Notably, at the same time that this Committee began this joint-endeavor to generate greater understanding of the opportunities and challenges of AI within the Legislature, the Little Hoover Commission (LHC) was simultaneously studying the same topic. The LHC began its process, which included both public hearings and roundtables, with a public hearing on January 25, 2018, entitled “Artificial Intelligence: Applications and Implications.” At that first hearing, the LHC indicated that it ultimately intended to produce a report and policy recommendations about how the State of California can approach AI.

Indeed, in November 2018, the LHC produced its report, Artificial Intelligence: A Roadmap for California, wherein it similarly recognized the possible benefits and potential misuses of AI:

Imagine using AI applications to predict where fires may occur, detect early-stage wildfires, or guide firefighters where best to fight a fire and save lives. Conceive of an environment where AI could promote biodiversity and water conservation, and protect endangered species. See educators using AI to improve student learning and increase graduation rates. Envision better detection of diseases, including cancer, and more finely-tuned effective treatments. Certainly, such visions must be tempered with appropriate privacy protections and robust laws aimed at preventing the misuse of data. In addition, this encouraging future, which is presently knocking at our door, will require not just foresight but insight, not just political will but political action, and not just one mind but a collaboration of minds in government, academia, and private industry. (See Chairman Pedro Nava Opening Letter, LHC Report #245, Artificial Intelligence: A Roadmap for California (Nov. 2018), p. 1; hereinafter “LHC Report.”)

Numerous bills on AI this year: This bill is one of many bills on AI this year. This Committee previously heard and approved AB 976 (Chau), which was supported by the LHC, and sought to create the Artificial Intelligence in State Government Services Commission. This Committee is also set to hear AB 459 (Kiley) which seeks to require the AB 976-proposed commission to develop various minimum standards for the use of AI in state government. Most recently, AB 1576 (Calderon) was recently referred to this Committee, to be heard at a future hearing. There are also bills in the Senate which could potentially also address similar issues. This bill would, similar to AB 976 (and potentially other bills this Committee has yet to hear), establish a state advisory commission to help study and advise on the incorporation of AI. In this regard, the bill is broader than AB 976 advisory commission, which would examine, specifically, the use of AI in state government services. The authors of each of these and other AI bills should work together, to ensure that duplicative commissions and responsibilities are not created in state law.

Staff notes that one key difference in this bill, that has not been seen in AB 976 or AB 459, is the establishment of a Chief AI Officer in the CDT to advise both the director of CDT and the California Workforce Development Board. This bill also incorporates the 23 Asilomar Principles that were the subject of a resolution last year, which expressed the support of the Legislature for those 23 principles as guiding values for the development of AI and of related public policy. (See ACR 215 (Kiley, Ch. 206, Stats. 2018).) As noted in the LHC Report, “[o]ver 1,200 AI and robotics researchers signed the principles, which are intended to promote the safe and beneficial development of AI.” They include, for example, principles on judicial transparency and personal privacy:

Any involvement by an autonomous system in judicial decision-making should provide a satisfactory explanation auditable by a competent human authority. (Principle #8.)
People should have the right to access, manage and control the data they generate, given AI systems’ power to analyze and utilize that data. (Principle #12.)

Lastly, to the extent that this bill requires that a state commission study the potential uses of AI and develop a policy framework that includes advice and recommendations in certain areas, including in establishing an ethical framework that is informed by the 23 Asilomar AI Principles and ensures the ethical use of AI, this bill is consistent with prior public policies supported by this Legislature and addresses a key question of the ethical use of AI. This could also greatly improve the ability of this Legislature to address an issue raised in the LHC Report. Namely, that “California state government is underprepared to take the lead [in AI]. It lacks any single clear leadership and focus on the development and use of AI technology and applications to improve internal and external operations and services within an ethical framework.” (Id. at 15.)

Chief AI Officer within CDT: As noted in Comment 4, above, this bill would require the director of CDT to appoint a Chief AI Officer within the department to evaluate the uses of AI in state government and to advise the CDT director on incorporating AI into state information technology strategic plans, policies, standards, and enterprise architecture. This appears consistent with the statutory responsibility of the CDT director to advise the Governor on the strategic management and direction of the state’s IT resources (many of which may be shaped or affected by uses of AI), to provide technology direction to agency and department chief information officers to ensure the integration of statewide technology initiatives, and to establish and enforce state IT strategic plans, policies, standards, and enterprise architecture. (See Gov. Code Sec. 11545(b).)

The inclusion of a Chief AI Officer in CDT is arguably not only appropriate to incorporate AI expertise more directly in the department that houses the State’s existing expertise in statewide IT resources and IT-related solutions and initiatives, but also consistent with activities undertaking by CDT in recent years to cultivate AI expertise within its Office of Digital Innovation (ODI). CDT originally launched ODI in 2016 to define an approach to government technology innovation that would drive the department forward as a thought leader and technology innovator in state government. To that end, ODI already appears to be considering some applications of AI within state government. A 2018 Techwire article, for example, noted that CDT’s Digital Web Services Network, led by ODI, was established as “a forum where state partners (and) local government can share information for digital services including policy, technology, tools and best practices” and would include a discussion around AI at its then-upcoming quarterly meeting. (Techwire, CDT Web Services Meeting to Focus on AI, Innovation (Oct. 17, 2018) <https://www.techwire.net/news/cdt.html> [as of Mar. 12, 2019].)

As such, the proposed Chief AI Officer could provide greater expertise in AI applications more broadly to help inform the CDT director in their responsibilities in shaping the use and direction of IT in the State. Having such expertise, through an individual who has visibility into the State’s use of AI as a whole, can prevent state entities from taking a needlessly siloed and potentially redundant or costly approach to AI. Inversely, it can ensure that the State’s use of AI is efficient and effective and in alignment with its overall IT management and strategy. Furthermore, a Chief AI Officer could potentially also be useful to specific state entities, such as the California Workforce Development Board, by advising that board in its review of statewide policies and programs related to workforce, education, training, and employment for the purpose of preparing and retraining workers whose jobs may be displaced or transformed by AI, as otherwise required under this bill.

AI definition: As noted in this Committee’s joint informational hearing on AI last year, “[t]hough first coined by a Dartmouth professor, John McCarthy, in the 1950s, there still does not appear to be any singular, consistent definition of [AI] in use today, over 60 years later.” One the one hand, providing definitions for bills such as this, which seek to establish commissions or AI-focused personnel within state government, may help to provide helpful parameters and greater understanding of how the Legislature understood AI at the time of that those entities and positions were created. On the other hand, given the varying definitions that exist even today, flexibility may be warranted, for example to avoid unduly limiting the new commission’s purposes under this bill. Leaving the term “undefined” with respect to any specific understanding of what is or is not “AI” at this given moment, may provide the Commission greater flexibility in making its recommendations.

As currently drafted, the bill would define the term to mean a form or application of technology that does either of the following:

- Performs cognitive functions commonly associated with human intelligence, including learning, problem solving, and pattern recognition.
- Simulates or engages in human behavior.

In contrast, the section of the LHC Report on “What is AI and What Opportunities Does it Offer?” describes AI, in part, as follows:

Artificial intelligence (AI) is a quality of any computer program (algorithms, data structures and data) that can sense, reason, act and adapt like humans. While many AI systems can also learn over time (e.g., a spam filter, a recommendation system), not all can do so. An expert system built to aid human reasoning, or an autopilot that can land an airplane, are examples of AI applications that do not learn. Whenever a computer program’s performance gains near-humanlike abilities, or better than human abilities, to sense, reason or act, we tend to call that AI. (Id. at 7, emphases added.)

The author may wish to consider whether the definition provided in this bill is as inclusive of different types of AI as the LHC Report description. For example, would the bill’s definition of AI as technology that simulates or engages in “human behavior,” be considered more narrow than the LHC Report discusses AI as “near humanlike like […] or better than human abilities”? A similar comparison could be made with how the bill defines AI as performing “cognitive functions” such as “learning, problem solving, and pattern recognition,” whereas the LHC Report describes its ability to “sense, reason, act, and adapt” like humans. What the implications of those differences are is not entirely clear, but they should be examined closely if the definition in the bill could be read to limit what the Commission considers as AI in its studies, or what the Chief AI Officer considers to be AI in its advisory roles.

Related legislation: AB 459 (Kiley) would require the Commission proposed by AB 976, below, to report to the Legislature on its recommended minimum standards for the use of AI in state government, as specified, on or before November 1, 2020. That bill contains contingent enactment language to ensure that it would take effect only if AB 976 is enacted and effective before January 1, 2020. This bill is also pending hearing in this Committee.

AB 976 (Chau et al.) See Comment 4. That bill is in the Assembly Accountability and Administrative Review Committee.

AB 1576 (Calderon) states the intent of the Legislature to enact legislation that would relate to identifying an appropriate state agency to analyze the possible impacts of AI technology on the state with a goal to ensuring that the state is ready to adopt and utilize the technology. This bill would require the Government Operations Agency Secretary to appoint an AI technology working group, to serve in an advisory capacity, as specified. The bill would require the working group to evaluate, among other things, the uses of AI technology by California-based businesses and associated risks (including privacy risks) with such usage.

SB 348 (Chang) would require the secretaries of certain state agencies to devise a strategic plan, as specified, to utilize artificial technology to improve state services. The bill would encourage the Governor to appoint a special adviser on AI to create a statewide strategic plan, as specified, to utilize artificial technology to improve state services. The bill also would encourage certain entities to designate a chief AI officer. This bill is pending hearing in the Senate Governmental Organization Committee.

SB 444 (Umberg) states the intent of the Legislature to enact legislation that would relate to civil actions and AI. This bill is currently in the Senate Rules Committee.

Double-referral: This bill is double-referred to the Assembly Labor and Employment Committee, where it will be heard if passed by this Committee.

REGISTERED SUPPORT / OPPOSITION:

Support

None on file

Opposition

None on file

Analysis Prepared by: Ronak Daylami / P. & C.P. / (916) 319-2200


Version: May 16, 2019	Policy Vote: G.O. 15 - 0
Urgency: No	Mandate: No
Hearing Date: August 30, 2019	Consultant: Debra Cooper

AB 594: Artificial intelligence.

GOVERNOR'S VETO

GV Bill Id:AB 594

SUMMARY:

Governor's Veto Message:

COMMENTS:

According to the Author:

Arguments in Support:

Arguments in Opposition:

FISCAL COMMENTS:

VOTES:

UPDATED:

CONCURRENCE IN SENATE AMENDMENTS

CSA1 Bill Id:AB 594

SUMMARY:

The Senate Amendments:

COMMENTS:

According to the Author:

Arguments in Support:

Arguments in Opposition:

FISCAL COMMENTS:

VOTES:

UPDATED:

THIRD READING

Bill No: AB 594

SENATE COMMITTEE ON APPROPRIATIONS

Bill Summary:

Fiscal Impact:

Author Amendments:

SENATE COMMITTEE ON

Bill Summary:

Fiscal Impact:

Background:

Proposed Law:

Related Legislation:

Staff Comments:

SENATE COMMITTEE ON GOVERNMENTAL ORGANIZATION

SUPPORT:

ASSEMBLY THIRD READING

ATR Bill Id:AB 594

SUMMARY:

Major Provisions

COMMENTS:

According to the Author:

Arguments in Support:

Arguments in Opposition:

FISCAL COMMENTS:

VOTES:

UPDATED:

ASSEMBLY COMMITTEE ON APPROPRIATIONS

ABPCA Bill Id:AB 594

ASSEMBLY COMMITTEE ON LABOR AND EMPLOYMENT

ABPCA Bill Id:AB 594 (

ASSEMBLY COMMITTEE ON PRIVACY AND CONSUMER PROTECTION

ABPCA Bill Id:AB 594 (