Date of Hearing: May 1, 2019

ASSEMBLY COMMITTEE ON APPROPRIATIONS

Lorena Gonzalez, Chair

ABPCA Bill Id:AB 459

Author:(Kiley) – As Amended Ver:April 8, 2019

Urgency: No State Mandated Local Program: No Reimbursable: No

SUMMARY:

This bill requires the Artificial Intelligence in State Government Services Commission (AI Commission) proposed to be created by AB 976 (Chau) of the 2018-2019 Legislative Session, to report to the Legislature on its recommended minimum standards for the use of AI in fostering accountability in state government services and prioritizing the safety and security of AI technologies used by state government.

FISCAL EFFECT:

Unknown. Possible onetime costs (GF) in the tens of thousands of dollars for the AI Commission to prepare and submit a report should it be funded. For example, the average cost for the Legislative Analyst’s Office to prepare a report with recommendations is approximately $30,000 to $50,000.

COMMENTS:

1. Purpose and Background. According to the author:

This bill is intended to ensure that future uses of AI tools within California state government are managed and implemented in a fair, secure, ethical, and beneficial manner. It’s imperative that we act now to stay ahead of the curve and be prepared to responsibly embrace the future.

AB 459 directs the AI Commission proposed to be created by AB 976 (Chau) of the 2018-2019 Legislative Session, to develop recommendations for the Legislature by November 1, 2020, on minimum standards for the use of AI in state government on topics such as accountability, security, privacy and other specified impacts. Enactment of this bill is contingent on enactment of AB 976.

1. Related Legislation.
   
   1. AB 976 (Chau) creates the AI Commission as an advisory body in state government. AB 976 is pending in this committee.
      
   2. AB 594 (Salas) establishes an advisory Commission on AI and a Chief AI Officer in state government. AB 594 is pending in this committee.

Analysis Prepared by: Kimberly Horiuchi / APPR. / (916) 319-2081

Date of Hearing: April 24, 2019

ASSEMBLY COMMITTEE ON ACCOUNTABILITY AND ADMINISTRATIVE REVIEW

Cottie Petrie-Norris, Chair

ABPCA Bill Id:AB 459 (

Author:Kiley) – As Amended Ver:April 8, 2019

SUBJECT: State government: Artificial Intelligence in State Government Services Commission: report

SUMMARY: Requires the Artificial Intelligence in State Government Services Commission (AI Commission) proposed to be established under AB 976 (Chau) of 2019 to report to the Legislature its recommended minimum standards for the use of AI in state government, as specified. Specifically, this bill:

1. Requires, on or before November 1, 2020, the AI Commission to report to the Legislature its recommended minimum standards for the use of AI in state government that must include all of the following:
   
   1. Fostering accountability in state government services, as specified;
   2. Prioritizing the safety and security of AI technologies used by state agencies, as specified;
   3. Protecting the privacy of California residents; and,
   4. Establishing processes to monitor the impact of AI technologies throughout California, as specified.
      
2. Specifies that this bill shall only take effect if AB 976 (Chau) of the 2019-2020 Regular Session is enacted and becomes effective on or before January 1, 2020.
3. Repeals the provisions of this bill on November 1, 2024.

EXISTING LAW establishes the California Department of Technology with broad responsibility and authority over all aspects of information technology (IT) and technology in California state government.

FISCAL EFFECT: Unknown

COMMENTS: This bill directs the AI Commission proposed by AB 976 (Chau) to develop recommendations for the Legislature by November 1, 2020, on minimum standards for the use of AI in state government on topics such as accountability, security, privacy, and other specified impacts.

According to the author, this bill is intended to ensure that “future uses of AI tools within California state government are managed and implemented in a fair, secure, ethical, and beneficial manner. It’s imperative that we act now to stay ahead of the curve and be prepared to responsibly embrace the future.”

This bill incorporates findings from a November 2018 report by the Little Hoover Commission entitled “Artificial Intelligence: A Roadmap for California.” The report notes that “AI poses four key decisions for California: how to support AI research and responsible AI use to grow the state’s economy; how to take advantage of advances in AI to enhance services to Californians; how to configure a new structure for lifelong education and training to respond to the inevitable disruption in the tasks or content of work; and, how to protect its values of privacy, transparency and accountability in this new economic era.”

The report includes a host of recommendations relating to AI and the importance of protecting core public values such as privacy, transparency, accountability, and bias prevention, among other things. All of these aspects of AI would be addressed in the minimum standards developed in this bill.

RELATED LEGISLATION:

1. AB 976 creates the AI Commission as an advisory body in state government that would provide the recommendations on AI minimum standards as required by this bill. Enactment of this bill is contingent upon the enactment of AB 976. AB 976 is pending in the Assembly Appropriations Committee.
2. AB 594 (Salas) establishes an advisory Commission on AI and a Chief AI Officer in state government to collectively help study and advise on incorporating the use of AI in state IT and the state’s workforce. This bill is scheduled to be heard in the Assembly Labor and Employment Committee on April 24, 2019.

DOUBLE REFERRAL: This bill was previously heard in the Assembly Privacy and Consumer Protection Committee on April 2, 2019, with a vote of 11-0.

REGISTERED SUPPORT / OPPOSITION:

Support

None on file

Opposition

None on file

Analysis Prepared by: Cassie Royce / A. & A.R. / (916) 319-3600

Date of Hearing: April 2, 2019

ASSEMBLY COMMITTEE ON PRIVACY AND CONSUMER PROTECTION

Ed Chau, Chair

ABPCA Bill Id:AB 459 (

Author:Kiley) – As Amended Ver:March 25, 2019

SUBJECT: State government: artificial intelligence commission

SUMMARY: This bill would require the Artificial Intelligence in State Government Services Commission to report to the Legislature its recommended minimum standards for the use of artificial intelligence (AI) in state government, as specified. This bill, which is set to sunset on November 1, 2024, would only take effect if AB 976 (Chau et al.) from this session is also enacted and takes effect on or before January 1, 2020. Specifically, this bill would:

1. Provide that, on or before November 1, 2020, the Artificial Intelligence in State Government Services Commission (Commission) must report to the Legislature, as specified under existing law, its recommended minimum standards for the use of AI in state government and that those minimum standards must include all of the following:
   • Fostering accountability in state government services, as specified.
   • Prioritizing the safety and security of AI technologies used by state agencies, as specified.

• Protecting the privacy of California residents.
• Establishing processes to monitor the impact of AI technologies throughout California, as specified.

1. Provide that the above provisions will be repealed on November 1, 2024, pursuant to specified existing law.
2. Provide that the above provisions will only become operative if AB 976 of the 2019-2020 Regular Session is enacted and become effective on or before January 1, 2020.

EXISTING LAW:

1. Requires reports to the Legislature to be submitted in printed copy to the Secretary of the Senate, electronically to the Chief Clerk of the Assembly, and as an electronic or printed copy to the Legislative Counsel, as specified. (Gov. Code Sec. 9795.)
2. Provides, in relevant part, that a bill that, as introduced or amended in either house of the Legislature, would require a state agency to submit a report on any subject to either house of the Legislature generally, a committee or office of either house of the Legislature, or the Legislative Counsel Bureau shall include a provision that repeals the reporting requirement, or makes the requirement inoperative, no later than a date four years following the date upon which the bill, as enacted, becomes operative or four years after the due date of any report required every four or more years. (Gov. Code Sec. 10231.5(a).)

FISCAL EFFECT: Unknown

COMMENTS:

1. Purpose of this bill: This bill seeks to have the Commission proposed by AB 976 (Chau) develop recommendations for the Legislature on minimum standards for the use of AI in state government on topics such as accountability, security, privacy, and impact in areas such as automation and labor displacement, as well as bias and discrimination. This is an author-sponsored bill.
2. Author’s statement: According to the author:

California has already established itself as a leader in AI policy. In 2018, the legislature passed my resolution in support of the Asilomar AI Principles, a set of guidelines for the safe and beneficial development and use of AI. That resolution, ACR 215, was only the first step in acknowledging the importance of Artificial Intelligence within state government.

AB 459 builds on ACR 215 by ensuring that future uses of AI tools within California state government are managed and implemented in a fair, secure, ethical, and beneficial manner. It’s imperative that we act now to stay ahead of the curve and be prepared to responsibly embrace the future.

AB 459 tasks the Artificial Intelligence in State Government Services Commission (to be created in AB 976, Chau) with creating a set of guidelines and standards for the use of artificial intelligence within state government.

Those recommended guidelines and standards would then be reported to the legislature, on or before November 1, 2020, to be discussed and adopted in future legislation.

1. The opportunities and challenges of AI: Last spring, this Committee held a joint informational hearing with the Assembly Select Committee on Emerging Technologies & Innovation on the topic of AI, to begin a preliminary discussion of the promises and challenges presented by AI. The overarching goal of the hearing was to bring members and staff a greater understanding of AI in order to engender more thoughtful public policy in the future. As recognized in the committees’ background paper on AI, the opportunities and challenges posed by AI are significant, and in many ways still being uncovered:

[…] AI is frequently associated with technologies linked to our smartphones, or new gadgets like virtual assistants or smart speakers like Alexa or Google Home. In cinema, it is often portrayed as “robot apocalypse.” For the Legislature, contemplating AI applications of the “future” frequently includes autonomous vehicles and concerns displacement of workers with the automation of jobs. Beyond such examples, however, it is not as obvious what AI looks like five years down the line, let alone [ten]. […]

For example, for many people, AI is not immediately associated with social justice. However, at the University of Southern California, the Center for Artificial Intelligence in Society (CAIS) has brought researchers together from around the world to focus on how computer science can be used to solve social problems. Indeed, from the CAIS’ perspective, AI can be used to improve society and fight social injustice. Their current projects include: AI for Cybersecurity; HIV prevention among homeless youth; Wildlife Conservation with drones; AI for Wildlife Conservation in Africa; Predictive modeling of tobacco use and prevention among abused children; Predictive models of vulnerability and housing prioritization for youth and families; Gang violence prevention using game theory; Social network-based substance abuse prevention for homeless youth; Predictive modeling for early identification of suicidal thinking among active duty service members; Network-based suicide prevention for college students; AI for public safety and security using game theory; and others.

At the same time, while AI may present unique solutions to social problems or even governmental ones, as indicated above, it may very well exacerbate others if not done with adequate safeguards in place. For example, governmental entities may turn to AI for useful applications in everything from enhancing delivery of services to better addressing public safety concerns. Consider how some states’ courts have sought to apply AI to conduct risk assessments (i.e. assessments of how likely a defendant is to commit future crimes) through the use of seemingly neutral algorithms. Already, concerns have been raised about how these algorithms may in fact reinforce or aggravate biases. (Citing Anwin, Larson, Mattu, and Kirchner, Machine Bias, ProPublica (May 23, 2016) [as of Mar. 4, 2018].)

Notably, at the same time that this Committee began this joint-endeavor to generate greater understanding of the opportunities and challenges of AI within the Legislature, the Little Hoover Commission (LHC) was simultaneously studying the same topic. The LHC began its process, which included both public hearings and roundtables, with a public hearing on January 25, 2018, entitled “Artificial Intelligence: Applications and Implications.” At that first hearing, the LHC indicated that it ultimately intended to produce a report and policy recommendations about how the State of California can approach AI.

Indeed, in November 2018, the LHC produced its report, Artificial Intelligence: A Roadmap for California, wherein it similarly recognized the possible benefits and potential misuses of AI:

Imagine using AI applications to predict where fires may occur, detect early-stage wildfires, or guide firefighters where best to fight a fire and save lives. Conceive of an environment where AI could promote biodiversity and water conservation, and protect endangered species. See educators using AI to improve student learning and increase graduation rates. Envision better detection of diseases, including cancer, and more finely-tuned effective treatments. Certainly, such visions must be tempered with appropriate privacy protections and robust laws aimed at preventing the misuse of data. In addition, this encouraging future, which is presently knocking at our door, will require not just foresight but insight, not just political will but political action, and not just one mind but a collaboration of minds in government, academia, and private industry. (See Chairman Pedro Nava Opening Letter, LHC Report #245, Artificial Intelligence: A Roadmap for California (Nov. 2018), p. 1; hereinafter “LHC Report.”)

This bill, contingent on the enactment of AB 976 (Chau et al.) which separately seeks to establish the Artificial Intelligence in State Government Services Commission (Commission), would require the proposed Commission to develop minimum standards for using AI in state government that would take into account, for example, privacy protections for Californians, and processes to monitor the impact of AI technologies on bias and discrimination.

1. Minimum standards on accountability, security, privacy, and impact of the use of AI in state government: According to the LHC Report, “[AI] is already changing the structure of goods and services in the economy, and altering the nature of work. This has major implications for our workforce and opens critical questions about our human values like privacy. [… ] AI poses four key decisions for California: (1) how to support AI research and responsible AI use to grow the state’s economy; (2) how to take advantage of advances in AI to enhance services to Californians; (3) how to configure a new structure for lifelong education and training to respond to the inevitable disruption in the tasks or content of work; and (4) how to protect its values of privacy, transparency and accountability in this new economic era.” (Id. at 6.) The LHC makes a host of recommendations in its report and in doing so also recognizes the importance of protecting core public values such as privacy, transparency, accountability, preventing against bias, among other things.

This bill appears to take into consideration the discussion in the LHC Report around such issues (see Comment 5, below, for more) by seeking to have the Commission establish minimum standards that relate to the following:

• • Fostering accountability in state government services, including the following:
• human responsibility for the use and impact of AI tools;
• an appeals process for decisions to use AI tools;
• recording and monitoring government use of AI tools; and,
• ensuring sufficient transparency and explainability of AI tools for use by state agencies.
  • Prioritizing the safety and security of AI technologies used by state agencies, including the following:
• measuring the reliability and robustness of AI tools; and,
• minimizing the potential for misuse.
• Protecting the privacy of California residents.
• Establishing processes to monitor the impact of AI technologies throughout California, including the following:
• automation and labor displacement;
• bias and discrimination;
• effectiveness and accuracy; and,
• unforeseen consequences.

Staff notes that when this Committee heard and passed AB 976, the Future of Life Institute wrote in support, but specifically suggested that it would be appropriate for the Commission to develop minimum standards for the use of AI technologies by the State of California that help foster accountability in the use of AI technologies for state government services, prioritize the safety and security of AI technologies used by state government, protect the privacy of California residents, and monitor the impacts of AI technologies throughout CA, in areas including such as automation and labor displacement, and bias and discrimination. That is precisely what this bill does.

1. LHC Report on privacy, transparency, and accountability: This bill requires the development of minimum standards that appear to encompass many of the LHC Report’s key discussion points regarding the protection of core public policy values in any implementation of AI.

The LHC report recognizes that “AI has the potential to revolutionize major areas of life for economic, social and environmental good” and provides many examples of how AI might assist in meeting social needs (e.g., reducing traffic accidents, improving efficiency in the use of energy and water resources, reducing risks of work-related injuries, improving access to education, or improving speed and accuracy in disease diagnosis). It also recognizes that in the realm of jobs, it could not only make many jobs easier by performing mundane tasks and allowing humans to perform more interpersonal and creative activities, but it will also drive the creation of new occupations which may help to counterbalance job displacement. For example, “[i]nvestments in renewable energy and mitigation of climate change may create demand for workers across several sectors, from manufacturing and construction to installation. Growth in the population of people age and older may create demand for occupations in the health care sector, including doctors, nurses, home health aides and nursing assistants.” (LHC Report, p. 8, internal footnotes omitted.)

That being said, the implementation of AI not only has exciting promises, but also comes with challenges and risks to certain values held in high regard as a matter of public policy, such as privacy, transparency, and accountability, non-discrimination, among other things. The LHC Report, for example, clearly recognizes that one major risk associated with AI is bias. “There is no doubt that the use of algorithms in decision making poses a risk of error and bias. The public policy question is what to do about these challenges and how to balance the benefits against the risks. For California government, the first task is to help educate and inform policymakers in state government, including within the executive branch and departments, agencies and commissions, on how algorithms are created, tested and used. As a state with the best and strongest set of research universities in AI, California should sponsor research and development in this area.” (Id. at 14.)

Examining the risks of machine bias, how it can lead to unlawful discrimination, and the implications of that bias, the LHC writes in its report that:

[AI] empowers machines to think like people. However, machines that [AI] are prone to incorporating the biases contained in data and of their human engineers. Machine bias occurs when different groups of people are treated unequally by machines on the basis of their gender, race and other characteristics. For example, an AI algorithm that is taught to diagnose skin cancer using images of white faces means the system will perform best for Caucasians as opposed to African-Americans. Machine bias could lead, whether intentionally or unintentionally, to unequal and unfair treatment for certain socioeconomic, racial, religious and other similar groups. It may identify people of color as being at higher risk for committing a crime or systematically exclude people with mental disabilities from being hired. Machine bias may perpetuate discriminatory outcomes and prevent society from gaining the benefits of AI-powered technology.

[…] AI algorithms do not require explicit data on race, gender or socioeconomic status to exhibit bias. Commercial lending and insurance algorithms, for example, may not ask direct questions about race or class identity, but these tools frequently incorporate other variables like ZIP code that would count against those living in poor communities. AI systems might not work well for all populations and can reflect prejudices deeply embedded in history and society. (Id. at 28-29; internal footnotes omitted.)

Beyond issues of bias, the LHC Report also discusses how privacy implications of AI, and the use of AI, are “well accepted.” “Political jurisdictions such as the European Union have implemented new rules that cover AI. Closer to home, the California Legislature passed and the Governor signed a new privacy statute, the California Consumer Privacy Act of 2018. The best next step for the incoming administration and Legislature [is] to examine over time that statute to determine how effectively it is working and whether it is a good fit for AI-related issues. It will be useful to review the AI-related privacy principles of non-profit groups, companies and their trade associations, as well as those recently drafted by Congressman Ro Khanna. These principles offer additional insights into AI-related privacy questions. Questions associated with transparency and accountability are complex and fraught with risks of unintended consequences. Moreover, rules in this area potentially can be overly inclusive, under cover key issues or insufficiently address unique challenges faced by specific sectors.” (Id. at 19.)

This bill would arguably help protect these and other major core values by requiring the AB 976 proposed Commission to develop minimum standards, including on: (1) the impact of AI in areas such as bias and discrimination, automation and labor displacement, effectiveness and accuracy, as well as unforeseen consequences; and (2) fostering accountability in state government services, in areas such as human responsibility for the use and impact of AI tools; recording and monitoring government use of AI tools; and ensuring sufficient transparency and explainability of AI tools^[1] for use by state agencies.

1. Technical amendments: The author has accepted the following amendments to correct drafting errors in the bill. The first ensures that the minimum standards address an appeals process for decisions “made by” AI tools, as opposed by “to use” AI tools, to foster accountability in state government, when using AI. The second corrects a misspelling of the word “explainability.”

Author’s amendments:

(1) On page 2, line 12, strike “to use” and insert “made by”

(2) On page 2, line 16, strike “exlainability” and insert “explainability”

1. Related legislation: AB 976 (Chau et al.) See Comments 3, 4, and 5. That bill is in the Assembly Accountability and Administrative Review Committee.

AB 594 (Salas) would enact the California Artificial Intelligence Act of 2020. The bill, no later than January 1, 2021, would the appointment of a Chief AI Officer within the Department of Technology to evaluate the uses of AI in state government and to advise the Director of Technology on incorporating artificial intelligence into state information technology strategic plans, policies, standards, and enterprise architecture. The bill would establish an advisory commission on AI, as specified, to study the potential uses of AI and develop a policy framework to manage the use of AI, as specified, and to submit to the Legislature on or before January 1, 2022, a report containing the findings, advice, and recommendations of the commission. This bill is also pending hearing in this Committee.

AB 1576 (Calderon) states the intent of the Legislature to enact legislation that would relate to identifying an appropriate state agency to analyze the possible impacts of AI technology on the state with a goal to ensuring that the state is ready to adopt and utilize the technology. This bill would require the Government Operations Agency Secretary to appoint an AI technology working group, to serve in an advisory capacity, as specified. The bill would require the working group to evaluate, among other things, the uses of AI technology by California-based businesses and associated risks (including privacy risks) with such usage.

SB 348 (Chang) would require the secretaries of certain state agencies to devise a strategic plan, as specified, to utilize artificial technology to improve state services. The bill would encourage the Governor to appoint a special adviser on AI to create a statewide strategic plan, as specified, to utilize artificial technology to improve state services. The bill also would encourage certain entities to designate a chief AI officer. This bill is pending hearing in the Senate Governmental Organization Committee.

SB 444 (Umberg) states the intent of the Legislature to enact legislation that would relate to civil actions and AI. This bill is currently in the Senate Rules Committee.

1. Double-referral: This bill is double-referred to the Assembly Accountability and Administrative Review Committee, where it will be heard if passed by this Committee.

REGISTERED SUPPORT / OPPOSITION:

Support

None on file

Opposition

None on file

Analysis Prepared by: Ronak Daylami / P. & C.P. / (916) 319-2200

1. Explainable AI refers largely to AI tools, techniques, or programs which can be trusted and easily understood by humans. ↑