Existing law requires the Secretary of Government Operations to develop a coordinated plan to, among other things, investigate the feasibility of, and obstacles to, developing standards and technologies for state departments to determine digital content provenance. For the purpose of informing that coordinated plan, existing law requires the secretary to evaluate, among other things, the impact of the proliferation of deepfakes, defined to mean audio or visual content that has been generated or manipulated by artificial intelligence that would falsely appear to be authentic or truthful and that features depictions of people appearing to say or do things they did not say or do without their consent, on state government, California-based businesses, and residents of the state.

This bill would require the Government Operations Agency, the Governors Office of Business and Economic Development, the California Privacy Protection Agency, and the Department of Technology to collaborate to establish the California Artificial Intelligence Research Hub (hub) in the Government Operations Agency, as prescribed. The bill would require the hub to serve as a centralized entity to facilitate collaboration between government agencies, academic institutions, and private sector partners to advance artificial intelligence research and development that seeks to harness the technologys full potential for public benefit while safeguarding privacy, advancing security, and addressing risks and potential harms to society, as prescribed.

Existing law requires the Secretary of Government Operations to develop a coordinated plan to, among other things, investigate the feasibility of, and obstacles to, developing standards and technologies for state departments to determine digital content provenance. For the purpose of informing that coordinated plan, existing law requires the secretary to evaluate, among other things, the impact of the proliferation of deepfakes, defined to mean audio or visual content that has been generated or manipulated by artificial intelligence that would falsely appear to be authentic or truthful and that features depictions of people appearing to say or do things they did not say or do without their consent, on state government, California-based businesses, and residents of the state.

This bill would require the Government Operations Agency, the Governors Office of Business and Economic Development, the California Privacy Protection Agency, and the Department of Technology to collaborate to establish the California Artificial Intelligence Research Hub (hub) in the Government Operations Agency, as prescribed. The bill would require the hub to serve as a centralized entity to facilitate collaboration between government agencies, academic institutions, and private sector partners to advance artificial intelligence research and development that seeks to harness the technologys full potential for public benefit while safeguarding privacy, advancing security, and addressing risks and potential harms to society, as prescribed.

Existing law requires the Secretary of Government Operations to develop a coordinated plan to, among other things, investigate the feasibility of, and obstacles to, developing standards and technologies for state departments to determine digital content provenance. For the purpose of informing that coordinated plan, existing law requires the secretary to evaluate, among other things, the impact of the proliferation of deepfakes, defined to mean audio or visual content that has been generated or manipulated by artificial intelligence that would falsely appear to be authentic or truthful and that features depictions of people appearing to say or do things they did not say or do without their consent, on state government, California-based businesses, and residents of the state.

This bill would require the Government Operations Agency, the Governors Office of Business and Economic Development, and the Department of Technology to collaborate to establish the California Artificial Intelligence Research Hub (hub) in the Government Operations Agency. Agency, as prescribed. The bill would require the hub to serve as a centralized entity to facilitate collaboration between government agencies, academic institutions, and private sector partners to advance artificial intelligence research and development that seeks to harness the technologys full potential for public benefit while safeguarding privacy, advancing security, and addressing risks and potential harms to society, as prescribed.

Existing law requires the Secretary of Government Operations to develop a coordinated plan to, among other things, investigate the feasibility of, and obstacles to, developing standards and technologies for state departments to determine digital content provenance. For the purpose of informing that coordinated plan, existing law requires the secretary to evaluate, among other things, the impact of the proliferation of deepfakes, defined to mean audio or visual content that has been generated or manipulated by artificial intelligence that would falsely appear to be authentic or truthful and that features depictions of people appearing to say or do things they did not say or do without their consent, on state government, California-based businesses, and residents of the state.

This bill would require the Government Operations Agency, the Governors Office of Business and Economic Development, and the Department of Technology to collaborate to establish the California Artificial Intelligence Research Hub (hub) in the Government Operations Agency. The bill would require the hub to serve as a centralized entity to facilitate collaboration between government agencies, academic institutions, and private sector partners to advance artificial intelligence research and development that seeks to harness the technologys full potential for public benefit while safeguarding privacy, advancing security, and addressing risks and potential harms to society, as prescribed.

Date of Hearing: August 7, 2024

ASSEMBLY COMMITTEE ON APPROPRIATIONS

Buffy Wicks, Chair

SBPCA Bill Id:SB 893

Author:(Padilla) – As Amended Ver:July 3, 2024

Urgency: No State Mandated Local Program: No Reimbursable: No

SUMMARY:

This bill requires the Government Operations Agency (GovOps), the Governor’s Office of Business and Economic Development (GO-Biz), the California Privacy Protection Agency (Privacy Agency), and the California Department of Technology (CDT) to establish the California Artificial Intelligence (AI) Research Hub (“the hub”) within GovOps.

Specifically, this bill:

1. Requires GovOps, GO-Biz, the Privacy Agency, and CDT to consult with academic institutions within the state in establishing the hub.
2. Requires the hub to do all of the following:
3. Increase lawful access to government data by developing a streamlined process for researchers at academic institutions to access data collected by state agencies, except for personal information and trade secrets. Requires the data access process to include an eligibility process that prioritizes security by limiting who can access the data and for what purpose.
4. Requires the Privacy Agency to review and approve all government data before it may be made available through the process created under a), above.
5. Support access to and development of AI computing capacity and technology by facilitating efforts to expand public and commercial computing infrastructure.
6. Spur innovation in AI applications for the benefit of the public.
7. Ensure the development of trustworthy AI technologies with a focus on transparency, fairness, and accountability.
8. Provide researchers with increased access to data and computing resources, education, and training opportunities in furtherance of applications of AI for benefit to the people of California.

FISCAL EFFECT:

1. Costs (General Fund) to GovOps, GO-Biz, and CDT to establish and operate the hub:
   1. In the first year of implementation, CDT reports costs of about $254,000 for one temporary management position, $2 million for external consultants, and $650,000 for GenAI talent practices. In subsequent years, CDT anticipates annual ongoing costs of $2 million for nine permanent positions, $1.75 million for external consultants, and $450,000 for GenAI talent practices. CDT notes it is challenging to hire GenAI talent in the public sector since salaries are often two or three times higher in the private sector, and private sector employers can offer bonuses, equity, and stock options. As a result, CDT reports, its fiscal estimates for this bill include significant funding for external consultants, and actual costs for consultants may be higher if the state does not provide sufficient funding for training of state employees. These challenges and additional cost pressures will likely be faced by each of the state agencies affected by this bill.
   2. GovOps reports ongoing costs of $1 million to $3 million annually.
   3. Costs of an unknown but potentially significant amount to GO-Biz. GO-Biz was unable to report costs for this bill for inclusion in this analysis. If GO-Biz must hire additional staff or consultants to fulfill its duties under this bill, it may incur costs in the hundreds of thousands to low millions of dollars annually ongoing.
2. Costs (General Fund) to the Privacy Agency to support development of the hub, help develop the data review process, and review and approve state agency data proposed for release. The Privacy Agency anticipates needing one additional permanent attorney position for an ongoing annual cost of $285,000. However, depending on how many agencies use the data release process established under this bill, one attorney position may not be sufficient and additional resources may be required. The Privacy Agency also notes there are multiple bills pending in the Legislature that would affect the agency, and the agency may need additional administrative resources for any increase in staffing.
3. Possible costs (General Fund, special funds) of an unknown but significant amount to state agencies to obtain approval from the Privacy Agency before releasing data to academic researchers. California has more than 200 agencies, including state departments, boards, commissions, bureaus, universities, and other entities. If each state agency that releases data to academic researchers is required to use the data release process established under the bill, aggregate costs statewide may be in the millions to tens of millions of dollars annually ongoing.

According to the Legislative Analyst’s Office, the General Fund faces a structural deficit in the tens of billions of dollars over the next several fiscal years.

COMMENTS:

1. Purpose. According to the author:

California is a global leader in technological advancement. Much of that leadership has been driven by our world-class higher education systems. Emerging AI technologies are costly and energy intensive, and require broad-based coordination among institutions and other sectors. Shared resources will be vital to the continued development of AI technology in California. The creation of the California Artificial Intelligence Research Hub allows us to pool and leverage the state’s financial resources and the intellectual firepower of our academic sector to democratize AI and stop it from becoming monopolized by proprietary interests alone – the tech titans.

1. Background. This bill requires several California agencies to work together to establish a central hub with the goal of facilitating collaboration between the government, academic researchers, and the private sector to provide more public access to and benefit from AI systems. The bill may also have major implications for state agencies that provide data to researchers at academic institutions. Under existing law, release of personal information maintained by state agencies is governed by the Information Practices Act, which permits release of such personal information only in specified circumstances, including to authorized researchers. (Civ. Code, § 1798.24, subds. (h) & (t).) Release of data maintained by state agencies is also governed by many agency- and data-specific statutes, including some that specifically address release of data to academic researchers. For example, the Penal Code governs the Department of Justice’s release of certain firearms data and criminal offender record information to researchers (see Pen. Code, §§ 11106, subd. (d) & 13202).

The stated focus of this bill is fostering innovation and development of infrastructure for AI systems that may benefit the public. Presumably, this includes dissemination of state data to academic researchers to train AI systems and inform AI research. However, the bill’s provisions regarding release of state data seem to extend well beyond these purposes. The bill requires the hub to establish a “streamlined process” by which data collected by state agencies is released to researchers at academic institutions, and requires any data released through this process to be reviewed and approved by the Privacy Agency before it is released. The text of the bill does not limit these provisions to data provided for the use of training AI systems or any other specific purpose, so it may apply to all data released to academic researchers by any state agency. If so, it will result in a potentially significant increase in workload to the Privacy Agency and to each agency that releases data to academic researchers.

Analysis Prepared by: Annika Carlson / APPR. / (916) 319-2081

Date of Hearing: July 2, 2024

ASSEMBLY COMMITTEE ON PRIVACY AND CONSUMER PROTECTION

Rebecca Bauer-Kahan, Chair

SBPCA Bill Id:SB 893

Author:(Padilla) – As Amended Ver:June 21, 2024

SENATE VOTE: 37-0

SUBJECT: California Artificial Intelligence Research Hub

SYNOPSIS

The recent explosive growth of the artificial intelligence (AI) industry in California is powered by access to information. Modern generative artificial intelligence (GenAI) systems are trained using nearly all text data that humanity has thus far produced, and the industries at the heart of the AI hype train are hungry for more. California’s state government serves as a central repository for information related to the state of California – all 156,000 square miles of it. In the right hands, this information represents an untapped goldmine.

This bill would create a mechanism for government data to be released to external researchers. In doing so, however, this bill creates significant privacy concerns. More than 39 million people currently live in California. Reproductive healthcare, immigration, welfare, and taxes – in order to serve its residents effectively, California frequently collects and retains various types of sensitive information from its residents. In return, Californians rely on government to not take advantage of or reveal these data. The Information Practices Act of 1977 (IPA), modeled after the Federal Privacy Act of 1974, codifies this concept: the IPA is the primary privacy statute governing the collection, maintenance, and disclosure of personal information by California state agencies. In its current form, this bill serves as a blanket exemption to the IPA.

Committee amendments narrow the scope of this bill by limiting the recipients of government data under this bill to researchers at academic institutions. Committee amendments also augment the bill’s privacy protections by prohibiting the release of personal information and requiring the California Consumer Privacy Agency to review and approve data prior to its release.

This bill is author sponsored. It is supported by the University of California and Stanford University, as well as a variety of industry associations including the California Chamber of Commerce, TechNet, and the Computer & Communications Industry Association.

Committee amendments, set forth below, narrow the bill’s focus and augment its privacy protections.

SUMMARY: Creates the California Artificial Intelligence Research Hub in the Government Operations Agency and tasks it with increasing access to government data and supporting the research into artificial intelligence. Specifically, this bill:

1. Requires the Government Operations Agency, the Governor’s Office of Business and Economic Development, the California Privacy Protection Agency, and the Department of Technology to collaborate to establish the California Artificial Intelligence Research Hub in the Government Operations Agency.
2. Permits the Government Operations Agency to collaborate with additional state agencies to establish the hub, as needed.
3. Declares that the hub shall serve as a centralized entity to facilitate collaboration between government agencies, academic institutions, and private sector partners to advance artificial intelligence research and development that seeks to harness the technology’s full potential for public benefit while safeguarding privacy, advancing security, and addressing risks and potential harms to society.
4. Requires the Government Operations Agency, the Governor’s Office of Business and Economic Development, the California Privacy Protection Agency, and the Department of Technology to consult with academic institutions within the state in establishing the hub.
5. Requires the California Artificial Intelligence Research Hub to do all of the following:
6. Increase lawful access to government data while protecting privacy and safeguarding access to data by developing a streamlined process for researchers at academic institutions to access data collected by state agencies.
7. Exempts trade secrets, as defined in Civil Code section 3426.1.
8. Support access to and development of artificial intelligence computing capacity and technology by building out public computing infrastructure, facilitating access to existing commercial computing infrastructure, or finding ways to reduce costs and other economic barriers research institutions may face in accessing computing infrastructure.
9. Spur innovation in artificial intelligence applications for the benefit of the public.
10. Ensure the development of trustworthy artificial intelligence technologies with a focus on transparency, fairness, and accountability.
11. Provide researchers with increased access to data and computing resources, education, and training opportunities in furtherance of applications of artificial intelligence for benefit to the people of California.

EXISTING LAW:

1. Provides, pursuant to the California Constitution, that all people are by nature free and independent and have inalienable rights. Among these the fundamental right to privacy. (Cal. Const. art. I, § 1.)
2. Establishes the Information Practices Act (IPA) of 1977, which generally enumerates the requirements applicable to state agencies that collect, maintain, and disclose personal information from California residents, including limitations on permissible disclosure, the rights of residents to know and access the information, and required accounting of disclosures of the information. (Civ. Code § 1798, et seq.)
3. States, in the IPA, that the “right to privacy is a personal and fundamental right protected by Section 1 of Article I of the Constitution of California and by the United States Constitution and that all individuals have a right of privacy in information pertaining to them.” Further states these findings of the Legislature:
   1. The right to privacy is being threatened by the indiscriminate collection, maintenance, and dissemination of personal information and the lack of effective laws and legal remedies.
   2. The increasing use of computers and other sophisticated information technology has greatly magnified the potential risk to individual privacy that can occur from the maintenance of personal information.
   3. In order to protect the privacy of individuals, it is necessary that the maintenance and dissemination of personal information be subject to strict limits. (Civ. Code § 1798.1.)
4. Requires that each state agency maintain in its records only personal information that is relevant and necessary to accomplish the purpose of the agency. (Civ. Code § 1798.14.)
5. Requires that each agency collect personal information to the greatest extent practicable directly from the individual who is the subject of the information rather than from another source. (Civ. Code § 1798.15.)
6. Prohibits an individual’s name and address from being distributed for commercial purposes, sold, or rented by an agency unless such action is specifically authorized by law. (Civ. Code § 1798.60.)
7. Defines “personal information,” for purposes of the IPA, as any information that is maintained by an agency that identifies or describes an individual, including, but not limited to, the individual’s name, social security number, physical description, home address, home telephone number, education, financial matters, and medical or employment history. (Civ. Code § 1798.3(a).)
8. Defines “agency”, for the purposes of the IPA, to mean every state office, officer, department, division, bureau, board, commission, or other state agency, except for the California Legislature, agencies within the judicial branch, the State Compensation Insurance Fund, and local agencies, defined to include: counties; cities, whether general law or chartered; cities and counties; school districts; municipal corporations; districts; political subdivisions; or any board, commission, or agency thereof; other local public agencies, or entities that are legislative bodies of a local agency as specified. (Civ. Code § 1798.3(b); Gov. Code § 6252(a).)
9. Requires each agency to keep an accurate accounting of the date, nature, and purpose of each disclosure of a record made pursuant to specified circumstances; and requires each agency to retain that accounting for at least three years after the disclosure, or until the record is destroyed, whichever is shorter. (Civ. Code §§ 1798.25 & 1798.27.)
10. Except as specified, endows each individual with the following rights: to inquire and be notified as to whether the agency maintains a record about them; to inspect all personal information in any record maintained by reference to an identifying particular of the individual; and to submit a request in writing to amend a record containing personal information pertaining to them maintained by an agency. (Civ. Code § 1798.30, et seq.)
11. Requires each state agency, when it provides by contract for the operation or maintenance of records containing personal information to accomplish an agency function, to cause, consistent with its authority, the requirements of the IPA to be applied to those records; and specifies that for purposes of enforcing penalties for violations of the IPA, any contractor and any employee of the contractor, shall be considered to be an employee of an agency. (Civ. Code § 1798.19.)
12. Establishes the Department of Technology within the Government Operations Agency, under the supervision of the Director of Technology. (Gov. Code § 11545(a).)
13. Requires, upon appropriation by the Legislature, the Secretary of the Government Operations Agency to evaluate the following:
14. The impact of the proliferation of deepfakes on state government, California-based businesses, and residents of the state.
15. The risks, including privacy risks, associated with the deployment of digital content forgery technologies and deepfakes on state and local government, California-based businesses, and residents of the state.
16. Potential privacy impacts of technologies allowing public verification of digital content provenance.
17. The impact of digital content forgery technologies and deepfakes on civic engagement, including voters.
18. The legal implications associated with the use of digital content forgery technologies, deepfakes, and technologies allowing public verification of digital content provenance.
19. The best practices for preventing digital content forgery and deepfake technology to benefit the state, California-based businesses, and California residents, including exploring whether and how the adoption of a digital content provenance standard could assist with reducing the proliferation of digital content forgeries and deepfakes. (Gov. Code § 11547.5(b).)
20. Requires the Secretary of the Government Operations Agency to develop a coordinated plan to accomplish all of the following:
21. Investigate the feasibility of, and obstacles to, developing standards and technologies for state departments for determining digital content provenance.
22. Increase the ability of internet companies, journalists, watchdog organizations, other relevant entities, and members of the public to meaningfully scrutinize and identify digital content forgeries and relay trust and information about digital content provenance to content consumers.
23. Develop or identify mechanisms for content creators to cryptographically certify authenticity of original media and nondeceptive manipulations.
24. Develop or identify mechanisms for content creators to enable the public to validate the authenticity of original media and nondeceptive manipulations to establish digital content provenance without materially compromising personal privacy or civil liberties. (Gov. Code § 11547.5(c).)
25. Expresses the intent of the Legislature that policies and procedures developed by the Department of Technology and Department of General Services pertaining to the acquisition of information technology (IT) goods and services provide for all of the following: the expeditious and value-effective acquisition of IT goods and services to satisfy state requirements; the acquisition of IT goods and services within a competitive framework; the delegation of authority by the Department of General Services to each state agency that has demonstrated to the Department of General Services’ satisfaction the ability to conduct value-effective IT goods and services acquisitions; and the review and resolution of protests submitted by any bidders with respect to any IT goods and services acquisitions. (Pub. Con. Code § 12101.)
26. Requires the Department of Technology, on or before September 1, 2024, to conduct, in coordination with other interagency bodies as it deems appropriate, a comprehensive inventory of all high-risk automated decisionmaking tools that have been proposed for use, development, or procurement by, or are being used, developed, or procured by, any state agency. (Gov. Code § 11546.45.5(b).)
27. Requires the Department of Technology, on or before January 1, 2025, and annually thereafter, to submit a report, as specified, of the comprehensive inventory to the Assembly Committee on Privacy and Consumer Protection and the Senate Committee on Governmental Organization. This requirement expires on January 1, 2029. (Gov. Code § 11546.45.5(d).)

FISCAL EFFECT: As currently in print this bill is keyed fiscal.

COMMENTS:

1. AI and GenAI. The development of GenAI is creating exciting opportunities to grow California’s economy and improve the lives of its residents. GenAI can generate compelling text, images and audio in an instant – but with novel technologies come novel safety concerns.

In brief, AI is the mimicking of human intelligence by artificial systems such as computers. AI uses algorithms – sets of rules – to transform inputs into outputs. Inputs and outputs can be anything a computer can process: numbers, text, audio, video, or movement. AI is not fundamentally different from other computer functions; its novelty lies in its application. Unlike normal computer functions, AI is able to accomplish tasks that are normally performed by humans.

AI that are trained on small, specific datasets in order to make recommendations and predictions are sometimes referred to as “predictive AI.” This differentiates them from GenAI, which are trained on massive datasets in order to produce detailed text and images. When Netflix suggests a TV show to a viewer, the recommendation is produced by predictive AI that has been trained on the viewing habits of Netflix users. When ChatGPT generates text in clear, concise paragraphs, it uses GenAI that has been trained on the written contents of the internet.

1. The importance of training data. There is a common saying in computer science: “garbage in, garbage out.” The performance of an AI product is directly impacted by the quality, quantity, and relevance of the data used to train it. Before training, datasets are often categorized to make them easier for AI to work with. Rigorously categorizing the data in a dataset becomes more difficult as the dataset becomes larger, but failing to organize its contents can lead to meaningless, false, or harmful outputs.

The biggest names in AI – OpenAI, Meta, and Google – understand AI’s critical need for data better than anyone else. According to a recent New York Times examination, the race to lead in the AI space has become a desperate hunt for digital data. To obtain that data, these tech companies have cut corners, ignored corporate policies and debated bending the law:

At Meta, which owns Facebook and Instagram, managers, lawyers and engineers last year discussed buying the publishing house Simon & Schuster to procure long works, according to recordings of internal meetings obtained by The Times. They also conferred on gathering copyrighted data from across the internet, even if that meant facing lawsuits. Negotiating licenses with publishers, artists, musicians and the news industry would take too long, they said.

Like OpenAI, Google transcribed YouTube videos to harvest text for its A.I. models, five people with knowledge of the company’s practices said. That potentially violated the copyrights to the videos, which belong to their creators.

Last year, Google also broadened its terms of service. One motivation for the change, according to members of the company’s privacy team and an internal message viewed by The Times, was to allow Google to be able to tap publicly available Google Docs, restaurant reviews on Google Maps and other online material for more of its A.I. products.

Meta and Google are privy to some of the most sensitive information in the world. In many developing countries, Facebook effectively is the internet. A tremendous number of Californians use Google, or Google Chrome, or Google Drive, or Google Cloud, or Gmail. In their race to obtain vast quantities of training data, major AI developers have not hesitated to move fast and break things. The Stanford Internet Observatory recently discovered that a common image training dataset known as LAION-5B contains many instances of child sexual abuse materials. Their study identified 3226 dataset entries of suspected child pornography, much of which was later confirmed as such by third parties. This dataset was built by automatically scraping the internet, and images containing child pornography were found to have originated from large, well-known websites such as Reddit, Twitter, Blogspot, and Wordpress, as well as mainstream adult sites such as XHamster and XVideos.

1. An AI never forgets. Just as humans cannot intentionally forget information they have learned, it is not currently possible to remove data from a trained AI. Unlike an Excel spreadsheet, which stores data in neat columns, AI stores data in the connections between “neurons” in a “neural network.” Every one of these connections is influenced by every piece of training data, and a large model like ChatGPT-4 is reported to have more than 1.7 trillion connections. It is not possible to specifically alter these connections in order to remove data without fundamentally changing the model; as a result, for data to be removed, the model must be retrained from scratch. ChatGPT-4 is estimated to have taken 4-7 months to train in the first place.

What happens when an AI is trained on extremely sensitive information – for example, an individual’s DNA sequence, or their social security number, or their intimate photos, or their immigration status? The same thing that happens when an AI is trained on any other type of information: the AI digests it, and then retains it forever. AI are fundamentally different from other forms of data storage. They are black holes in the information ecosystem, with “training” as their event horizons. Once data has crossed this threshold it cannot be removed.

1. The California Consumer Privacy Act (CCPA) and the California Privacy Rights Act (CPRA). In 2018, the Legislature enacted the CCPA (AB 375 (Chau, Chap. 55, Stats. 2018)), which gives consumers certain rights regarding their personal information, such as the right to: (1) know what personal information about them is collected and sold; (2) request the categories and specific pieces of personal information the business collects about them; and (3) opt out of the sale of their personal information, or opt in, in the case of minors under 16 years of age.

Subsequently, in 2020, California voters passed Proposition 24, the California Privacy Rights Act (CPRA), which established additional privacy rights for Californians. With the passage of the CCPA and the CPRA, California now has the most comprehensive laws in the country when it comes to protecting consumers’ rights to privacy.

In addition, Proposition 24 created the California Privacy Protection Agency (Privacy Agency) in California, vested with full administrative power, authority, and jurisdiction to implement and enforce the CCPA and the CPRA. The Privacy Agency’s responsibilities include updating existing regulations, and adopting new regulations.

1. The Information Practices Act. The Information Practices Act of 1977 (IPA; Civ. Code § 1798, et seq.), modeled after the Federal Privacy Act of 1974, is the primary privacy statute governing the collection, maintenance, and disclosure of personal information by California state agencies. Along with the substantive provisions of the IPA, the Legislature codified findings and declarations upon its passage justifying the need for the consistent limits on the maintenance and dissemination of personal information by government agencies.

Generally, the IPA places several conditions and restrictions on the collection, maintenance, and disclosure of the personal information of Californians held by state agencies, including a prohibition on the disclosure of an individual’s personal information without the individual’s consent except in specified circumstances. In addition, the IPA requires that along with any form requesting personal information from an individual, an agency provide notice of information pertaining to the individual’s rights with respect to their personal information, the purposes for which the personal information will be used, and any foreseeable disclosures of that personal information.

The IPA also provides individuals with certain rights to be informed of what personal information an agency holds relating to that individual; to access and inspect that personal information; and to request corrections to that personal information, subject to specified exceptions. Finally, when state agencies contract with private entities for services, the contractors are typically governed by the IPA, with few additional privacy protections generally stipulated in the contracts themselves.

1. What this bill would do. This bill would require the Government Operations Agency, in collaboration with the Governor’s Office of Business and Economic Development and the California Privacy Protection Agency, to collaborate to create the California Artificial Intelligence Research Hub in the Government Operations Agency. The California Artificial Intelligence Research Hub would be tasked with increasing lawful access to government data, excluding trade secrets, and with generally supporting the development of infrastructure related to artificial intelligence.
2. Author’s statement:

California is a global leader in technological advancement. Much of that leadership has been driven by our world-class higher education systems. Emerging AI technologies are costly and energy intensive, and require broad-based coordination among institutions and other sectors. Shared resources will be vital to the continued development of AI technology in California. The creation of the California Artificial Intelligence Research Hub allows us to pool and leverage the state’s financial resources and the intellectual firepower of our academic sector to democratize AI and stop it from becoming monopolized by proprietary interests alone – the tech titans.

1. Committee amendments. Three committee amendments serve to narrow this bill’s focus and augment its privacy protections. The first limits access to government data to researchers at academic institutions. The second prohibits the release of personal information, as defined in Section 1798.3 of the Civil Code. The third requires the Privacy Agency to review and approve for release any government data prior to those data being made available. The amended text follows:

(e) The hub shall do all of the following:

(1) (A) (i) Increase lawful access to government data while protecting privacy and safeguarding access to data by developing a streamlined process for researchers at academic institutions to access data collected by state agencies.

(ii) Lawful access to government data increased pursuant to clause (i) shall not include access to trade secrets, as defined in Section 3426.1 of the Civil Code, obtained by the state.

(iii) Lawful access to government data increased pursuant to clause (i) shall not include access to personal information, as defined in Section 1798.3 of the Civil Code, obtained by the state.

(B) In complying with subparagraph (A), the hub shall create a process for eligibility that prioritizes security by limiting who can access the data and for what purpose.

(C) Any government data made available pursuant to subparagraph (A) must first be reviewed and approved for release by the California Privacy Protection Agency.

1. Related legislation. AB 302 (Ward, Ch. 800, Stats. 2023) required CDT in coordination with other interagency bodies, to conduct a comprehensive inventory of all high-risk ADS used by state agencies on or before September 1, 2024, and report the findings to the Legislature by January 1, 2025, and annually thereafter, as specified.

SB 896 (Dodd, 2024) would largely codify Governor Newsom’s executive order on the use of Generative artificial intelligence (GenAI). The bill requires assessments of the beneficial uses, potential harms, and risks to critical infrastructure of GenAI. The bill calls for the development of guidelines for public sector procurement, uses, and required trainings for the use of GenAI. The bill places obligations on state entities with respect to the use of GenAI and ADS. SB 896 is currently in this Committee.

SB 1047 (Wiener, 2024) establishes the Frontier Model Division in the Government Operations Agency and tasks it with overseeing of the most advances artificial intelligence models. SB 1047 is currently pending in the Assembly Judiciary Committee.

ARGUMENTS IN SUPPORT:

Writing on behalf of an industry coalition, Computer & Communications Industry Association states:

SB 893 outlines several key responsibilities that would fall under the Hub which would allow it to function as a central facilitator that fosters collaboration among government agencies, academic institutions, and private sector partners to drive forward artificial intelligence research and development. The measure also seeks to responsibly support access to and development of artificial intelligence computing capacity by finding ways to reduce costs and other economic barriers research institutions may face in accessing computing infrastructure. The co-signed organizations believe this is an effective way to promote innovation for public benefit while providing protections for consumer privacy and promoting equity. We believe that SB 893 strikes the right balance of acknowledging the potential immense benefits of artificial intelligence while also safeguarding privacy, advancing security, and addressing risks and potential harms to society.

REGISTERED SUPPORT / OPPOSITION:

Support

California Chamber of Commerce
Computer & Communications Industry Association
Engine
R Street Institute
San Diego Regional Chamber of Commerce
Security Industry Association
Southwest California Legislative Council
Stanford University
TechNet
Technology Industry Association of California (TECHCA)
Tri County Chamber Alliance
University of California

Opposition

None on file.

Analysis Prepared by: Slater Sharp / P. & C.P. / (916) 319-2200

 THIRD READING  

Bill No: SB 893

Author: Padilla (D), et al.

Amended: 3/19/24

Vote: 21

SENATE GOVERNMENTAL ORG. COMMITTEE: 14-0, 4/9/24

AYES: Dodd, Wilk, Alvarado-Gil, Archuleta, Bradford, Glazer, Jones, Nguyen, Ochoa Bogh, Padilla, Portantino, Rubio, Seyarto, Smallwood-Cuevas

NO VOTE RECORDED: Ashby, Roth

SENATE JUDICIARY COMMITTEE: 11-0, 4/16/24

AYES: Umberg, Wilk, Allen, Ashby, Caballero, Durazo, Laird, Min, Niello, Stern, Wahab

SENATE APPROPRIATIONS COMMITTEE: 7-0, 5/16/24

AYES: Caballero, Jones, Ashby, Becker, Bradford, Seyarto, Wahab

SUBJECT: California Artificial Intelligence Research Hub

SOURCE: Author

DIGEST:

This bill requires the Government Operations Agency (GovOps), the Governor’s Office of Business and Economic Development (GO-Biz), and the California Department of Technology (CDT) to collaborate and establish the California Artificial Intelligence Research Hub (Hub) to serve as a centralized entity to facilitate collaboration between government agencies, academic institutions, and private sector partners to advance artificial intelligence (AI) research and development, as specified.

ANALYSIS:

Existing law:

1. Requires the Secretary of GovOps to develop a coordinated plan to, among other things, investigate the feasibility of, and obstacles to, developing standards and technologies for state departments to determine digital content provenance.
2. Requires the Secretary of GovOps, for purposes of informing that coordinated plan, to evaluate, among other things, the impact of the proliferation of deepfakes, defined to mean audio or visual content that has been generated or manipulated by AI that would falsely appear to be authentic or truthful and that features depictions of people appearing to say or do things they did not say or do without their consent, on state government, California-based businesses, and residents of the state.
3. Defines “trade secrets” to mean information, including a formula, pattern, compilation, program, device, method, technique, or process that:
   1. Derives independent economic value, actual or potential, from not being generally known to the public or other persons who can obtain economic value from its disclosure or use; and
   2. Is the subject of efforts that are reasonable under the circumstances to maintain its secrecy.

This bill:

1. Requires GovOps, GO-Biz, and CDT to collaborate to establish the California AI Research Hub in GovOps, and authorizes GovOps to collaborate with additional state agencies to establish the Hub, as needed.
2. Authorizes GovOps to collaborate with additional state agencies to establish the hub, as needed.
3. Requires the Hub to serve as a centralized entity to facilitate collaboration between government agencies, academic institutions, and private sector partners to advance AI research and development that seeks to harness the technology’s full potential for public benefit while safeguarding privacy, advancing security, and addressing risks and potential harms to society.
4. Requires GovOps, GO-Biz, and CDT to consult with academic institutions within the state in establishing the Hub.
5. Requires the Hub to do all of the following:
   1. Increase lawful access to government data while protecting privacy and safeguarding access to data by developing a streamlined process for researchers to access data collected by state agencies. Lawful access to government data increased pursuant to this bill shall not include access to trade secrets, as defined, obtained by the state. In complying with this requirement, the Hub shall create a process for eligibility that prioritizes security by limiting who can access the data and for what purpose.
   2. Support the access to, and development of, AI computing capacity and technology by building out public computing infrastructure, facilitating access to existing commercial computing infrastructure, or finding ways to reduce costs and other economic barriers research institutions may face in accessing computing infrastructure.
   3. Spur innovation in AI applications for the benefit of the public.
   4. Ensure the development of trustworthy AI technologies with a focus on transparency, fairness, and accountability.
   5. Provide researchers with increased access to data and computing resources, education, and training opportunities in furtherance of applications of AI for benefit to the people of California.
6. Provides that lawful access to government data increased pursuant to this bill shall not include access to trade secrets, as specified.

Background

Author Statement. According to the author’s office, “California is a global leader in technological advancement. Much of that leadership has been driven by our world-class higher education systems. Emerging AI technologies are costly and energy intensive, and require broad-based coordination among institutions and other sectors. Shared resources will be vital to the continued development of AI technology in California. The creation of the California Artificial Intelligence Research Hub allows us to pool and leverage the state’s financial resources and the intellectual firepower of our academic sector to democratize AI and stop it from becoming monopolized by proprietary interests alone – the tech titans.”

Navigating the AI Landscape. In the rapidly evolving landscape of technology, AI stands at the forefront, bringing a new era of innovation and application across various sectors. AI, defined broadly, is the capability of machines to perform tasks that typically require human intelligence, encompasses technologies that can process information, learn, reason, recognize patterns, and make decisions. The transformative potential of AI is vast, affecting disparate industries from healthcare to transportation, and its development is a subject of both enthusiasm and scrutiny. Given the technology's capacity to revolutionize the way we live and work, governments around the world and across the nation are seeking to nurture its growth while addressing the ethical, privacy, and security concerns it raises.

At the federal level in the United States, efforts to consolidate and amplify AI research and development have been underway. As part of the National Artificial Intelligence Initiative Act of 2020, Congress established the National Artificial Intelligence Research Resource (NAIRR) Task Force to “investigate the feasibility and advisability of developing” the NAIRR as a national AI research cyberinfrastructure, and “to propose a roadmap detailing [how the NAIRR] should be established and sustained.” The recent CHIPS and Science Act of 2022 reinforced the importance of democratizing access to a national AI research cyberinfrastructure, via investments that will accelerate development of advanced computing—from next-generation graphics processing units to high-density memory chips—as well as steps to actively engage broad and diverse U.S. talent in frontier science and engineering, including AI.

The NAIRR Task Force’s final report in January 2023 was the culmination of an 18-month effort to develop a vision and implementation plan for establishing the NAIRR. According to that report, the “NAIRR should be established with four measureable goals in mind, namely to (1) spur innovation, (2) increase diversity of talent, (3) improve capacity, and (4) advance trustworthy AI.”

In October of 2023, President Joe Biden issued the Executive Order on the Safe, Secure, and Trustworthy Development and Use of Artificial Intelligence. In that executive order, the President required the U.S. National Science Foundation (NSF) to launch a pilot program implementing the NAIRR, consistent with past recommendations of the NAIRR Task Force. The program is required to pursue the infrastructure, governance mechanisms, and user interfaces to pilot an initial integration of distributed computational, data, model, and training resources to be made available to the research community in support of AI-related research and development.

The NAIRR pilot is a vision for a shared national research infrastructure for responsible discovery and innovation in AI. Led by the NSF in partnership with 10 other federal agencies and 25 non-governmental partners, the pilot makes available government-funded, industry and other contributed resources in support of the nation’s research and education community. The NAIRR is a concept for a national infrastructure that connects U.S. researchers to computational, data, software, model and training resources they need to participate in AI research.

The NAIRR pilot will run for two years, beginning January 24, 2024. The pilot will broadly support fundamental, translational and use-inspired AI-related research with particular emphasis on societal changes. Initial priority topics include safe, secure and trustworthy AI; human health; and environment and infrastructure. A broader array of priority areas will be supported as the pilot progresses. The pilot will also support educators to train students on responsible use and development of AI technologies by providing access to infrastructure and training resources.

Blueprint for an AI Bill of Rights. In October 2022, The White House’s Office of Science and Technology Policy (OSTP) released the Blueprint for an AI Bill of Rights: Making Automated Systems Work for the American People (Blueprint) as a framework after a year of public engagement to inform the initiative. The Blueprint is a white paper published by OSTP intended to support the development of policies and practices that protect civil rights and promote democratic values in the building, deployment, and governance of automated systems. This non-binding document does not constitute U.S. government policy, does not superseded, modify, or direct an interpretation of any existing statute, regulation, policy, or international instrument.

The appropriate application of the principles set forth in the white paper depends on the context in which automated systems are being utilized. The five identified principles in the Blueprint help provide guidance whenever automated systems can meaningfully impact the public’s rights, opportunities, or access to critical needs. Specifically, the Blueprint identifies the following five principles: safe and effective systems; algorithmic discrimination protection; data privacy; notice and explanation; and human alternatives, consideration, and fallback.

SCR 17 (Dodd, Resolution Chapter 135, Statutes of 2023) affirmed the California Legislature’s commitment to President Biden’s vision for a safe AI and the principles outlined in the blueprint. Additionally, that resolution expressed the Legislature’s commitment to examining and implementing those principles in its legislation and policies related to the use and deployment of automated systems.

Governor Newsom’s Executive Order N-12-23. In September 2023, Governor Newsom signed Executive Order (EO) N-12-23 to prepare California for the Progress of AI. As a global hub for generative AI (GenAI), the EO announcement notes that California is “the natural leader in this emerging field of technology – tools that could very well change the world. To capture its benefits for the good of society, but also to protect against its potential harms, Governor Newsom issued an executive order today laying out how California’s measured approach will focus on shaping the future of ethical, transparent, and trustworthy AI, while remaining the world’s AI leader.”

Specifically, the EO requires the establishment of a formal partnership with the University of California, Berkeley and Stanford University to consider and evaluate the impacts of GenAI on California and what efforts the state should undertake to advance its leadership in this industry. The state and the institutions will develop and host a joint summit in 2024 to engage in meaningful discussions about the impacts of GenAI on California and its workforce.

California AI Research Hub Act. This bill requires GovOps, GO-Biz, and CDT to collaborate to establish the Hub in GovOps. Further, the bill specifically authorizes GovOps to collaborate with additional state agencies to establish the Hub, as needed.

SB 893 bill requires the Hub to serve as a centralized entity to facilitate collaboration between government agencies, academic institutions, and private sector partners to advance AI research and development that seeks to harness the technology’s full potential for public benefit while safeguarding privacy, advancing security, and addressing risks and potential harms to society.

This bill focuses the Hub on five general objectives including data access and privacy, computing infrastructure, innovation and public benefit, trustworthy AI development, and education and training. Specifically, this bill requires the hub to do all of the following:

1. Increase lawful access to government data while protecting privacy and safeguarding access to data by developing a streamlined process for researchers to access data collected by state agencies. Lawful access to government data increased pursuant to this bill shall not include access to trade secrets, as defined, obtained by the state. In complying with this requirement, the Hub must create a process for eligibility that prioritizes security by limiting who can access the data and for what purpose.
2. Support the access to, and development of, AI computing capacity and technology by building out public computing infrastructure, facilitating access to existing commercial computing infrastructure, or finding ways to reduce costs and other economic barriers research institutions may face in accessing computing infrastructure.
3. Spur innovation in AI applications for the benefit of the public.
4. Ensure the development of trustworthy AI technologies with a focus on transparency, fairness, and accountability.
5. Provide researchers with increased access to data and computing resources, education, and training opportunities in furtherance of applications of AI for benefit to the people of California.

This bill specifies that lawful access to government data increased pursuant to the above does not include access to trade secrets, as defined, obtained by the state. For the purposes of this bill, “trade secrets” mean information, including a formula, pattern, compilation, program, device, method, technique, or process that: derives independent economic value, actual or potential, from not being generally known to the public or other persons who can obtain economic value from its disclosure or use; and is the subject of efforts that are reasonable under the circumstances to maintain its secrecy.

Related/Prior Legislation

SB 892 (Padilla, 2024) requires CDT to develop and adopt regulations to create an AI risk management standard, consistent with specified publications regarding AI risk management, as specified. (Pending on the Senate Floor)

SB 896 (Dodd, 2024) the AI Accountability Act requires state agencies to advance safeguards and consumer protections around AI by incorporating and building upon recent directives from President Joe Biden and Governor Gavin Newsom. (Pending on the Senate Floor)

SB 1047 (Wiener, 2024) the Safe and Secure Innovation for Frontier AI Systems Act, among other things, requires the developer of a covered model, as defined, to determine whether it can make a positive safety determination before initiating training of that covered model, as specified, and creates the Frontier Model Division within CDT, as specified. This bill requires CDT to commission consultants, as prescribed, to create a public cloud computing cluster, to be known as CalCompute, with the primary focus of conducting research into the safe and secure deployment of large-scale AI models and fostering equitable innovation that includes, among other things, a fully owned and hosted cloud platform. (Pending on the Senate Floor)

SCR 17 (Dodd, Resolution Chapter 135, Statutes of 2023) committed the Legislature to examining and implementing the principals outlined in the Blueprint for an Artificial Intelligence Bill of Rights, published in October 2022 by the White House Office of Science and Technology Policy.

FISCAL EFFECT: Appropriation: No Fiscal Com.: Yes Local: No

According to the Senate Appropriations Committee, unknown significant one-time and ongoing costs ranging into the millions of dollars for GovOps, GO-Biz, and CDT to establish the hub and build out public computing infrastructure (General Fund). One-time costs may include equipment and other IT operating expenses, as well as outreach to solicit feedback from industry and academic institutions. Ongoing costs may include additional specialized staffing resources to build and maintain public computing infrastructure. Other operating expenses may include convening working groups, conducting meetings, and responding to or engaging with members of the public, industry, academic institutions, and other stakeholders.

SUPPORT: (Verified 5/16/24)

California Chamber of Commerce
California Institute of Technology
Chamber of Progress
Computer & Communications Industry Association
Computer and Communications Industry Association
Engine
Los Angeles Area Chamber of Commerce
R Street Institute
Security Industry Association
Silicon Valley Leadership Group
Stanford University
University of California

OPPOSITION: (Verified 5/16/24)

None received

ARGUMENTS IN SUPPORT: In support of the bill, the University of California (UC) writes that the “goal of SB 893 is to have various state agencies and departments to collaborate with academic institutions, including the University of California campuses, across the state in establishing the California Artificial Intelligence Research Hub. Through this hub, the state intends to facilitate access to computing resources and data for academic institutions while safeguarding privacy, advancing security, and addressing risks and potential risks to society.”

Prepared by: Brian Duke / G.O. / (916) 651-1530

5/17/24 10:09:09

**** END ****

SENATE COMMITTEE ON

Senator Anna Caballero, Chair

2023 - 2024 Regular Session

SB 893 (Padilla) - California Artificial Intelligence Research Hub

Bill Summary:

SB 893 establishes the California Artificial Intelligence Research Hub in the Government Operations Agency (GovOps) to serve as a centralized entity to facilitate collaboration between government agencies, academic institutions, and private sector partners to advance artificial intelligence (AI) research and development, as specified.

Fiscal Impact:

• Unknown significant one-time and ongoing costs ranging into the millions of dollars for GovOps, the Governor’s Office of Business and Economic Development (GO-Biz), and the California Department of Technology (CDT) to establish the hub and build out public computing infrastructure (General Fund). One-time costs may include equipment and other IT operating expenses, as well as outreach to solicit feedback from industry and academic institutions. Ongoing costs may include additional specialized staffing resources to build and maintain public computing infrastructure. Other operating expenses may include convening working groups, conducting meetings, and responding to or engaging with members of the public, industry, academic institutions, and other stakeholders.

Background:

Generally, AI is the capability of machines to perform tasks that typically require human intelligence, encompasses technologies that can process information, learn, reason, recognize patterns, and make decisions. The transformative potential of AI is vast, affecting disparate industries from healthcare to transportation, and its development is a subject of both enthusiasm and scrutiny. Given the technology's capacity to revolutionize the way we live and work, governments around the world and across the nation are seeking to nurture its growth while addressing the ethical, privacy, and security concerns it raises.

At the federal level in the United States, efforts to consolidate and amplify AI research and development have been underway. As part of the National Artificial Intelligence Initiative Act of 2020, Congress established the National Artificial Intelligence Research Resource (NAIRR) Task Force to “investigate the feasibility and advisability of developing” the NAIRR as a national AI research cyberinfrastructure, and “to propose a roadmap detailing [how the NAIRR] should be established and sustained.” The recent Creating Helpful Incentives to Produce Semiconductors (CHIPS) and Science Act of 2022 reinforced the importance of democratizing access to a national AI research cyberinfrastructure, via investments that will accelerate development of advanced computing—from next-generation graphics processing units to high-density memory chips—as well as steps to actively engage broad and diverse U.S. talent in frontier science and engineering, including AI.

The NAIRR Task Force’s final report in January 2023 was the culmination of an 18-month effort to develop a vision and implementation plan for establishing the NAIRR. According to that report, the “NAIRR should be established with four measureable goals in mind, namely to (1) spur innovation, (2) increase diversity of talent, (3) improve capacity, and (4) advance trustworthy AI.”

In October of 2023, President Joe Biden issued the Executive Order on the Safe, Secure, and Trustworthy Development and Use of Artificial Intelligence. In that executive order, the President required the U.S. National Science Foundation (NSF) to launch a pilot program implementing the NAIRR, consistent with past recommendations of the NAIRR Task Force. The program is required to pursue the infrastructure, governance mechanisms, and user interfaces to pilot an initial integration of distributed computational, data, model, and training resources to be made available to the research community in support of AI-related research and development.

Proposed Law:

• Establishes the California Artificial Intelligence Research Hub in the GovOps Agency to serve as a centralized entity to facilitate collaboration between government agencies, academic institutions, and private sector partners to advance AI research and development that seeks to harness the technology’s full potential for public benefit while safeguarding privacy, advancing security, and addressing risks and potential harms to society.
• Requires GovOps, GO-Biz, and CDT to establish the hub in consultation with academic institutions.
• Requires the hub to:
  • Increase lawful access to government data while protecting privacy and safeguarding access to data by developing a streamlined process for researchers to access data collected by state agencies.
  • Support the access to, and development of, AI computing capacity and technology by building out public computing infrastructure, facilitating access to existing commercial computing infrastructure, or finding ways to reduce costs and other economic barriers research institutions may face in accessing computing infrastructure.
  • Spur innovation in AI applications for the benefit of the public.
  • Ensure the development of trustworthy AI technologies with a focus on transparency, fairness, and accountability.
• Provide researchers with increased access to data and computing resources, education, and training opportunities in furtherance of applications of AI for benefit to the people of California.

Related Legislation:

SB 892 (Padilla) requires CDT to develop and adopt regulations for an AI risk management standard regarding state procurement and use of automated decision systems (ADS), as specified. SB 892 is pending in this committee.

SB 896 (Dodd, 2024) establishes the Artificial Intelligence Accountability Act, which tasks specified state entities with developing and periodically evaluating general guidelines for the use, regulation, and identification of potential risks of incorporating AI and generative AI (GenAI) within state government operations. SB 896 is pending in this committee.

SB 942 (Becker, 2024) the California AI Transparency Act, among other things, requires a covered provider to create an AI detection tool by which a person can query the covered provide as to the extent to which text, image, video, audio, or multimedia content was created, in whole or in part, by a GenAI system, as specified. SB 942 is pending in this committee.

SB 1047 (Wiener, 2024) requires developers of powerful AI models, and those providing the computer power to train such models, to put appropriate safeguards and policies into place to prevent critical harms. Additionally, SB 1047 establishes a state entity to oversee the development of these models and calls for the creation of a public cloud computer cluster known as CalCompute, as specified. SB 1047 is pending in this committee.

-- END --

SENATE JUDICIARY COMMITTEE

Senator Thomas Umberg, Chair

2023-2024 Regular Session

Bill No: SB 893 (Author:Padilla)

Version: March 19, 2024

Hearing Date: April 16, 2024

Fiscal: Yes

Urgency: No

Consultant: CK

SUBJECT

California Artificial Intelligence Research Hub

DIGEST

This bill directs state entities to establish the California Artificial Intelligence Research Hub to serve as a centralized entity to facilitate collaboration between government agencies, academic institutions, and private sector partners to advance artificial intelligence research and development.

EXECUTIVE SUMMARY

The transformative power of AI is unquestionable, and it offers numerous benefits for society, including state government. It can enhance efficiency and effectiveness in various sectors freeing up human resources. AI can improve decision-making processes by analyzing vast amounts of data to identify patterns, trends, and potential insights. This capability enables state governments to make more informed policy decisions, allocate resources more efficiently, and enhance public services. However, as with most technologies, there are also inherent risks and challenges. For instance, concerns about data privacy and security, as Generative AI systems rely on vast amounts of sensitive information. Ensuring transparency, accountability, and fairness in AI systems is crucial to mitigate these risks and maintain public trust. Additionally, there are concerns that the development of this technology and the massive resources and energy it requires will leave government and academic institutions in the dark.

In order to counteract this imbalance, this bill calls for the Government Operations Agency (GovOps), the Governor’s Office of Business and Economic Development (GO-Biz), and the California Department of Technology (CDT) to establish the California AI Research Hub (“the Hub”). Through collaborations between public and private sector partners, the Hub will serve to harness AI’s full potential for public benefit while addressing the risks and harms highlighted above.

The bill is author-sponsored and supported by the California Institute of Technology and various industry organizations. No timely opposition was received by the Committee. The bill passed out of the Senate Governmental Organization Committee on a vote of 14 to 0.

PROPOSED CHANGES TO THE LAW

Existing law:

1. Establishes CDT within GovOps, under the supervision of the Director of Technology (Director), also known as the State Chief Information Officer. (Gov. Code Sec. 11545(a).)
2. Requires, upon appropriation by the Legislature, the Secretary of GovOps (Secretary) to evaluate the following:
   1. The impact of the proliferation of deepfakes on state government, California-based businesses, and residents of the state.
   2. The risks, including privacy risks, associated with the deployment of digital content forgery technologies and deepfakes on state and local government, California-based businesses, and residents of the state.
   3. Potential privacy impacts of technologies allowing public verification of digital content provenance.
   4. The impact of digital content forgery technologies and deepfakes on civic engagement, including voters.
   5. The legal implications associated with the use of digital content forgery technologies, deepfakes, and technologies allowing public verification of digital content provenance.
   6. The best practices for preventing digital content forgery and deepfake technology to benefit the state, California-based businesses, and California residents, including exploring whether and how the adoption of a digital content provenance standard could assist with reducing the proliferation of digital content forgeries and deepfakes. (Gov’t Code § 11547.5(b).)
3. Requires the Secretary to develop a coordinated plan to accomplish all of the following:
   1. Investigate the feasibility of, and obstacles to, developing standards and technologies for state departments for determining digital content provenance.
   2. Increase the ability of internet companies, journalists, watchdog organizations, other relevant entities, and members of the public to meaningfully scrutinize and identify digital content forgeries and relay trust and information about digital content provenance to content consumers.
   3. Develop or identify mechanisms for content creators to cryptographically certify authenticity of original media and nondeceptive manipulations.
   4. Develop or identify mechanisms for content creators to enable the public to validate the authenticity of original media and nondeceptive manipulations to establish digital content provenance without materially compromising personal privacy or civil liberties. (Gov’t Code § 11547.5(c).)
4. Expresses the intent of the Legislature that policies and procedures developed by CDT and Department of General Services (DGS) pertaining to the acquisition of IT goods and services provide for all of the following: the expeditious and value-effective acquisition of IT goods and services to satisfy state requirements; the acquisition of IT goods and services within a competitive framework; the delegation of authority by DGS to each state agency that has demonstrated to DGS’s satisfaction the ability to conduct value-effective IT goods and services acquisitions; and the review and resolution of protests submitted by any bidders with respect to any IT goods and services acquisitions. (Pub. Con. Code § 12101.)
5. Requires CDT, on or before September 1, 2024, to conduct, in coordination with other interagency bodies as it deems appropriate, a comprehensive inventory of all high-risk ADS that have been proposed for use, development, or procurement by, or are being used, developed, or procured by, any state agency. (Gov’t Code § 11546.45.5(b).)
6. Requires CDT, on or before January 1, 2025, and annually thereafter, to submit a report, as specified, of the comprehensive inventory to the Assembly Committee on Privacy and Consumer Protection and the Senate Committee on Governmental Organization. This requirement expires on January 1, 2029. (Gov’t Code § 11546.45.5(d).)

This bill:

1. Requires GovOps, GO-Biz, and CDT to collaborate to establish the California Artificial Intelligence Research Hub in GovOps. The Hub shall serve as a centralized entity to facilitate collaboration between government agencies, academic institutions, and private sector partners to advance AI research and development that seeks to harness the technology’s full potential for public benefit while safeguarding privacy, advancing security, and addressing risks and potential harms to society.
2. Authorizes collaboration with additional state agencies and academic institutions within the state in establishing the Hub.
3. Directs the Hub to accomplish the following:
   1. Increase lawful access to government data while protecting privacy and safeguarding access to data by developing a streamlined process for researchers to access data collected by state agencies, excluding trade secrets. A process for eligibility shall prioritize security by limiting who can access the data and for what purpose.
   2. Support the access to, and development of, AI computing capacity and technology by building out public computing infrastructure, facilitating access to existing commercial computing infrastructure, or finding ways to reduce costs and other economic barriers research institutions may face in accessing computing infrastructure.
   3. Spur innovation in AI applications for the benefit of the public.
   4. Ensure the development of trustworthy AI technologies with a focus on transparency, fairness, and accountability.
   5. Provide researchers with increased access to data and computing resources, education, and training opportunities in furtherance of applications of AI for benefit to the people of California.

COMMENTS

1. Frameworks for responsible development and accountability in AI

Owing to recent advances in processing power and the rise of big data, AI’s capacity and the scope of its applications have expanded rapidly, impacting how we communicate, interact, entertain ourselves, travel, transact, and consume media. For good or ill, its transformative potential seems boundless. With these recent dramatic advances in the capabilities of AI systems, the need for frameworks for accountability and responsible development have become ever more urgent.

In January of 2017, AI researchers, economists, legal scholars, ethicists, and philosophers met in Asilomar, California to discuss principles for managing the responsible development of AI. The collaboration resulted in the Asilomar Principles. Aspirational rather than prescriptive, these 23 principles were intended to initiate and frame a dialogue by providing direction and guidance for policymakers, researchers, and developers. Its endorsers include 1,200 leading experts in the field of AI, including DeepMind founder Demis Hassabis and the late Stephen Hawking.

The Legislature subsequently adopted ACR 215 (Kiley, Ch. 206, Stats. 2018), which added the State of California to that list by endorsing the Asilomar Principles as guiding values for the development of artificial intelligence and related public policy. In broad strokes, those principles aim to do the following:

• Research issues: create beneficial AI; direct funding toward beneficial innovation; maintain constructive and healthy exchanges between AI researchers and policymakers; promote a culture of trust, cooperation, and transparency among researchers and developers of AI; and avoid corner-cutting on safety standards.
• Ethics and values: promote safety, failure transparency, judicial transparency, and responsible innovation; align human values with innovation; protect privacy and liberty; ensure that the benefits and prosperity created by AI are broadly shared; maintain human control over AI; develop AI that supports rather than subverts social and civil processes; and avoid an AI arms race.
• Longer-term issues: avoid assumptions regarding the capabilities of AI; give AI its due attention; and recognize that its risks are potentially catastrophic or existential. [emphasis added]

As directed by the National AI Initiative Act of 2020, NIST developed the AI Risk Management Framework to assist entities designing, developing, deploying, and using AI systems to help manage the many risks of AI and promote trustworthy and responsible development and use of AI systems. That framework highlights the serious risks at play and the uniquely challenging nature of addressing them in this context.

Seeking to establish a framework for California, Governor Gavin Newsom issued Executive Order N-12-23 to “to study the development, use, and risks of artificial intelligence (AI) technology throughout the state and to develop a deliberate and responsible process for evaluation and deployment of AI within state government.”^[1]

The executive order includes the following provisions:

• Risk-Analysis Report: Directs state agencies and departments to perform a joint risk-analysis of potential threats to and vulnerabilities of California’s critical energy infrastructure by the use of GenAI.
• Procurement Blueprint: To support a safe, ethical, and responsible innovation ecosystem inside state government, agencies will issue general guidelines for public sector procurement, uses, and required training for application of GenAI – building on the White House’s Blueprint for an AI Bill of Rights and the National Institute for Science and Technology’s AI Risk Management Framework. State agencies and departments will consider procurement and enterprise use opportunities where GenAI can improve the efficiency, effectiveness, accessibility, and equity of government operations.
• Beneficial Uses of GenAI Report: Direct state agencies and departments to develop a report examining the most significant and beneficial uses of GenAI in the state. The report will also explain the potential harms and risks for communities, government, and state government workers.
• Deployment and Analysis Framework: Develop guidelines for agencies and departments to analyze the impact that adopting GenAI tools may have on vulnerable communities. The state will establish the infrastructure needed to conduct pilots of GenAI projects, including California Department of Technology approved environments or “sandboxes” to test such projects.
• State Employee Training: To support California’s state government workforce and prepare for the next generation of skills needed to thrive in the GenAI economy, agencies will provide trainings for state government workers to use state-approved GenAI to achieve equitable outcomes, and will establish criteria to evaluate the impact of GenAI to the state government workforce.
• Legislative Engagement: Engage with legislative partners and key stakeholders, including academic institutions, in a formal process to develop policy recommendations for responsible use of AI, including any guidelines, criteria, reports, and/or training.
• Evaluate Impacts of AI on an Ongoing Basis: Periodically evaluate for potential impact of GenAI on regulatory issues under the respective agency, department, or board’s authority and recommend necessary updates as a result of this evolving technology.

1. Ensuring equitable access and development of AI technology

As industry races toward developing larger, more powerful AI models and seeks to commodify the seemingly infinite applications of AI, concerns are growing about the diminishing role that researchers, academic institutions, and more public-focused entities are playing in the development of AI. As reported by the Washington Post:

As such tech behemoths as Meta, Google and Microsoft funnel billions of dollars into AI, a massive resources gap is building with even the country’s richest universities. Meta aims to procure 350,000 of the specialized computer chips — called GPUs — that are essential to run the gargantuan calculations needed for AI models. In contrast, Stanford’s Natural Language Processing Group has 68 GPUs for all of its work.

To obtain the expensive computing power and data required to research AI systems, scholars frequently partner with tech employees. Meanwhile, tech firms’ eye-popping salaries are draining academia of star talent.

Big tech companies now dominate breakthroughs in the field. In 2022, the tech industry created 32 significant machine learning models, while academics produced three, a significant reversal from 2014, when the majority of AI breakthroughs originated in universities, according to a Stanford report.

Researchers say this lopsided power dynamic is shaping the field in subtle ways, pushing AI scholars to tailor their research for commercial use. Last month, Meta CEO Mark Zuckerberg announced that the company’s independent AI research lab would move closer to its product team, ensuring “some level of alignment” between the groups, he said.

“The public sector is now significantly lagging in resources and talent compared to that of industry,” said [Fei-Fei] Li, a former Google employee and the co-director of the Stanford Institute for Human-Centered AI. “This will have profound consequences because industry is focused on developing technology that is profit-driven, whereas public-sector AI goals are focused on creating public goods.”

…

As Silicon Valley races to build chatbots and image generators, it is drawing would-be computer science professors with high salaries and the chance to work on interesting AI problems. Nearly 70 percent of people with PhDs in AI end up in private industry compared with 21 percent of graduates two decades ago, according to a 2023 report.^[2]

The bill seeks to address this by directing GovOps, GO-Biz, and CDT to collaborate with other state entities and academic institutions to establish the California Artificial Intelligence Research Hub. The Hub’s stated goal is to serve as a centralized entity to facilitate collaboration between government agencies, academic institutions, and private sector partners to advance AI research and development that seeks to harness the technology’s full potential for public benefit while safeguarding privacy, advancing security, and addressing risks and potential harms to society.

In carrying out this mission, the bill directs the Hub to facilitate greater access to government data within lawful means. Critically, the bill makes clear that privacy protections and security measures are required and that access to trade secrets is excluded. The process for access must include both use limitations and provide restrictions on who can access it.

Among more lofty general goals, the bill urges the Hub to support access to, and development of public computing capacity and technology. This includes supporting researchers with increased access to computing resources, data, education, and training opportunities.

According to the author:

California is a global leader in technological advancement. Much of that leadership has been driven by our world-class higher education systems. Emerging AI technologies are costly and energy intensive, and require broad-based coordination among institutions and other sectors. Shared resources will be vital to the continued development of AI technology in California. The creation of the California Artificial Intelligence Research Hub allows us to pool and leverage the state’s financial resources and the intellectual firepower of our academic sector to democratize AI and stop it from becoming monopolized by proprietary interests alone – the tech titans.

This bill borrows from the direction being taken at the federal level. In his Executive Order on the Safe, Secure, and Trustworthy Development and Use of Artificial Intelligence, President Biden called for the launch of a pilot program implementing the National AI Research Resource (NAIRR), consistent with past recommendations of the NAIRR Task Force.^[3] The goal was to “pursue the infrastructure, governance mechanisms, and user interfaces to pilot an initial integration of distributed computational, data, model, and training resources to be made available to the research community in support of AI-related research and development.”

NAIRR is “a vision for a shared national research infrastructure for responsible discovery and innovation in AI.” The Director of the National Science Foundation is directed to identify Federal and private sector computational, data, software, and training resources appropriate for inclusion in the NAIRR pilot program, to collaborate with other federal entities, establish at least four new National AI Research Institutes, in addition to the 25 currently funded, and to enhance existing successful training programs for scientists, with the goal of training 500 new researchers by 2025 capable of meeting the rising demand for AI talent.

1. Stakeholder positions

A coalition of organizations, including the Chamber of Progress and the Computer & Communications Industry Association, write in support of the bill:

SB 893 outlines several key responsibilities that would fall under the Hub which would allow it to function as a central facilitator that fosters collaboration among government agencies, academic institutions, and private sector partners to drive forward artificial intelligence research and development. The measure also seeks to responsibly support access to and development of artificial intelligence computing capacity by finding ways to reduce costs and other economic barriers research institutions may face in accessing computing infrastructure. The co-signed organizations believe this is an effective way to promote innovation for public benefit while providing protections for consumer privacy and promoting equity. We believe that SB 893 strikes the right balance of acknowledging the potential immense benefits of artificial intelligence while also safeguarding privacy, advancing security, and addressing risks and potential harms to society.

Writing in support, the California Institute of Technology states:

The State of California, bolstered by its extensive network of research institutions dedicated to the public good, is well-positioned to lead in the development and research of Al technologies, with the aim of optimizing public welfare benefits. However, this endeavor requires resources beyond the scope of any single public entity, including comprehensive data access and advanced computing technology and infrastructure. Notably, other nations such as China and the members of the European Union already have made significant public investments in AI. The imperative to bridge the computing and data access gap for academic institutions in the United States has been highlighted by Stanford's Institute for Human-Centered Artificial Intelligence (HAI) and the leaders of 22 prominent universities.

As the world's fourth-largest economy, California possesses the market influence necessary to spearhead the development of ethical and responsible Al technologies that prioritize the public interest. This includes addressing the disparities in computing resources, facilitating data accessibility, addressing bias, and leveraging the state's esteemed network of research institutions. Building upon the Governor's executive order, which mandates a partnership between UC Berkeley's College of Computing, Data Science, and Society and Stanford University's HAI to organize a summit, Senate Bill 893 seeks to solidify further the State's partnerships with academic institutions through the establishment of the California Al Research Hub.

SUPPORT

California Institute of Technology

Chamber of Progress

Computer and Communications Industry Association

Engine

Los Angeles Area Chamber of Commerce

R Street Institute

Security Industry Association

OPPOSITION

None received

RELATED LEGISLATION

Pending Legislation:

SB 721 (Becker, 2023) establishes the California Interagency AI Working Group tasked with delivering a report to the Legislature regarding AI. Group members are to be of varied expertise and shall take input from a broad range of stakeholders to, among other things, recommend a definition of AI and determine the relevant agencies to develop and oversee AI policy and implementation of that policy. SB 721 is currently in the Assembly Privacy and Consumer Protection Committee.

SB 892 (Padilla, 2024) requires CDT to establish an AI risk management standard regarding procurement and use of automated decisionmaking systems (ADS) that is informed by leading established standards. The standard must detail specified procedures for assessing and controlling risks, prohibited use cases, and an assessment for impact on vulnerable communities. State agencies are prohibited from entering into contracts for ADS unless the contract contains specified provisions. SB 892 is currently in this Committee.

SB 896 (Dodd, 2024) largely codifies Governor Newsom’s executive order on the use of Generative artificial intelligence (GenAI). The bill requires assessments of the beneficial uses, potential harms, and risks to critical infrastructure of GenAI. The bill calls for the development of guidelines for public sector procurement, uses, and required trainings for the use of GenAI. The bill places obligations on state entities with respect to the use of GenAI and ADS. SB 896 is currently in this Committee.

SB 942 (Becker, 2024) establishes the California AI Transparency Act, which, among other things, requires a covered provider, as defined, to create an AI detection tool by which a person can query the covered provider as to the extent to which text, image, video, audio, or multimedia content was created, in whole or in part, by a generative AI system, as defined, provided by the covered provider that meets certain criteria. Covered providers are required to include in AI-generated content a visible disclosure that, among other things, includes a clear and conspicuous notice, that identifies the content as generated by AI. SB 942 requires a covered provider to register with CDT and provide them a URL to any AI detection tool it has created. SB 942 is currently in this Committee.

SCR 17 (Dodd, 2023) affirms the California Legislature’s commitment to President Biden’s vision for a safe AI and the principles outlined in the “Blueprint for an AI Bill of Rights” and expresses the Legislature’s commitment to examining and implementing those principles in its legislation and policies related to the use and deployment of automated systems. SCR 17 is currently in the Assembly Privacy and Consumer Protection Committee.

Prior Legislation:

SB 313 (Dodd, 2023) would have established the Office of Artificial Intelligence. It would have required state agencies to disclose when they are using generative AI to communicate with a person and to provide them an option to speak with a natural person at the agency. SB 313 died in the Senate Appropriations Committee.

AB 302 (Ward, Ch. 800, Stats. 2023) required CDT in coordination with other interagency bodies, to conduct a comprehensive inventory of all high-risk ADS used by state agencies on or before September 1, 2024, and report the findings to the Legislature by January 1, 2025, and annually thereafter, as specified.

AB 13 (Chau, 2021) would have established the Automated Decision Systems Accountability Act, which, in the context of the State’s procurement policies, promotes oversight over ADS that pose a high risk of adverse impacts on individual rights. The bill was eventually gutted and amended to address a different topic.

SB 444 (Umberg, 2019) would have requested the Regents of the University of California (UC) to enact a resolution authorizing the law schools at UC Berkeley and UC Irvine to participate in a pilot project to develop AI or machine-learning solutions to address access to justice issues faced by self-representing litigants in their respective courts. The bill died in the Assembly Higher Education Committee.

AB 1576 (Calderon, 2019) would have required the Secretary of Government Operations to appoint participants to an AI working group to evaluate the uses, risks, benefits, and legal implications associated with the development and deployment of AI by California-based businesses. The bill was held on the Senate Appropriations Committee suspense file.

SJR 6 (Chang, Res. Ch. 112, Stats. 2019) urged the President and the Congress of the United States to develop a comprehensive AI Advisory Committee and to adopt a comprehensive AI policy.

ACR 215 (Kiley, Resolution Ch. 206, Stats. 2018) See Comment 1.

PRIOR VOTES:

Senate Governmental Organization Committee (Ayes 14, Noes 0)

**************

1. Press Release, Governor Newsom Signs Executive Order to Prepare California for the Progress of Artificial Intelligence (September 6, 2023) Office of Governor Gavin Newsom, https://www.gov.ca.gov/2023/09/06/governor-newsom-signs-executive-order-to-prepare-california-for-the-progress-of-artificial-intelligence/. All internet citations are current as of April 10, 2024. ↑

2. Naomi Nix, Cat Zakrzewski & Gerrit De Vynck, Silicon Valley is pricing academics out of AI research (March 10, 2024) The Washington Post, https://www.washingtonpost.com/technology/2024/03/10/big-tech-companies-ai-research/. ↑

3. Executive Order on the Safe, Secure, and Trustworthy Development and Use of Artificial Intelligence (October 30, 2023) The White House, https://www.whitehouse.gov/briefing-room/presidential-actions/2023/10/30/executive-order-on-the-safe-secure-and-trustworthy-development-and-use-of-artificial-intelligence/; see also National Artificial Intelligence Research Resource Pilot, NSF, https://new.nsf.gov/focus-areas/artificial-intelligence/nairr. ↑

SENATE COMMITTEE ON GOVERNMENTAL ORGANIZATION

Senator Bill Dodd

Chair

2023 - 2024 Regular

Bill No: 893 Hearing Date: 4/9/2024

SUBJECT: California Artificial Intelligence Research Hub

DIGEST: This bill requires the Government Operations Agency (GovOps), the Governor’s Office of Business and Economic Development (GO-Biz), and the California Department of Technology (CDT) to collaborate and establish the California Artificial Intelligence Research Hub (Hub) to serve as a centralized entity to facilitate collaboration between government agencies, academic institutions, and private sector partners to advance artificial intelligence (AI) research and development, as specified.

ANALYSIS:

Existing law:

1. Requires the Secretary of GovOps to develop a coordinated plan to, among other things, investigate the feasibility of, and obstacles to, developing standards and technologies for state departments to determine digital content provenance.
2. Requires the Secretary of GovOps, for purposes of informing that coordinated plan, to evaluate, among other things, the impact of the proliferation of deepfakes, defined to mean audio or visual content that has been generated or manipulated by AI that would falsely appear to be authentic or truthful and that features depictions of people appearing to say or do things they did not say or do without their consent, on state government, California-based businesses, and residents of the state.
3. Defines “trade secrets” to mean information, including a formula, pattern, compilation, program, device, method, technique, or process that:
   1. Derives independent economic value, actual or potential, from not being generally known to the public or other persons who can obtain economic value from its disclosure or use; and
   2. Is the subject of efforts that are reasonable under the circumstances to maintain its secrecy.
4. Defines “trade secret” to mean information, including a formula, pattern, compilation, program, device, method technique, or process, as specified.

This bill:

1. Requires GovOps, GO-Biz, and CDT to collaborate to establish the California AI Research Hub in GovOps, and authorizes GovOps to collaborate with additional state agencies to establish the Hub, as needed.
2. Authorizes GovOps to collaborate with additional state agencies to establish the hub, as needed.
3. Requires the Hub to serve as a centralized entity to facilitate collaboration between government agencies, academic institutions, and private sector partners to advance AI research and development that seeks to harness the technology’s full potential for public benefit while safeguarding privacy, advancing security, and addressing risks and potential harms to society.
4. Requires GovOps, GO-Biz, and CDT to consult with academic institutions within the state in establishing the Hub.
5. Requires the Hub to do all of the following:
   1. Increase lawful access to government data while protecting privacy and safeguarding access to data by developing a streamlined process for researchers to access data collected by state agencies. Lawful access to government data increased pursuant to this bill shall not include access to trade secrets, as defined, obtained by the state. In complying with this requirement, the Hub shall create a process for eligibility that prioritizes security by limiting who can access the data and for what purpose.
   2. Support the access to, and development of, AI computing capacity and technology by building out public computing infrastructure, facilitating access to existing commercial computing infrastructure, or finding ways to reduce costs and other economic barriers research institutions may face in accessing computing infrastructure.
   3. Spur innovation in AI applications for the benefit of the public.
   4. Ensure the development of trustworthy AI technologies with a focus on transparency, fairness, and accountability.
   5. Provide researchers with increased access to data and computing resources, education, and training opportunities in furtherance of applications of AI for benefit to the people of California.
6. Provides that lawful access to government data increased pursuant to this bill shall not include access to trade secrets, as specified.

Background

Author Statement. According to the author’s office, “California is a global leader in technological advancement. Much of that leadership has been driven by our world-class higher education systems. Emerging AI technologies are costly and energy intensive, and require broad-based coordination among institutions and other sectors. Shared resources will be vital to the continued development of AI technology in California. The creation of the California Artificial Intelligence Research Hub allows us to pool and leverage the state’s financial resources and the intellectual firepower of our academic sector to democratize AI and stop it from becoming monopolized by proprietary interests alone – the tech titans.”

Navigating the AI Landscape. In the rapidly evolving landscape of technology, AI stands at the forefront, bringing a new era of innovation and application across various sectors. AI, defined broadly, is the capability of machines to perform tasks that typically require human intelligence, encompasses technologies that can process information, learn, reason, recognize patterns, and make decisions. The transformative potential of AI is vast, affecting disparate industries from healthcare to transportation, and its development is a subject of both enthusiasm and scrutiny. Given the technology's capacity to revolutionize the way we live and work, governments around the world and across the nation are seeking to nurture its growth while addressing the ethical, privacy, and security concerns it raises.

At the federal level in the United States, efforts to consolidate and amplify AI research and development have been underway. As part of the National Artificial Intelligence Initiative Act of 2020, Congress established the National Artificial Intelligence Research Resource (NAIRR) Task Force to “investigate the feasibility and advisability of developing” the NAIRR as a national AI research cyberinfrastructure, and “to propose a roadmap detailing [how the NAIRR] should be established and sustained.” The recent CHIPS and Science Act of 2022 reinforced the importance of democratizing access to a national AI research cyberinfrastructure, via investments that will accelerate development of advanced computing—from next-generation graphics processing units to high-density memory chips—as well as steps to actively engage broad and diverse U.S. talent in frontier science and engineering, including AI.

The NAIRR Task Force’s final report in January 2023 was the culmination of an 18-month effort to develop a vision and implementation plan for establishing the NAIRR. According to that report, the “NAIRR should be established with four measureable goals in mind, namely to (1) spur innovation, (2) increase diversity of talent, (3) improve capacity, and (4) advance trustworthy AI.”

In October of 2023, President Joe Biden issued the Executive Order on the Safe, Secure, and Trustworthy Development and Use of Artificial Intelligence. In that executive order, the President required the U.S. National Science Foundation (NSF) to launch a pilot program implementing the NAIRR, consistent with past recommendations of the NAIRR Task Force. The program is required to pursue the infrastructure, governance mechanisms, and user interfaces to pilot an initial integration of distributed computational, data, model, and training resources to be made available to the research community in support of AI-related research and development.

The NAIRR pilot is a vision for a shared national research infrastructure for responsible discovery and innovation in AI. Led by the NSF in partnership with 10 other federal agencies and 25 non-governmental partners, the pilot makes available government-funded, industry and other contributed resources in support of the nation’s research and education community. The NAIRR is a concept for a national infrastructure that connects U.S. researchers to computational, data, software, model and training resources they need to participate in AI research.

The NAIRR pilot will run for two years, beginning January 24, 2024. The pilot will broadly support fundamental, translational and use-inspired AI-related research with particular emphasis on societal changes. Initial priority topics include safe, secure and trustworthy AI; human health; and environment and infrastructure. A broader array of priority areas will be supported as the pilot progresses. The pilot will also support educators to train students on responsible use and development of AI technologies by providing access to infrastructure and training resources.

Blueprint for an AI Bill of Rights. In October 2022, The White House’s Office of Science and Technology Policy (OSTP) released the Blueprint for an AI Bill of Rights: Making Automated Systems Work for the American People (Blueprint) as a framework after a year of public engagement to inform the initiative. The Blueprint is a white paper published by OSTP intended to support the development of policies and practices that protect civil rights and promote democratic values in the building, deployment, and governance of automated systems. This non-binding document does not constitute U.S. government policy, does not superseded, modify, or direct an interpretation of any existing statute, regulation, policy, or international instrument.

The appropriate application of the principles set forth in the white paper depends on the context in which automated systems are being utilized. The five identified principles in the Blueprint help provide guidance whenever automated systems can meaningfully impact the public’s rights, opportunities, or access to critical needs. Specifically, the Blueprint identifies the following five principles: safe and effective systems; algorithmic discrimination protection; data privacy; notice and explanation; and human alternatives, consideration, and fallback.

SCR 17 (Dodd, Res. Chapter 135, Statutes of 2023) affirmed the California Legislature’s commitment to President Biden’s vision for a safe AI and the principles outlined in the blueprint. Additionally, that resolution expressed the Legislature’s commitment to examining and implementing those principles in its legislation and policies related to the use and deployment of automated systems.

Governor Newsom’s Executive Order N-12-23. In September 2023, Governor Newsom signed Executive Order (EO) N-12-23 to prepare California for the Progress of AI. As a global hub for generative AI (GenAI), the EO announcement notes that California is “the natural leader in this emerging field of technology – tools that could very well change the world. To capture its benefits for the good of society, but also to protect against its potential harms, Governor Newsom issued an executive order today laying out how California’s measured approach will focus on shaping the future of ethical, transparent, and trustworthy AI, while remaining the world’s AI leader.”

Specifically, the EO requires the establishment of a formal partnership with the University of California, Berkeley and Stanford University to consider and evaluate the impacts of GenAI on California and what efforts the state should undertake to advance its leadership in this industry. The state and the institutions will develop and host a joint summit in 2024 to engage in meaningful discussions about the impacts of GenAI on California and its workforce.

Additionally, the EO requires the development of guidelines for agencies and departments to analyze the impact that adopting GenAI tools may have on vulnerable communities. The state will establish the infrastructure needed to conduct pilots of GenAI projects, including CDT-approved environments or “sandboxes” to test such projects.

California AI Research Hub Act. This bill requires GovOps, GO-Biz, and CDT to collaborate to establish the Hub in GovOps. Further, the bill specifically authorizes GovOps to collaborate with additional state agencies to establish the Hub, as needed.

SB 893 bill requires the Hub to serve as a centralized entity to facilitate collaboration between government agencies, academic institutions, and private sector partners to advance AI research and development that seeks to harness the technology’s full potential for public benefit while safeguarding privacy, advancing security, and addressing risks and potential harms to society.

This bill focuses the Hub on five general objectives including data access and privacy, computing infrastructure, innovation and public benefit, trustworthy AI development, and education and training. Specifically, this bill requires the hub to do all of the following:

1. Increase lawful access to government data while protecting privacy and safeguarding access to data by developing a streamlined process for researchers to access data collected by state agencies. Lawful access to government data increased pursuant to this bill shall not include access to trade secrets, as defined, obtained by the state. In complying with this requirement, the Hub must create a process for eligibility that prioritizes security by limiting who can access the data and for what purpose.
2. Support the access to, and development of, AI computing capacity and technology by building out public computing infrastructure, facilitating access to existing commercial computing infrastructure, or finding ways to reduce costs and other economic barriers research institutions may face in accessing computing infrastructure.
3. Spur innovation in AI applications for the benefit of the public.
4. Ensure the development of trustworthy AI technologies with a focus on transparency, fairness, and accountability.
5. Provide researchers with increased access to data and computing resources, education, and training opportunities in furtherance of applications of AI for benefit to the people of California.

This bill specifies that lawful access to government data increased pursuant to the above does not include access to trade secrets, as defined, obtained by the state. For the purposes of this bill, “trade secrets” mean information, including a formula, pattern, compilation, program, device, method, technique, or process that: derives independent economic value, actual or potential, from not being generally known to the public or other persons who can obtain economic value from its disclosure or use; and is the subject of efforts that are reasonable under the circumstances to maintain its secrecy.

The author contends that “California, along with our vast network of institutions generating research can be at the forefront of the development and research of AI technologies as a way to ensure we maximize the benefits for the public good. To make this happen, resources are needed that are not available to one single public entity including access to sufficient data or access to the necessary computing technology and infrastructure. Other countries such as China, as well as the European union have already prioritized public investment heavily and the need to address the gap in access to computing and data for academic entities in the United States has been identified by Stanford’s Institute for Human-Centered Artificial Intelligence (HAI) and 22 presidents and provosts of 22 top universities in the country.”

Prior/Related Legislation

SB 892 (Padilla, 2024) requires CDT to develop and adopt regulations to create an AI risk management standard, consistent with specified publications regarding AI risk management, as specified. (Pending in the Senate Governmental Organization Committee)

SB 896 (Dodd, 2024) the AI Accountability Act requires state agencies to advance safeguards and consumer protections around AI by incorporating and building upon recent directives from President Joe Biden and Governor Gavin Newsom. (Pending in the Senate Governmental Organization Committee)

SB 942 (Becker, 2024) establishes the California AI Transparency Act, which among other things, requires a covered provider to create an AI detection tool by which a person can query the covered provider as to the extent to which text, image, video, audio, or multimedia content was created, in whole or in part, by a generative AI system, as specified. (Pending in the Senate Judiciary Committee)

SB 1047 (Wiener, 2024) the Safe and Secure Innovation for Frontier AI Systems Act, among other things, requires the developer of a covered model, as defined, to determine whether it can make a positive safety determination before initiating training of that covered model, as specified, and creates the Frontier Model Division within CDT, as specified. This bill requires CDT to commission consultants, as prescribed, to create a public cloud computing cluster, to be known as CalCompute, with the primary focus of conducting research into the safe and secure deployment of large-scale AI models and fostering equitable innovation that includes, among other things, a fully owned and hosted cloud platform. (Pending in the Senate Judiciary Committee)

SB 1235 (Gonzalez, 2024) requires an unspecified public institution of higher education to establish the AI Deepfake Working Group to evaluate and advise the Legislature and the public on the relevant issues and impacts of AI and deepfakes, as specified. (Pending in the Senate Education Committee)

SB 1288 (Becker, 2024) requires the Superintendent of Public Instruction, in consultation with the State Board of Education, to convene a working group for the purpose of evaluating AI-enabled teaching and learning practices, as specified. (Pending in the Senate Education Committee)

AB 2652 (Muratsuchi, 2024) requires the Superintendent of Public Instruction, in consultation with the State Board of Education, to convene a working group for the purpose of exploring how AI and other forms of similarly advanced technology are currently being used in education, identifying how they may be used in the future, and developing best practices to ensure that those technologies advanced, rather than harm, educational quality, as specified. (Pending in the Assembly Education Committee)

AB 3058 (Low, 2024) establishes the California Unconditional Benefit Income Pilot Program for the purpose of providing assistance to individuals who are unemployed because of automation or AI, as specified. (Pending in the Assembly Privacy and Consumer Protection Committee)

SB 313 (Dodd, 2023) would have established the Office of AI within CDT and would have required a state agency to disclose when it utilizes AI to communicate with a person and provide instructions on how to communicate with a real person from the state agency, as specified. (Held on the Senate Appropriations Committee Suspense File)

SB 721 (Becker, 2023) establishes the California Interagency AI Working Group consisting of 11 members with expertise in computer science, AI, technology, workforce development, and data privacy to report to the Legislature regarding the implications of AI, as specified. (Pending in the Assembly Privacy and Consumer Protection Committee)

SCR 17 (Dodd, Res. Chapter 135, Statutes of 2023) commits the Legislature to examining and implementing the principals outlined in the Blueprint for an Artificial Intelligence Bill of Rights, published in October 2022 by the White House Office of Science and Technology Policy.

SB 1216 (Gonzalez, Chapter 885, Statutes of 2022) requires, upon appropriation by the Legislature, the Secretary of GovOps to evaluate, among other things, the impact the proliferation of deepfakes has on government, businesses, and residents of the state, as specified.

FISCAL EFFECT: Appropriation: Fiscal Com.: Local:

SUPPORT:

California Institute of Technology

Chamber of Progress

Computer and Communications Industry Association

Engine

Los Angeles Area Chamber of Commerce

R Street Institute

Security Industry Association

Stanford University

University of California

OPPOSITION:

None received

ARGUMENTS IN SUPPORT: According to the California Institute of Technology, “[AI] offers substantial economic and societal benefits if strategically harnessed. It is crucial that AI research and development be conducted within a framework of robust ethical principles and protective measures that ensure privacy, enhance security, and mitigate potential harm. Currently, the vanguard of AI technology research and development is led by private corporations such as Google, Microsoft, and Meta, primarily due to the extensive resources required. The development of sophisticated models akin to ChatGPT necessitates computational investments in the range of hundreds of millions of dollars, alongside vast data repositories and specialized expertise, which are increasingly the purview of these private entities.”

DUAL REFERRAL: Senate Governmental Organization Committee & Senate Judiciary Committee