Existing law requires the Department of Technology, in coordination with other interagency bodies, to conduct, on or before September 1, 2024, a comprehensive inventory of all high-risk automated decision systems, as defined, that have been proposed for use, development, or procurement by, or are being used, developed, or procured by, state agencies, as defined.

This bill would require, on or before January 1, 2026, and before each time thereafter that a generative artificial intelligence system or service, as defined, or a substantial modification to a generative artificial intelligence system or service, released on or after January 1, 2022, is made available to Californians for use, regardless of whether the terms of that use include compensation, a developer of the system or service to post on the developers internet website documentation, as specified, regarding the data used to train the generative artificial intelligence system or service. The bill would require that this documentation include, among other requirements, a high-level summary of the datasets used in the development of the system or service, as specified.

Existing law requires the Department of Technology, in coordination with other interagency bodies, to conduct, on or before September 1, 2024, a comprehensive inventory of all high-risk automated decision systems, as defined, that have been proposed for use, development, or procurement by, or are being used, developed, or procured by, state agencies, as defined.

This bill would require, on or before January 1, 2026, and before each time thereafter that a generative artificial intelligence system or service, as defined, or a substantial modification to a generative artificial intelligence system or service, released on or after January 1, 2022, is made available to Californians for use, regardless of whether the terms of that use include compensation, a developer of the system or service to post on the developers internet website documentation, as specified, regarding the data used to train the generative artificial intelligence system or service. The bill would require that this documentation include, among other requirements, a high-level summary of the datasets used in the development of the system or service, as specified.

Existing law requires the Department of Technology, in coordination with other interagency bodies, to conduct, on or before September 1, 2024, a comprehensive inventory of all high-risk automated decision systems, as defined, that have been proposed for use, development, or procurement by, or are being used, developed, or procured by, state agencies, as defined.

This bill would require, on or before January 1, 2026, and before each time thereafter that an a generative artificial intelligence system or service, as defined, or a substantial modification to a generative artificial intelligence system or service, released on or after January 1, 2022, is made available to Californians for use, regardless of whether the terms of that use include compensation, a developer of the system or service to post on the developers internet website documentation, as specified, regarding the data used to train the generative artificial intelligence system or service, including, service. The bill would require that this documentation include, among other requirements, a high-level summary of the datasets used in the development of the system or service, as specified. For an artificial intelligence system or service made available for use before January 1, 2025, the bill would instead require the high-level summary to use information reasonably available to the developer and would require a developer who is unable to locate information for a high-level summary to post a description of the methods used to search for information, as specified.

Existing law requires the Department of Technology, in coordination with other interagency bodies, to conduct, on or before September 1, 2024, a comprehensive inventory of all high-risk automated decision systems, as defined, that have been proposed for use, development, or procurement by, or are being used, developed, or procured by, state agencies, as defined.

This bill would require, on or before January 1, 2026, and before each time thereafter that an artificial intelligence system or service, as defined, is made available to Californians for use, regardless of whether the terms of that use include compensation, a developer of the system or service to post on the developers internet website documentation, as specified, regarding the data used to train the artificial intelligence system or service. service, including, among other requirements, a high-level summary of the datasets used in the development of the system or service, as specified. For an artificial intelligence system or service made available for use before January 1, 2025, the bill would instead require the high-level summary to use information reasonably available to the developer and would require a developer who is unable to locate information for a high-level summary to post a description of the methods used to search for information, as specified.

Existing law requires the Department of Technology, in coordination with other interagency bodies, to conduct, on or before September 1, 2024, a comprehensive inventory of all high-risk automated decision systems, as defined, that have been proposed for use, development, or procurement by, or are being used, developed, or procured by, state agencies, as defined.

This bill would require, on or before January 1, 2026, and before each time thereafter that an artificial intelligence system or service, as defined, is made available to Californians for use, regardless of whether the terms of that use include compensation, a developer of the system or service to post on the developers internet website documentation, as specified, regarding the data used to train the artificial intelligence system or service, as defined. service.

Existing law requires the Department of Technology, in coordination with other interagency bodies, to conduct, on or before September 1, 2024, a comprehensive inventory of all high-risk automated decision systems, as defined, that have been proposed for use, development, or procurement by, or are being used, developed, or procured by, state agencies, as defined.

This bill would require, on or before January 1, 2026, a developer, as defined, of and before each time thereafter that an artificial intelligence system or service, as defined, is made available to Californians for use, regardless of whether the terms of that use include compensation, a developer of the system or service to post on the developers internet website documentation documentation, as specified, regarding the data used to train the artificial intelligence system or service, as specified. defined.

Existing law requires the Department of Technology, in coordination with other interagency bodies, to conduct, on or before September 1, 2024, a comprehensive inventory of all high-risk automated decision systems, as defined, that have been proposed for use, development, or procurement by, or are being used, developed, or procured by, state agencies, as defined.

This bill would require, on or before January 1, 2026, a developer, as defined, of an artificial intelligence system or service, as defined, made available to Californians for use, regardless of whether the terms of that use include compensation, to post on the developers internet website documentation regarding the data used to train the artificial intelligence system or service, as specified.

CONCURRENCE IN SENATE AMENDMENTS

CSA1 Bill Id:AB 2013

Author:(Irwin)

As Amended Ver:August 19, 2024

Majority vote

SUMMARY

Requires a developer of a generative artificial intelligence (GenAI) system or service to publicly disclose specific information related to the system or service’s training data, except as provided.

Senate Amendments

1. Scope of the bill is now limited to GenAI only, rather than all artificial intelligence (AI) systems and services.
2. Exempts GenAI systems developed exclusively for use by affiliates, as defined in Civ. Code Section 1799.1a, and hospital medical staff members.
3. Limits disclosure requirements to the original developers of a GenAI system, even if a third party further trains the system through fine-tuning.
4. Limits the bill’s scope to GenAI systems released on or after January 1, 2022.
5. Requires disclosure of “general ranges” of numbers of data points included in training datasets, rather than specific values.
6. Requires disclosure of “whether there was any cleaning, processing, or other modification of datasets by the developer,” rather than a description of those modifications.
7. Removes a requirement that if datasets have been merged with other datasets, the developer shall include the disclosure required by the bill for the original datasets.
8. Additionally exempts GenAI systems trained for the following purposes:
   1. GenAI systems with the sole purpose of the operation of aircraft in the national airspace.
   2. GenAI systems developed for national security, military, or defense purposes that are only made available to a federal entity.

COMMENTS

Background. The development of AI is creating exciting opportunities to grow California’s economy and improve the lives of its residents. AI uses algorithms – sets of rules – to transform inputs into outputs. Inputs and outputs can be anything a computer can process: numbers, text, audio, video, or movement.

Training is the principle innovation that allows modern AI to be both efficient and versatile. During training, a naïve AI is exposed to data and allowed to automatically explore its structure. As the AI explores, it alters itself in an attempt to better represent the data. There is a common saying in computer science: “garbage in, garbage out.” The performance of an AI product is directly impacted by the quality, quantity, and relevance of the data used to train it.

AI that are trained on small, specific datasets in order to make recommendations and predictions are sometimes referred to as “predictive AI.” This differentiates them from GenAI, which are trained on massive datasets in order to produce detailed text and images. When Netflix suggests a TV show to a viewer, the recommendation is produced by predictive AI that has been trained on the viewing habits of Netflix users. When ChatGPT generates text in clear, concise paragraphs, it uses GenAI that has been trained on the written contents of the internet.

What this bill would do. This bill would require developers of GenAI systems and services to publicly disclose specified information about the datasets used to train, test, and validate their models, except as provided.

Analysis. The present bill furthers consumer protection in California by granting the state’s residents insight into how the GenAI systems and services they engage with are trained. Overall, this bill would impose modest requirements on developers of GenAI systems and services in exchange for providing Californians with a fuller understanding of the state’s GenAI information ecosystem.

According to the Author

Artificial Intelligence has become nearly unavoidable in Californians’ daily lives, with new exciting generative AI tools being introduced daily, and the companies who make up the cornerstones of our digital lives either adopting AI or identifying their existing tools as falling under the AI umbrella. However consumer confidence in AI systems has not grown at the same rapid pace as industry adoption. Many consumers have valid questions about how these AI systems and services are created, and if they truly are better than what they seek to replace.

To build consumer confidence we need to start with the foundations, and for AI that is the selection of training data. AB 2013 provides transparency to consumers of AI systems and services by providing important documentation about the data used to train the services and systems they are being offered, including if synthetic data has or is being used to fill gaps in data sources.

Consumers may use this knowledge to better evaluate if they have confidence in the AI system or service, compare competing systems and services, or put into place mitigation measures to address any shortcomings of the particular system or service.

Arguments in Support

The Concept Art Association writes:

It is essential that consumers know where and how their data (often unknowingly) is being taken, used and sold by these AI companies. AB 2013 lays an imperative first stone on the path to protection for all Californians, and it is a solid concrete action we can take right now to begin bringing some transparency to what so far has mostly been an unscrupulous, opaque and predatory model for data acquisition.

Arguments in Opposition

Software & Information Industry Association takes an “Oppose unless Amended” position, writing:

Generally speaking, our concerns with the approach taken in AB 2013 have specifically centered around overburdensome mandates, overbroad scope (applying both retroactively and to non-high risk models), the technical feasibility of some of the disclosure requirements (including its assignment of responsibilities and the unique challenges presented for different types of developers in meeting the standards set in this bill), insufficient clarity around key terms, exposure of trade secrets or intellectual property, and potential exposure to liability.

FISCAL COMMENTS

As currently in print this bill is keyed nonfiscal.

VOTES:

ASM PRIVACY AND CONSUMER PROTECTION: 8-1-2

YES: Bauer-Kahan, Bryan, Irwin, Lowenthal, Ortega, Ward, Wicks, Wilson

NO: Dixon

ABS, ABST OR NV: Joe Patterson, Chen

ASSEMBLY FLOOR: 56-8-16

YES: Addis, Aguiar-Curry, Alvarez, Bains, Bauer-Kahan, Bennett, Berman, Boerner, Bonta, Bryan, Calderon, Juan Carrillo, Wendy Carrillo, Connolly, Mike Fong, Friedman, Garcia, Gipson, Grayson, Haney, Hart, Holden, Irwin, Jones-Sawyer, Kalra, Lee, Low, Lowenthal, Maienschein, McCarty, McKinnor, Muratsuchi, Ortega, Pacheco, Papan, Pellerin, Petrie-Norris, Quirk-Silva, Ramos, Rendon, Reyes, Blanca Rubio, Santiago, Schiavo, Soria, Ting, Valencia, Villapudua, Waldron, Ward, Weber, Wicks, Wilson, Wood, Zbur, Robert Rivas

NO: Alanis, Davies, Dixon, Vince Fong, Gallagher, Hoover, Joe Patterson, Sanchez

ABS, ABST OR NV: Arambula, Cervantes, Chen, Megan Dahle, Essayli, Flora, Gabriel, Jackson, Lackey, Mathis, Stephanie Nguyen, Jim Patterson, Luz Rivas, Rodriguez, Ta, Wallis

SENATE FLOOR: 39-0-1

YES: Allen, Alvarado-Gil, Archuleta, Ashby, Atkins, Becker, Blakespear, Bradford, Caballero, Cortese, Dahle, Dodd, Durazo, Eggman, Glazer, Gonzalez, Grove, Hurtado, Jones, Laird, Limón, McGuire, Menjivar, Newman, Nguyen, Niello, Ochoa Bogh, Padilla, Portantino, Roth, Rubio, Seyarto, Skinner, Smallwood-Cuevas, Stern, Umberg, Wahab, Wiener, Wilk

ABS, ABST OR NV: Min

UPDATED

VERSION: August 19, 2024

CONSULTANT: Slater Sharp / P. & C.P. / (916) 319-2200 FN: 0004750

CONCURRENCE IN SENATE AMENDMENTS

CSA1 Bill Id:AB 2013

Author:(Irwin)

As Amended Ver:August 19, 2024

Majority vote

SUMMARY

Requires a developer of a generative artificial intelligence (GenAI) system or service to publicly disclose specific information related to the system or service’s training data, except as provided.

Senate Amendments

1. Scope of the bill is now limited to GenAI only, rather than all artificial intelligence (AI) systems and services.
2. Exempts GenAI systems developed exclusively for use by affiliates, as defined in Civ. Code § 1799.1a, and hospital medical staff members.
3. Limits disclosure requirements to the original developers of a GenAI system, even if a third party further trains the system through fine-tuning.
4. Limits the bill’s scope to GenAI systems released on or after January 1, 2022.
5. Requires disclosure of “general ranges” of numbers of data points included in training datasets, rather than specific values.
6. Requires disclosure of “whether there was any cleaning, processing, or other modification of datasets by the developer,” rather than a description of those modifications.
7. Removes a requirement that if datasets have been merged with other datasets, the developer shall include the disclosure required by the bill for the original datasets.
8. Additionally exempts GenAI systems trained for the following purposes:
   1. GenAI systems with the sole purpose of the operation of aircraft in the national airspace.
   2. GenAI systems developed for national security, military, or defense purposes that are only made available to a federal entity.

COMMENTS

Background. The development of AI is creating exciting opportunities to grow California’s economy and improve the lives of its residents. AI uses algorithms – sets of rules – to transform inputs into outputs. Inputs and outputs can be anything a computer can process: numbers, text, audio, video, or movement.

Training is the principle innovation that allows modern AI to be both efficient and versatile. During training, a naïve AI is exposed to data and allowed to automatically explore its structure. As the AI explores, it alters itself in an attempt to better represent the data. There is a common saying in computer science: “garbage in, garbage out.” The performance of an AI product is directly impacted by the quality, quantity, and relevance of the data used to train it.

AI that are trained on small, specific datasets in order to make recommendations and predictions are sometimes referred to as “predictive AI.” This differentiates them from GenAI, which are trained on massive datasets in order to produce detailed text and images. When Netflix suggests a TV show to a viewer, the recommendation is produced by predictive AI that has been trained on the viewing habits of Netflix users. When ChatGPT generates text in clear, concise paragraphs, it uses GenAI that has been trained on the written contents of the internet.

What this bill would do. This bill would require developers of GenAI systems and services to publicly disclose specified information about the datasets used to train, test, and validate their models, except as provided.

Analysis. The present bill furthers consumer protection in California by granting the state’s residents insight into how the GenAI systems and services they engage with are trained. Overall, this bill would impose modest requirements on developers of GenAI systems and services in exchange for providing Californians with a fuller understanding of the state’s GenAI information ecosystem.

According to the Author

Artificial Intelligence has become nearly unavoidable in Californians’ daily lives, with new exciting generative AI tools being introduced daily, and the companies who make up the cornerstones of our digital lives either adopting AI or identifying their existing tools as falling under the AI umbrella. However consumer confidence in AI systems has not grown at the same rapid pace as industry adoption. Many consumers have valid questions about how these AI systems and services are created, and if they truly are better than what they seek to replace.

To build consumer confidence we need to start with the foundations, and for AI that is the selection of training data. AB 2013 provides transparency to consumers of AI systems and services by providing important documentation about the data used to train the services and systems they are being offered, including if synthetic data has or is being used to fill gaps in data sources.

Consumers may use this knowledge to better evaluate if they have confidence in the AI system or service, compare competing systems and services, or put into place mitigation measures to address any shortcomings of the particular system or service.

Arguments in Support

The Concept Art Association writes:

It is essential that consumers know where and how their data (often unknowingly) is being taken, used and sold by these AI companies. AB 2013 lays an imperative first stone on the path to protection for all Californians, and it is a solid concrete action we can take right now to begin bringing some transparency to what so far has mostly been an unscrupulous, opaque and predatory model for data acquisition.

Arguments in Opposition

Software & Information Industry Association takes an “Oppose unless Amended” position, writing:

Generally speaking, our concerns with the approach taken in AB 2013 have specifically centered around overburdensome mandates, overbroad scope (applying both retroactively and to non-high risk models), the technical feasibility of some of the disclosure requirements (including its assignment of responsibilities and the unique challenges presented for different types of developers in meeting the standards set in this bill), insufficient clarity around key terms, exposure of trade secrets or intellectual property, and potential exposure to liability.

FISCAL COMMENTS

As currently in print this bill is keyed nonfiscal.

VOTES:

ASM PRIVACY AND CONSUMER PROTECTION: 8-1-2

YES: Bauer-Kahan, Bryan, Irwin, Lowenthal, Ortega, Ward, Wicks, Wilson

NO: Dixon

ABS, ABST OR NV: Joe Patterson, Chen

ASSEMBLY FLOOR: 56-8-16

YES: Addis, Aguiar-Curry, Alvarez, Bains, Bauer-Kahan, Bennett, Berman, Boerner, Bonta, Bryan, Calderon, Juan Carrillo, Wendy Carrillo, Connolly, Mike Fong, Friedman, Garcia, Gipson, Grayson, Haney, Hart, Holden, Irwin, Jones-Sawyer, Kalra, Lee, Low, Lowenthal, Maienschein, McCarty, McKinnor, Muratsuchi, Ortega, Pacheco, Papan, Pellerin, Petrie-Norris, Quirk-Silva, Ramos, Rendon, Reyes, Blanca Rubio, Santiago, Schiavo, Soria, Ting, Valencia, Villapudua, Waldron, Ward, Weber, Wicks, Wilson, Wood, Zbur, Robert Rivas

NO: Alanis, Davies, Dixon, Vince Fong, Gallagher, Hoover, Joe Patterson, Sanchez

ABS, ABST OR NV: Arambula, Cervantes, Chen, Megan Dahle, Essayli, Flora, Gabriel, Jackson, Lackey, Mathis, Stephanie Nguyen, Jim Patterson, Luz Rivas, Rodriguez, Ta, Wallis

UPDATED

VERSION: August 19, 2024

CONSULTANT: Slater Sharp / P. & C.P. / (916) 319-2200 FN: 0004750

 THIRD READING  

Bill No: AB 2013

Author: Irwin (D)

Amended: 8/19/24 in Senate

Vote: 21

SENATE JUDICIARY COMMITTEE: 10-1, 6/25/24

AYES: Umberg, Wilk, Allen, Ashby, Caballero, Durazo, Laird, Roth, Stern, Wahab

NOES: Niello

ASSEMBLY FLOOR: 56-8, 5/20/24 - See last page for vote

SUBJECT: Generative artificial intelligence: training data transparency

SOURCE: Author

DIGEST:

This bill requires developers of generative artificial intelligence (GenAI) systems or services that are made available for Californians to use to post on their website documentation regarding the data used to train the system or service, including high-level summaries of the datasets used.

Senate Floor Amendments of 8/19/24 restructure definitions and relax the documentation requirements on developers.

ANALYSIS:

Existing law:

1. Establishes the California Consumer Privacy Act (CCPA), which grants consumers certain rights with regard to their personal information, including enhanced notice, access, and disclosure; the right to deletion; the right to restrict the sale of information; and protection from discrimination for exercising these rights. It places attendant obligations on businesses to respect those rights. (Civil (Civ.) Code § 1798.100 et seq.)
2. Defines “personal information” as information that identifies, relates to, describes, is reasonably capable of being associated with, or could reasonably be linked, directly or indirectly, with a particular consumer or household. The CCPA provides a nonexclusive series of categories of information deemed to be personal information, including identifiers, biometric information, and geolocation data. (Civ. Code § 1798.140(v).) The CCPA defines and provides additional protections for sensitive personal information, as defined, that reveals specified personal information about consumers. (Civ. Code § 1798.140(ae).)
3. Defines “aggregate consumer information” to mean information that relates to a group or category of consumers, from which individual consumer identities have been removed, that is not linked or reasonably linkable to any consumer or household, including via a device. “Aggregate consumer information” does not mean one or more individual consumer records that have been deidentified. (Civ. Code § 1798.140(b).)
4. Establishes the California Privacy Rights Act (CPRA), which amends the CCPA and creates the Privacy Protection Agency (PPA), which is charged with implementing these privacy laws, promulgating regulations, and carrying out enforcement actions. (Civ. Code § 798.100 et seq.; Proposition 24 (2020).)

This bill:

1. Requires a developer, on or before January 1, 2026, and before each time thereafter that a GenAI system or service, or a substantial modification to such a system or service, as provided, is made publicly available to Californians for use, regardless of whether the terms of that use include compensation, to post on the developer’s website documentation regarding the data used by the developer to train the GenAI system or service, including a high-level summary of the datasets used in the development of the GenAI system or service, including specified information.
2. Clarifies that a developer shall not be required to post documentation regarding the data used to train a GenAI system or service for any of the following:
   1. A GenAI system or service whose sole purpose is to help ensure security and integrity as defined in subdivision (ac) of Section 1798.140.
   2. A GenAI system or service whose sole purpose is the operation of an aircraft in the national airspace.
   3. A GenAI system or service developed for national security, military, or defense purposes that is made available only to a federal entity.

Background

Owing to recent advances in processing power and the rise of big data, AI’s capacity and the scope of its applications have expanded rapidly, impacting how we communicate, interact, entertain ourselves, travel, transact business, and consume media. It has been used to accelerate productivity, achieve efficiencies, liberate us from drudgery, write our college essay, connect with each other, and live longer, fuller lives. It has also been used to constrain personal autonomy, compromise privacy and security, foment social upheaval, exacerbate inequality, spread misinformation, and subvert democracy. For good or ill, its transformative potential seems boundless.

Ultimately, AI systems are only as good as the data used to train them. However, there is very little transparency in what data is used to train these systems and that lack of transparency hamstrings efforts to address and adequately identify many of the issues being raised by AI’s rapid development. This bill seeks to establish basic transparency requirements for developers of GenAI systems or services that are made available in California. Developers are required to post documentation regarding the data used to train the system or service, including high-level summaries of the datasets used in developing the system or service.

This bill is author-sponsored. It is supported by a variety of organizations, including the California Labor Federation and Transparency Coalition.AI. It is opposed by various industry and business associations, including the California Chamber of Commerce.

Comments

Requiring transparency about the training data used for AI systems helps identify and mitigate biases, addressing hallucinations and other problematic outputs, and shines the light on various other issues, such as privacy and copyright concerns. A team of experts from both industry and academia at the Shorenstein Center created a documentation framework for AI which highlights the importance:

While often categorized as technical, AI systems and their underlying data and models are sociotechnical. In other words, they combine the technical infrastructure and design with the social context in which they are designed, developed, evaluated, and deployed. Accountability for these systems and their impacts requires transparency around their design and creation and how they are intended to be used. In recent years, alongside the exponential increase in data collection and the efforts to develop increasingly powerful machine learning models, there have been notable efforts calling attention to the need for documentation to accompany datasets, models, and AI systems, and to account for the process of creating them.

Documentation is worthwhile for various stakeholders. It improves the understanding of practitioners creating or building datasets, models, or AI systems, which opens up opportunities to reflect on implicit and explicit decisions, ultimately enhancing the reliability of the systems they create. For organizations, it enables knowledge transfer across silos and encourages responsible use. Further, it provides information to users and potentially affected communities that can be used to determine the appropriateness of an AI system or its underlying data or models, thus helping inform consumer choice, advocacy work, regulation development, and regulation enforcement. It also enables recourse in the event of harms caused by or inquiries into the AI system, and accountability regarding who might be held responsible for those harms.^[1]

A recent article in the Harvard Data Science Review highlights the importance of transparency and barriers to achieving it without regulation:

Knowing what is in the data sets used to train models and how they have been compiled is vitally important. Without this information, the work of developers, researchers, and ethicists to address biases or remove harmful content from the data is hampered. Information about training data is also vital to lawmakers’ attempts to assess whether foundation models have ingested personal data or copyrighted material. Further downstream, the intended operators of AI systems and those impacted by their use are far more likely to trust them if they understand how they have been developed.

However, in undertaking their analysis, Schaul et al. (2023) concluded that “many companies do not document the contents of their training data—even internally—for fear of finding personal information about identifiable individuals, copyrighted material and other data grabbed without consent.”^[2]

Various sectors are calling on lawmakers to provide some measure of transparency in this space. A group of media organizations and outlets, including the Associated Press and Gannett, recently issued an open letter calling on regulators to require transparency as to the makeup of all training sets used to create AI models.^[3] At the federal level, Representative Anna Eshoo and Representative Don Beyer introduced the AI Foundation Model Transparency Act, which would direct the Federal Trade Commission — in consultation with the National Institute of Standards and Technology and the White House Office of Science and Technology Policy — to “establish standards for making publicly available information about the training data and algorithms used in artificial intelligence foundation models.”^[4]

Requiring baseline transparency for AI systems in California. This bill begins to address the issue of training data transparency by requiring developers of GenAI systems and services made available to Californians for use to post documentation of the data used to train the system or service. This includes a high-level summary of the datasets used, including:

• The sources or owners of the datasets.
• A description of the types of data points within the datasets, as provided.
• Whether the datasets include any data protected by copyright, trademark, or patent, requiring the purchase or licensure of the data, or whether the datasets are entirely in the public domain.
• Whether the datasets were purchased or licensed by the developer.
• Whether the datasets include personal information.

Developers must also disclose whether the system or service used or uses synthetic data generation in its development.

This bill makes clear that training includes testing, validating, and fine tuning by the developer of the GenAI system or service.

According to the author:

Artificial Intelligence has become nearly unavoidable in Californians’ daily lives, with new exciting generative AI tools being introduced daily, and the companies who make up the cornerstones of our digital lives either adopting AI or identifying their existing tools as falling under the AI umbrella. However consumer confidence in AI systems has not grown at the same rapid pace as industry adoption. Many consumers have valid questions about how these AI systems and services are created, and if they truly are better than what they seek to replace.

To build consumer confidence we need to start with the foundations, and for AI that is the selection of training data. AB 2013 provides transparency to consumers of AI systems and services by providing important documentation about the data used to train the services and systems they are being offered, including if synthetic data has or is being used to fill gaps in data sources.

Consumers may use this knowledge to better evaluate if they have confidence in the AI system or service, compare competing systems and services, or put into place mitigation measures to address any shortcomings of the particular system or service.

FISCAL EFFECT: Appropriation: No Fiscal Com.: No Local: No

SUPPORT: (Verified 8/20/24)

California Democratic Party

California Labor Federation, AFL-CIO

Center for AI and Digital Policy

Concept Art Association

Los Angeles County Democratic Party

Oakland Privacy

Perk Advocacy

Santa Monica Democratic Club

Secure Justice

Transparency Coalition.AI

OPPOSITION: (Verified 8/20/24)

Chamber of Progress

Software & Information Industry Association

TechNet

ARGUMENTS IN SUPPORT: The California Labor Federation argues:

AB 2013 increases public transparency by requiring the developer of an AI system that makes predictions, recommendations, or decisions to publish a description of the datasets used to train the system. The set of required disclosures includes the source of the dataset, who owns it, definitional categories, when the data was collected and when it has been used, whether the data set was purchased, licensed, or found in the public domain, and whether the collected data is being used to synthesize new data sets. AB 2013 provides the public with the information to address AI systems utilizing nonconsensual personal information and training data riddled with implicit and explicit biases.

ARGUMENTS IN OPPOSITION: Chamber of Progress writes:

Requiring online platforms to disclose data used to train their artificial intelligence (AI) systems and services on their website stifles competition in the digital marketplace. A healthy, competitive marketplace is essential to promote quality services for consumers and encourages platforms to innovate. The disclosure requirement risks revealing important business information and strategies, even when platforms specifically note that the datasets are protected intellectual property. Additionally, the language “but not be limited to” in such requirements makes the expectations placed on online platforms unclear.

ASSEMBLY FLOOR: 56-8, 5/20/24

AYES: Addis, Aguiar-Curry, Alvarez, Bains, Bauer-Kahan, Bennett, Berman, Boerner, Bonta, Bryan, Calderon, Juan Carrillo, Wendy Carrillo, Connolly, Mike Fong, Friedman, Garcia, Gipson, Grayson, Haney, Hart, Holden, Irwin, Jones-Sawyer, Kalra, Lee, Low, Lowenthal, Maienschein, McCarty, McKinnor, Muratsuchi, Ortega, Pacheco, Papan, Pellerin, Petrie-Norris, Quirk-Silva, Ramos, Rendon, Reyes, Blanca Rubio, Santiago, Schiavo, Soria, Ting, Valencia, Villapudua, Waldron, Ward, Weber, Wicks, Wilson, Wood, Zbur, Robert Rivas

NOES: Alanis, Davies, Dixon, Vince Fong, Gallagher, Hoover, Joe Patterson, Sanchez

NO VOTE RECORDED: Arambula, Cervantes, Chen, Megan Dahle, Essayli, Flora, Gabriel, Jackson, Lackey, Mathis, Stephanie Nguyen, Jim Patterson, Luz Rivas, Rodriguez, Ta, Wallis

Prepared by: Christian Kurpiewski / JUD. / (916) 651-4113

8/20/24 17:27:54

**** END ****

1. Kasia Chmielinski, et al., The CLeAR Documentation Framework for AI Transparency: Recommendations for Practitioners & Context for Policymakers (May 21, 2024) Shorenstein Center on Media, Politics, and Public Policy, https://shorensteincenter.org/wp-content/uploads/2024/05/CleAR_KChmielinski_FINAL.pdf. ↑

2. Jack Hardinges, et al., We Must Fix the Lack of Transparency Around the Data Used to Train Foundation Models (December 13, 2023) Harvard Data Science Review, https://hdsr.mitpress.mit.edu/pub/xau9dza3/release/2. ↑

3. Bailey Schulz, Will AI deepen distrust in news? Gannett, other media organizations want more regulations (August 9, 2023) USA Today, https://www.usatoday.com/story/tech/news/2023/08/09/ai-regulations-media-gannett/70551555007/. ↑

4. Edward Graham, Bill sets transparency standards for AI models, including use of copyrighted material (January 2, 2024) Nextgov/FCW, https://www.nextgov.com/artificial-intelligence/2024/01/bill-sets-transparency-standards-ai-models-including-use-copyrighted-material/393052/. ↑

 THIRD READING  

Bill No: AB 2013

Author: Irwin (D)

Amended: 6/17/24 in Senate

Vote: 21

SENATE JUDICIARY COMMITTEE: 10-1, 6/25/24

AYES: Umberg, Wilk, Allen, Ashby, Caballero, Durazo, Laird, Roth, Stern, Wahab

NOES: Niello

ASSEMBLY FLOOR: 56-8, 5/20/24 - See last page for vote

SUBJECT: Artificial intelligence: training data transparency

SOURCE: Author

DIGEST:

This bill requires developers of artificial intelligence (AI) systems or services that are made available for Californians to use to post on their website documentation regarding the data used to train the system or service, including high-level summaries of the datasets used.

ANALYSIS:

Existing law:

1. Establishes the California Consumer Privacy Act (CCPA), which grants consumers certain rights with regard to their personal information, including enhanced notice, access, and disclosure; the right to deletion; the right to restrict the sale of information; and protection from discrimination for exercising these rights. It places attendant obligations on businesses to respect those rights. (Civil (Civ.) Code § 1798.100 et seq.)
2. Defines “personal information” as information that identifies, relates to, describes, is reasonably capable of being associated with, or could reasonably be linked, directly or indirectly, with a particular consumer or household. The CCPA provides a nonexclusive series of categories of information deemed to be personal information, including identifiers, biometric information, and geolocation data. (Civ. Code § 1798.140(v).) The CCPA defines and provides additional protections for sensitive personal information, as defined, that reveals specified personal information about consumers. (Civ. Code § 1798.140(ae).)
3. Defines “aggregate consumer information” to mean information that relates to a group or category of consumers, from which individual consumer identities have been removed, that is not linked or reasonably linkable to any consumer or household, including via a device. “Aggregate consumer information” does not mean one or more individual consumer records that have been deidentified. (Civ. Code § 1798.140(b).)
4. Establishes the California Privacy Rights Act (CPRA), which amends the CCPA and creates the Privacy Protection Agency (PPA), which is charged with implementing these privacy laws, promulgating regulations, and carrying out enforcement actions. (Civ. Code § 798.100 et seq.; Proposition 24 (2020).)

This bill:

1. Requires the developer of an AI system or service, on or before January 1, 2026, and before each time thereafter that the system or service is made publicly available to Californians for use, regardless of whether the terms of that use include compensation, to post on the developer’s website documentation regarding the data used by the developer to train the AI system or service, including a high-level summary of the datasets used in the development of the AI system or service, including specified information.
2. Clarifies that a developer shall not be required to post documentation regarding the data used to train an AI system or service for any of the following:
   1. An AI system or service whose sole purpose is to help ensure security and integrity as defined in subdivision (ac) of Section 1798.140.
   2. An AI system or service whose sole purpose is the operation of an aircraft in the national airspace.
   3. An AI system or service developed for national security, military, or defense purposes that is made available only to a federal entity.
3. Provides that for an AI system or service made available before January 1, 2025, the high-level summary must use information reasonable available to the developer, as provided.
4. Defines the following terms:
   1. “Artificial intelligence” means an engineered or machine-based system that varies in its level of autonomy and that can, for explicit or implicit objectives, infer from the input it receives how to generate outputs that can influence physical or virtual environments.
   2. “Developer” means a person, partnership, state or local government agency, or corporation that designs, codes, or produces an artificial intelligence system or service, or substantially modifies an artificial intelligence system or service for use by a third party for free or for a fee.
   3. “Synthetic data generation” means a process in which seed data are used to create artificial data that have some of the statistical characteristics of the seed data.
   4. “Train an artificial intelligence system or service” includes testing, validating, or fine tuning by the developer of the AI system or service.

Background

Owing to recent advances in processing power and the rise of big data, AI’s capacity and the scope of its applications have expanded rapidly, impacting how we communicate, interact, entertain ourselves, travel, transact business, and consume media. It has been used to accelerate productivity, achieve efficiencies, liberate us from drudgery, write our college essay, connect with each other, and live longer, fuller lives. It has also been used to constrain personal autonomy, compromise privacy and security, foment social upheaval, exacerbate inequality, spread misinformation, and subvert democracy. For good or ill, its transformative potential seems boundless.

Ultimately, AI systems are only as good as the data used to train them. However, there is very little transparency in what data is used to train these systems and that lack of transparency hamstrings efforts to address and adequately identify many of the issues being raised by AI’s rapid development.

This bill seeks to establish basic transparency requirements for developers of AI systems or services that are made available in California. Developers are required to post documentation regarding the data used to train the AI system or service, including high-level summaries of the datasets used in developing the system or service.

This bill is author-sponsored. It is supported by a variety of organizations, including the California Labor Federation and Transparency Coalition.AI. It is opposed by various industry and business associations, including the California Chamber of Commerce.

Comment

Requiring transparency about the training data used for AI systems helps identify and mitigate biases, addressing hallucinations and other problematic outputs, and shines the light on various other issues, such as privacy and copyright concerns. A team of experts from both industry and academia at the Shorenstein Center created a documentation framework for AI which highlights the importance:

While often categorized as technical, AI systems and their underlying data and models are sociotechnical. In other words, they combine the technical infrastructure and design with the social context in which they are designed, developed, evaluated, and deployed. Accountability for these systems and their impacts requires transparency around their design and creation and how they are intended to be used. In recent years, alongside the exponential increase in data collection and the efforts to develop increasingly powerful machine learning models, there have been notable efforts calling attention to the need for documentation to accompany datasets, models, and AI systems, and to account for the process of creating them.

Documentation is worthwhile for various stakeholders. It improves the understanding of practitioners creating or building datasets, models, or AI systems, which opens up opportunities to reflect on implicit and explicit decisions, ultimately enhancing the reliability of the systems they create. For organizations, it enables knowledge transfer across silos and encourages responsible use. Further, it provides information to users and potentially affected communities that can be used to determine the appropriateness of an AI system or its underlying data or models, thus helping inform consumer choice, advocacy work, regulation development, and regulation enforcement. It also enables recourse in the event of harms caused by or inquiries into the AI system, and accountability regarding who might be held responsible for those harms.^[1]

A recent article in the Harvard Data Science Review highlights the importance of transparency and barriers to achieving it without regulation:

Knowing what is in the data sets used to train models and how they have been compiled is vitally important. Without this information, the work of developers, researchers, and ethicists to address biases or remove harmful content from the data is hampered. Information about training data is also vital to lawmakers’ attempts to assess whether foundation models have ingested personal data or copyrighted material. Further downstream, the intended operators of AI systems and those impacted by their use are far more likely to trust them if they understand how they have been developed.

However, in undertaking their analysis, Schaul et al. (2023) concluded that “many companies do not document the contents of their training data—even internally—for fear of finding personal information about identifiable individuals, copyrighted material and other data grabbed without consent.”^[2]

Various sectors are calling on lawmakers to provide some measure of transparency in this space. A group of media organizations and outlets, including the Associated Press and Gannett, recently issued an open letter calling on regulators to require transparency as to the makeup of all training sets used to create AI models.^[3] At the federal level, Representative Anna Eshoo and Representative Don Beyer introduced the AI Foundation Model Transparency Act, which would direct the Federal Trade Commission — in consultation with the National Institute of Standards and Technology and the White House Office of Science and Technology Policy — to “establish standards for making publicly available information about the training data and algorithms used in artificial intelligence foundation models.”^[4]

Requiring baseline transparency for AI systems in California. This bill begins to address the issue of training data transparency by requiring developers of AI systems and services made available to Californians for use to post documentation of the data used to train the system or service. This includes a high-level summary of the datasets used, including:

• The sources or owners of the datasets.
• A clear definition of each category associated to data points within the datasets, including the format of data points and sample values.
• Whether the datasets include any data protected by copyright, trademark, or patent, requiring the purchase or licensure of the data, or whether the datasets are entirely in the public domain.
• Whether the datasets were purchased or licensed by the developer.
• Whether the datasets include personal information.

Developers must also disclose whether the system or service used or uses synthetic data generation in its development.

This bill makes clear that training includes testing, validating, and fine tuning by the developer of the AI system or service.

According to the author:

Artificial Intelligence has become nearly unavoidable in Californians’ daily lives, with new exciting generative AI tools being introduced daily, and the companies who make up the cornerstones of our digital lives either adopting AI or identifying their existing tools as falling under the AI umbrella. However consumer confidence in AI systems has not grown at the same rapid pace as industry adoption. Many consumers have valid questions about how these AI systems and services are created, and if they truly are better than what they seek to replace.

To build consumer confidence we need to start with the foundations, and for AI that is the selection of training data. AB 2013 provides transparency to consumers of AI systems and services by providing important documentation about the data used to train the services and systems they are being offered, including if synthetic data has or is being used to fill gaps in data sources.

Consumers may use this knowledge to better evaluate if they have confidence in the AI system or service, compare competing systems and services, or put into place mitigation measures to address any shortcomings of the particular system or service.

FISCAL EFFECT: Appropriation: No Fiscal Com.: No Local: No

SUPPORT: (Verified 6/28/24)

California Democratic Party

California Labor Federation, AFL-CIO

Concept Art Association

Los Angeles County Democratic Party

Oakland Privacy

Perk Advocacy

Santa Monica Democratic Club

Secure Justice

Transparency Coalition.AI

OPPOSITION: (Verified 6/28/24)

American Property Casualty Insurance Association

California Bankers Association

California Chamber of Commerce

California Land Title Association

Chamber of Progress

Computer & Communications Industry Association

Insights Association

National Association of Mutual Insurance Companies

Personal Insurance Federation of California

Software & Information Industry Association

TechNet

ARGUMENTS IN SUPPORT: The California Labor Federation argues:

AB 2013 increases public transparency by requiring the developer of an AI system that makes predictions, recommendations, or decisions to publish a description of the datasets used to train the system. The set of required disclosures includes the source of the dataset, who owns it, definitional categories, when the data was collected and when it has been used, whether the data set was purchased, licensed, or found in the public domain, and whether the collected data is being used to synthesize new data sets. AB 2013 provides the public with the information to address AI systems utilizing nonconsensual personal information and training data riddled with implicit and explicit biases.

ARGUMENTS IN OPPOSITION: Chamber of Progress writes:

Requiring online platforms to disclose data used to train their artificial intelligence (AI) systems and services on their website stifles competition in the digital marketplace. A healthy, competitive marketplace is essential to promote quality services for consumers and encourages platforms to innovate. The disclosure requirement risks revealing important business information and strategies, even when platforms specifically note that the datasets are protected intellectual property. Additionally, the language “but not be limited to” in such requirements makes the expectations placed on online platforms unclear.

ASSEMBLY FLOOR: 56-8, 5/20/24

AYES: Addis, Aguiar-Curry, Alvarez, Bains, Bauer-Kahan, Bennett, Berman, Boerner, Bonta, Bryan, Calderon, Juan Carrillo, Wendy Carrillo, Connolly, Mike Fong, Friedman, Garcia, Gipson, Grayson, Haney, Hart, Holden, Irwin, Jones-Sawyer, Kalra, Lee, Low, Lowenthal, Maienschein, McCarty, McKinnor, Muratsuchi, Ortega, Pacheco, Papan, Pellerin, Petrie-Norris, Quirk-Silva, Ramos, Rendon, Reyes, Blanca Rubio, Santiago, Schiavo, Soria, Ting, Valencia, Villapudua, Waldron, Ward, Weber, Wicks, Wilson, Wood, Zbur, Robert Rivas

NOES: Alanis, Davies, Dixon, Vince Fong, Gallagher, Hoover, Joe Patterson, Sanchez

NO VOTE RECORDED: Arambula, Cervantes, Chen, Megan Dahle, Essayli, Flora, Gabriel, Jackson, Lackey, Mathis, Stephanie Nguyen, Jim Patterson, Luz Rivas, Rodriguez, Ta, Wallis

Prepared by: Christian Kurpiewski / JUD. / (916) 651-4113

6/28/24 15:19:06

**** END ****

1. Kasia Chmielinski, et al., The CLeAR Documentation Framework for AI Transparency: Recommendations for Practitioners & Context for Policymakers (May 21, 2024) Shorenstein Center on Media, Politics, and Public Policy, https://shorensteincenter.org/wp-content/uploads/2024/05/CleAR_KChmielinski_FINAL.pdf. ↑

2. Jack Hardinges, et al., We Must Fix the Lack of Transparency Around the Data Used to Train Foundation Models (December 13, 2023) Harvard Data Science Review, https://hdsr.mitpress.mit.edu/pub/xau9dza3/release/2. ↑

3. Bailey Schulz, Will AI deepen distrust in news? Gannett, other media organizations want more regulations (August 9, 2023) USA Today, https://www.usatoday.com/story/tech/news/2023/08/09/ai-regulations-media-gannett/70551555007/. ↑

4. Edward Graham, Bill sets transparency standards for AI models, including use of copyrighted material (January 2, 2024) Nextgov/FCW, https://www.nextgov.com/artificial-intelligence/2024/01/bill-sets-transparency-standards-ai-models-including-use-copyrighted-material/393052/. ↑

SENATE JUDICIARY COMMITTEE

Senator Thomas Umberg, Chair

2023-2024 Regular Session

Bill No: AB 2013 (Author:Irwin)

Version: June 17, 2024

Hearing Date: June 25, 2024

Fiscal: No

Urgency: No

Consultant: CK

SUBJECT

Artificial intelligence: training data transparency

DIGEST

This bill requires developers of artificial intelligence (AI) systems or services that are made available for Californians to use to post on their website documentation regarding the data used to train the system or service, including high-level summaries of the datasets used.

EXECUTIVE SUMMARY

Owing to recent advances in processing power and the rise of big data, AI’s capacity and the scope of its applications have expanded rapidly, impacting how we communicate, interact, entertain ourselves, travel, transact business, and consume media. It has been used to accelerate productivity, achieve efficiencies, liberate us from drudgery, write our college essay, connect with each other, and live longer, fuller lives. It has also been used to constrain personal autonomy, compromise privacy and security, foment social upheaval, exacerbate inequality, spread misinformation, and subvert democracy. For good or ill, its transformative potential seems boundless.

Ultimately, AI systems are only as good as the data used to train them. However, there is very little transparency in what data is used to train these systems and that lack of transparency hamstrings efforts to address and adequately identify many of the issues being raised by AI’s rapid development.

This bill seeks to establish basic transparency requirements for developers of AI systems or services that are made available in California. Developers are required to post documentation regarding the data used to train the AI system or service, including high-level summaries of the datasets used in developing the system or service.

The bill is author-sponsored. It is supported by a variety of organizations, including the California Labor Federation and Transparency Coalition.AI. It is opposed by various industry and business associations, including the California Chamber of Commerce.

PROPOSED CHANGES TO THE LAW

Existing law:

1. Establishes the California Consumer Privacy Act (CCPA), which grants consumers certain rights with regard to their personal information, including enhanced notice, access, and disclosure; the right to deletion; the right to restrict the sale of information; and protection from discrimination for exercising these rights. It places attendant obligations on businesses to respect those rights. (Civ. Code § 1798.100 et seq.)
2. Defines “personal information” as information that identifies, relates to, describes, is reasonably capable of being associated with, or could reasonably be linked, directly or indirectly, with a particular consumer or household. The CCPA provides a nonexclusive series of categories of information deemed to be personal information, including identifiers, biometric information, and geolocation data. (Civ. Code § 1798.140(v).) The CCPA defines and provides additional protections for sensitive personal information, as defined, that reveals specified personal information about consumers. (Civ. Code § 1798.140(ae).)
3. Defines “aggregate consumer information” to mean information that relates to a group or category of consumers, from which individual consumer identities have been removed, that is not linked or reasonably linkable to any consumer or household, including via a device. “Aggregate consumer information” does not mean one or more individual consumer records that have been deidentified. (Civ. Code § 1798.140(b).)
4. Defines “security and integrity” as the ability of:
   1. Networks or information systems to detect security incidents that compromise the availability, authenticity, integrity, and confidentiality of stored or transmitted personal information.
   2. Businesses to detect security incidents, resist malicious, deceptive, fraudulent, or illegal actions and to help prosecute those responsible for those actions.
   3. Businesses to ensure the physical safety of natural persons. (Civ. Code § 1798.140(ac).
5. Establishes the California Privacy Rights Act (CPRA), which amends the CCPA and creates the Privacy Protection Agency (PPA), which is charged with implementing these privacy laws, promulgating regulations, and carrying out enforcement actions. (Civ. Code § 798.100 et seq.; Proposition 24 (2020).)
6. Permits amendment of the CPRA by a majority vote of each house of the Legislature and the signature of the Governor, provided such amendments are consistent with and further the purpose and intent of this act as set forth therein. (Proposition 24 § 25 (2020).)

This bill:

1. Requires the developer of an AI system or service, on or before January 1, 2026, and before each time thereafter that the system or service is made publicly available to Californians for use, regardless of whether the terms of that use include compensation, to post on the developer’s website documentation regarding the data used by the developer to train the AI system or service, including, but not be limited to, all of the following:
   1. A high-level summary of the datasets used in the development of the AI system or service, including, but not limited to:
      1. The sources or owners of the datasets.
      2. A description of how the datasets further the intended purpose of the system or service.
      3. The number of data points included in the datasets, which may be in general ranges, and with estimated figures for dynamic datasets.
      4. A clear definition of each category associated to data points within the datasets, including the format of data points and sample values.
      5. Whether the datasets include any data protected by copyright, trademark, or patent, or whether the datasets are entirely in the public domain.
      6. Whether the datasets were purchased or licensed by the developer.
      7. Whether the datasets include personal information, as defined in subdivision (v) of Section 1798.140.
      8. Whether the datasets include aggregate consumer information, as defined in subdivision (b) Section 1798.140.
      9. A description of any cleaning, processing, or other modification to the datasets by the developer, including the intended purpose of those efforts in relation to the system or service.
      10. The time period during which the data in the datasets were collected, including a notice if the data collection is ongoing.
      11. The dates the datasets were first and last used during the development of the system or service.
      12. Whether the system or service used or continuously uses synthetic data generation in its development. A developer may include a description of the functional need or desired purpose of the synthetic data in relation to the intended purpose of the system or service.
2. Clarifies that a developer shall not be required to post documentation regarding the data used to train an AI system or service for any of the following:
   1. An AI system or service whose sole purpose is to help ensure security and integrity as defined in subdivision (ac) of Section 1798.140.
   2. An AI system or service whose sole purpose is the operation of an aircraft in the national airspace.
   3. An AI system or service developed for national security, military, or defense purposes that is made available only to a federal entity.
3. Provides that for an AI system or service made available before January 1, 2025, the high-level summary must use information reasonable available to the developer, as provided.
4. Defines the following terms:
   1. “Artificial intelligence” means an engineered or machine-based system that varies in its level of autonomy and that can, for explicit or implicit objectives, infer from the input it receives how to generate outputs that can influence physical or virtual environments.
   2. “Developer” means a person, partnership, state or local government agency, or corporation that designs, codes, or produces an artificial intelligence system or service, or substantially modifies an artificial intelligence system or service for use by a third party for free or for a fee.
   3. “Synthetic data generation” means a process in which seed data are used to create artificial data that have some of the statistical characteristics of the seed data.
   4. “Train an artificial intelligence system or service” includes testing, validating, or fine tuning by the developer of the AI system or service.

COMMENTS

1. Training data and transparency

As stated, training data is the veritable secret sauce for AI systems. With the race to build bigger and better AI systems and models, a battle over data is also being waged:

The race to lead A.I. has become a desperate hunt for the digital data needed to advance the technology. To obtain that data, tech companies including OpenAI, Google and Meta have cut corners, ignored corporate policies and debated bending the law, according to an examination by The New York Times.

At Meta, which owns Facebook and Instagram, managers, lawyers and engineers last year discussed buying the publishing house Simon & Schuster to procure long works, according to recordings of internal meetings obtained by The Times. They also conferred on gathering copyrighted data from across the internet, even if that meant facing lawsuits. Negotiating licenses with publishers, artists, musicians and the news industry would take too long, they said.

Like OpenAI, Google transcribed YouTube videos to harvest text for its A.I. models, five people with knowledge of the company’s practices said. That potentially violated the copyrights to the videos, which belong to their creators.^[1]

Requiring transparency about the training data used for AI systems helps identify and mitigate biases, addressing hallucinations and other problematic outputs, and shines the light on various other issues, such as privacy and copyright concerns. A team of experts from both industry and academia at the Shorenstein Center created a documentation framework for AI which highlights the importance:

While often categorized as technical, AI systems and their underlying data and models are sociotechnical. In other words, they combine the technical infrastructure and design with the social context in which they are designed, developed, evaluated, and deployed. Accountability for these systems and their impacts requires transparency around their design and creation and how they are intended to be used. In recent years, alongside the exponential increase in data collection and the efforts to develop increasingly powerful machine learning models, there have been notable efforts calling attention to the need for documentation to accompany datasets, models, and AI systems, and to account for the process of creating them.

Documentation is worthwhile for various stakeholders. It improves the understanding of practitioners creating or building datasets, models, or AI systems, which opens up opportunities to reflect on implicit and explicit decisions, ultimately enhancing the reliability of the systems they create. For organizations, it enables knowledge transfer across silos and encourages responsible use. Further, it provides information to users and potentially affected communities that can be used to determine the appropriateness of an AI system or its underlying data or models, thus helping inform consumer choice, advocacy work, regulation development, and regulation enforcement. It also enables recourse in the event of harms caused by or inquiries into the AI system, and accountability regarding who might be held responsible for those harms.^[2]

A recent article in the Harvard Data Science Review highlights the importance of transparency and barriers to achieving it without regulation:

Knowing what is in the data sets used to train models and how they have been compiled is vitally important. Without this information, the work of developers, researchers, and ethicists to address biases or remove harmful content from the data is hampered. Information about training data is also vital to lawmakers’ attempts to assess whether foundation models have ingested personal data or copyrighted material. Further downstream, the intended operators of AI systems and those impacted by their use are far more likely to trust them if they understand how they have been developed.

However, in undertaking their analysis, Schaul et al. (2023) concluded that “many companies do not document the contents of their training data—even internally—for fear of finding personal information about identifiable individuals, copyrighted material and other data grabbed without consent.”

In public, companies have used different arguments to justify the lack of transparency around their training data. In documentation published at the launch of its GPT-4 model, OpenAI (2023) stated that it would not share detailed information about ‘data set construction’ and other aspects of the model’s development due to “the competitive landscape and the safety implications of large-scale models.” The decision not to disclose the data used to train the model was roundly criticized by a number of leading researchers (Xiang, 2023). A recent op-ed in the Guardian argued that companies are using ‘speculative fears’ to “stop people asking awkward questions about how this particular technological sausage has been made” (Naughton, 2023).^[3]

Various sectors are calling on lawmakers to provide some measure of transparency in this space. A group of media organizations and outlets, including the Associated Press and Gannett, recently issued an open letter calling on regulators to require transparency as to the makeup of all training sets used to create AI models.^[4] At the federal level, Representative Anna Eshoo and Representative Don Beyer introduced the AI Foundation Model Transparency Act, which would direct the Federal Trade Commission — in consultation with the National Institute of Standards and Technology and the White House Office of Science and Technology Policy — to “establish standards for making publicly available information about the training data and algorithms used in artificial intelligence foundation models.”^[5]

1. Requiring baseline transparency for AI systems in California

This bill begins to address the issue of training data transparency by requiring developers of AI systems and services made available to Californians for use to post documentation of the data used to train the system or service. This includes a high-level summary of the datasets used, including:

• The sources or owners of the datasets.
• A clear definition of each category associated to data points within the datasets, including the format of data points and sample values.
• Whether the datasets include any data protected by copyright, trademark, or patent, requiring the purchase or licensure of the data, or whether the datasets are entirely in the public domain.
• Whether the datasets were purchased or licensed by the developer.
• Whether the datasets include personal information.

Developers must also disclose whether the system or service used or uses synthetic data generation in its development.

The bill makes clear that training includes testing, validating, and fine tuning by the developer of the AI system or service.

According to the author:

Artificial Intelligence has become nearly unavoidable in Californians’ daily lives, with new exciting generative AI tools being introduced daily, and the companies who make up the cornerstones of our digital lives either adopting AI or identifying their existing tools as falling under the AI umbrella. However consumer confidence in AI systems has not grown at the same rapid pace as industry adoption. Many consumers have valid questions about how these AI systems and services are created, and if they truly are better than what they seek to replace.

To build consumer confidence we need to start with the foundations, and for AI that is the selection of training data. AB 2013 provides transparency to consumers of AI systems and services by providing important documentation about the data used to train the services and systems they are being offered, including if synthetic data has or is being used to fill gaps in data sources.

Consumers may use this knowledge to better evaluate if they have confidence in the AI system or service, compare competing systems and services, or put into place mitigation measures to address any shortcomings of the particular system or service.

1. Stakeholder positions

A coalition of industry groups in opposition, including TechNet, writes:

We note that the bill defines “training” to include testing, validating, or fine tuning by the developer of an AI system or service. We are concerned that amendments expanded the scope of the bill even further and effectively captures all data, regardless of risk level. The bill should be narrowed to scope in only high-risk AI systems. Mandating disclosures for low-risk AI unnecessarily burdens businesses for little to no benefit to the public. A system or service is not a "high-risk” AI system or service, for example, if it is only intended to either perform a narrow procedural task or detect decision-making patterns or deviations from prior decision-making patterns but not meant to replace or influence the previously completed human assessment without proper human review.

For disclosures to be meaningful and not overly burdensome, amendments are also needed to narrow various definitions, starting with the bill’s definition of “artificial intelligence system or service”. AB 2013’s current definition of AI system or service is over broad, arguably capturing regression-based models and even the most rudimentary prediction models or machine-based systems that generate content and make decisions using solely linear functions. Such issues can be addressed by recognizing that the system or service must be capable of “operating with varying levels of autonomy,” in line with the OECD AI definition.

Chamber of Progress writes in opposition:

Requiring online platforms to disclose data used to train their artificial intelligence (AI) systems and services on their website stifles competition in the digital marketplace. A healthy, competitive marketplace is essential to promote quality services for consumers and encourages platforms to innovate. The disclosure requirement risks revealing important business information and strategies, even when platforms specifically note that the datasets are protected intellectual property. Additionally, the language “but not be limited to” in such requirements makes the expectations placed on online platforms unclear.

Writing in support, the California Labor Federation argues:

Artificial intelligence systems have the capability of producing a wide range of outputs ranging from decisions on whether an individual ought to be hired to evaluating an employee’s performance. These decisions are monumentally impactful on the lives of everyday Californians, yet the public is not privy to the data used to train the AI systems affecting their livelihoods. From a public transparency view, this is basic information that the public is entitled to in order to understand whether their personal information may have been used as training data, the potential efficacy of the AI system, and what its outputs are predicated upon. Workers and the public cannot be left in the dark when technology of this magnitude is impacting their jobs and lives.

AB 2013 increases public transparency by requiring the developer of an AI system that makes predictions, recommendations, or decisions to publish a description of the datasets used to train the system. The set of required disclosures includes the source of the dataset, who owns it, definitional categories, when the data was collected and when it has been used, whether the data set was purchased, licensed, or found in the public domain, and whether the collected data is being used to synthesize new data sets. AB 2013 provides the public with the information to address AI systems utilizing nonconsensual personal information and training data riddled with implicit and explicit biases.

SUPPORT

California Democratic Party

California Labor Federation, AFL-CIO

Concept Art Association

Los Angeles County Democratic Party

Oakland Privacy

Perk Advocacy

Santa Monica Democratic Club

Secure Justice

Transparency Coalition.AI

OPPOSITION

American Property Casualty Insurance Association

California Bankers Association

California Chamber of Commerce

California Land Title Association

Chamber of Progress

Computer & Communications Industry Association

Insights Association

National Association of Mutual Insurance Companies

Personal Insurance Federation of California

Software & Information Industry Association

TechNet

RELATED LEGISLATION

Pending Legislation: AB 2877 (Bauer-Kahan, 2024) prohibits CCPA covered-businesses that are the developers of AI systems or tools from using the personal information of consumers under the age of 16 to train AI systems or services without first obtaining affirmative authorization, and even with such authorization the data must be de-identified and aggregated before it is used to train. AB 2877 is currently in this Committee.

AB 3204 (Bauer-Kahan, 2024) requires data digesters to register with the agency, pay a registration fee, and provide specified information, prescribe penalties for a failure to register as required by these provisions, require the California Privacy Protection Agency to create a page on its internet website where this registration information is accessible to the public, and create a fund known as the “Data Digester Registry Fund.” AB 3204 was held by the Assembly Appropriations Committee.

Prior Legislation: None known.

PRIOR VOTES:

Assembly Floor (Ayes 56, Noes 8)

Assembly Privacy and Consumer Protection Committee (Ayes 8, Noes 1)

**************

1. Cade Metz, et al., How Tech Giants Cut Corners to Harvest Data for A.I. (April 6, 2024) The New York Times, https://www.nytimes.com/2024/04/06/technology/tech-giants-harvest-data-artificial-intelligence.html. All internet citations are current as of June 16, 2024. ↑

2. Kasia Chmielinski, et al., The CLeAR Documentation Framework for AI Transparency: Recommendations for Practitioners & Context for Policymakers (May 21, 2024) Shorenstein Center on Media, Politics, and Public Policy, https://shorensteincenter.org/wp-content/uploads/2024/05/CleAR_KChmielinski_FINAL.pdf. ↑

3. Jack Hardinges, et al., We Must Fix the Lack of Transparency Around the Data Used to Train Foundation Models (December 13, 2023) Harvard Data Science Review, https://hdsr.mitpress.mit.edu/pub/xau9dza3/release/2. ↑

4. Bailey Schulz, Will AI deepen distrust in news? Gannett, other media organizations want more regulations (August 9, 2023) USA Today, https://www.usatoday.com/story/tech/news/2023/08/09/ai-regulations-media-gannett/70551555007/. ↑

5. Edward Graham, Bill sets transparency standards for AI models, including use of copyrighted material (January 2, 2024) Nextgov/FCW, https://www.nextgov.com/artificial-intelligence/2024/01/bill-sets-transparency-standards-ai-models-including-use-copyrighted-material/393052/. ↑

ASSEMBLY THIRD READING

ATR Bill Id:AB 2013

Author:(Irwin)

As Amended Ver:May 2, 2024

Majority vote

SUMMARY

Requires a developer of an artificial intelligence (AI) system or service to publicly disclose specific information related to the system or service's training data.

Major Provisions

1. Requires a developer of an AI system or service to post documentation related to its training data to the developer's internet website on or before January 1, 2026, and before each time thereafter that an AI system or service is made available to Californians.
2. Requires documentation related to training data to contain a high-level summary of the datasets used to develop the AI system or service.
3. Requires a developer of a system or service to disclose whether the system or service used or uses synthetic data generation in its development.
4. Exempts AI systems or services whose sole purpose is to help ensure security and integrity.

COMMENTS

Background. The development of AI is creating exciting opportunities to grow California's economy and improve the lives of its residents. AI uses algorithms – sets of rules – to transform inputs into outputs. Inputs and outputs can be anything a computer can process: numbers, text, audio, video, or movement.

Training is the principle innovation that allows modern AI to be both efficient and versatile. During training, a naïve AI is exposed to data and allowed to automatically explore its structure. As the AI explores, it alters itself in an attempt to better represent the data. There is a common saying in computer science: "garbage in, garbage out." The performance of an AI product is directly impacted by the quality, quantity, and relevance of the data used to train it.

Just as humans cannot intentionally forget information they have learned, it is not currently possible to remove data from a trained AI. Unlike an Excel spreadsheet, which stores data in neat columns, AI stores data in the connections between "neurons" in a "neural network." Every one of these connections is influenced by every piece of training data. It is not possible to specifically alter these connections in order to remove data without fundamentally changing the model; as a result, for data to be removed, the model must be retrained from scratch.

This bill requires AI developers disclose "whether [their] system or service used or continuously uses synthetic data generation in its development." Synthetic data is artificially generated data that is created, rather than collected from real-world events. It is designed to mimic the statistical properties of authentic data, and can be useful for training AI when actual data may be limited, sensitive, or biased. However, there are risks associated with relying on synthetic data. First, synthetic datasets may not perfectly replicate the complexity and variability of real-world data. This discrepancy can lead to models that perform well when tested in isolation, but falter in real-world applications. Second, training on synthetic data can introduce biases into a system's output if the dataset is not carefully designed. Third, training an AI system on its own outputs can lead to a phenomenon known as "model collapse," where errors and biases become continuously amplified until the AI's outputs are no longer correct or useful. The intentional use of synthetic data to train AI may expedite this process.

What this bill would do. This bill would require developers of AI systems and services to publicly disclose specified information about the datasets used to train, test, and validate their models.

Analysis. The present bill furthers consumer protection in California by granting the state's residents insight into how the AI systems and services they engage with are trained. Overall, this bill would impose modest requirements on developers of AI systems and services in exchange for providing Californians with a fuller understanding of the state's AI information ecosystem. This is consistent with the policy aims of a related bill this Committee recently passed, AB 3204 (Bauer-Kahan), which would require organizations that train AI using personal information to register with the California Privacy Protection Agency. The two bills are broadly compatible: AB 3204 would create a registry to identify the universe of entities that train AI with personal information, while this bill would require more specific disclosures by a subset of those entities to better understand the types of data being used to train AI.

According to the Author

Artificial Intelligence has become nearly unavoidable in Californians' daily lives, with new exciting generative AI tools being introduced daily, and the companies who make up the cornerstones of our digital lives either adopting AI or identifying their existing tools as falling under the AI umbrella. However consumer confidence in AI systems has not grown at the same rapid pace as industry adoption. Many consumers have valid questions about how these AI systems and services are created, and if they truly are better than what they seek to replace.

To build consumer confidence we need to start with the foundations, and for AI that is the selection of training data. [This bill] provides transparency to consumers of AI systems and services by providing important documentation about the data used to train the services and systems they are being offered, including if synthetic data has or is being used to fill gaps in data sources.

Consumers may use this knowledge to better evaluate if they have confidence in the AI system or service, compare competing systems and services, or put into place mitigation measures to address any shortcomings of the particular system or service.

Arguments in Support

Oakland Privacy writes:

From a public transparency view, this is basic and minimal information that the public is entitled to in order to be able to understand the uses to which their personal information may be applied, the potential efficacy of the AI system, and what its output is predicated upon.

Secure Justice writes:

We believe [this bill] is a pragmatic proposal that will greatly increase public awareness into the data sets being used to train artificial intelligence models, and further does not impose an undue burden on the developers subject to such a requirement.

Concept Art Association writes:

[This bill] lays an imperative first stone on the path to protection for all Californians, and it is a solid concrete action we can take right now to begin bringing some transparency to what so far has mostly been an unscrupulous, opaque and predatory model for data acquisition.

Arguments in Opposition

California Chamber of Commerce, taking an "oppose unless amended" position, writes on behalf of an industry coalition:

Unfortunately, as currently drafted, we have significant concerns with the approach taken in [this bill], and specifically around overburdensome mandates, the technical feasibility of the bill's transparency measures (including its assignment of responsibilities and the unique challenges presented for different types of developers in meeting the standards set in this bill), insufficient clarity around key terms, and potential exposure to liability. Moreover, we are heavily concerned about AB 2013's failure to provide protections for trade secrets and intellectual property, though we do not believe that is the intended outcome of this bill. While it may not be obvious on its face, the expertise and judgment as well as selection of data and datasets is part of what differentiates providers, thereby causing significant concern among developers as to the potential of this bill to undermine their intellectual property and harm competition. And lastly, we question whether the disclosure of training data will result in any substantial benefit when it comes to determining an AI model's performance for a particular use case. Stated another way, simply because a model has been trained on certain data does not mean it will perform as needed in a specific use case.

Chamber of Progress writes:

Requiring online platforms to disclose data used to train their artificial intelligence (AI) systems and services on their website stifles competition in the digital marketplace. A healthy competition marketplace is essential to ensure better quality of services for consumers and encourages platforms to innovate. The disclosure requirement risks revealing important business information and strategies. Additionally, the inclusion of "but not be limited to" in such requirements makes the expectations placed on online platforms unclear.

FISCAL COMMENTS

As currently in print, this bill is keyed nonfiscal.

VOTES

ASM PRIVACY AND CONSUMER PROTECTION: 8-1-2

YES: Bauer-Kahan, Bryan, Irwin, Lowenthal, Ortega, Ward, Wicks, Wilson

NO: Dixon

ABS, ABST OR NV: Joe Patterson, Chen

UPDATED

VERSION: May 2, 2024

CONSULTANT: Slater Sharp / P. & C.P. / (916) 319-2200 FN: 0002863

Date of Hearing: April 30, 2024

ASSEMBLY COMMITTEE ON PRIVACY AND CONSUMER PROTECTION

Rebecca Bauer-Kahan, Chair

ABPCA Bill Id:AB 2013 (

Author:Irwin) – As Amended Ver:April 22, 2024

AS PROPOSED TO BE AMENDED

SUBJECT: Artificial intelligence: training data transparency

SYNOPSIS

There is a common saying in data science: “garbage in, garbage out.” When it comes to artificial intelligence (AI), data is everything. Whether an AI’s outputs are useful and fair depends entirely on the data used to train it – at present, however, Californians have no insight into which data are used to train which products. As a result, they cannot make informed decisions when purchasing AI products to help maintain their businesses, or when exchanging sensitive personal information for services that will ostensibly improve their quality of life.

This bill would require developers of AI systems and services to publicly disclose specified information related to the datasets used to train their products. In doing so, this bill would allow Californians to make informed decisions about the AI systems they purchase and engage with.

This bill is author-sponsored and supported by Oakland Privacy, Secure Justice, Transparency Coalition.ai, Concept Art Association, and Santa Monica Democratic Club. A coalition of industry associations, including California Chamber of Commerce and Technet, takes an “oppose unless amended” position. The bill is opposed by Chamber of Progress.

SUMMARY: Requires a developer of an AI system or service to publicly disclose specific information related to the system or service’s training data. Specifically, this bill:

1. Requires a developer of an AI system or service to post documentation related to its training data to the developer’s internet website on or before January 1, 2026, and before each time thereafter than an AI system or service is made available to Californians.
2. Requires documentation related to training data to contain a description of each dataset used to develop the AI system or service, including:
3. The source or owner of the dataset.
4. A description of how the dataset furthers the intended purpose of the system or service.
5. The number of data points included in the dataset, with estimated figures for dynamic datasets.
6. A clear definition of each category associated with data points within the dataset, including the format of data points and sample values.
7. Whether the dataset includes any data protected by copyright, trademark, or patent, requiring the purchase or licensure of the data, or whether the dataset is entirely in the public domain.
8. Whether the data was purchased or licensed by the developer.
9. Whether the dataset includes personal information.
10. Whether the dataset includes aggregate consumer information.
11. A description of any cleaning, processing, or modification to the dataset by the developer, including the intended purpose of those efforts.
12. The time period during which the data was collected.
13. Whether data collection is ongoing.
14. The dates the dataset was first and last used during development of the AI system or service.
15. Requires a developer of a system or service to disclose whether the system or service used or uses synthetic data generation in its development.
16. Exempts AI systems or services whose sole purpose is to help ensure security and integrity.

EXISTING LAW:

1. Provides, pursuant to the California Constitution, that all people are by nature free and independent and have inalienable rights. Among these are the fundamental right to privacy. (Cal. Const. art. I, § 1.)
2. States that the “right to privacy is a personal and fundamental right protected by Section 1 of Article I of the Constitution of California and by the United States Constitution and that all individuals have a right of privacy in information pertaining to them.” Further states these findings of the Legislature:
3. The right to privacy is being threatened by the indiscriminate collection, maintenance, and dissemination of personal information and the lack of effective laws and legal remedies.
4. The increasing use of computers and other sophisticated information technology has greatly magnified the potential risk to individual privacy that can occur from the maintenance of personal information.
5. In order to protect the privacy of individuals, it is necessary that the maintenance and dissemination of personal information be subject to strict limits. (Civ. Code § 1798.1.)
6. Defines “personal information” to mean information that identifies, relates to, describes, is reasonably capable of being associated with, or could reasonably be linked, directly or indirectly, with a particular consumer or household. States that personal information includes, but is not limited to, the following if it identifies, relates to, describes, is reasonably capable of being associated with, or could be reasonably linked, directly or indirectly, with a particular consumer or household (Civ. Code § 1798.140(v)):
7. Identifiers such as a real name, alias, postal address, unique personal identifier, online identifier, Internet Protocol address, email address, account name, social security number, driver’s license number, passport number, or other similar identifiers.
8. Any personal information described in Section 1798.80(e).
9. Characteristics of protected classifications under California or federal law.
10. Commercial information, including records of personal property, products or services purchased, obtained, or considered, or other purchasing or consuming histories or tendencies.
11. Biometric information.
12. Internet or other electronic network activity information, including, but not limited to, browsing history, search history, and information regarding a consumer’s interaction with an internet website application, or advertisement.
13. Geolocation data.
14. Audio, electronic, visual, thermal, olfactory, or similar information.
15. Professional or employment-related information.
16. Education information, defined as information that is not publicly available personally identifiable information as defined in the Family Educational Rights and Privacy Act (20 U.S.C. Sec. 1232g; 34 C.F.R. Part 99).
17. Inferences drawn from any of the information identified in this subdivision to create a profile about a consumer reflecting the consumer’s preferences, characteristics, psychological trends, predispositions, behavior, attitudes, intelligence, abilities, and aptitudes.
18. Sensitive personal information.
19. Defines biometric information to mean an individual’s physiological, biological, or behavioral characteristics, including information pertaining to an individual’s deoxyribonucleic acid (DNA), that is used or is intended to be used singly or in combination with each other or with other identifying data, to establish individual identity. (Civ. Code § 1798.140(c).)
20. Further defines “personal information” to include any information that identifies, relates to, describes, or is capable of being associated with, a particular individual, including, but not limited to, his or her name, signature, social security number, physical characteristics or description, address, telephone number, passport number, driver’s license or state identification card number, insurance policy number, education, employment, employment history, bank account number, credit card number, debit card number, or any other financial information, medical information, or health insurance information. (Civ. Code § 1798.80(e).)
21. States that personal information does not include publicly available information that is lawfully made available to the general public from federal, state, or local government records.
22. Defines sensitive personal information to mean any of the following:
23. Personal information that reveals:
24. A consumer’s social security, driver’s license, state identification card, or passport number.
25. A consumer’s account log-in, financial account, debit card, or credit card number in combination with any required security or access code, password, or credentials allowing access to an account.
26. A consumer’s precise geolocation.
27. A consumer’s racial or ethnic origin, citizenship or immigration status, religious or philosophical beliefs, or union membership.
28. The contents of a consumer’s mail, email, and text messages unless the business is the intended recipient of the communication.
29. A consumer’s genetic data.
30. The processing of biometric information for the purpose of uniquely identifying a consumer.
31. Personal information collected and analyzed concerning a consumer’s health.
32. Personal information collected and analyzed concerning a consumer’s sex life or sexual orientation. (Civ. Code § 1798.140(ae).)
33. Defines “aggregate personal information” to mean information that relates to a group or category of consumers, from which individual consumer identities have been removed, that is not linked or reasonably linkable to any consumer or household, including via a device. Excludes from this definition one or more individual consumer records that have been deidentified. (Civ. Code § 1798.140(b).)
34. Defines “security and integrity” to mean the ability of:
35. Networks or information systems to detect security incidents that compromise the availability, authenticity, integrity, and confidentiality of stored or transmitted personal information.
36. Businesses to detect security incidents, resist malicious, deceptive, fraudulent, or illegal actions and to help prosecute those responsible for those actions.
37. Businesses to ensure the physical safety of natural persons. (Civ. Code § 1798.140(ac).)

FISCAL EFFECT: As currently in print, this bill is keyed nonfiscal.

COMMENTS:

1) Artificial intelligence. The development of AI is creating exciting opportunities to grow California’s economy and improve the lives of its residents. AI can generate compelling text and convincing images in an instant. It can automate painstaking tasks, identify subtle patterns in large datasets, and make accurate predictions in the face of incomplete information. But with novel technologies come novel safety concerns. The present bill furthers consumer protection in California by granting the state’s residents insight into how the AI systems and services they engage with are trained.

1. The importance of training. AI uses algorithms – sets of rules – to transform inputs into outputs. Inputs and outputs can be anything a computer can process: numbers, text, audio, video, or movement. This is because AI is not fundamentally different from other computer functions. Its novelty lies in its application: unlike normal computer functions, AI is able to accomplish tasks that are normally performed by humans.

Training is the secret sauce of machine learning; it is the principle innovation that allows modern AI to be both efficient and versatile. During training, a naïve AI is exposed to data and allowed to automatically explore its structure. As the AI explores, it alters itself in an attempt to better represent the data. Each piece of data affects every part of an AI. In a sense, AI “digest” and integrate the data they train on in order to learn, just as humans digest and integrate the foods we eat in order to grow.

AI that are trained on small, specific datasets in order to make recommendations and predictions are sometimes called “predictive AI.” This differentiates them from “generative AI,” which are trained on massive datasets in order to produce detailed text and images. When Netflix suggests a TV show to a viewer, the recommendation is produced by predictive AI that has been trained on the viewing habits of Netflix users. When ChatGPT generates text in clear, concise paragraphs, it uses generative AI that has been trained on the written contents of the internet.

1. Haphazard training data. There is a common saying in computer science: “garbage in, garbage out.” The performance of an AI product is directly impacted by the quality, quantity, and relevance of the data used to train it. Before training, datasets are often categorized to make them easier for AI to work with. Rigorously categorizing the data in a dataset becomes more difficult as the dataset becomes larger, but failing to organize its contents can lead to meaningless, false, or harmful outputs.

The biggest names in AI – OpenAI, Meta, and Google – understand AI’s critical need for data better than anyone else. According to a recent New York Times examination, the race to lead in the AI space has become a desperate hunt for digital data. To obtain that data, these tech companies have cut corners, ignored corporate policies and debated bending the law:

At Meta, which owns Facebook and Instagram, managers, lawyers and engineers last year discussed buying the publishing house Simon & Schuster to procure long works, according to recordings of internal meetings obtained by The Times. They also conferred on gathering copyrighted data from across the internet, even if that meant facing lawsuits. Negotiating licenses with publishers, artists, musicians and the news industry would take too long, they said.

Like OpenAI, Google transcribed YouTube videos to harvest text for its A.I. models, five people with knowledge of the company’s practices said. That potentially violated the copyrights to the videos, which belong to their creators.

Last year, Google also broadened its terms of service. One motivation for the change, according to members of the company’s privacy team and an internal message viewed by The Times, was to allow Google to be able to tap publicly available Google Docs, restaurant reviews on Google Maps and other online material for more of its A.I. products.^[1]

In their race to obtain vast quantities of training data, major AI developers have not hesitated to move fast and break things. The Stanford Internet Observatory recently discovered that a common image training dataset known as LAION-5B contains many instances of child sexual abuse materials. Their study identified 3226 dataset entries of suspected child pornography, much of which was later confirmed as such by third parties.^[2] This dataset was built by automatically scraping the internet, and images containing child pornography were found to have originated from large, well-known websites such as Reddit, Twitter, Blogspot, and Wordpress, as well as mainstream adult sites such as XHamster and XVideos.

1. An AI never forgets. Just as humans cannot intentionally forget information they have learned, it is not currently possible to remove data from a trained AI.^[3] Unlike an Excel spreadsheet, which stores data in neat columns, AI stores data in the connections between “neurons” in a “neural network.” Every one of these connections is influenced by every piece of training data, and a large model like ChatGPT-4 is reported to have more than 1.7 trillion connections.^[4] It is not possible to specifically alter these connections in order to remove data without fundamentally changing the model; as a result, for data to be removed, the model must be retrained from scratch. ChatGPT-4 is estimated to have taken 4-7 months to train.^[5]
2. Synthetic data. AB 2013 requires AI developers disclose “whether [their] system or service used or continuously uses synthetic data generation in its development.” Synthetic data is artificially generated data that is created, rather than collected from real-world events. It is designed to mimic the statistical properties of authentic data, and can be useful for training AI when actual data may be limited, sensitive, or biased. Having already gobbled up most of the high-quality data that humanity has produced, major generative AI developers have begun looking to synthetic data:

OpenAI’s Mr. Altman had a plan to deal with the looming data shortage.

Companies like his, he said at the May conference, would eventually train their A.I. on text generated by A.I. — otherwise known as synthetic data.

Since an A.I. model can produce humanlike text, Mr. Altman and others have argued, the systems can create additional data to develop better versions of themselves. This would help developers build increasingly powerful technology and reduce their dependence on copyrighted data.

“As long as you can get over the synthetic data event horizon, where the model is smart enough to make good synthetic data, everything will be fine,” Mr. Altman said.

A.I. researchers have explored synthetic data for years. But building an A.I system that can train itself is easier said than done. A.I. models that learn from their own outputs can get caught in a loop where they reinforce their own quirks, mistakes and limitations.

“The data these systems need is like a path through the jungle,” said Jeff Clune, a former OpenAI researcher who now teaches computer science at the University of British Columbia. “If they only train on synthetic data, they can get lost in the jungle.”^[6]

There are risks associated with relying on synthetic data. First, synthetic datasets may not perfectly replicate the complexity and variability of real-world data. This discrepancy can lead to models that perform well when tested in isolation, but falter in real-world applications. Second, training on synthetic data can introduce biases into a system’s output if the dataset is not carefully designed. Third, training an AI system on its own outputs can lead to a phenomenon known as “model collapse,” where errors and biases become continuously amplified until the AI’s outputs are no longer correct or useful. A recent Scientific American article likens this problem to the scramble to obtain low-radioactivity metal in the 20^th-century:

The possibility of AI models tainting themselves may be a bit analogous to a certain 20th-century dilemma. After the first atomic bombs were detonated at World War II’s end, decades of nuclear testing spiced Earth’s atmosphere with a dash of radioactive fallout. When that air entered newly-made steel, it brought elevated radiation with it. For particularly radiation-sensitive steel applications, such as Geiger counter consoles, that fallout poses an obvious problem: it won’t do for a Geiger counter to flag itself. Thus, a rush began for a dwindling supply of low-radiation metal. Scavengers scoured old shipwrecks to extract scraps of prewar steel. Now some insiders believe a similar cycle is set to repeat in generative AI—with training data instead of steel.

Researchers can watch AI’s poisoning in action. For instance, start with a language model trained on human-produced data. Use the model to generate some AI output. Then use that output to train a new instance of the model and use the resulting output to train a third version, and so forth. With each iteration, errors build atop one another. The 10th model, prompted to write about historical English architecture, spews out gibberish about jackrabbits.^[7]

Model collapse will become of a more pressing issue as more of the internet’s content is AI-generated. The intentional use of synthetic data to train AI may expedite this process.

6) What this bill would do. This bill would require developers of AI systems and services to publicly disclose specified information about the datasets used to train, test, and validate their models.

1. Author’s statement. According to the author:

Artificial Intelligence has become nearly unavoidable in Californians’ daily lives, with new exciting generative AI tools being introduced daily, and the companies who make up the cornerstones of our digital lives either adopting AI or identifying their existing tools as falling under the AI umbrella. However consumer confidence in AI systems has not grown at the same rapid pace as industry adoption. Many consumers have valid questions about how these AI systems and services are created, and if they truly are better than what they seek to replace.

To build consumer confidence we need to start with the foundations, and for AI that is the selection of training data. AB 2013 provides transparency to consumers of AI systems and services by providing important documentation about the data used to train the services and systems they are being offered, including if synthetic data has or is being used to fill gaps in data sources.

Consumers may use this knowledge to better evaluate if they have confidence in the AI system or service, compare competing systems and services, or put into place mitigation measures to address any shortcomings of the particular system or service.

1. Analysis. Overall, this bill would impose modest requirements on developers of AI systems and services in exchange for providing Californians with a fuller understanding of the state’s AI information ecosystem. This is consistent with the policy aims of a related bill this Committee recently passed, AB 3204 (Bauer-Kahan), which would require organizations that train AI using personal information to register with the California Privacy Protection Agency. The two bills are broadly compatible: AB 3204 would create a registry to identify the universe of entities that train AI with personal information, while this bill would require more specific disclosures by a subset of those entities to better understand the types of data being used to train AI.

An industry coalition takes an “oppose unless amended” position on AB 2013; their opposition letter, penned by the California Chamber of Commerce, describes four perceived weaknesses of the bill:

Opposition claim #1: AB 2013 should clearly delineate what is and is not considered “training” and narrow the scope of AI systems or services subject to these transparency measures to high-risk AI systems.

The bill defines training as “testing, validating, or fine tuning the artificial intelligence system or service.” The author may wish to instead tie the definition of training to the provided definition of artificial intelligence; for example, AB 3204 defines training to mean “exposing artificial intelligence to data in order to alter the relationship between inputs and outputs.” The requirement to describe datasets used to test, validate, or fine-tune an AI system or service could be placed elsewhere in the bill, as they are not fundamental to “training.” The author may also wish to provide a definition for “fine-tune” somewhere in the bill.

Opposition claim #2: AB 2013’s definition of “developer” is both overbroad and vague and should include guardrails to address compliance challenges.

The bill defines “developer” to mean “a person, partnership, state or local government agency, or corporation that designs, codes, or produces an artificial intelligence system or service, or substantially modifies an artificial intelligence system or service for use by a third party for free or for a fee.” This definition is broad, but not vague: any entity who produces or substantially modifies an AI system or service through training is required to provide information about the datasets they used.

Opposition claim #3: AB 2013 should not apply to AI systems and services that were in use prior to the bill’s effective date.

AB 2013 is oriented towards consumer protection, and it is not clear why products that are already available to Californian consumers should be broadly exempted from the bill’s requirements. The coalition letter specifies one particular descriptor that may be genuinely hard for developers to comply with: the “dates the dataset was first and last used during the development of the system or service.” Developers of existing systems, unaware of this bill’s provisions, may simply have no record of when training began and ended for particular datasets. The author may wish to include a narrow exemption for existing systems on this point.

Opposition claim #4: AB 2013 should expressly preclude any private right of action.

This bill does not outline specific enforcement mechanisms for its provisions. In their absence, enforcement will likely occur on the basis of California’s Unfair Competition Law.^[8] It is not clear why the bill would need to specifically preclude a Private Right of Action. Bills are generally written to permit or require specific actions, rather than to laboriously outline actions that are not meant to be taken.

Writing in support of the bill, Oakland Privacy describes the importance of training data transparency:

Visibility of data sources is one transparency factor in a number of multi-factor AI transparency models being designed. While it is far from the only one (and many focus on AI explainability and AI auditability as crucial factors as well), the ubiquitous presence of data set transparency on all of these models indicates the consensus that responsible AI always includes disclosure of the data sets used to train the system.

One reason data set transparency is important is to root out illegally or unethically sourced data sets. A notorious example is Clearview AI which scraped social media to establish a database now described as comprising almost 20 billion images – larger than the population of the earth. This data was publicly available, but protected by community standards policies which forbade large-scale scraping for profit. Similarly, industries like journalism (NY Times v OpenAI) and playwrights and screenwriters are exploring the legal limits of AI data set use. While the laws that will come to govern the use of data sets for artificial intelligence are wildly unsettled at the moment, a robust transparency mandate can be informative for ongoing efforts to set limits and determine the rules of the road going forward, as well as identifying rogue players in the system crossing lines that we decide should not be crossed. Whatever our society comes to decide constitutes acceptable use, it is unlikely to be “everything and anything”.

Oakland Privacy also points out perceived challenges for AB 2013:

We think the challenges in AB 2013 will be defining the formats for the disclosures. How this is done is likely dependent on the intentions of the bill. If the intent is for the disclosed information to be accessible to developers, engineers and scientists, then highly technical disclosures may meet the intention of the bill, and allow for some checks and balances. However, if the intent is to provide reasonably tech-savvy members of the public with actionable information, then we expect there will have to be some prescriptions regarding disclosure formats to avoid such highly technical and abstruse disclosure documents that they are virtually useless to anyone but a highly trained engineer.

1. Committee amendments. Two proposed committee amendments would clarify and adjust the scope of this bill. First, the definition of “artificial intelligence system or service” would be replaced with the definition for “artificial intelligence” in other AI-related bills that have passed through this committee:

(a) “Artificial intelligence system or service” or “system or service” means a machine-based system or service that can, for a given set of human-defined objectives, generate content and make predictions, recommendations, or decisions influencing a real or virtual environment.

(a) “Artificial intelligence” means an engineered or machine-based system that varies in its level of autonomy and that can, for explicit or implicit objectives, infer from the input it receives how to generate outputs that can influence physical or virtual environments.

Second, the bill in print requires developers of AI systems to provide a wealth of specific information related to each dataset used to train a model. Each individual description required by this bill may not constitute proprietary information, but their combination may unintentionally allow competitors to infer the precise data that goes into the training of a compliant developer’s AI system or service. To avoid this issue, a proposed amendment would require a “high level summary of the datasets” used to develop a given system or service:

(a) A description high-level summary of each dataset the datasets used in the development of the system or service, including, but not limited to…

1. Related legislation. AB 3204 (Bauer-Kahan, 2024) would require organizations that train AI using personal information to register with the California Privacy Protection Agency. In registering, organizations would be required to pay an annual fee and disclose the categories of personal information they use to train AI. This bill is currently pending in the Appropriations Committee.

ARGUMENTS IN SUPPORT:

Oakland Privacy writes:

From a public transparency view, this is basic and minimal information that the public is entitled to in order to be able to understand the uses to which their personal information may be applied, the potential efficacy of the AI system, and what its output is predicated upon.

Secure Justice writes:

We believe AB 2013 is a pragmatic proposal that will greatly increase public awareness into the data sets being used to train artificial intelligence models, and further does not impose an undue burden on the developers subject to such a requirement.

Concept Art Association writes:

AB 2013 lays an imperative first stone on the path to protection for all Californians, and it is a solid concrete action we can take right now to begin bringing some transparency to what so far has mostly been an unscrupulous, opaque and predatory model for data acquisition.

ARGUMENTS IN OPPOSITION:

California Chamber of Commerce, taking an “oppose unless amended” position, writes on behalf of an industry coalition:

Unfortunately, as currently drafted, we have significant concerns with the approach taken in AB 2013, and specifically around overburdensome mandates, the technical feasibility of the bill’s transparency measures (including its assignment of responsibilities and the unique challenges presented for different types of developers in meeting the standards set in this bill), insufficient clarity around key terms, and potential exposure to liability. Moreover, we are heavily concerned about AB 2013’s failure to provide protections for trade secrets and intellectual property, though we do not believe that is the intended outcome of this bill. While it may not be obvious on its face, the expertise and judgment as well as selection of data and datasets is part of what differentiates providers, thereby causing significant concern among developers as to the potential of this bill to undermine their intellectual property and harm competition. And lastly, we question whether the disclosure of training data will result in any substantial benefit when it comes to determining an AI model’s performance for a particular use case. Stated another way, simply because a model has been trained on certain data does not mean it will perform as needed in a specific use case.

Chamber of Progress writes:

Requiring online platforms to disclose data used to train their artificial intelligence (AI) systems and services on their website stifles competition in the digital marketplace. A healthy competition marketplace is essential to ensure better quality of services for consumers and encourages platforms to innovate. The disclosure requirement risks revealing important business information and strategies. Additionally, the inclusion of “but not be limited to” in such requirements makes the expectations placed on online platforms unclear.

REGISTERED SUPPORT / OPPOSITION:

Support

Concept Art Association
Oakland Privacy
Santa Monica Democratic Club
Secure Justice
Transparency Coalition.ai

Opposition

Chamber of Progress

Oppose Unless Amended

California Bankers Association
California Chamber of Commerce
California Land Title Association
Insights Association
National Association of Mutual Insurance Companies
Personal Insurance Federation of California
Software & Information Industry Association
Technet

Analysis Prepared by: Slater Sharp / P. & C.P. / (916) 319-2200

1. Cade Metz, Cecilia Kang, Sheera Frenkel, Stuart A. Thompson and Nico Grant, “How Tech Giants Cut Corners to Harvest Data for A.I.,” New York Times, Apr. 6, 2024, https://www.nytimes.com/2024/04/06/technology/tech-giants-harvest-data-artificial-intelligence.html. ↑

2. David Thiel, “Identifying and Eliminating CSAM in Generative ML Training Data and Models,” Stanford Internet Observatory, Dec. 23, 2023. ↑

3. Stephen Pastis, “A.I.’s un-learning problem: Researchers say it’s virtually impossible to make an A.I. model ‘forget’ the things it learns from private user data,” Yahoo! Finance, Aug. 30, 2023, finance.yahoo.com/news/un-learning-problem-researchers-virtually-164342971.html. ↑

4. Reed Albergotti, “Microsoft pushes the boundaries of small AI models with big breakthrough,” SEMAFOR, Nov. 1, 2023, www.semafor.com/article/11/01/2023/microsoft-pushes-the-boundaries-of-small-ai-models. ↑

5. Stephen McAleese, “Retrospective on ‘GPT-4 Predictions’ After the Release of GPT-4,” LESSWRONG, Mar. 17, 2023, https://www.lesswrong.com/posts/iQx2eeHKLwgBYdWPZ/retrospective-on-gpt-4-predictions-after-the-release-of-gpt. ↑

6. Cade Metz, Cecilia Kang, Sheera Frenkel, Stuart A. Thompson and Nico Grant, “How Tech Giants Cut Corners to Harvest Data for A.I.,” New York Times, Apr. 6, 2024, https://www.nytimes.com/2024/04/06/technology/tech-giants-harvest-data-artificial-intelligence.html. ↑

7. Rahul Rao, “AI-Generated Data Can Poison Future AI Models,” Scientific American, Jul. 28, 2023, https://www.scientificamerican.com/article/ai-generated-data-can-poison-future-ai-models/ ↑

8. Bus. & Prof. Code § 17200 et seq. ↑