SB 942: California AI Transparency Act.
- Session Year: 2023-2024
- House: Senate
- Latest Version Date: 2024-09-19
Current Status:
Passed
(2024-09-19: Chaptered by Secretary of State. Chapter 291, Statutes of 2024.)
Introduced
In Committee
First Chamber
In Committee
Second Chamber
Enacted
Existing law requires the Secretary of Government Operations to develop a coordinated plan to, among other things, investigate the feasibility of, and obstacles to, developing standards and technologies for state departments to determine digital content provenance. For the purpose of informing that coordinated plan, existing law requires the secretary to evaluate, among other things, the impact of the proliferation of deepfakes, defined to mean audio or visual content that has been generated or manipulated by artificial intelligence that would falsely appear to be authentic or truthful and that features depictions of people appearing to say or do things they did not say or do without their consent, on state government, California-based businesses, and residents of the state.
This bill, the California AI Transparency Act, would, among other things, require a covered provider, as defined, to make available an artificial intelligence (AI) detection tool at no cost to the user that meets certain criteria, including that the AI detection tool is publicly accessible. The bill would require a covered provider to offer the user an option to include a manifest disclosure in image, video, or audio content, or content that is any combination thereof, created or altered by the covered providers generative artificial intelligence (GenAI) system that, among other things, identifies content as AI-generated content and is clear, conspicuous, appropriate for the medium of the content, and understandable to a reasonable person. The bill would require a covered provider to include a latent disclosure in AI-generated image, video, audio content, or content that is any combination thereof, created by the covered providers GenAI system that, among other things, to the extent that it is technically feasible and reasonable conveys certain information, either directly or through a link to a permanent internet website, regarding the provenance of the content. The bill would require a covered provider that knows a third-party licensee modified a licensed GenAI system such that it is no longer capable of including the disclosures described above in content the system creates or alters to revoke the license within 96 hours of discovering the licensees action and would require a third-party licensee to cease using a licensed GenAI system after the license for the system has been revoked by the covered provider.
This bill would make a covered provider that violates these provisions liable for a civil penalty in the amount of $5,000 per violation to be collected in a civil action filed by the Attorney General, a city attorney, or a county counsel, as prescribed. The bill would, for a violation by a third-party licensee of the requirement to cease using a licensed GenAI system after the license of the system has been revoked, authorize the Attorney General, a county counsel, or a city attorney to bring a civil action for injunctive relief and reasonable attorneys fees and costs.
This bill would make its provisions operative on January 1, 2026.
Existing law requires the Secretary of Government Operations to develop a coordinated plan to, among other things, investigate the feasibility of, and obstacles to, developing standards and technologies for state departments to determine digital content provenance. For the purpose of informing that coordinated plan, existing law requires the secretary to evaluate, among other things, the impact of the proliferation of deepfakes, defined to mean audio or visual content that has been generated or manipulated by artificial intelligence that would falsely appear to be authentic or truthful and that features depictions of people appearing to say or do things they did not say or do without their consent, on state government, California-based businesses, and residents of the state.
This bill, the California AI Transparency Act, would, among other things, require a covered provider, as defined, to make available an artificial intelligence (AI) detection tool at no cost to the user that meets certain criteria, including that the AI detection tool is publicly accessible. The bill would require a covered provider to offer the user an option to include a manifest disclosure in image, video, or audio content, or content that is any combination thereof, created or altered by the covered providers generative artificial intelligence (GenAI) system that, among other things, identifies content as AI-generated content and is clear, conspicuous, appropriate for the medium of the content, and understandable to a reasonable person. The bill would require a covered provider to include a latent disclosure in AI-generated image, video, audio content, or content that is any combination thereof, created by the covered providers GenAI system that, among other things, to the extent that it is technically feasible and reasonable conveys certain information, either directly or through a link to a permanent internet website, regarding the provenance of the content. The bill would require a covered provider that knows a third-party licensee modified a licensed GenAI system such that it is no longer capable of including the disclosures described above in content the system creates or alters to revoke the license within 96 hours of discovering the licensees action and would require a third-party licensee to cease using a licensed GenAI system after the license for the system has been revoked by the covered provider.
This bill would make a covered provider that violates these provisions liable for a civil penalty in the amount of $5,000 per violation to be collected in a civil action filed by the Attorney General, a city attorney, or a county counsel, as prescribed. The bill would, for a violation by a third-party licensee of the requirement to cease using a licensed GenAI system after the license of the system has been revoked, authorize the Attorney General, a county counsel, or a city attorney to bring a civil action for injunctive relief and reasonable attorneys fees and costs.
This bill would make its provisions operative on January 1, 2026.
Existing law requires the Secretary of Government Operations to develop a coordinated plan to, among other things, investigate the feasibility of, and obstacles to, developing standards and technologies for state departments to determine digital content provenance. For the purpose of informing that coordinated plan, existing law requires the secretary to evaluate, among other things, the impact of the proliferation of deepfakes, defined to mean audio or visual content that has been generated or manipulated by artificial intelligence that would falsely appear to be authentic or truthful and that features depictions of people appearing to say or do things they did not say or do without their consent, on state government, California-based businesses, and residents of the state.
This bill, the California AI Transparency Act, would, among other things, require a covered provider, as defined, to create and make available an artificial intelligence (AI) detection tool at no cost to the user that meets certain criteria, including that the AI detection tool is publicly accessible. The bill would require a covered provider to offer the user an option to include a manifest disclosure in image, video, or audio content, or content that is any combination thereof, created or altered by the covered providers generative artificial intelligence (GenAI) system that, among other things, identifies content as AI-generated content and is clear, conspicuous, appropriate for the medium of the content, and understandable to a reasonable person. The bill would require a covered provider to include a latent disclosure in AI-generated image, video, audio content, or content that is any combination thereof, created by the covered providers GenAI system that, among other things, to the extent that it is technically feasible and reasonable conveys certain information, either directly or through a link to a permanent internet website, regarding the provenance of the content. The bill would require a covered provider that knows a third-party licensee modified a licensed GenAI system such that it is no longer capable of including the disclosures described above in content the system creates or alters to revoke the license within 72 96 hours of discovering the licensees action and would require a third-party licensee to cease using a licensed GenAI system after the license for the system has been revoked by the covered provider.
This bill would make a covered provider that violates these provisions liable for a civil penalty in the amount of $5,000 per violation to be collected in a civil action filed by the Attorney General, a city attorney, or a county counsel, as prescribed. The bill would, for a violation by a third-party licensee of the requirement to cease using a licensed GenAI system after the license of the system has been revoked, authorize the Attorney General, a county counsel, or a city attorney to bring a civil action for injunctive relief and reasonable attorneys fees and costs.
This bill would make its provisions operative on January 1, 2026.
Existing law requires the Secretary of Government Operations to develop a coordinated plan to, among other things, investigate the feasibility of, and obstacles to, developing standards and technologies for state departments to determine digital content provenance. For the purpose of informing that coordinated plan, existing law requires the secretary to evaluate, among other things, the impact of the proliferation of deepfakes, defined to mean audio or visual content that has been generated or manipulated by artificial intelligence that would falsely appear to be authentic or truthful and that features depictions of people appearing to say or do things they did not say or do without their consent, on state government, California-based businesses, and residents of the state.
This bill, the California AI Transparency Act, would, among other things, require a covered provider, as defined, to create and make freely available an artificial intelligence (AI) detection tool at no cost to the user that meets certain criteria, including that the AI detection tool is publicly accessible through the covered providers internet website and its mobile application, as applicable. accessible. The bill would require a covered provider to offer the user an option to include a manifest disclosure in image, video, audio, or other digital content or audio content, or content that is any combination thereof, created or altered by the covered providers generative artificial intelligence (GenAI) system that, among other things, identifies content as AI-generated content and is clear, conspicuous, appropriate for the medium of the content, and understandable to a reasonable person. The bill would require a covered provider to include a latent disclosure in AI-generated image, video, audio, or other digital content audio content, or content that is any combination thereof, created by the covered providers GenAI system that, among other things, to the extent that it is technically feasible conveys certain information, either directly or through a link to a permanent internet website, regarding the provenance of the content. The bill would require a covered provider that knows a third-party licensee modified a licensed GenAI system such that it is no longer capable of including the disclosures described above in content the system creates or alters to revoke the license within 72 hours of discovering the licensees action and would require a third-party licensee to cease using a licensed GenAI system after the license for the system has been revoked by the covered provider.
This bill would make a covered provider that violates these provisions liable for a civil penalty in the amount of $5,000 per violation to be collected in a civil action filed by the Attorney General, a city attorney, or a county counsel, as prescribed. The bill would, for a violation by a third-party licensee of the requirement to cease using a licensed GenAI system after the license of the system has been revoked, authorize the Attorney General, a county counsel, or a city attorney to bring a civil action for injunctive relief and reasonable attorneys fees and costs.
This bill would make its provisions operative on January 1, 2026.
Existing law requires the Secretary of Government Operations to develop a coordinated plan to, among other things, investigate the feasibility of, and obstacles to, developing standards and technologies for state departments to determine digital content provenance. For the purpose of informing that coordinated plan, existing law requires the secretary to evaluate, among other things, the impact of the proliferation of deepfakes, defined to mean audio or visual content that has been generated or manipulated by artificial intelligence that would falsely appear to be authentic or truthful and that features depictions of people appearing to say or do things they did not say or do without their consent, on state government, California-based businesses, and residents of the state.
This bill, the California AI Transparency Act, would, among other things, require a covered provider, as defined, to create and make freely available an artificial intelligence (AI) detection tool that meets certain criteria, including that the AI detection tool is publicly accessible through the covered providers internet website and its mobile application, as applicable. The bill would require a covered provider to offer the user an option to include a manifest disclosure in image, video, audio, or other digital content created or altered by the covered providers generative artificial intelligence (GenAI) system that, among other things, identifies content as AI-generated content and is clear, conspicuous, appropriate for the medium of the content, and understandable to a reasonable person. The bill would require a covered provider to include a latent disclosure in AI-generated image, video, audio, or other digital content created by the covered providers GenAI system that, among other things, conveys certain information, either directly or through a link to a permanent internet website, regarding the provenance of the content. The bill would require a covered provider that knows a third-party licensee modified a licensed GenAI system such that it is no longer capable of including the disclosures described above in content the system creates or alters to revoke the license within 72 hours of discovering the licensees action and would require a third-party licensee to cease using a licensed GenAI system after the license for the system has been revoked by the covered provider.
This bill would make a covered provider that violates these provisions liable for a civil penalty in the amount of $5,000 per violation to be collected in a civil action filed only by the Attorney General, a city attorney, or a county counsel, as prescribed. The bill would, for a violation by a third-party licensee of the requirement to cease using a licensed GenAI system after the license of the system has been revoked, authorize the Attorney General, a district attorney, a county counsel, or a city attorney to bring a civil action for injunctive relief and reasonable attorneys fees and costs.
Existing law requires the Secretary of Government Operations to develop a coordinated plan to, among other things, investigate the feasibility of, and obstacles to, developing standards and technologies for state departments to determine digital content provenance. For the purpose of informing that coordinated plan, existing law requires the secretary to evaluate, among other things, the impact of the proliferation of deepfakes, defined to mean audio or visual content that has been generated or manipulated by artificial intelligence that would falsely appear to be authentic or truthful and that features depictions of people appearing to say or do things they did not say or do without their consent, on state government, California-based businesses, and residents of the state.
This bill, the California AI Transparency Act, would, among other things, require a covered provider, as defined, to create and make freely available an AI artificial intelligence (AI) detection tool by which a person can query the covered provider as to the extent to which text, image, video, audio, or multimedia content was created, in whole or in part, by a generative AI system, as defined, provided by the covered provider that meets certain criteria, including that the AI detection tool is publicly accessible and available via a uniform resource locator (URL) on through the covered providers internet website and through its mobile application, as applicable. The bill would also require a covered provider to offer the user an option to include a manifest disclosure in AI-generated image, text, video, audio, or multimedia other digital content created or altered by a generative AI system it provides a visible disclosure the covered providers generative artificial intelligence (GenAI) system that, among other things, includes a clear and conspicuous notice, as appropriate for the medium of the content, that identifies the content as generated by AI, such that the disclosure is not avoidable, is understandable to a reasonable person, and is not contradicted, mitigated by, or inconsistent with anything else in the communication. identifies content as AI-generated content and is clear, conspicuous, appropriate for the medium of the content, and understandable to a reasonable person. The bill would require a covered provider to include a latent disclosure in AI-generated image, video, audio, or other digital content created by the covered providers GenAI system that, among other things, conveys certain information, either directly or through a link to a permanent internet website, regarding the provenance of the content. The bill would require a covered provider that knows a third-party licensee modified a licensed GenAI system such that it is no longer capable of including the disclosures described above in content the system creates or alters to revoke the license within 72 hours of discovering the licensees action and would require a third-party licensee to cease using a licensed GenAI system after the license for the system has been revoked by the covered provider.
This bill would make a covered provider that violates these provisions liable for a civil penalty in the amount of $5,000 per violation to be collected in a civil action filed only by the Attorney General, as prescribed. The bill would, for a violation by a third-party licensee of the requirement to cease using a licensed GenAI system after the license of the system has been revoked, authorize the Attorney General, a district attorney, a county counsel, or a city attorney to bring a civil action for injunctive relief and reasonable attorneys fees and costs.
Existing law requires the Secretary of Government Operations to develop a coordinated plan to, among other things, investigate the feasibility of, and obstacles to, developing standards and technologies for state departments to determine digital content provenance. For the purpose of informing that coordinated plan, existing law requires the secretary to evaluate, among other things, the impact of the proliferation of deepfakes, defined to mean audio or visual content that has been generated or manipulated by artificial intelligence that would falsely appear to be authentic or truthful and that features depictions of people appearing to say or do things they did not say or do without their consent, on state government, California-based businesses, and residents of the state.
This bill, the California AI Transparency Act, would, among other things, require a covered provider, as defined, to create an AI detection tool by which a person can query the covered provider as to the extent to which text, image, video, audio, or multimedia content was created, in whole or in part, by a generative AI system, as defined, provided by the covered provider that meets certain criteria, including that the AI detection tool is publicly accessible and available via a uniform resource locator (URL) on the covered providers internet website and through its mobile application, as applicable. The act bill would also require a covered provider to include in AI-generated image, text, video, or multimedia content created by a generative AI system it provides a visible disclosure that, among other things, includes a clear and conspicuous notice, as appropriate for the medium of the content, that identifies the content as generated by AI, such that the disclosure is not avoidable, is understandable to a reasonable person, and is not contradicted, mitigated by, or inconsistent with anything else in the communication.
The act would create the Generative AI Registry Fund and would require moneys in the fund to be made available, only upon appropriation by the Legislature, to the Department of Technology for the purposes of the act.The act would require a covered provider to register with the department and provide to the department a URL to any AI detection tool it has created. The act would authorize the department to charge a registration fee, which shall be deposited into the Generative AI Registry Fund, to a covered provider, as specified. The act would require the department to create and display on its internet website the Generative AI Registry that displays the name of any covered provider registered with the department and a link to the covered providers AI detection tool. The act would provide that This bill would make a covered provider that violates the act is these provisions liable for a civil penalty in the amount of $5,000 per violation to be collected in a civil action filed only by the Attorney General, as prescribed.
Existing law requires the Secretary of Government Operations to develop a coordinated plan to, among other things, investigate the feasibility of, and obstacles to, developing standards and technologies for state departments to determine digital content provenance. For the purpose of informing that coordinated plan, existing law requires the secretary to evaluate, among other things, the impact of the proliferation of deepfakes, defined to mean audio or visual content that has been generated or manipulated by artificial intelligence that would falsely appear to be authentic or truthful and that features depictions of people appearing to say or do things they did not say or do without their consent, on state government, California-based businesses, and residents of the state.
This bill, the California AI Transparency Act, would, among other things, require a covered provider, as defined, to create an AI detection tool by which a person can query the covered provider as to the extent to which text, image, video, audio, or multimedia content was created, in whole or in part, by a generative AI system, as defined, provided by the covered provider that meets certain criteria, including that the AI detection tool is publicly accessible and available via a uniform resource locator (URL) on the covered providers internet website and through its mobile application, as applicable. The act would also require a covered provider to include in AI-generated image, text, video, or multimedia content created by a generative AI system it provides a visible disclosure that, among other things, includes a clear and conspicuous notice, as appropriate for the medium of the content, that identifies the content as generated by AI, such that the disclosure is not avoidable, is understandable to a reasonable person, and is not contradicted, mitigated by, or inconsistent with anything else in the communication.
The act would create the Generative AI Registry Fund and would require moneys in the fund to be made available, only upon appropriation by the Legislature, to the Department of Technology for the purposes of the act.
The act would require a covered provider to register with the department and provide to the department a URL to any AI detection tool it has created. The act would authorize the department to charge a registration fee, which shall be deposited into the Generative AI Registry Fund, to a covered provider, as specified. The act would require the department to create and display on its internet website the Generative AI Registry that displays the name of any covered provider registered with the department and a link to the covered providers AI detection tool. The act would provide that a covered provider that violates the act is liable for a civil penalty in the amount of $5,000 per violation to be collected in a civil action filed only by the Attorney General, as prescribed.
Existing law imposes certain obligations in the context of particular transactions, and provides mechanisms to enforce those obligations. This bill would state the intent of the Legislature to enact legislation that would establish a mechanism to allow consumers to easily determine whether images, audio, video, or text was created by generative artificial intelligence.Existing law imposes certain obligations in the context of particular transactions, and provides mechanisms to enforce those obligations.
This bill would state the intent of the Legislature to enact legislation that would establish a mechanism to allow consumers to easily determine whether images, audio, video, or text was created by generative artificial intelligence.
This chapter shall be known as the California AI Transparency Act.
22757.1.As used in this chapter:
(a)Artificial intelligence or AI means an engineered or machine-based system that varies in its level of autonomy and that can, for explicit or implicit objectives, infer from the input it receives how to generate outputs that can influence physical or virtual environments.
(b)Covered provider means a person that creates, codes, or otherwise produces a generative artificial intelligence system that has over 1,000,000 monthly visitors or users and is publicly accessible within the geographic boundaries of the state.
(c)Generative artificial intelligence system or GenAI system means an artificial intelligence that can generate derived synthetic content, including text, images, video, and audio, that emulates the structure and characteristics of the systems training data.
(d)Latent means present but not manifest.
(e)Manifest means easily perceived, understood, or recognized by a natural person.
(f)Metadata means structural or descriptive information about data.
(g)Personal information has the same meaning as defined in Section 1798.140 of the Civil Code.
(h)Personal provenance data means provenance data that contains either of the following:
(1)Personal information.
(2)Unique device, system, or service information that is reasonably capable of being associated with a particular user.
(i)Provenance data means data that is embedded into digital content, or that is included in the digital contents metadata, for the purpose of verifying the digital contents authenticity, origin, or history of modification.
(j)System provenance data means provenance data that is not reasonably capable of being associated with a particular user and that contains either of the following:
(1)Information regarding the type of device, system, or service that was used to generate a piece of digital content.
(2)Information related to content authenticity.
22757.2.(a)A covered provider shall make available an AI detection tool at no cost to the user that meets all of the following criteria:
(1)The tool allows a user to assess whether image, video, or audio content, or content that is any combination thereof, was created or altered by the covered providers GenAI system.
(2)The tool outputs any system provenance data that is detected in the content.
(3)The tool does not output any personal provenance data that is detected in the content.
(4)(A)Subject to subparagraph (B), the tool is publicly accessible.
(B)A covered provider may impose reasonable limitations on access to the tool to prevent, or respond to, demonstrable risks to the security or integrity of its GenAI system.
(5)The tool allows a user to upload content or provide a uniform resource locator (URL) linking to online content.
(6)The tool supports an application programming interface that allows a user to invoke the tool without visiting the covered providers internet website.
(b)A covered provider shall collect user feedback related to the efficacy of the covered providers AI detection tool and incorporate relevant feedback into any attempt to improve the efficacy of the tool.
(c)A covered provider shall not do any of the following:
(1)(A)Except as provided in subparagraph (B), collect or retain personal information from users of the covered providers AI detection tool.
(B)(i)A covered provider may collect and retain the contact information of a user who submits feedback pursuant to subdivision (b) if the user opts in to being contacted by the covered provider.
(ii)User information collected pursuant to clause (i) shall be used only to evaluate and improve the efficacy of the covered providers AI detection tool.
(2)Retain any content submitted to the AI detection tool for longer than is necessary to comply with this section.
(3)Retain any personal provenance data from content submitted to the AI detection tool by a user.
22757.3.(a)A covered provider shall offer the user the option to include a manifest disclosure in image, video, or audio content, or content that is any combination thereof, created or altered by the covered providers GenAI system that meets all of the following criteria:
(1)The disclosure identifies content as AI-generated content.
(2)The disclosure is clear, conspicuous, appropriate for the medium of the content, and understandable to a reasonable person.
(3)The disclosure is permanent or extraordinarily difficult to remove, to the extent it is technically feasible.
(b)A covered provider shall include a latent disclosure in AI-generated image, video, or audio content, or content that is any combination thereof, created by the covered providers GenAI system that meets all of the following criteria:
(1)To the extent that it is technically feasible and reasonable, the disclosure conveys all of the following information, either directly or through a link to a permanent internet website:
(A)The name of the covered provider.
(B)The name and version number of the GenAI system that created or altered the content.
(C)The time and date of the contents creation or alteration.
(D)A unique identifier.
(2)The disclosure is detectable by the covered providers AI detection tool.
(3)The disclosure is consistent with widely accepted industry standards.
(4)The disclosure is permanent or extraordinarily difficult to remove, to the extent it is technically feasible.
(c)(1)If a covered provider licenses its GenAI system to a third party, the covered provider shall require by contract that the licensee maintain the systems capability to include a disclosure required by subdivision (b) in content the system creates or alters.
(2)If a covered provider knows that a third-party licensee modified a licensed GenAI system such that it is no longer capable of including a disclosure required by subdivision (b) in content the system creates or alters, the covered provider shall revoke the license within 96 hours of discovering the licensees action.
(3)A third-party licensee shall cease using a licensed GenAI system after the license for the system has been revoked by the covered provider pursuant to paragraph (2).
22757.4.(a)(1)A covered provider that violates this chapter shall be liable for a civil penalty in the amount of five thousand dollars ($5,000) per violation to be collected in a civil action filed by the Attorney General, a city attorney, or a county counsel.
(2)A prevailing plaintiff in an action brought pursuant to this subdivision shall be entitled to all reasonable attorneys costs and fees.
(b)Each day that a covered provider is in violation of this chapter shall be deemed a discrete violation.
(c)For a violation by a third-party licensee of paragraph (3) of subdivision (c) of Section 22757.3, the Attorney General, a county counsel, or a city attorney may bring a civil action for both of the following:
(1)Injunctive relief.
(2)Reasonable attorneys fees and costs.
22757.5.This chapter does not apply to any product, service, internet website, or application that provides exclusively non-user-generated video game, television, streaming, movie, or interactive experiences.
22757.6.This chapter shall become operative on January 1, 2026.
This chapter shall be known as the California AI Transparency Act.
22757.1.As used in this chapter:
(a)Artificial intelligence or AI means an engineered or machine-based system that varies in its level of autonomy and that can, for explicit or implicit objectives, infer from the input it receives how to generate outputs that can influence physical or virtual environments.
(b)Covered provider means a person that creates, codes, or otherwise produces a generative artificial intelligence system that has over 1,000,000 monthly visitors or users and is publicly accessible within the geographic boundaries of the state.
(c)Generative artificial intelligence system or GenAI system means an artificial intelligence that can generate derived synthetic content, including text, images, video, and audio, that emulates the structure and characteristics of the systems training data.
(d)Latent means present but not manifest.
(e)Manifest means easily perceived, understood, or recognized by a natural person.
(f)Metadata means structural or descriptive information about data.
(g)Personal information has the same meaning as defined in Section 1798.140 of the Civil Code.
(h)Personal provenance data means provenance data that contains either of the following:
(1)Personal information.
(2)Unique device, system, or service information that is reasonably capable of being associated with a particular user.
(i)Provenance data means data that is embedded into digital content, or that is included in the digital contents metadata, for the purpose of verifying the digital contents authenticity, origin, or history of modification.
(j)System provenance data means provenance data that is not reasonably capable of being associated with a particular user and that contains either of the following:
(1)Information regarding the type of device, system, or service that was used to generate a piece of digital content.
(2)Information related to content authenticity.
22757.2.(a)A covered provider shall make available an AI detection tool at no cost to the user that meets all of the following criteria:
(1)The tool allows a user to assess whether image, video, or audio content, or content that is any combination thereof, was created or altered by the covered providers GenAI system.
(2)The tool outputs any system provenance data that is detected in the content.
(3)The tool does not output any personal provenance data that is detected in the content.
(4)(A)Subject to subparagraph (B), the tool is publicly accessible.
(B)A covered provider may impose reasonable limitations on access to the tool to prevent, or respond to, demonstrable risks to the security or integrity of its GenAI system.
(5)The tool allows a user to upload content or provide a uniform resource locator (URL) linking to online content.
(6)The tool supports an application programming interface that allows a user to invoke the tool without visiting the covered providers internet website.
(b)A covered provider shall collect user feedback related to the efficacy of the covered providers AI detection tool and incorporate relevant feedback into any attempt to improve the efficacy of the tool.
(c)A covered provider shall not do any of the following:
(1)(A)Except as provided in subparagraph (B), collect or retain personal information from users of the covered providers AI detection tool.
(B)(i)A covered provider may collect and retain the contact information of a user who submits feedback pursuant to subdivision (b) if the user opts in to being contacted by the covered provider.
(ii)User information collected pursuant to clause (i) shall be used only to evaluate and improve the efficacy of the covered providers AI detection tool.
(2)Retain any content submitted to the AI detection tool for longer than is necessary to comply with this section.
(3)Retain any personal provenance data from content submitted to the AI detection tool by a user.
22757.3.(a)A covered provider shall offer the user the option to include a manifest disclosure in image, video, or audio content, or content that is any combination thereof, created or altered by the covered providers GenAI system that meets all of the following criteria:
(1)The disclosure identifies content as AI-generated content.
(2)The disclosure is clear, conspicuous, appropriate for the medium of the content, and understandable to a reasonable person.
(3)The disclosure is permanent or extraordinarily difficult to remove, to the extent it is technically feasible.
(b)A covered provider shall include a latent disclosure in AI-generated image, video, or audio content, or content that is any combination thereof, created by the covered providers GenAI system that meets all of the following criteria:
(1)To the extent that it is technically feasible and reasonable, the disclosure conveys all of the following information, either directly or through a link to a permanent internet website:
(A)The name of the covered provider.
(B)The name and version number of the GenAI system that created or altered the content.
(C)The time and date of the contents creation or alteration.
(D)A unique identifier.
(2)The disclosure is detectable by the covered providers AI detection tool.
(3)The disclosure is consistent with widely accepted industry standards.
(4)The disclosure is permanent or extraordinarily difficult to remove, to the extent it is technically feasible.
(c)(1)If a covered provider licenses its GenAI system to a third party, the covered provider shall require by contract that the licensee maintain the systems capability to include a disclosure required by subdivision (b) in content the system creates or alters.
(2)If a covered provider knows that a third-party licensee modified a licensed GenAI system such that it is no longer capable of including a disclosure required by subdivision (b) in content the system creates or alters, the covered provider shall revoke the license within 96 hours of discovering the licensees action.
(3)A third-party licensee shall cease using a licensed GenAI system after the license for the system has been revoked by the covered provider pursuant to paragraph (2).
22757.4.(a)(1)A covered provider that violates this chapter shall be liable for a civil penalty in the amount of five thousand dollars ($5,000) per violation to be collected in a civil action filed by the Attorney General, a city attorney, or a county counsel.
(2)A prevailing plaintiff in an action brought pursuant to this subdivision shall be entitled to all reasonable attorneys costs and fees.
(b)Each day that a covered provider is in violation of this chapter shall be deemed a discrete violation.
(c)For a violation by a third-party licensee of paragraph (3) of subdivision (c) of Section 22757.3, the Attorney General, a county counsel, or a city attorney may bring a civil action for both of the following:
(1)Injunctive relief.
(2)Reasonable attorneys fees and costs.
22757.5.This chapter does not apply to any product, service, internet website, or application that provides exclusively non-user-generated video game, television, streaming, movie, or interactive experiences.
22757.6.This chapter shall become operative on January 1, 2026.
This chapter shall be known as the California AI Transparency Act.
22757.1.As used in this chapter:
(a)Artificial intelligence or AI means an engineered or machine-based system that varies in its level of autonomy and that can, for explicit or implicit objectives, infer from the input it receives how to generate outputs that can influence physical or virtual environments.
(b)Covered provider means a person that creates, codes, or otherwise produces a generative artificial intelligence system that has over 1,000,000 monthly visitors or users and is publicly accessible within the geographic boundaries of the state.
(c)Generative artificial intelligence system or GenAI system means an artificial intelligence that can generate derived synthetic content, including text, images, video, and audio, that emulates the structure and characteristics of the systems training data.
(d)Latent means present but not manifest.
(e)Manifest means easily perceived, understood, or recognized by a natural person.
(f)Metadata means structural or descriptive information about data.
(g)Personal information has the same meaning as defined in Section 1798.140 of the Civil Code.
(h)Personal provenance data means provenance data that contains either of the following:
(1)Personal information.
(2)Unique device, system, or service information that is reasonably capable of being associated with a particular user.
(i)Provenance data means data that is embedded into digital content, or that is included in the digital contents metadata, for the purpose of verifying the digital contents authenticity, origin, or history of modification.
(j)System provenance data means provenance data that is not reasonably capable of being associated with a particular user and that contains either of the following:
(1)Information regarding the type of device, system, or service that was used to generate a piece of digital content.
(2)Information related to content authenticity.
22757.2.(a)A covered provider shall create and make available an AI detection tool at no cost to the user that meets all of the following criteria:
(1)The tool allows a user to assess whether image, video, or audio content, or content that is any combination thereof, was created or altered by the covered providers GenAI system.
(2)The tool outputs any system provenance data that is detected in the content.
(3)The tool does not output any personal provenance data that is detected in the content.
(4)The (A)Subject to subparagraph (B), the tool is publicly accessible.
(B)A covered provider may impose reasonable limitations on access to the tool to prevent, or respond to, demonstrable risks to the security or integrity of its GenAI system.
(5)The tool allows a user to upload content or provide a uniform resource locator (URL) linking to online content.
(6)The tool supports an application programming interface that allows a user to invoke the tool without visiting the covered providers internet website.
(b)A covered provider shall collect user feedback related to the efficacy of the covered providers AI detection tool and incorporate relevant feedback into any attempt to improve the efficacy of the tool.
(c)A covered provider shall not do any of the following:
(1)(A)Except as provided in subparagraph (B), collect or retain personal information from users of the covered providers AI detection tool.
(B)(i)A covered provider may collect and retain the contact information of a user who submits feedback pursuant to subdivision (b) if the user opts in to being contacted by the covered provider.
(ii)User information collected pursuant to clause (i) shall be used only to evaluate and improve the efficacy of the covered providers AI detection tool.
(2)Retain any content submitted to the AI detection tool for longer than is necessary to comply with this section.
(3)Retain any personal provenance data from content submitted to the AI detection tool by a user.
22757.3.(a)A covered provider shall offer the user the option to include a manifest disclosure in image, video, or audio content, or content that is any combination thereof, created or altered by the covered providers GenAI system that meets all of the following criteria:
(1)The disclosure identifies content as AI-generated content.
(2)The disclosure is clear, conspicuous, appropriate for the medium of the content, and understandable to a reasonable person.
(3)The disclosure is permanent or extraordinarily difficult to remove, to the extent it is technically feasible.
(b)A covered provider shall include a latent disclosure in AI-generated image, video, or audio content, or content that is any combination thereof, created by the covered providers GenAI system that meets all of the following criteria:
(1)To the extent that it is technically feasible, feasible and reasonable, the disclosure conveys all of the following information, either directly or through a link to a permanent internet website:
(A)The name of the covered provider.
(B)The name and version number of the GenAI system that created or altered the content.
(C)The time and date of the contents creation or alteration.
(D)A unique identifier.
(2)The disclosure is detectable by the covered providers AI detection tool.
(3)The disclosure is consistent with widely accepted industry standards.
(4)The disclosure is permanent or extraordinarily difficult to remove, to the extent it is technically feasible.
(c)(1)If a covered provider licenses its GenAI system to a third party, the covered provider shall require by contract that the licensee maintain the systems capability to include a disclosure required by subdivision (b) in content the system creates or alters.
(2)If a covered provider knows that a third-party licensee modified a licensed GenAI system such that it is no longer capable of including a disclosure required by subdivision (b) in content the system creates or alters, the covered provider shall revoke the license within 72 96 hours of discovering the licensees action.
(3)A third-party licensee shall cease using a licensed GenAI system after the license for the system has been revoked by the covered provider pursuant to paragraph (2).
22757.4.(a)(1)A covered provider that violates this chapter shall be liable for a civil penalty in the amount of five thousand dollars ($5,000) per violation to be collected in a civil action filed by the Attorney General, a city attorney, or a county counsel.
(2)A prevailing plaintiff in an action brought pursuant to this subdivision shall be entitled to all reasonable attorneys costs and fees.
(b)Each day that a covered provider is in violation of this chapter shall be deemed a discrete violation.
(c)For a violation by a third-party licensee of paragraph (3) of subdivision (c) of Section 22757.3, the Attorney General, a county counsel, or a city attorney may bring a civil action for both of the following:
(1)Injunctive relief.
(2)Reasonable attorneys fees and costs.
22757.5.This chapter does not apply to any product, service, internet website, or application that provides exclusively non-user-generated video game, television, streaming, movie, or interactive experiences.
22757.5.22757.6.This chapter shall become operative on January 1, 2026.
This chapter shall be known as the California AI Transparency Act.
22757.1.As used in this chapter:
(a)Artificial intelligence or AI means an engineered or machine-based system that varies in its level of autonomy and that can, for explicit or implicit objectives, infer from the input it receives how to generate outputs that can influence physical or virtual environments.
(b)Covered provider means a person that creates, codes, or otherwise produces a generative artificial intelligence system that has over 1,000,000 monthly visitors or users and is publicly accessible within the geographic boundaries of the state.
(c)Generative artificial intelligence intelligence system or GenAI GenAI system means an artificial intelligence that can generate derived synthetic content, including text, images, video, and audio, that emulates the structure and characteristics of the systems training data.
(d)Latent means present but not manifest.
(e)Manifest means easily perceived, understood, or recognized by a natural person.
(d)(f)Metadata means structural or descriptive information about data.
(e)(g)Personal information has the same meaning as defined in Section 1798.140 of the Civil Code.
(f)(h)Personal provenance data means provenance data that contains either of the following:
(1)Personal information.
(2)Unique device, system, or service information that is reasonably capable of being associated with a particular user.
(g)(i)Provenance data means data that is embedded into digital content, or that is included in the digital contents metadata, for the purpose of verifying the digital contents authenticity, origin, or history of modification.
(h)(j)System provenance data means provenance data that is not reasonably capable of being associated with a particular user and that contains either of the following:
(1)Information regarding the type of device, system, or service that was used to generate a piece of digital content.
(2)Information that provides proof of related to content authenticity.
22757.2.(a)A covered provider shall create and make freely available an AI detection tool at no cost to the user that meets all of the following criteria:
(1)The tool allows a user to assess whether image, video, audio, or audio content, or other digital content content that is any combination thereof, was created or altered by the covered providers GenAI system.
(2)The tool allows a user to determine which parts of the content were created or altered by the covered providers GenAI system.(3)(2)The tool outputs any system provenance data that is detected in the content.
(4)(3)The tool does not output any personal provenance data that is detected in the content.
(5)The tool is publicly accessible through the covered providers internet website and its mobile application, as applicable.(4)The tool is publicly accessible.
(6)(5)The tool allows a user to upload content or provide a uniform resource locator (URL) linking to online content.
(7)(6)The tool supports an application programming interface that allows a user to invoke the tool without visiting the covered providers internet website.
(b)A covered provider shall collect user feedback related to the efficacy of the covered providers AI detection tool and incorporate that relevant feedback into any attempt to improve the efficacy of the tool.
(c)A covered provider shall not do any of the following:
(1)(A)Except as provided in subparagraph (B), collect or retain personal information from users of the covered providers AI detection tool.
(B)(i)A covered provider may collect and retain the contact information of a user who submits feedback pursuant to subdivision (b) if the user opts in to being contacted by the covered provider.
(ii)User information collected pursuant to clause (i) shall be used only to evaluate and improve the efficacy of the covered providers AI detection tool.
(2)Retain any content submitted to the AI detection tool for longer than is necessary to comply with this section.
(3)Retain any personal provenance data from content submitted to the AI detection tool by a user.
22757.3.(a)A covered provider shall offer the user the option to include a manifest disclosure in image, video, audio, or other digital content or audio content, or content that is any combination thereof, created or altered by the covered providers GenAI system that meets all of the following criteria:
(1)The disclosure identifies content as AI-generated content.
(2)The disclosure is clear, conspicuous, appropriate for the medium of the content, and understandable to a reasonable person.
(3)The disclosure is permanent or extraordinarily difficult to remove, to the extent it is technically feasible.
(b)A covered provider shall include a latent disclosure in AI-generated image, video, audio, or other digital content or audio content, or content that is any combination thereof, created by the covered providers GenAI system that meets all of the following criteria:
(1)The To the extent that it is technically feasible, the disclosure conveys all of the following information, either directly or through a link to a permanent internet website:
(A)The name of the covered provider.
(B)The name and version number of the GenAI system that created or altered the content.
(C)The time and date of the contents creation or alteration.
(D)Which parts of the content were created or altered by the GenAI system.(E)(D)A unique identifier.
(2)The disclosure is detectable by the covered providers AI detection tool.
(3)The disclosure is consistent with widely accepted industry standards.
(4)The disclosure is permanent or extraordinarily difficult to remove, to the extent it is technically feasible.
(c)(1)If a covered provider licenses its GenAI system to a third party, the covered provider shall require by contract that the licensee maintain the systems capability to include a disclosure required by subdivision (b) in content the system creates or alters.
(2)If a covered provider knows that a third-party licensee modified a licensed GenAI system such that it is no longer capable of including a disclosure required by subdivision (b) in content the system creates or alters, the covered provider shall revoke the license within 72 hours of discovering the licensees action.
(3)A third-party licensee shall cease using a licensed GenAI system after the license for the system has been revoked by the covered provider pursuant to paragraph (2).
22757.4.(a)(1)A covered provider that violates this chapter shall be liable for a civil penalty in the amount of five thousand dollars ($5,000) per violation to be collected in a civil action filed by the Attorney General, a city attorney, or a county counsel.
(2)A prevailing plaintiff in an action brought pursuant to this subdivision shall be entitled to all reasonable attorneys costs and fees.
(b)Each day that a covered provider is in violation of this chapter shall be deemed a discrete violation.
(c)For a violation by a third-party licensee of paragraph (3) of subdivision (c) of Section 22757.3, the Attorney General, a county counsel, or a city attorney may bring a civil action for both of the following:
(1)Injunctive relief.
(2)Reasonable attorneys fees and costs.
22757.5.This chapter shall become operative on January 1, 2026.
This chapter shall be known as the California AI Transparency Act.
22757.1.As used in this chapter:
(a)Artificial intelligence or AI means an engineered or machine-based system that varies in its level of autonomy and that can, for explicit or implicit objectives, infer from the input it receives how to generate outputs that can influence physical or virtual environments.
(b)Covered provider means a person that creates, codes, or otherwise produces a generative artificial intelligence system that has over 1,000,000 monthly visitors or users and is publicly accessible within the geographic boundaries of the state.
(c)Generative artificial intelligence or GenAI means an artificial intelligence that can generate derived synthetic content, including text, images, video, and audio, that emulates the structure and characteristics of the systems training data.
(d)Metadata means structural or descriptive information about data.
(e)Personal information has the same meaning as defined in Section 1798.140 of the Civil Code.
(f)Personal provenance data means provenance data that contains either of the following:
(1)Personal information.
(2)Unique device, system, or service information that is reasonably capable of being associated with a particular user.
(g)Provenance data means data that is embedded into digital content, or that is included in the digital contents metadata, for the purpose of verifying the digital contents authenticity, origin, or history of modification.
(h)System provenance data means provenance data that is not reasonably capable of being associated with a particular user and that contains either of the following:
(1)Information regarding the type of device, system, or service that was used to generate a piece of digital content.
(2)Information that provides proof of content authenticity.
22757.2.(a)A covered provider shall create and make freely available an AI detection tool that meets all of the following criteria:
(1)The tool allows a user to assess whether image, video, audio, or other digital content was created or altered by the covered providers GenAI system.
(2)The tool allows a user to determine which parts of the content were created or altered by the covered providers GenAI system.
(3)The tool outputs any system provenance data that is detected in the content.
(4)The tool does not output any personal provenance data that is detected in the content.
(5)The tool is publicly accessible through the covered providers internet website and its mobile application, as applicable.
(6)The tool allows a user to upload content or provide a uniform resource locator (URL) linking to online content.
(7)The tool supports an application programming interface that allows a user to invoke the tool without visiting the covered providers internet website.
(b)A covered provider shall collect user feedback related to the efficacy of the covered providers AI detection tool and incorporate that feedback into any attempt to improve the efficacy of the tool.
(c)A covered provider shall not do any of the following:
(1)(A)Except as provided in subparagraph (B), collect or retain personal information from users of the covered providers AI detection tool.
(B)(i)A covered provider may collect and retain the contact information of a user who submits feedback pursuant to subdivision (b) if the user opts in to being contacted by the covered provider.
(ii)User information collected pursuant to clause (i) shall be used only to evaluate and improve the efficacy of the covered providers AI detection tool.
(2)Retain any content submitted to the AI detection tool for longer than is necessary to comply with this section.
(3)Retain any personal provenance data from content submitted to the AI detection tool by a user.
22757.3.(a)A covered provider shall offer the user the option to include a manifest disclosure in image, video, audio, or other digital content created or altered by the covered providers GenAI system that meets all of the following criteria:
(1)The disclosure identifies content as AI-generated content.
(2)The disclosure is clear, conspicuous, appropriate for the medium of the content, and understandable to a reasonable person.
(3)The disclosure is permanent or extraordinarily difficult to remove, to the extent it is technically feasible.
(b)A covered provider shall include a latent disclosure in AI-generated image, video, audio, or other digital content created by the covered providers GenAI system that meets all of the following criteria:
(1)The disclosure conveys all of the following information, either directly or through a link to a permanent internet website:
(A)The name of the covered provider.
(B)The name and version number of the GenAI system that created or altered the content.
(C)The time and date of the contents creation or alteration.
(D)Which parts of the content were created or altered by the GenAI system.
(E)A unique identifier.
(2)The disclosure is detectable by the covered providers AI detection tool.
(3)The disclosure is consistent with widely accepted industry standards.
(4)The disclosure is permanent or extraordinarily difficult to remove, to the extent it is technically feasible.
(c)(1)If a covered provider licenses its GenAI system to a third party, the covered provider shall require by contract that the licensee maintain the systems capability to include a disclosure required by subdivision (b) in content the system creates or alters.
(2)If a covered provider knows that a third-party licensee modified a licensed GenAI system such that it is no longer capable of including a disclosure required by subdivision (b) in content the system creates or alters, the covered provider shall revoke the license within 72 hours of discovering the licensees action.
(3)A third-party licensee shall cease using a licensed GenAI system after the license for the system has been revoked by the covered provider pursuant to paragraph (2).
22757.4.(a)(1)A covered provider that violates this chapter shall be liable for a civil penalty in the amount of five thousand dollars ($5,000) per violation to be collected in a civil action filed only by the Attorney General. Attorney General, a city attorney, or a county counsel.
(2)A prevailing plaintiff in an action brought pursuant to this subdivision shall be entitled to all reasonable attorneys costs and fees.
(b)Each day that a covered provider is in violation of this chapter shall be deemed a discrete violation.
(c)For a violation by a third-party licensee of paragraph (3) of subdivision (c) of Section 22757.3, the Attorney General, a district attorney, a county counsel, or a city attorney may bring a civil action for both of the following:
(1)Injunctive relief.
(2)Reasonable attorneys fees and costs.
This chapter shall be known as the California AI Transparency Act.
22757.1.As used in this chapter:
(a)AI detection tool means the tool required by Section 22757.2.(b)Artificial intelligence or AI means a machine-based system that can, for a given set of human-defined objectives, make predictions, recommendations, or decisions influencing real or virtual environments by using machine-based inputs and human-based inputs to perceive real and virtual environments, abstract its perceptions into models through analysis in an automated manner, and use model inference to formulate options for information or action.(c)AI-generated content means any form of digital content that is created with deep learning or machine learning processes.(d)Business has the same meaning as defined in Section 1798.140 of the Civil Code.(a)Artificial intelligence or AI means an engineered or machine-based system that varies in its level of autonomy and that can, for explicit or implicit objectives, infer from the input it receives how to generate outputs that can influence physical or virtual environments.
(e)(b)Covered provider is a business means a person that provides creates, codes, or otherwise produces a generative AI artificial intelligence system that has, on average over the preceding 12 months, has over 1,000,000 monthly visitors or users and is publicly accessible within the geographic boundaries of the state.
(f)Department means the Department of Technology.(g)(c)Generative AI system refers to deep learning models that can generate text, images, and other content based on the data they were trained on. artificial intelligence or GenAI means an artificial intelligence that can generate derived synthetic content, including text, images, video, and audio, that emulates the structure and characteristics of the systems training data.
(h)(d)Metadata means structural or descriptive information about data, including content, format, source, rights, accuracy, provenance, frequency, periodicity, granularity, publisher or responsible party, contact information, method of collection, and other descriptions. data.
(i)Model means a component of an information system that implements artificial intelligence technology and uses computational, statistical, or machine learning techniques to produce outputs from a given set of inputs.(j)Person means a natural person located within the geographic boundaries of the state.(k)(e)Personal information has the same meaning as defined in Section 1798.140 of the Civil Code.
(f)Personal provenance data means provenance data that contains either of the following:
(1)Personal information.
(2)Unique device, system, or service information that is reasonably capable of being associated with a particular user.
(g)Provenance data means data that is embedded into digital content, or that is included in the digital contents metadata, for the purpose of verifying the digital contents authenticity, origin, or history of modification.
(h)System provenance data means provenance data that is not reasonably capable of being associated with a particular user and that contains either of the following:
(1)Information regarding the type of device, system, or service that was used to generate a piece of digital content.
(2)Information that provides proof of content authenticity.
22757.2.(a)A covered provider shall create and make freely available an AI detection tool by which a person can query the covered provider as to the extent to which text, image, video, audio, or multimedia content was created, in whole or in part, by a generative AI system provided by the covered provider that meets all of the following criteria:
(1)The tool allows a user to assess whether image, video, audio, or other digital content was created or altered by the covered providers GenAI system.
(2)The tool allows a user to determine which parts of the content were created or altered by the covered providers GenAI system.
(3)The tool outputs any system provenance data that is detected in the content.
(4)The tool does not output any personal provenance data that is detected in the content.
(1)(5)The AI detection tool shall be is publicly accessible and available via a uniform resource locator (URL) on through the covered providers internet website and through its mobile application, as applicable.
(2)(6)The AI detection tool shall allow allows a person user to upload content or a URL. provide a uniform resource locator (URL) linking to online content.
(3)(7)The AI detection tool shall support supports an application programming interface that allows a person user to invoke the AI detection tool without visiting the covered providers internet website.
(4)The AI detection tool(b)A covered provider shall allow a person to provide collect user feedback if the person believes the AI detection tool is not properly identifying content that was created by the covered provider. related to the efficacy of the covered providers AI detection tool and incorporate that feedback into any attempt to improve the efficacy of the tool.
(b)In complying with this section, a(c)A covered provider shall not do any of the following:
(1)Reveal personal information that identifies who utilized the covered providers generative AI system to create AI-generated content that was submitted to the covered providers AI detection tool.(2)(1)(A)Subject to Except as provided in subparagraph (B), collect and or retain personal information when a person utilizes from users of the covered providers AI detection tool.
(B)(i)A covered provider may collect and retain the contact information of a person user who submitted submits feedback pursuant to paragraph (4) of subdivision (a). subdivision (b) if the user opts in to being contacted by the covered provider.
(ii)User information collected pursuant to clause (i) shall be used only to evaluate and improve the efficacy of the covered providers AI detection tool.
(3)(2)Retain any content submitted to the AI detection tool for longer than is necessary to comply with this section.
(3)Retain any personal provenance data from content submitted to the AI detection tool by a user.
22757.3.(a)A covered provider shall offer the user the option to include in AI-generated a manifest disclosure in image, text, video, audio, or multimedia other digital content created or altered by a generative AI the covered providers GenAI system it provides a visible disclosure that meets all of the following criteria:
(1)The disclosure shall include a clear and conspicuous notice, as appropriate for the medium of the content, that identifies the content as generated by AI, such that the disclosure is not avoidable, is understandable to a reasonable person, and is not contradicted, mitigated by, or inconsistent with anything else in the communication.(1)The disclosure identifies content as AI-generated content.
(2)The disclosure is clear, conspicuous, appropriate for the medium of the content, and understandable to a reasonable person.
(2)(3)The disclosure shall, to the extent technically feasible, be is permanent or extraordinarily difficult to remove. remove, to the extent it is technically feasible.
(3)The outputs metadata information shall include an identification of the content as being generated by AI, the identity of the tool used to create the content, and the date and time the content was created.(b)A covered provider shall include in AI-generated image, audio, video, or multimedia content created by a generative AI system an imperceptible disclosure that is machine detectable and is, to the extent technically feasible, permanent or difficult to remove.(c)A covered provider shall implement reasonable procedures to prevent downstream use of a generative AI system it provides without the disclosure required by this section, including by doing both of the following:(1)Requiring by contract that third-party licensees of the generative AI system refrain from removing a required disclosure.(2)Terminating access to the generative AI system when the covered provider has reason to believe that a third-party licensee has removed a required disclosure.(d)At least once every two years, the department shall review this section and make recommendations to the Legislature regarding any amendments needed to account for changing technology and standards.(b)A covered provider shall include a latent disclosure in AI-generated image, video, audio, or other digital content created by the covered providers GenAI system that meets all of the following criteria:
(1)The disclosure conveys all of the following information, either directly or through a link to a permanent internet website:
(A)The name of the covered provider.
(B)The name and version number of the GenAI system that created or altered the content.
(C)The time and date of the contents creation or alteration.
(D)Which parts of the content were created or altered by the GenAI system.
(E)A unique identifier.
(2)The disclosure is detectable by the covered providers AI detection tool.
(3)The disclosure is consistent with widely accepted industry standards.
(4)The disclosure is permanent or extraordinarily difficult to remove, to the extent it is technically feasible.
(c)(1)If a covered provider licenses its GenAI system to a third party, the covered provider shall require by contract that the licensee maintain the systems capability to include a disclosure required by subdivision (b) in content the system creates or alters.
(2)If a covered provider knows that a third-party licensee modified a licensed GenAI system such that it is no longer capable of including a disclosure required by subdivision (b) in content the system creates or alters, the covered provider shall revoke the license within 72 hours of discovering the licensees action.
(3)A third-party licensee shall cease using a licensed GenAI system after the license for the system has been revoked by the covered provider pursuant to paragraph (2).
22757.4.(a)A covered provider that violates this chapter shall be liable for a civil penalty in the amount of five thousand dollars ($5,000) per violation to be collected in a civil action filed only by the Attorney General.
(b)Each day that a covered provider is in violation of this chapter shall be deemed a discrete violation.
(c)For a violation by a third-party licensee of paragraph (3) of subdivision (c) of Section 22757.3, the Attorney General, a district attorney, a county counsel, or a city attorney may bring a civil action for both of the following:
(1)Injunctive relief.
(2)Reasonable attorneys fees and costs.
This chapter shall be known as the California AI Transparency Act.
22757.1.As used in this chapter:
(a)AI detection tool means the tool required by Section 22757.2.
(b)Artificial intelligence or AI means a machine-based system that can, for a given set of human-defined objectives, make predictions, recommendations, or decisions influencing real or virtual environments by using machine-based inputs and human-based inputs to perceive real and virtual environments, abstract its perceptions into models through analysis in an automated manner, and use model inference to formulate options for information or action.
(c)AI-generated content means any form of digital content that is created with deep learning or machine learning processes.
(d)Business has the same meaning as defined in Section 1798.140 of the Civil Code.
(e)Covered provider is a business that provides a generative AI system that has, on average over the preceding 12 months, over 1,000,000 monthly visitors or users and is publicly accessible within the geographic boundaries of the state.
(f)Department means the Department of Technology.
(g)Generative AI system refers to deep learning models that can generate text, images, and other content based on the data they were trained on.
(h)Metadata means structural or descriptive information about data, including content, format, source, rights, accuracy, provenance, frequency, periodicity, granularity, publisher or responsible party, contact information, method of collection, and other descriptions.
(i)Model means a component of an information system that implements artificial intelligence technology and uses computational, statistical, or machine learning techniques to produce outputs from a given set of inputs.
(j)Person means a natural person located within the geographic boundaries of the state.
(k)Personal information has the same meaning as defined in Section 1798.140 of the Civil Code.
(l)Registry means the Generative AI Registry created by the department pursuant to Section 22757.4.22757.2.(a)A covered provider shall create an AI detection tool by which a person can query the covered provider as to the extent to which text, image, video, audio, or multimedia content was created, in whole or in part, by a generative AI system provided by the covered provider that meets all of the following criteria:
(1)The AI detection tool shall be publicly accessible and available via a uniform resource locator (URL) on the covered providers internet website and through its mobile application, as applicable.
(2)The AI detection tool shall allow a person to upload content or a URL.
(3)The AI detection tool shall support an application programming interface that allows a person to invoke the AI detection tool without visiting the covered providers internet website.
(4)The AI detection tool shall allow a person to provide feedback if the person believes the AI detection tool is not properly identifying content that was created by the covered provider.
(b)In complying with this section, a covered provider shall not do any of the following:
(1)Reveal personal information that identifies who utilized the covered providers generative AI system to create AI-generated content that was submitted to the covered providers AI detection tool.
(2)(A)Subject to subparagraph (B), collect and retain personal information when a person utilizes the covered providers AI detection tool.
(B)A covered provider may collect and retain the contact information of a person who submitted feedback pursuant to paragraph (4) of subdivision (a).
(3)Retain any content submitted to the AI detection tool for longer than is necessary to comply with this section.
22757.3.(a)A covered provider shall include in AI-generated image, text, video, or multimedia content created by a generative AI system it provides a visible disclosure that meets all of the following criteria:
(1)The disclosure shall include a clear and conspicuous notice, as appropriate for the medium of the content, that identifies the content as generated by AI, such that the disclosure is not avoidable, is understandable to a reasonable person, and is not contradicted, mitigated by, or inconsistent with anything else in the communication.
(2)The disclosure shall, to the extent technically feasible, be permanent or difficult to remove.
(3)The outputs metadata information shall include an identification of the content as being generated by AI, the identity of the tool used to create the content, and the date and time the content was created.
(b)A covered provider shall include in AI-generated image, audio, video, or multimedia content created by a generative AI system an imperceptible disclosure that is machine detectable and is, to the extent technically feasible, permanent or difficult to remove.
(c)A covered provider shall implement reasonable procedures to prevent downstream use of a generative AI system it provides without the disclosure required by this section, including by doing both of the following:
(1)Requiring by contract that third-party licensees of the generative AI system refrain from removing a required disclosure.
(2)Terminating access to the generative AI system when the covered provider has reason to believe that a third-party licensee has removed a required disclosure.
(d)At least once every two years, the department shall review this section and make recommendations to the Legislature regarding any amendments needed to account for changing technology and standards.
22757.4.(a)A covered provider shall register with the department and provide to the department a URL to any AI detection tool it has created.(b)(1)The department shall create and display on its internet website the Generative AI Registry.(2)The registry shall display both of the following:(A)The name of any covered provider registered with the department.(B)A link to the covered providers AI detection tool.(c)(1)The department may adopt regulations necessary to perform its duties under this section.(2)The department may charge a registration fee, which shall be deposited into the Generative AI Registry Fund, to a covered provider, but only to the extent that the registration fee is necessary to cover the costs of administering the registry.22757.5.(a)There is hereby created in the State Treasury the Generative AI Registry Fund.(b)Moneys in the fund shall be made available, only upon appropriation by the Legislature, to the department for the purposes of this chapter.22757.6. 22757.4.(a)A covered provider that violates this chapter shall be liable for a civil penalty in the amount of five thousand dollars ($5,000) per violation to be collected in a civil action filed only by the Attorney General.
(b)Each day that a covered provider is in violation of this chapter shall be deemed a discrete violation.
This chapter shall be known as the California AI Transparency Act.
22757.1.As used in this chapter:
(a)AI detection tool means the tool required by Section 22757.2.
(b)Artificial intelligence or AI means a machine-based system that can, for a given set of human-defined objectives, make predictions, recommendations, or decisions influencing real or virtual environments by using machine-based inputs and human-based inputs to perceive real and virtual environments, abstract its perceptions into models through analysis in an automated manner, and use model inference to formulate options for information or action.
(c)AI-generated content means any form of digital content that is created with deep learning or machine learning processes.
(d)Business has the same meaning as defined in Section 1798.140 of the Civil Code.
(e)Covered provider is a business that provides a generative AI system that has, on average over the preceding 12 months, over 1,000,000 monthly visitors or users and is publicly accessible within the geographic boundaries of the state.
(f)Department means the Department of Technology.
(g)Generative AI system refers to deep learning models that can generate text, images, and other content based on the data they were trained on.
(h)Metadata means structural or descriptive information about data, including content, format, source, rights, accuracy, provenance, frequency, periodicity, granularity, publisher or responsible party, contact information, method of collection, and other descriptions.
(i)Model means a component of an information system that implements artificial intelligence technology and uses computational, statistical, or machine learning techniques to produce outputs from a given set of inputs.
(j)Person means a natural person located within the geographic boundaries of the state.
(k)Personal information has the same meaning as defined in Section 1798.140 of the Civil Code.
(l)Registry means the Generative AI Registry created by the department pursuant to Section 22757.4.
22757.2.(a)A covered provider shall create an AI detection tool by which a person can query the covered provider as to the extent to which text, image, video, audio, or multimedia content was created, in whole or in part, by a generative AI system provided by the covered provider that meets all of the following criteria:
(1)The AI detection tool shall be publicly accessible and available via a uniform resource locator (URL) on the covered providers internet website and through its mobile application, as applicable.
(2)The AI detection tool shall allow a person to upload content or a URL.
(3)The AI detection tool shall support an application programming interface that allows a person to invoke the AI detection tool without visiting the covered providers internet website.
(4)The AI detection tool shall allow a person to provide feedback if the person believes the AI detection tool is not properly identifying content that was created by the covered provider.
(b)In complying with this section, a covered provider shall not do any of the following:
(1)Reveal personal information that identifies who utilized the covered providers generative AI system to create AI-generated content that was submitted to the covered providers AI detection tool.
(2)(A)Subject to subparagraph (B), collect and retain personal information when a person utilizes the covered providers AI detection tool.
(B)A covered provider may collect and retain the contact information of a person who submitted feedback pursuant to paragraph (4) of subdivision (a).
(3)Retain any content submitted to the AI detection tool for longer than is necessary to comply with this section.
22757.3.(a)A covered provider shall include in AI-generated image, text, video, or multimedia content created by a generative AI system it provides a visible disclosure that meets all of the following criteria:
(1)The disclosure shall include a clear and conspicuous notice, as appropriate for the medium of the content, that identifies the content as generated by AI, such that the disclosure is not avoidable, is understandable to a reasonable person, and is not contradicted, mitigated by, or inconsistent with anything else in the communication.
(2)The disclosure shall, to the extent technically feasible, be permanent or difficult to remove.
(3)The outputs metadata information shall include an identification of the content as being generated by AI, the identity of the tool used to create the content, and the date and time the content was created.
(b)A covered provider shall include in AI-generated image, audio, video, or multimedia content created by a generative AI system an imperceptible disclosure that is machine detectable and is, to the extent technically feasible, permanent or difficult to remove.
(c)A covered provider shall implement reasonable procedures to prevent downstream use of a generative AI system it provides without the disclosure required by this section, including by doing both of the following:
(1)Requiring by contract that third-party licensees of the generative AI system refrain from removing a required disclosure.
(2)Terminating access to the generative AI system when the covered provider has reason to believe that a third-party licensee has removed a required disclosure.
(d)At least once every two years, the department shall review this section and make recommendations to the Legislature regarding any amendments needed to account for changing technology and standards.
22757.4.(a)A covered provider shall register with the department and provide to the department a URL to any AI detection tool it has created.
(b)(1)The department shall create and display on its internet website the Generative AI Registry.
(2)The registry shall display both of the following:
(A)The name of any covered provider registered with the department.
(B)A link to the covered providers AI detection tool.
(c)(1)The department may adopt regulations necessary to perform its duties under this section.
(2)The department may charge a registration fee, which shall be deposited into the Generative AI Registry Fund, to a covered provider, but only to the extent that the registration fee is necessary to cover the costs of administering the registry.
22757.5.(a)There is hereby created in the State Treasury the Generative AI Registry Fund.
(b)Moneys in the fund shall be made available, only upon appropriation by the Legislature, to the department for the purposes of this chapter.
22757.6.(a)A covered provider that violates this chapter shall be liable for a civil penalty in the amount of five thousand dollars ($5,000) per violation to be collected in a civil action filed only by the Attorney General.
(b)Each day that a covered provider is in violation of this chapter shall be deemed a discrete violation.
SECTION 1.It is the intent of the Legislature to enact legislation that would establish a mechanism to allow consumers to easily determine whether images, audio, video, or text was created by generative artificial intelligence.It is the intent of the Legislature to enact legislation that would establish a mechanism to allow consumers to easily determine whether images, audio, video, or text was created by generative artificial intelligence.
|
Date
|
Status
|
|---|---|
| Sep 19, 2024 | Chaptered by Secretary of State. Chapter 291, Statutes of 2024. |
| Sep 19, 2024 | Approved by the Governor. |
| Sep 9, 2024 | Enrolled and presented to the Governor at 3 p.m. |
| Aug 29, 2024 | Assembly amendments concurred in. (Ayes 35. Noes 2. Page 5636.) Ordered to engrossing and enrolling. |
| Aug 28, 2024 | In Senate. Concurrence in Assembly amendments pending. |
| Aug 28, 2024 | Read third time. Passed. (Ayes 73. Noes 0. Page 6715.) Ordered to the Senate. |
| Aug 23, 2024 | Ordered to third reading. |
| Aug 23, 2024 | Read third time and amended. |
| Aug 20, 2024 | Read second time. Ordered to third reading. |
| Aug 19, 2024 | Read second time and amended. Ordered to second reading. |
| Aug 15, 2024 | From committee: Do pass as amended. (Ayes 11. Noes 0.) (August 15). |
| Aug 7, 2024 | August 7 set for first hearing. Placed on suspense file. |
| Jul 3, 2024 | Read second time and amended. Re-referred to Com. on APPR. |
| Jul 2, 2024 | From committee: Do pass as amended and re-refer to Com. on APPR. (Ayes 9. Noes 1.) (July 2). |
| Jun 20, 2024 | Read second time and amended. Re-referred to Com. on JUD. |
| Jun 19, 2024 | From committee: Do pass as amended and re-refer to Com. on JUD. (Ayes 8. Noes 0.) (June 18). |
| Jun 3, 2024 | Referred to Coms. on P. & C.P. and JUD. |
| May 22, 2024 | In Assembly. Read first time. Held at Desk. |
| May 21, 2024 | Read third time. Passed. (Ayes 32. Noes 1. Page 4074.) Ordered to the Assembly. |
| May 20, 2024 | Read second time. Ordered to third reading. |
| May 16, 2024 | Read second time and amended. Ordered to second reading. |
| May 16, 2024 | From committee: Do pass as amended. (Ayes 5. Noes 0. Page 3964.) (May 16). |
| May 10, 2024 | Set for hearing May 16. |
| May 6, 2024 | May 6 hearing: Placed on APPR suspense file. |
| Apr 26, 2024 | Set for hearing May 6. |
| Apr 23, 2024 | From committee: Do pass and re-refer to Com. on APPR. (Ayes 11. Noes 1. Page 3721.) (April 23). Re-referred to Com. on APPR. |
| Apr 22, 2024 | Set for hearing April 23. |
| Apr 17, 2024 | From committee: Do pass and re-refer to Com. on G.O. (Ayes 10. Noes 1. Page 3634.) (April 16). Re-referred to Com. on G.O. |
| Apr 5, 2024 | Set for hearing April 16. |
| Apr 3, 2024 | Re-referred to Coms. on JUD. and G.O. |
| Mar 20, 2024 | From committee with author's amendments. Read second time and amended. Re-referred to Com. on RLS. |
| Feb 14, 2024 | Referred to Com. on RLS. |
| Jan 18, 2024 | From printer. May be acted upon on or after February 17. |
| Jan 17, 2024 | Introduced. Read first time. To Com. on RLS. for assignment. To print. |
|
Date
|
Description
|
Location
|
Votes
|
|
|---|---|---|---|---|
| Aug 29, 2024 | Unfinished Business SB942 Becker Concurrence | Senate Floor |
Passed |
|
| Aug 28, 2024 | SB 942 Becker Senate Third Reading By Kalra | Assembly Floor |
Passed |
|
| Aug 15, 2024 | Do pass as amended. | Assembly Standing Committee on Appropriations |
Passed |
|
| Jul 2, 2024 | Do pass as amended and be re-referred to the Committee on [Appropriations] | Assembly Standing Committee on Judiciary |
Passed |
|
| Jun 18, 2024 | Do pass as amended and be re-referred to the Committee on [Judiciary] | Assembly Standing Committee on Privacy and Consumer Protection |
Passed |
|
| May 21, 2024 | Senate 3rd Reading SB942 Becker | Senate Floor |
Passed |
|
| May 16, 2024 | Do pass as amended | Senate Standing Committee on Appropriations |
Passed |
|
| May 6, 2024 | Placed on suspense file | Senate Standing Committee on Appropriations |
Passed |
|
| Apr 23, 2024 | Do pass, but first be re-referred to the Committee on [Appropriations] | Senate Standing Committee on Governmental Organization |
Passed |
|
| Apr 16, 2024 | Do pass, but first be re-referred to the Committee on [Governmental Organization] | Senate Standing Committee on Judiciary |
Passed |
This list shows positions taken on this bill by organizations on the date displayed, which is linked to the original source.
Positions by individuals are not included.
|
Organization
|
Position
|
Date
|
|---|---|---|
| California Chamber Of Commerce | ** Against | |
| Computer And Communications Industry Association | ** Against | |
| NetChoice | ** Against | |
| Oakland Privacy | ** Support | |
| Protection of the Educational Rights of Kids - Advocacy | ** Support | |
| Technet | ** Against | |
| Transparency Coalition.ai | ** Support |
SENATE RULES COMMITTEE Office of Senate Floor Analyses (916) 651-1520 Fax: (916) 327-4478 | SB 942 |
UNFINISHED BUSINESS
Bill No: SB 942
Author: Becker (D)
Amended: 8/23/24
Vote: 21
SENATE JUDICIARY COMMITTEE: 10-1, 4/16/24
AYES: Umberg, Allen, Ashby, Caballero, Durazo, Laird, Min, Niello, Stern, Wahab
NOES: Wilk
SENATE GOVERNMENTAL ORG. COMMITTEE: 11-1, 4/23/24
AYES: Dodd, Alvarado-Gil, Archuleta, Ashby, Bradford, Glazer, Padilla, Portantino, Roth, Rubio, Smallwood-Cuevas
NOES: Wilk
NO VOTE RECORDED: Jones, Nguyen, Ochoa Bogh, Seyarto
SENATE APPROPRIATIONS COMMITTEE: 5-0, 5/16/24
AYES: Caballero, Ashby, Becker, Bradford, Wahab
NO VOTE RECORDED: Jones, Seyarto
SENATE FLOOR: 32-1, 5/21/24
AYES: Alvarado-Gil, Archuleta, Ashby, Atkins, Becker, Blakespear, Bradford, Caballero, Cortese, Dodd, Durazo, Glazer, Gonzalez, Hurtado, Laird, Limón, McGuire, Menjivar, Min, Newman, Niello, Ochoa Bogh, Padilla, Portantino, Roth, Rubio, Skinner, Smallwood-Cuevas, Stern, Umberg, Wahab, Wiener
NOES: Wilk
NO VOTE RECORDED: Allen, Dahle, Eggman, Grove, Jones, Nguyen, Seyarto
ASSEMBLY FLOOR: 73-0, 8/28/24 - See last page for vote
SUBJECT: California AI Transparency Act
SOURCE: Author
DIGEST:
This bill places obligations on businesses that provide generative artificial intelligence (AI) systems to make accessible tools to detect whether specified content was generated by those systems. These “covered providers” are required to offer visible, and include imperceptible, markings on AI-generated content to identify it as such.
Assembly Amendments
amend the relevant definitions of the bill, remove some requirements on providers, expand exemptions, and delay the operative date.
ANALYSIS:
Existing law:
- Defines “deepfake” as audio or visual content that has been generated or manipulated by artificial intelligence which would falsely appear to be authentic or truthful and which features depictions of people appearing to say or do things they did not say or do without their consent. (Gov't Code § 11547.5.)
- Defines “digital content forgery” as the use of technologies, including artificial intelligence and machine learning techniques, to fabricate or manipulate audio, visual, or text content with the intent to mislead. (Gov't Code § 11547.5.)
- Defines “digital content provenance” as the verifiable chronology of the original piece of digital content, such as an image, video, audio recording, or electronic document. (Gov't Code § 11547.5.)
- Requires, upon appropriation by the Legislature, the Secretary of Government Operations Agency (GovOps) to evaluate, among other things, the impact of the proliferation of deepfakes on state government, California-based businesses, and residents of the state.
- The risks, including privacy risks, associated with the deployment of digital content forgery technologies and deepfakes on state and local government, California-based businesses, and residents of the state.
- Potential privacy impacts of technologies allowing public verification of digital content provenance.
- The impact of digital content forgery technologies and deepfakes on civic engagement, including voters.
- The legal implications associated with the use of digital content forgery technologies, deepfakes, and technologies allowing public verification of digital content provenance.
- The best practices for preventing digital content forgery and deepfake technology to benefit the state, California-based businesses, and California residents, including exploring whether and how the adoption of a digital content provenance standard could assist with reducing the proliferation of digital content forgeries and deepfakes. (Gov’t Code § 11547.5(b).)
- Requires the Secretary of GovOps to develop a coordinated plan to accomplish all of the following:
- Investigate the feasibility of, and obstacles to, developing standards and technologies for state departments for determining digital content provenance.
- Increase the ability of internet companies, journalists, watchdog organizations, other relevant entities, and members of the public to meaningfully scrutinize and identify digital content forgeries and relay trust and information about digital content provenance to content consumers.
- Develop or identify mechanisms for content creators to cryptographically certify authenticity of original media and nondeceptive manipulations.
- Develop or identify mechanisms for content creators to enable the public to validate the authenticity of original media and nondeceptive manipulations to establish digital content provenance without materially compromising personal privacy or civil liberties. (Gov’t Code § 11547.5(c).)
- Establishes CDT within GovOps, under the supervision of the Director of Technology (Director), also known as the State Chief Information Officer. (Gov. Code Sec. 11545(a).)
- Provides that the duties of the Director include:
- Advising the Governor on the strategic management and direction of the state’s information technology (IT) resources.
- Establishing and enforcing state IT strategic plans, policies, standards, and enterprise architecture, as specified.
- Minimizing overlap, redundancy, and cost in state IT operations by promoting the efficient and effective use of information technology.
- Providing technology direction to agency and department chief information officers to ensure the integration of statewide technology initiatives, compliance with IT policies and standards, and the promotion of the alignment and effective management of IT services.
- Working to improve organizational maturity and capacity in the effective management of IT; and establishing performance management and improvement processes to ensure state IT systems and services are efficient and effective. (Gov. Code § 11545(b).)
This bill:
- Establishes the California AI Transparency Act.
- Requires a covered provider to make an AI detection tool available at no cost by which a person can assess whether content was created or altered by the provider’s GenAI system.
- Prohibits a covered provider from doing any of the following in carrying out the duties above:
- Collect or retain personal information when a person utilizes the covered provider’s AI detection tool, except that it may collect and retain the contact information of a person who submitted feedback.
- Retain any content submitted to the AI detection tool for longer than is necessary to comply with this law.
- Requires a covered provider to offer users the option to include in AI-generated image, video, or audio content created by its own generative AI system a manifest disclosure that meets specified criteria, including that it identifies the content as AI-generated content.
- Requires a covered provider to include in AI-generated image, audio, and video content created by its generative AI system a latent disclosure that is detectable by the tool specified above and is, to the extent technically feasible, permanent or extraordinarily difficult to remove.
- Provides that a covered provider that violates the above provisions is liable for a civil penalty in the amount of $5,000 per violation to be collected in a civil action filed by the Attorney General, a city attorney, or a county counsel. Each day that a covered provider is in violation shall be deemed a discrete violation.
- Provides that it does not apply to any product, service, internet website, or application that provides exclusively non-user-generated video game, television, streaming, movie, or interactive experiences.
- Becomes operative on January 1, 2026.
Background
Certain forms of media – audio recordings, video recordings, and still images – can be powerful evidence of the truth. While such media have always been susceptible to some degree of manipulation, fakes were relatively easy to detect. The rapid advancement of AI technology, specifically the wide-scale introduction of generative AI models, has made it drastically cheaper and easier to produce synthetic content created, audio, images, text, and video recordings that are not real, but that are so realistic that they are virtually impossible to distinguish from authentic content, including so-called “deepfakes.”
This bill works to ensure that providers of these GenAI systems are equipping consumers with a tool to identify when specific content has been generated by their systems. This required tool must be made available to the public, as provided. Covered providers are also required to place latent disclsoures on this generated content to identify it as such and to be able to trace it back to the covered providers’ systems. Providers must also offer users the option to place manifest disclosures on content generated by these systems. Concerns have been raised that the relevant technology to achieve this is still in its nascent stage and its reliability is insufficient to impose mandates based on it.
This bill is author-sponsored. The bill is supported by Transparency Coalition.AI. No opposition has been received. For a more thorough discussion of the bill, please see the Senate Judiciary Committee analysis, which is incorporated herein by reference.
Comment
The author states:
Generative Artificial Intelligence (AI) is advancing at an unprecedented pace, revolutionizing industries and everyday life. However, this rapid progress has also sparked a pressing need for regulatory frameworks to govern its use. While AI offers innovative solutions that drive economic growth and efficiency, it simultaneously presents complex challenges related to misinformation, manipulation, bias, and ownership of generated content.
One of the most concerning aspects of unregulated AI models is their potential to produce “deepfake” content, such as manipulated images and videos, which can be used to deceive or manipulate individuals and society at large. Deepfake pornography and fabricated political messages have already emerged as serious threats, highlighting the urgency of implementing effective regulations.
As AI-generated content approaches a level of realism that makes it indistinguishable from genuine content, the need for transparency becomes paramount. It is crucial for industry leaders to take responsibility for clearly labeling AI-generated content and developing tools that can accurately identify such content, especially when there is no visible disclosure. This transparency is essential for maintaining trust in AI technologies and ensuring that users can make informed decisions about the content they consume.
SB 942 stands as a critical legislative response in our ongoing regulatory arms race with artificial intelligence.
FISCAL EFFECT: Appropriation: No Fiscal Com.: Yes Local: No
According to the Senate Appropriations Committee:
- Unknown cost for CDT to review covered providers’ disclosures at least once every two years and provide recommendations based on changing technology and standards. To the extent that CDT would need additional or specialized staff to perform oversight activities related to reviewing disclosures, there may be significant General Fund costs.
- The Department of Justice (DOJ) reports costs of $359,000 for 3.0 personnel years (PYs) in Fiscal Year (FY) 2024-25 and $637,000 for 3.0 PYs in FY 2025-26 and ongoing for its Consumer Protection Section within the Public Rights Division to address an anticipated increase in litigation and other potential legal challenges (Unfair Competition Law Fund).
According to the Assembly Appropriations Committee:
- Cost pressures (General Fund) to the Department of Justice (DOJ) of an unknown but potentially significant amount if the Attorney General brings civil actions to enforce violations of this bill. If DOJ files enforcement actions under this bill, the department may experience a significant workload increase in excess of $150,000 annually, depending on the number of enforcement actions filed and the amount of time needed for each case. If DOJ chooses not to pursue enforcement as authorized by this bill, DOJ would likely not incur any costs.
- Cost pressures (Trial Court Trust Fund, General Fund) of an unknown but potentially significant amount to the courts to adjudicate civil enforcement actions filed to enforce this bill. Actual costs will depend on the number of cases filed and the amount of court time needed to resolve each case. It generally costs approximately $1,000 to operate a courtroom for one hour. Although courts are not funded on the basis of workload, increased pressure on the Trial Court Trust Fund may create a need for increased funding for courts from the General Fund. The 2024-25 state budget provides $37.3 million ongoing General Fund to backfill declining revenue to the Trial Court Trust Fund.
According to the Legislative Analyst’s Office, the General Fund faces a structural deficit in the tens of billions of dollars over the next several fiscal years.
SUPPORT: (Verified 8/28/24)
Transparency Coalition.AI
OPPOSITION: (Verified 8/28/24)
None received
ARGUMENTS IN SUPPORT: Transparency Coalition.AI writes:
SB 942 will provide the public with critical insight into which content (text, image, video, audio, or multimedia content) was created, in whole or in part, by a generative AI system.
Prepared by: Christian Kurpiewski / JUD. / (916) 651-4113
8/29/24 13:32:05
**** END ****
SENATE RULES COMMITTEE Office of Senate Floor Analyses (916) 651-1520 Fax: (916) 327-4478 | SB 942 |
UNFINISHED BUSINESS
Bill No: SB 942
Author: Becker (D)
Amended: 8/23/24
Vote: 21
SENATE JUDICIARY COMMITTEE: 10-1, 4/16/24
AYES: Umberg, Allen, Ashby, Caballero, Durazo, Laird, Min, Niello, Stern, Wahab
NOES: Wilk
SENATE GOVERNMENTAL ORG. COMMITTEE: 11-1, 4/23/24
AYES: Dodd, Alvarado-Gil, Archuleta, Ashby, Bradford, Glazer, Padilla, Portantino, Roth, Rubio, Smallwood-Cuevas
NOES: Wilk
NO VOTE RECORDED: Jones, Nguyen, Ochoa Bogh, Seyarto
SENATE APPROPRIATIONS COMMITTEE: 5-0, 5/16/24
AYES: Caballero, Ashby, Becker, Bradford, Wahab
NO VOTE RECORDED: Jones, Seyarto
SENATE FLOOR: 32-1, 5/21/24
AYES: Alvarado-Gil, Archuleta, Ashby, Atkins, Becker, Blakespear, Bradford, Caballero, Cortese, Dodd, Durazo, Glazer, Gonzalez, Hurtado, Laird, Limón, McGuire, Menjivar, Min, Newman, Niello, Ochoa Bogh, Padilla, Portantino, Roth, Rubio, Skinner, Smallwood-Cuevas, Stern, Umberg, Wahab, Wiener
NOES: Wilk
NO VOTE RECORDED: Allen, Dahle, Eggman, Grove, Jones, Nguyen, Seyarto
ASSEMBLY VOTE: 48-0, 8/28/24 – Roll call vote not available
SUBJECT: California AI Transparency Act
SOURCE: Author
DIGEST:
This bill places obligations on businesses that provide generative artificial intelligence (AI) systems to make accessible tools to detect whether specified content was generated by those systems. These “covered providers” are required to offer visible, and include imperceptible, markings on AI-generated content to identify it as such.
Assembly Amendments
amend the relevant definitions of the bill, remove some requirements on providers, expand exemptions, and delay the operative date.
ANALYSIS:
Existing law:
- Defines “deepfake” as audio or visual content that has been generated or manipulated by artificial intelligence which would falsely appear to be authentic or truthful and which features depictions of people appearing to say or do things they did not say or do without their consent. (Gov't Code § 11547.5.)
- Defines “digital content forgery” as the use of technologies, including artificial intelligence and machine learning techniques, to fabricate or manipulate audio, visual, or text content with the intent to mislead. (Gov't Code § 11547.5.)
- Defines “digital content provenance” as the verifiable chronology of the original piece of digital content, such as an image, video, audio recording, or electronic document. (Gov't Code § 11547.5.)
- Requires, upon appropriation by the Legislature, the Secretary of Government Operations Agency (GovOps) to evaluate, among other things, the impact of the proliferation of deepfakes on state government, California-based businesses, and residents of the state.
- The risks, including privacy risks, associated with the deployment of digital content forgery technologies and deepfakes on state and local government, California-based businesses, and residents of the state.
- Potential privacy impacts of technologies allowing public verification of digital content provenance.
- The impact of digital content forgery technologies and deepfakes on civic engagement, including voters.
- The legal implications associated with the use of digital content forgery technologies, deepfakes, and technologies allowing public verification of digital content provenance.
- The best practices for preventing digital content forgery and deepfake technology to benefit the state, California-based businesses, and California residents, including exploring whether and how the adoption of a digital content provenance standard could assist with reducing the proliferation of digital content forgeries and deepfakes. (Gov’t Code § 11547.5(b).)
- Requires the Secretary of GovOps to develop a coordinated plan to accomplish all of the following:
- Investigate the feasibility of, and obstacles to, developing standards and technologies for state departments for determining digital content provenance.
- Increase the ability of internet companies, journalists, watchdog organizations, other relevant entities, and members of the public to meaningfully scrutinize and identify digital content forgeries and relay trust and information about digital content provenance to content consumers.
- Develop or identify mechanisms for content creators to cryptographically certify authenticity of original media and nondeceptive manipulations.
- Develop or identify mechanisms for content creators to enable the public to validate the authenticity of original media and nondeceptive manipulations to establish digital content provenance without materially compromising personal privacy or civil liberties. (Gov’t Code § 11547.5(c).)
- Establishes CDT within GovOps, under the supervision of the Director of Technology (Director), also known as the State Chief Information Officer. (Gov. Code Sec. 11545(a).)
- Provides that the duties of the Director include:
- Advising the Governor on the strategic management and direction of the state’s information technology (IT) resources.
- Establishing and enforcing state IT strategic plans, policies, standards, and enterprise architecture, as specified.
- Minimizing overlap, redundancy, and cost in state IT operations by promoting the efficient and effective use of information technology.
- Providing technology direction to agency and department chief information officers to ensure the integration of statewide technology initiatives, compliance with IT policies and standards, and the promotion of the alignment and effective management of IT services.
- Working to improve organizational maturity and capacity in the effective management of IT; and establishing performance management and improvement processes to ensure state IT systems and services are efficient and effective. (Gov. Code § 11545(b).)
This bill:
- Establishes the California AI Transparency Act.
- Requires a covered provider to make an AI detection tool available at no cost by which a person can assess whether content was created or altered by the provider’s GenAI system.
- Prohibits a covered provider from doing any of the following in carrying out the duties above:
- Collect or retain personal information when a person utilizes the covered provider’s AI detection tool, except that it may collect and retain the contact information of a person who submitted feedback.
- Retain any content submitted to the AI detection tool for longer than is necessary to comply with this law.
- Requires a covered provider to offer users the option to include in AI-generated image, video, or audio content created by its own generative AI system a manifest disclosure that meets specified criteria, including that it identifies the content as AI-generated content.
- Requires a covered provider to include in AI-generated image, audio, and video content created by its generative AI system a latent disclosure that is detectable by the tool specified above and is, to the extent technically feasible, permanent or extraordinarily difficult to remove.
- Provides that a covered provider that violates the above provisions is liable for a civil penalty in the amount of $5,000 per violation to be collected in a civil action filed by the Attorney General, a city attorney, or a county counsel. Each day that a covered provider is in violation shall be deemed a discrete violation.
- Provides that it does not apply to any product, service, internet website, or application that provides exclusively non-user-generated video game, television, streaming, movie, or interactive experiences.
- Becomes operative on January 1, 2026.
Background
Certain forms of media – audio recordings, video recordings, and still images – can be powerful evidence of the truth. While such media have always been susceptible to some degree of manipulation, fakes were relatively easy to detect. The rapid advancement of AI technology, specifically the wide-scale introduction of generative AI models, has made it drastically cheaper and easier to produce synthetic content created, audio, images, text, and video recordings that are not real, but that are so realistic that they are virtually impossible to distinguish from authentic content, including so-called “deepfakes.”
This bill works to ensure that providers of these GenAI systems are equipping consumers with a tool to identify when specific content has been generated by their systems. This required tool must be made available to the public, as provided. Covered providers are also required to place latent disclsoures on this generated content to identify it as such and to be able to trace it back to the covered providers’ systems. Providers must also offer users the option to place manifest disclosures on content generated by these systems. Concerns have been raised that the relevant technology to achieve this is still in its nascent stage and its reliability is insufficient to impose mandates based on it.
This bill is author-sponsored. The bill is supported by Transparency Coalition.AI. No opposition has been received. For a more thorough discussion of the bill, please see the Senate Judiciary Committee analysis, which is incorporated herein by reference.
Comment
The author states:
Generative Artificial Intelligence (AI) is advancing at an unprecedented pace, revolutionizing industries and everyday life. However, this rapid progress has also sparked a pressing need for regulatory frameworks to govern its use. While AI offers innovative solutions that drive economic growth and efficiency, it simultaneously presents complex challenges related to misinformation, manipulation, bias, and ownership of generated content.
One of the most concerning aspects of unregulated AI models is their potential to produce “deepfake” content, such as manipulated images and videos, which can be used to deceive or manipulate individuals and society at large. Deepfake pornography and fabricated political messages have already emerged as serious threats, highlighting the urgency of implementing effective regulations.
As AI-generated content approaches a level of realism that makes it indistinguishable from genuine content, the need for transparency becomes paramount. It is crucial for industry leaders to take responsibility for clearly labeling AI-generated content and developing tools that can accurately identify such content, especially when there is no visible disclosure. This transparency is essential for maintaining trust in AI technologies and ensuring that users can make informed decisions about the content they consume.
SB 942 stands as a critical legislative response in our ongoing regulatory arms race with artificial intelligence.
FISCAL EFFECT: Appropriation: No Fiscal Com.: Yes Local: No
According to the Senate Appropriations Committee:
- Unknown cost for CDT to review covered providers’ disclosures at least once every two years and provide recommendations based on changing technology and standards. To the extent that CDT would need additional or specialized staff to perform oversight activities related to reviewing disclosures, there may be significant General Fund costs.
- The Department of Justice (DOJ) reports costs of $359,000 for 3.0 personnel years (PYs) in Fiscal Year (FY) 2024-25 and $637,000 for 3.0 PYs in FY 2025-26 and ongoing for its Consumer Protection Section within the Public Rights Division to address an anticipated increase in litigation and other potential legal challenges (Unfair Competition Law Fund).
According to the Assembly Appropriations Committee:
- Cost pressures (General Fund) to the Department of Justice (DOJ) of an unknown but potentially significant amount if the Attorney General brings civil actions to enforce violations of this bill. If DOJ files enforcement actions under this bill, the department may experience a significant workload increase in excess of $150,000 annually, depending on the number of enforcement actions filed and the amount of time needed for each case. If DOJ chooses not to pursue enforcement as authorized by this bill, DOJ would likely not incur any costs.
- Cost pressures (Trial Court Trust Fund, General Fund) of an unknown but potentially significant amount to the courts to adjudicate civil enforcement actions filed to enforce this bill. Actual costs will depend on the number of cases filed and the amount of court time needed to resolve each case. It generally costs approximately $1,000 to operate a courtroom for one hour. Although courts are not funded on the basis of workload, increased pressure on the Trial Court Trust Fund may create a need for increased funding for courts from the General Fund. The 2024-25 state budget provides $37.3 million ongoing General Fund to backfill declining revenue to the Trial Court Trust Fund.
According to the Legislative Analyst’s Office, the General Fund faces a structural deficit in the tens of billions of dollars over the next several fiscal years.
SUPPORT: (Verified 8/28/24)
Transparency Coalition.AI
OPPOSITION: (Verified 8/28/24)
None received
ARGUMENTS IN SUPPORT: Transparency Coalition.AI writes:
SB 942 will provide the public with critical insight into which content (text, image, video, audio, or multimedia content) was created, in whole or in part, by a generative AI system.
Prepared by: Christian Kurpiewski / JUD. / (916) 651-4113
8/28/24 22:09:21
**** END ****
SENATE THIRD READING
STR Bill Id:SB 942
Author:(Becker)
As Amended Ver:August 23, 2024
Majority vote
SUMMARY
Beginning in 2026, requires the developers of generative artificial intelligence (GenAI) systems to both include provenance disclosures in the content their systems produce, and make tools available to identify GenAI-content produced by their systems.
Major Provisions
- Defines "covered provider" to mean a person that creates, codes, or otherwise produces a GenAI system that has over 1,000,000 monthly visitors or users and is publicly accessible within the geographic boundaries of the state.
- Requires a covered provider to make available (but not necessarily create) an artificial intelligence (AI) detection tool that can be used to assess whether digital content was created or altered by the covered provider's GenAI system, and that has various specified characteristics.
- Requires a covered provider to offer a user of a GenAI tool the option to include a manifest (visible) disclosure in generated content, with various specified properties.
- Requires a covered provider include a latent (invisible) disclosure in generated content, with various specified properties.
- Requires that if a covered provider licenses their GenAI system to a third party, the covered provider shall require by contract that the licensee maintain the system's capability to include a latent disclosure. Requires that if a covered provider knows a licensee has removed the capability of a licensed GenAI system to include a latent disclosure, the covered provider shall revoke the licensee's contract within 96 hours. Requires that a licensee cease using a licensed GenAI system after its license has been revoked.
- Provides that a covered provider that violates the chapter be liable for a civil penalty of $5000 to be collected in a civil action filed by the Attorney General only. Provides that the Attorney General, a district attorney, a county counsel, or a city attorney may bring a civil action for injunctive relief and reasonable attorney's fees and costs against a third party licensee who violates this chapter.
COMMENTS
Background. The development of GenAI is creating exciting opportunities to grow California's economy and improve the lives of its residents. GenAI can generate compelling text, images and audio in an instant – but with novel technologies come novel safety concerns. As generative artificial intelligence (GenAI) technologies become more accessible, online content that appears real – but is actually false – threatens to flood social media and other large online platforms. The unmitigated spread of synthetic content threatens to harm individual Californians in numerous ways, such as through the proliferation of nonconsensual deepfake pornography, scams, and the distribution of targeted election disinformation.
What this bill would do. This bill has four basic components. First, it requires GenAI system providers to include disclosures in content produced by their systems. The bill describes two types of disclosures: manifest and latent. Manifest disclosures are optional, and primarily serve to clearly identify content as AI-generated. Latent disclosures are mandatory, and serve as an imperceptible indicator that content is AI-generated. These disclosures are required to be interpretable by a provider's AI detection tool, detailed below.
Second, it requires GenAI system providers to make available (but not necessarily create) AI detection tools that can identify content created by the providers' systems. When a user uploads content to be analyzed, the tool is required to output information related to the system-of-origin of the content, and withhold any personal information included in the disclosure. A covered provider may impose reasonable limitations on access to the tool to prevent, or respond to, demonstrable risks to the security or integrity of its GenAI system.
Third, it requires that GenAI system providers who license their systems to third parties contractually obligate licensees to retain the systems' disclosure capabilities. If a provider knows that a licensee has stripped a licensed system of its disclosure capabilities, the provider is required to revoke their license within 96 hours. A licensee who has had their license revoked must stop using the system.
Fourth, the bill outlines enforcement mechanisms. Under the bill's enforcement scheme, a GenAI system provider who violates the bill's provisions is liable for a civil penalty of $5000, to be collected in a civil action filed only by the Attorney General. A third party licensee who violates the bill's provisions may have a civil action brought against them by the Attorney General, a district attorney, a county counsel, or a city attorney, for injunctive relief and reasonable attorney's costs and fees.
Analysis. The issue of content authentication can theoretically be solved in three steps:
- Require all content produced by GenAI to be labeled "fake."
- Require all content produced by cameras and other recording devices to be labeled "real."
- Require all social media platforms to prominently display these labels.
SB 942 relates to step (1) of this solution. SB 942 requires that latent (imperceptible to the human eye) "disclosures" be included in content produced by GenAI systems. Aside from requiring disclosures in GenAI content, the bill requires GenAI providers make available at no cost tools to identify content generated by their systems.
Furthermore, this bill requires that any GenAI provider who licenses their system to a third party must require, as a term of the contract, that the third party not remove their system's disclosure capability. In order to contend with open-source concerns, this is not a physical limitation: the licensee could, if they desired, remove the disclosure capability. But if a covered provider discovers that a licensee has done so, the bill requires that the provider revoke the third party's license within 96 hours. A licensee who continues to use a GenAI system once this license has been revoked may have a civil action brought against them for injunctive relief and reasonable costs and attorney's fees.
Following recent amendments, the bill no longer applies to any “product, service, internet website, or application that provides exclusively non-user-generated video game, television, streaming, movie, or interactive experiences.” Previously, the bill may have inadvertently captured innocuous uses of GenAI in tightly constrained environments – for example, the creation of user avatars in video games that otherwise do not involve GenAI. The addition of this language is meant to acknowledge and exempt such incidental use cases
According to the Author
As AI-generated content approaches a level of realism that makes it indistinguishable from genuine content, the need for transparency becomes paramount. It is crucial for industry leaders to take responsibility for clearly labeling AI-generated content and developing tools that can accurately identify such content, especially when there is no visible disclosure. The transparency SB 942 would provide is essential for establishing trust in AI technologies and ensuring that users can make informed decisions about the content they consume.
Arguments in Support
Oakland Privacy writes:
Operationally, SB 942 calls for the makers of generative AI products to bear the primary responsibility for the labeling of content and defers from any specifics regarding the disclosures except that they be visible, and as permanent and understandable as possible. Given that generative AI is in its infancy and provenance labeling even more so, the lack of specificity about the technical requirements is understandable. It may even be desirable, as no method is perfect, and all are subject to change.
Arguments in Opposition
Writing on behalf of a coalition of trade associations, Technet takes an oppose-unless-amended position:
Content provenance and watermarking are still incredibly unreliable and, in many cases, easy to break. Researchers at the University of Maryland were able to break all the currently available watermarking methods. Some can be avoided by simple cropping, resizing, or screenshotting an image. More concerning, these researchers were able to insert fake watermarks and credentials into images, creating false positives. We've provided amendments that recognize that this technology is still in the early stages of development and adoption by building in a technical feasibility standard.
FISCAL COMMENTS
According to the Assembly Appropriations Committee analysis of this bill:
- Cost pressures (General Fund) to the Department of Justice (DOJ) of an unknown but potentially significant amount if the Attorney General brings civil actions to enforce violations of this bill. If DOJ files enforcement actions under this bill, the department may experience a significant workload increase in excess of $150,000 annually, depending on the number of enforcement actions filed and the amount of time needed for each case. If DOJ chooses not to pursue enforcement as authorized by this bill, DOJ would likely not incur any costs.
- Cost pressures (Trial Court Trust Fund, General Fund) of an unknown but potentially significant amount to the courts to adjudicate civil enforcement actions filed to enforce this bill. Actual costs will depend on the number of cases filed and the amount of court time needed to resolve each case. It generally costs approximately $1,000 to operate a courtroom for one hour. Although courts are not funded on the basis of workload, increased pressure on the Trial Court Trust Fund may create a need for increased funding for courts from the General Fund. The 2024-25 state budget provides $37.3 million ongoing General Fund to backfill declining revenue to the Trial Court Trust Fund.
According to the Legislative Analyst's Office, the General Fund faces a structural deficit in the tens of billions of dollars over the next several fiscal years.
VOTES
SENATE FLOOR: 32-1-7
YES: Alvarado-Gil, Archuleta, Ashby, Atkins, Becker, Blakespear, Bradford, Caballero, Cortese, Dodd, Durazo, Glazer, Gonzalez, Hurtado, Laird, Limón, McGuire, Menjivar, Min, Newman, Niello, Ochoa Bogh, Padilla, Portantino, Roth, Rubio, Skinner, Smallwood-Cuevas, Stern, Umberg, Wahab, Wiener
NO: Wilk
ABS, ABST OR NV: Allen, Dahle, Eggman, Grove, Jones, Nguyen, Seyarto
ASM PRIVACY AND CONSUMER PROTECTION: 8-0-3
YES: Bauer-Kahan, Bryan, Irwin, Lowenthal, Ortega, Ward, Wicks, Wilson
ABS, ABST OR NV: Joe Patterson, Dixon, Hoover
ASM JUDICIARY: 9-1-2
YES: Kalra, Bauer-Kahan, Bryan, Connolly, Haney, Maienschein, McKinnor, Pacheco, Reyes
NO: Joe Patterson
ABS, ABST OR NV: Dixon, Sanchez
ASM APPROPRIATIONS: 11-0-4
YES: Wicks, Arambula, Bryan, Calderon, Wendy Carrillo, Mike Fong, Grayson, Haney, Hart, Pellerin, Villapudua
ABS, ABST OR NV: Sanchez, Dixon, Jim Patterson, Ta
UPDATED
VERSION: August 23, 2024
CONSULTANT: Slater Sharp / P. & C.P. / (916) 319-2200 FN: 0004605
SENATE THIRD READING
STR Bill Id:SB 942
Author:(Becker)
As Amended Ver:August 23, 2024
Majority vote
SUMMARY
Beginning in 2026, requires the developers of generative artificial intelligence (GenAI) systems to both include provenance disclosures in the content their systems produce, and make tools available to identify GenAI-content produced by their systems.
Major Provisions
- Defines "covered provider" to mean a person that creates, codes, or otherwise produces a GenAI system that has over 1,000,000 monthly visitors or users and is publicly accessible within the geographic boundaries of the state.
- Requires a covered provider to make available (but not necessarily create) an artificial intelligence (AI) detection tool that can be used to assess whether digital content was created or altered by the covered provider's GenAI system, and that has various specified characteristics.
- Requires a covered provider to offer a user of a GenAI tool the option to include a manifest (visible) disclosure in generated content, with various specified properties.
- Requires a covered provider include a latent (invisible) disclosure in generated content, with various specified properties.
- Requires that if a covered provider licenses their GenAI system to a third party, the covered provider shall require by contract that the licensee maintain the system's capability to include a latent disclosure. Requires that if a covered provider knows a licensee has removed the capability of a licensed GenAI system to include a latent disclosure, the covered provider shall revoke the licensee's contract within 96 hours. Requires that a licensee cease using a licensed GenAI system after its license has been revoked.
- Provides that a covered provider that violates the chapter be liable for a civil penalty of $5000 to be collected in a civil action filed by the Attorney General only. Provides that the Attorney General, a district attorney, a county counsel, or a city attorney may bring a civil action for injunctive relief and reasonable attorney's fees and costs against a third party licensee who violates this chapter.
COMMENTS
Background. The development of GenAI is creating exciting opportunities to grow California's economy and improve the lives of its residents. GenAI can generate compelling text, images and audio in an instant – but with novel technologies come novel safety concerns. As generative artificial intelligence (GenAI) technologies become more accessible, online content that appears real – but is actually false – threatens to flood social media and other large online platforms. The unmitigated spread of synthetic content threatens to harm individual Californians in numerous ways, such as through the proliferation of nonconsensual deepfake pornography, scams, and the distribution of targeted election disinformation.
What this bill would do. This bill has four basic components. First, it requires GenAI system providers to include disclosures in content produced by their systems. The bill describes two types of disclosures: manifest and latent. Manifest disclosures are optional, and primarily serve to clearly identify content as AI-generated. Latent disclosures are mandatory, and serve as an imperceptible indicator that content is AI-generated. These disclosures are required to be interpretable by a provider's AI detection tool, detailed below.
Second, it requires GenAI system providers to make available (but not necessarily create) AI detection tools that can identify content created by the providers' systems. When a user uploads content to be analyzed, the tool is required to output information related to the system-of-origin of the content, and withhold any personal information included in the disclosure. A covered provider may impose reasonable limitations on access to the tool to prevent, or respond to, demonstrable risks to the security or integrity of its GenAI system.
Third, it requires that GenAI system providers who license their systems to third parties contractually obligate licensees to retain the systems' disclosure capabilities. If a provider knows that a licensee has stripped a licensed system of its disclosure capabilities, the provider is required to revoke their license within 96 hours. A licensee who has had their license revoked must stop using the system.
Fourth, the bill outlines enforcement mechanisms. Under the bill's enforcement scheme, a GenAI system provider who violates the bill's provisions is liable for a civil penalty of $5000, to be collected in a civil action filed only by the Attorney General. A third party licensee who violates the bill's provisions may have a civil action brought against them by the Attorney General, a district attorney, a county counsel, or a city attorney, for injunctive relief and reasonable attorney's costs and fees.
Analysis. The issue of content authentication can theoretically be solved in three steps:
- Require all content produced by GenAI to be labeled "fake."
- Require all content produced by cameras and other recording devices to be labeled "real."
- Require all social media platforms to prominently display these labels.
SB 942 relates to step (1) of this solution. SB 942 requires that latent (imperceptible to the human eye) "disclosures" be included in content produced by GenAI systems. Aside from requiring disclosures in GenAI content, the bill requires GenAI providers make available at no cost tools to identify content generated by their systems.
Furthermore, this bill requires that any GenAI provider who licenses their system to a third party must require, as a term of the contract, that the third party not remove their system's disclosure capability. In order to contend with open-source concerns, this is not a physical limitation: the licensee could, if they desired, remove the disclosure capability. But if a covered provider discovers that a licensee has done so, the bill requires that the provider revoke the third party's license within 96 hours. A licensee who continues to use a GenAI system once this license has been revoked may have a civil action brought against them for injunctive relief and reasonable costs and attorney's fees.
According to the Author
As AI-generated content approaches a level of realism that makes it indistinguishable from genuine content, the need for transparency becomes paramount. It is crucial for industry leaders to take responsibility for clearly labeling AI-generated content and developing tools that can accurately identify such content, especially when there is no visible disclosure. The transparency SB 942 would provide is essential for establishing trust in AI technologies and ensuring that users can make informed decisions about the content they consume.
Arguments in Support
Oakland Privacy writes:
Operationally, SB 942 calls for the makers of generative AI products to bear the primary responsibility for the labeling of content and defers from any specifics regarding the disclosures except that they be visible, and as permanent and understandable as possible. Given that generative AI is in its infancy and provenance labeling even more so, the lack of specificity about the technical requirements is understandable. It may even be desirable, as no method is perfect, and all are subject to change.
Arguments in Opposition
Writing on behalf of a coalition of trade associations, Technet takes an oppose-unless-amended position:
Content provenance and watermarking are still incredibly unreliable and, in many cases, easy to break. Researchers at the University of Maryland were able to break all the currently available watermarking methods. Some can be avoided by simple cropping, resizing, or screenshotting an image. More concerning, these researchers were able to insert fake watermarks and credentials into images, creating false positives. We've provided amendments that recognize that this technology is still in the early stages of development and adoption by building in a technical feasibility standard.
FISCAL COMMENTS
According to the Assembly Appropriations Committee analysis of this bill:
- Cost pressures (General Fund) to the Department of Justice (DOJ) of an unknown but potentially significant amount if the Attorney General brings civil actions to enforce violations of this bill. If DOJ files enforcement actions under this bill, the department may experience a significant workload increase in excess of $150,000 annually, depending on the number of enforcement actions filed and the amount of time needed for each case. If DOJ chooses not to pursue enforcement as authorized by this bill, DOJ would likely not incur any costs.
- Cost pressures (Trial Court Trust Fund, General Fund) of an unknown but potentially significant amount to the courts to adjudicate civil enforcement actions filed to enforce this bill. Actual costs will depend on the number of cases filed and the amount of court time needed to resolve each case. It generally costs approximately $1,000 to operate a courtroom for one hour. Although courts are not funded on the basis of workload, increased pressure on the Trial Court Trust Fund may create a need for increased funding for courts from the General Fund. The 2024-25 state budget provides $37.3 million ongoing General Fund to backfill declining revenue to the Trial Court Trust Fund.
According to the Legislative Analyst's Office, the General Fund faces a structural deficit in the tens of billions of dollars over the next several fiscal years.
VOTES
SENATE FLOOR: 32-1-7
YES: Alvarado-Gil, Archuleta, Ashby, Atkins, Becker, Blakespear, Bradford, Caballero, Cortese, Dodd, Durazo, Glazer, Gonzalez, Hurtado, Laird, Limón, McGuire, Menjivar, Min, Newman, Niello, Ochoa Bogh, Padilla, Portantino, Roth, Rubio, Skinner, Smallwood-Cuevas, Stern, Umberg, Wahab, Wiener
NO: Wilk
ABS, ABST OR NV: Allen, Dahle, Eggman, Grove, Jones, Nguyen, Seyarto
ASM PRIVACY AND CONSUMER PROTECTION: 8-0-3
YES: Bauer-Kahan, Bryan, Irwin, Lowenthal, Ortega, Ward, Wicks, Wilson
ABS, ABST OR NV: Joe Patterson, Dixon, Hoover
ASM JUDICIARY: 9-1-2
YES: Kalra, Bauer-Kahan, Bryan, Connolly, Haney, Maienschein, McKinnor, Pacheco, Reyes
NO: Joe Patterson
ABS, ABST OR NV: Dixon, Sanchez
ASM APPROPRIATIONS: 11-0-4
YES: Wicks, Arambula, Bryan, Calderon, Wendy Carrillo, Mike Fong, Grayson, Haney, Hart, Pellerin, Villapudua
ABS, ABST OR NV: Sanchez, Dixon, Jim Patterson, Ta
UPDATED
VERSION: August 23, 2024
CONSULTANT: Slater Sharp / P. & C.P. / (916) 319-2200 FN: 0004605
SENATE THIRD READING
STR Bill Id:SB 942
Author:(Becker)
As Amended Ver:August 19, 2024
Majority vote
SUMMARY
Beginning in 2026, requires the developers of generative artificial intelligence (GenAI) systems to both include provenance disclosures in the content their systems produce, and make tools available to identify GenAI-content produced by their systems.
Major Provisions
- Defines "covered provider" to mean a person that creates, codes, or otherwise produces a GenAI system that has over 1,000,000 monthly visitors or users and is publicly accessible within the geographic boundaries of the state.
- Requires a covered provider to create and make available an artificial intelligence (AI) detection tool that can be used to assess whether digital content was created or altered by the covered provider's GenAI system, and that has various specified characteristics.
- Requires a covered provider to offer a user of a GenAI tool the option to include a manifest (visible) disclosure in generated content, with various specified properties.
- Requires a covered provider include a latent (invisible) disclosure in generated content, with various specified properties.
- Requires that if a covered provider licenses their GenAI system to a third party, the covered provider shall require by contract that the licensee maintain the system's capability to include a latent disclosure. Requires that if a covered provider knows a licensee has removed the capability of a licensed GenAI system to include a latent disclosure, the covered provider shall revoke the licensee's contract. Requires that a licensee cease using a licensed GenAI system after its license has been revoked.
- Provides that a covered provider that violates the chapter be liable for a civil penalty of $5000 to be collected in a civil action filed by the Attorney General only. Provides that the Attorney General, a district attorney, a county counsel, or a city attorney may bring a civil action for injunctive relief and reasonable attorney's fees and costs against a third party licensee who violates this chapter.
COMMENTS
Background. The development of GenAI is creating exciting opportunities to grow California's economy and improve the lives of its residents. GenAI can generate compelling text, images and audio in an instant – but with novel technologies come novel safety concerns. As generative artificial intelligence (GenAI) technologies become more accessible, online content that appears real – but is actually false – threatens to flood social media and other large online platforms. The unmitigated spread of synthetic content threatens to harm individual Californians in numerous ways, such as through the proliferation of nonconsensual deepfake pornography, scams, and the distribution of targeted election disinformation.
What this bill would do. This bill has four basic components. First, it requires GenAI system providers to include disclosures in content produced by their systems. The bill describes two types of disclosures: manifest and latent. Manifest disclosures are optional, and primarily serve to clearly identify content as AI-generated. Latent disclosures are mandatory, and serve as an imperceptible indicator that content is AI-generated. These disclosures are required to be interpretable by a provider's AI detection tool, detailed below.
Second, it requires GenAI system providers create and make available AI detection tools that can identify content created by the providers' systems. When a user uploads content to be analyzed, the tool is required to output information related to the system-of-origin of the content, and withhold any personal information included in the disclosure.
Third, it requires that GenAI system providers who license their systems to third parties contractually obligate licensees to retain the systems' disclosure capabilities. If a provider knows that a licensee has stripped a licensed system of its disclosure capabilities, the provider is required to revoke their license. A licensee who has had their license revoked must stop using the system.
Fourth, the bill outlines enforcement mechanisms. Under the bill's enforcement scheme, a GenAI system provider who violates the bill's provisions is liable for a civil penalty of $5000, to be collected in a civil action filed only by the Attorney General. A third party licensee who violates the bill's provisions may have a civil action brought against them by the Attorney General, a district attorney, a county counsel, or a city attorney, for injunctive relief and reasonable attorney's costs and fees.
Analysis. The issue of content authentication can theoretically be solved in three steps:
- Require all content produced by GenAI to be labeled "fake."
- Require all content produced by cameras and other recording devices to be labeled "real."
- Require all social media platforms to prominently display these labels.
This bill relates to step 1) of this solution. This bill requires that latent (imperceptible to the human eye) "disclosures" be included in content produced by GenAI systems. Aside from requiring disclosures in GenAI content, the bill requires GenAI providers develop and make freely available tools to identify content generated by their systems.
Furthermore, this bill requires that any GenAI provider who licenses their system to a third party must require, as a term of the contract, that the third party not remove their system's disclosure capability. In order to contend with open-source concerns, this is not a physical limitation: the licensee could, if they desired, remove the disclosure capability. But if a covered provider discovers that a licensee has done so, the bill requires that the provider revoke the third party's license. A licensee who continues to use a GenAI system once this license has been revoked may have a civil action brought against them for injunctive relief and reasonable costs and attorney's fees.
According to the Author
As AI-generated content approaches a level of realism that makes it indistinguishable from genuine content, the need for transparency becomes paramount. It is crucial for industry leaders to take responsibility for clearly labeling AI-generated content and developing tools that can accurately identify such content, especially when there is no visible disclosure. The transparency SB 942 would provide is essential for establishing trust in AI technologies and ensuring that users can make informed decisions about the content they consume.
Arguments in Support
Oakland Privacy writes:
Operationally, SB 942 calls for the makers of generative AI products to bear the primary responsibility for the labeling of content and defers from any specifics regarding the disclosures except that they be visible, and as permanent and understandable as possible. Given that generative AI is in its infancy and provenance labeling even more so, the lack of specificity about the technical requirements is understandable. It may even be desirable, as no method is perfect, and all are subject to change.
Arguments in Opposition
Writing on behalf of a coalition of trade associations, Technet takes an oppose-unless-amended position and writes:
Content provenance and watermarking are still incredibly unreliable and, in many cases, easy to break. Researchers at the University of Maryland were able to break all the currently available watermarking methods. Some can be avoided by simple cropping, resizing, or screenshotting an image. More concerning, these researchers were able to insert fake watermarks and credentials into images, creating false positives. We've provided amendments that recognize that this technology is still in the early stages of development and adoption by building in a technical feasibility standard.
FISCAL COMMENTS
- Cost pressures (General Fund) to the Department of Justice (DOJ) of an unknown but potentially significant amount if the Attorney General brings civil actions to enforce violations of this bill. If DOJ files enforcement actions under this bill, the department may experience a significant workload increase in excess of $150,000 annually, depending on the number of enforcement actions filed and the amount of time needed for each case. If DOJ chooses not to pursue enforcement as authorized by this bill, DOJ would likely not incur any costs.
- Cost pressures (Trial Court Trust Fund, General Fund) of an unknown but potentially significant amount to the courts to adjudicate civil enforcement actions filed to enforce this bill. Actual costs will depend on the number of cases filed and the amount of court time needed to resolve each case. It generally costs approximately $1,000 to operate a courtroom for one hour. Although courts are not funded on the basis of workload, increased pressure on the Trial Court Trust Fund may create a need for increased funding for courts from the General Fund. The 2024-25 state budget provides $37.3 million ongoing General Fund to backfill declining revenue to the Trial Court Trust Fund.
According to the Legislative Analyst's Office, the General Fund faces a structural deficit in the tens of billions of dollars over the next several fiscal years.
VOTES
SENATE FLOOR: 32-1-7
YES: Alvarado-Gil, Archuleta, Ashby, Atkins, Becker, Blakespear, Bradford, Caballero, Cortese, Dodd, Durazo, Glazer, Gonzalez, Hurtado, Laird, Limón, McGuire, Menjivar, Min, Newman, Niello, Ochoa Bogh, Padilla, Portantino, Roth, Rubio, Skinner, Smallwood-Cuevas, Stern, Umberg, Wahab, Wiener
NO: Wilk
ABS, ABST OR NV: Allen, Dahle, Eggman, Grove, Jones, Nguyen, Seyarto
ASM PRIVACY AND CONSUMER PROTECTION: 8-0-3
YES: Bauer-Kahan, Bryan, Irwin, Lowenthal, Ortega, Ward, Wicks, Wilson
ABS, ABST OR NV: Joe Patterson, Dixon, Hoover
ASM JUDICIARY: 9-1-2
YES: Kalra, Bauer-Kahan, Bryan, Connolly, Haney, Maienschein, McKinnor, Pacheco, Reyes
NO: Joe Patterson
ABS, ABST OR NV: Dixon, Sanchez
ASM APPROPRIATIONS: 11-0-4
YES: Wicks, Arambula, Bryan, Calderon, Wendy Carrillo, Mike Fong, Grayson, Haney, Hart, Pellerin, Villapudua
ABS, ABST OR NV: Sanchez, Dixon, Jim Patterson, Ta
UPDATED
VERSION: August 19, 2024
CONSULTANT: Slater Sharp / P. & C.P. / (916) 319-2200 FN: 0004155
Date of Hearing: August 7, 2024
ASSEMBLY COMMITTEE ON APPROPRIATIONS
Buffy Wicks, Chair
SBPCA Bill Id:SB 942
Author:(Becker) – As Amended Ver:July 3, 2024
Policy Committee: | Privacy and Consumer Protection | Vote: | 8 - 0 |
Judiciary | 9 - 1 | ||
|
|
Urgency: No State Mandated Local Program: No Reimbursable: No
SUMMARY:
This bill requires the provider of a generative artificial intelligence (GenAI) system to include provenance disclosure in original content produced by the system, and requires such a provider to make tools available that can identify GenAI content produced by its system.
The bill governs a provider that that creates, codes, or otherwise produces a GenAI system that has over one million monthly visitors or users and is publicly accessible within the geographic boundaries of the state (a “covered provider”).
Specifically, among other provisions, this bill:
- Requires a covered provider to create and make freely available an AI detection tool that, among other requirements, allows a user to assess whether a piece of digital content was created or altered by the provider’s GenAI system, outputs system provenance data but not personal provenance data in the content, and is publicly accessible through the provider’s website and mobile application.
- Requires a covered provider to offer the user the option to include a manifest disclosure in image, video, audio, or other digital content created or altered by the covered provider’s GenAI system.
- Requires a covered provider to include a latent disclosure in generated content, as specified, and requires a covered provider that licenses its GenAI system to a third party to mandate by contract that the licensee maintain the system’s capability to include a latent disclosure.
- Requires a covered provider that licenses its GenAI system to take specified steps within 72 hours if the provider discovers the licensee has removed the system’s latent disclosure capability.
- Authorizes the Attorney General, a city attorney, or a county counsel to file a civil action to enforce a violation by a covered provider, and specifies a civil penalty of $5,000 per day a covered provider is deemed in violation.
FISCAL EFFECT:
- Cost pressures (General Fund) to the Department of Justice (DOJ) of an unknown but potentially significant amount if the Attorney General brings civil actions to enforce violations of this bill. If DOJ files enforcement actions under this bill, the department may experience a significant workload increase in excess of $150,000 annually, depending on the number of enforcement actions filed and the amount of time needed for each case. If DOJ chooses not to pursue enforcement as authorized by this bill, DOJ would likely not incur any costs.
- Cost pressures (Trial Court Trust Fund, General Fund) of an unknown but potentially significant amount to the courts to adjudicate civil enforcement actions filed to enforce this bill. Actual costs will depend on the number of cases filed and the amount of court time needed to resolve each case. It generally costs approximately $1,000 to operate a courtroom for one hour. Although courts are not funded on the basis of workload, increased pressure on the Trial Court Trust Fund may create a need for increased funding for courts from the General Fund. The 2024-25 state budget provides $37.3 million ongoing General Fund to backfill declining revenue to the Trial Court Trust Fund.
According to the Legislative Analyst’s Office, the General Fund faces a structural deficit in the tens of billions of dollars over the next several fiscal years.
COMMENTS:
- Purpose. According to the author:
As AI-generated content approaches a level of realism that makes it indistinguishable from genuine content, the need for transparency becomes paramount. It is crucial for industry leaders to take responsibility for clearly labeling AI-generated content and developing tools that can accurately identify such content, especially when there is no visible disclosure. The transparency SB 942 would provide is essential for establishing trust in AI technologies and ensuring that users can make informed decisions about the content they consume.
- Background. Although images have long been manipulated or faked, the proliferation of GenAI systems that can generate and modify images and videos has dramatically increased the amount of convincing but ultimately fake content circulating in the public domain. This emerging technology has already resulted in creation of nonconsensual pornography, scams targeting individuals and corporations, and political propaganda and misinformation.
This bill requires covered providers of GenAI systems to include latent disclosures in content generated or altered by their systems, and create tools that permit users to detect GenAI content and add visible disclosures to GenAI content. The bill authorizes enforcement of violations by the Attorney General, city attorneys, and county counsel, and specifies civil penalties of $5,000 per violation. Each day a covered provider is in violation of the bill’s provisions is considered a discrete violation, meaning a covered provider may be liable for a $5,000 penalty for each day of noncompliance.
Analysis Prepared by: Annika Carlson / APPR. / (916) 319-2081
Date of Hearing: July 2, 2024
ASSEMBLY COMMITTEE ON JUDICIARY
Ash Kalra, Chair
SBPCA Bill Id:SB 942
Author:(Becker) – As Amended Ver:June 20, 2024
As Proposed to be Amended
SENATE VOTE: 32-1
SUBJECT: California AI Transparency Act
KEY ISSUE: SHOULD THE STATE MANDATE that AI DEVELOPERS CREATE AND PROVIDE PUBLIC ACCESS TO AI DETECTION TOOLS, AND INCORPORATE VISIBLE AND IMPERCEPTIBLE DISCLOSURES IN AI-GENERATED CONTENT?
SYNOPSIS
As generative artificial intelligence (GenAI) technologies become more widespread, the proliferation of artificially generated synthetic content that mimics reality poses significant threats to individuals and society. This content, which can appear highly realistic, risks inundating social media and other large online platforms with false information. Such unregulated spread of synthetic media can harm Californians in various ways, including the creation and dissemination of nonconsensual deepfake pornography, perpetration of scams, and the spread of targeted election disinformation.
To tackle the issue of GenAI-produced content, this bill proposes a comprehensive three-step plan. First, all AI-generated content must be clearly labeled as such. Second, all genuine content, such as video and audio recordings, should be labeled to verify its authenticity. Third, large online platforms must be required to prominently display these labels, helping users distinguish between artificial and real content. This bill requires GenAI providers to embed machine-readable disclosures into the content produced by their systems. These disclosures, which include information about the content's origin, must be designed to be difficult to remove. Providers are also required to develop and freely distribute software that enables users to identify AI-generated content. Additionally, GenAI providers who license their systems to other content creating platforms must ensure that licensees maintain the systems' disclosure capabilities through contractual requirements. The proposed amendments, discussed in this analysis, are technical and ensure consistency within the enforcement provisions of this bill.
This bill is supported by organizations such as Protection of the Educational Rights of Kids Advocacy and Transparency Coalition.AI. However, the bill faces opposition from a coalition of industry associations, including TechNet, NetChoice, the Computer and Communications Industry Association, and the California Chamber of Commerce. These industry groups argue that federal standards would be more effective than state-specific regulations and express concerns about the current technological feasibility of the bill's requirements. The bill was previously heard and approved by the Committee on Privacy and Consumer Protection by a vote of eight to zero.
SUMMARY: Requires the developers of generative artificial intelligence (GenAI) systems to both include provenance disclosures in the original content their systems produce and make tools available to identify GenAI-content produced by their systems. Specifically, this bill:
- Defines “Generative artificial intelligence” or “GenAI” to mean artificial intelligence that can generate derived synthetic content, including text, images, video, and audio, that emulates the structure and characteristics of the system’s training data.
- Defines “covered provider” to mean a person that creates, codes, or otherwise produces a GenAI system that has over 1,000,000 monthly visitors or users and is publicly accessible within the geographic boundaries of the state.
- Defines “provenance data” to mean data that is embedded into digital content, or that is included in the digital content’s metadata, for the purpose of verifying the digital content’s authenticity, origin, or history of modification.
- Defines “personal provenance data” to mean provenance data that contains either personal information or unique device, system, or service information that is reasonably capable of being associated with a particular user.
- Defines “system provenance data” to mean provenance data that is not reasonably capable of being associated with a particular user and that contains either information regarding the type of device, system, or service that was used to generate a piece of digital content, or information that provides proof of content authenticity.
- Requires a covered provider to create and make freely available an AI detection tool that complies with the following:
- Allows a user to assess whether image, video, audio, or other digital content was created or altered by the covered provider’s GenAI system;
- Allows a user to determine which parts of the content were created or altered by the covered provider’s GenAI system;
- Outputs any system provenance data that is detected in the content;
- Does not output any personal provenance data that is detected in the content;
- Is publicly accessible through the covered provider’s internet website and its mobile application, as applicable;
- Allows a user to upload content or provide a uniform resource locator (URL) linking to online content; and
- Supports an application programming interface that allows a user to invoke the tool without visiting the covered provider’s internet website.
- Requires a covered provider to collect user feedback related to the efficacy of the AI detection tool, as specified.
- Prohibits a covered provider from collecting or retaining personal information from users of the covered provider’s AI detection tool, or retaining personal provenance data from user-uploaded content, except as specified.
- Requires a covered provider to offer the user the option to include a manifest disclosure in image, video, audio, or other digital content created or altered by the covered provider’s GenAI system that meets all of the following criteria:
- The disclosure identifies content as AI-generated content;
- The disclosure is clear, conspicuous, appropriate for the medium of the content, and understandable to a reasonable person;
- The disclosure is permanent or extraordinarily difficult to remove, to the extent it is technically feasible.
- Requires a covered provider to include a latent disclosure in generated content, with various specified characteristics, such as conveying information like the name of the covered provider, the name and version number of the GenAI system, the time and date of the content’s creation or alteration, and a unique identifier.
- Requires, if a covered provider licenses their GenAI system to a third party, the covered provider to mandate by contract that the licensee maintain the system’s capability to include a latent disclosure.
- Requires that if a covered provider knows a licensee has removed the capability of a licensed GenAI system to include a latent disclosure, the covered provider to revoke the licensee’s contract within 72 hours of discovering the licensee’s action.
- Requires that a licensee cease using a licensed GenAI system after its license has been revoked.
- Provides that a covered provider that violates the bill be liable for a civil penalty of $5000 per violation, to be collected in a civil action filed by the Attorney General, a city attorney, or county counsel. Each day that a covered provider is in violation of this bill is to be deemed a discrete violation.
- Provides that the Attorney General, a county counsel, or a city attorney may bring a civil action for injunctive relief and reasonable attorney’s fees and costs against a third-party licensee who violates this chapter.
- Mandates that a prevailing plaintiff in an action brought pursuant to this bill is to be entitled to all attorneys’ costs and fees.
EXISTING LAW:
- Provides, pursuant to the California Constitution, that all people are by nature free and independent and have inalienable rights. Among these are the fundamental right to privacy. (California Constitution Article. I, Section 1.)
- States that the “right to privacy is a personal and fundamental right protected by Section 1 of Article I of the Constitution of California and by the United States Constitution and that all individuals have a right of privacy in information pertaining to them.” Further states these findings of the Legislature:
- The right to privacy is being threatened by the indiscriminate collection, maintenance, and dissemination of personal information and the lack of effective laws and legal remedies.
- The increasing use of computers and other sophisticated information technology has greatly magnified the potential risk to individual privacy that can occur from the maintenance of personal information.
- In order to protect the privacy of individuals, it is necessary that the maintenance and dissemination of personal information be subject to strict limits. (Civil Code Section 1798.1.)
- Defines “deepfake” to mean audio or visual content that has been generated or manipulated by artificial intelligence (AI) which would falsely appear to be authentic or truthful and which features depictions of people appearing to say or do things they did not say or do without their consent. Requires the Secretary of Government Operations to evaluate the impact of the proliferation of deepfakes on the state. (Government Code Section 11547.5.)
- Defines “personal information” to mean information that identifies, relates to, describes, is reasonably capable of being associated with, or could reasonably be linked, directly or indirectly, with a particular consumer or household. States that personal information includes, but is not limited to, the following:
- Identifiers such as a real name, alias, postal address, unique personal identifier, online identifier, Internet Protocol address, email address, account name, social security number, driver’s license number, passport number, or other similar identifiers;
- Information that identifies, relates to, describes, or is capable of being associated with, a particular individual, as specified;
- Characteristics of protected classifications under California or federal law;
- Commercial information, including records of personal property, products or services purchased, obtained, or considered, or other purchasing or consuming histories or tendencies;
- Biometric information;
- Internet or other electronic network activity information, including, but not limited to, browsing history, search history, and information regarding a consumer’s interaction with an internet website application, or advertisement;
- Geolocation data;
- Audio, electronic, visual, thermal, olfactory, or similar information;
- Professional or employment-related information;
- Education information, defined as information that is not publicly available personally identifiable information as defined in the Family Educational Rights and Privacy Act;
- Inferences drawn from any of the information identified in this subdivision to create a profile about a consumer reflecting the consumer’s preferences, characteristics, psychological trends, predispositions, behavior, attitudes, intelligence, abilities, and aptitudes; and
- Sensitive personal information. (Civil Code Section 1798.140 (v).)
FISCAL EFFECT: As currently in print this bill is keyed fiscal.
COMMENTS: With significant advancements in generative artificial intelligence (GenAI) technology, distinguishing between human-generated and machine-generated content has become increasingly challenging. GenAI technologies, which can produce highly realistic synthetic media, pose substantial risks related to misinformation, fraud, and privacy violations. The unchecked spread of AI-generated content can have serious implications for society and democracy, including the proliferation of nonconsensual “deepfake” pornography, financial scams, and targeted election disinformation. This measure seeks to establish a framework to better enable Californians to identify GenAI developed content. In support of the bill the author states:
Generative Artificial Intelligence (AI) is advancing at an unprecedented pace, revolutionizing industries and everyday life. However, this rapid progress has also sparked a pressing need for regulatory frameworks to govern its use. While AI offers innovative solutions that drive economic growth and efficiency, it simultaneously presents complex challenges related to misinformation, manipulation, bias, and ownership of generated content.
One of the most concerning aspects of unregulated AI models is their potential to produce “deepfake” content, such as manipulated images and videos, which can be used to deceive or manipulate individuals and society at large. Deepfake pornography and fabricated political messages have already emerged as serious threats, highlighting the urgency of implementing effective regulations.
As AI-generated content approaches a level of realism that makes it indistinguishable from genuine content, the need for transparency becomes paramount. It is crucial for industry leaders to take responsibility for clearly labeling AI-generated content and developing tools that can accurately identify such content, especially when there is no visible disclosure. The transparency SB 942 would provide is essential for establishing trust in AI technologies and ensuring that users can make informed decisions about the content they consume.
Generative Artificial Intelligence vs. Predictive Artificial Intelligence. The development of GenAI offers significant potential to enhance California’s economy and improve residents' lives by rapidly generating text, images, and audio. However, these advancements come with new safety concerns. Artificial intelligence, broadly speaking, involves computers mimicking human intelligence through algorithms that process various inputs into outputs, such as text or video. Predictive AI uses specific datasets to make recommendations, while GenAI utilizes vast datasets to produce detailed content. For instance, when a streaming service suggests a TV show to a viewer, the recommendation is produced by predictive AI that has been trained on the viewing habits of the service’s users. Conversely, when a service like ChatGPT generates text in clear, concise paragraphs, it uses generative AI that has been trained on the written contents of the internet. ChatGPT is an example of an open-sourced tool, meaning it is accessible to the public. Researchers and developers can also access its code and parameters. This accessibility increases transparency, but it has downsides: when a tool’s code and parameters can be easily accessed, they can be easily altered, and open-source tools have the potential to be used for nefarious purposes.
The need for this bill is further highlighted by various instances and research underscoring the threats posed by unregulated GenAI. While image manipulation and video doctoring have long existed, the advent of GenAI, in combination with the abundance of open sourced GenAI tools available, has drastically simplified these processes, enabling the creation of convincing fake content with minimal effort. GenAI’s ability to generate realistic imagery, video, and audio can pose significant risks when used maliciously, as they can produce “deepfakes” pornography, targeted propaganda, and other disinformation that can harm individuals and destabilize societies.
For instance, earlier this year voters in New Hampshire received phone calls from an AI-generated voice mimicking President Joe Biden misleadingly urging them not to vote in the primary election in order to save their vote for the upcoming general election in November. (Em Steck & Andrew Kaczynski, Fake Joe Biden robocall urges New Hampshire voters not to vote in Tuesday’s Democratic primary (January 22, 2024) CNN, https://www.cnn.com/2024/01/22/politics/fake-joe-biden-robocall/index.html.) AI-generated political disinformation poses a threat to democratic processes by spreading false information that can influence voter behavior. Not only can these fraudulent efforts potentially impact the upcoming election, deepfake scams have been effective in the private business sector as well. An employee of a multinational financial firm was defrauded of $25 million when he was invited to participate in a video call with several other colleagues, including the company’s CFO—or so he was told. All of the voices and people he saw on the call were AI-generated, and were convincing enough for the victim employee to agree to administer a secret transfer of $25 million. (WRAL, Finance Worker Pays Out $25 Million After Video Call with Deepfake Chief Financial Officer (February 4, 2024), https://www.wral.com/story/finance-worker-pays-out-25-
Potentially most troubling is the unethical use of AI tools against children in California. In April 2024, a Los Angeles area high school had to investigate and expel students who were behind the creation and circulation of fake sexual exploitative photos of sixteen fellow eighth-grade students. If a California student shares a nude photo of a classmate without consent, the student could be prosecuted under state laws dealing with child pornography and disorderly conduct, however, these laws would not necessarily apply to AI-generated "deepfake" photos. (FOX 11 Los Angeles, Fairfax High School Investigates AI-Generated Photos (June 23, 2024), https://www.foxla.com/news/fairfax-high-school-ai-generated-photos-lausd.) These challenges underscore the need for regulatory frameworks to govern the use of GenAI, ensuring that the public is protected from the potential harms of synthetic media.
This bill seeks to address these issues by establishing clear requirements for developers of GenAI systems. The bill mandates that AI-generated content must include provenance disclosures to verify its authenticity and origin. Additionally, it requires developers to provide tools that enable users to detect AI-generated content. The bill also seeks to ensure that those that generate open-source GenAI products police the third party licensees who use the software. To ensure targeted comprehensive coverage under this bill, any person or entity that produces a GenAI system with over 1,000,000 monthly users accessible within California must follow the provisions of the bill. This applies to any AI system that produces synthetic content, including text, images, videos, and audio, emulating the training data's structure and characteristics. This definition is crucial as it specifically targets the types of AI that can create misleading or harmful content, ensuring major players in the AI industry are regulated without overburdening smaller developers. These measures aim to enhance transparency and accountability, ensuring that consumers are informed about the nature of the content they encounter online.
The required disclosures should be embedded in the produced content and act as the providers’ unique identification marker. Under this bill, AI-generated content must include machine-readable disclosures with information such as the provider’s name, the GenAI system version, creation date, and a unique identifier. This requirement ensures that the origin of the content can be traced back to the specific GenAI system responsible for its creation. These “disclosures” are to be included in content produced by GenAI systems. The measure envisions two means to implement disclosures required by the bill – manifest (visible), and latent (imperceptible to the human eye). Disclosures should be permanent or difficult to remove and detectable by the provider's own systems. This measure ensures that even if visible labels are removed, the content can still be identified as AI-generated through technical means and the use of the latent disclosure. Providers must also offer users the option to include visible disclosures in AI-generated content, indicating it was created by AI. Visible disclosures provide immediate transparency, allowing users to easily recognize AI-generated content. The bill describes the minimum information that these disclosures must convey:
- The name of the covered provider.
- The name and version number of the GenAI system used.
- The time and date of the content’s creation or alteration.
- Which parts of the content were created or altered by the GenAI system.
- A unique identifier.
The bill requires providers to create publically accessible AI detection tools. Covered providers must create and make available AI detection tools that allow users to assess whether content has been created or altered by their GenAI system. By mandating these tools, this bill empowers consumers to identify AI-generated content, thereby reducing the spread of misinformation and enhancing accountability. The tools can be uniquely created to be tailored within the providers’ system but must meet the following criteria:
- The AI detection tool shall be publicly accessible and available via a uniform resource locator (URL) on the covered provider’s internet website and through its mobile application, as applicable.
- The AI detection tool shall allow a person to upload content or a URL.
- The AI detection tool shall support an application programming interface (API) that allows a person to invoke the AI detection tool without visiting the covered provider’s website.
- The AI detection tool shall allow a person to provide feedback if the person believes the AI detection tool is not properly identifying content that was created by the provider.
In theory, a person who views a video circulating on social media conveying President Joe Biden telling voters not to vote in the primary election could use a provider’s AI detection tool to upload and analyze the video. By examining the embedded machine-readable disclosures, the user could identify the provider's name, the GenAI system used, and the creation date, concluding that the video was produced by AI. This process would reveal that the video is not genuine, thus, in theory, helping to prevent the spread of misinformation.
The proposed amendments aim to enhance and align the bill’s current enforcement provisions. Providers licensing their GenAI systems must ensure that licensees maintain the system's disclosure capabilities. If these essential capabilities are removed, licenses must be promptly revoked to uphold the integrity of the regulations. A failure to do so would expose a covered provider to civil penalties of $5,000 per each day of non-compliance. These stringent enforcement mechanisms are crucial for ensuring consistent compliance, deterring potential violations, and maintaining the effectiveness and trustworthiness of the regulatory framework.
The bill in print specifies that civil actions against covered providers who violate the provisions of this bill requiring oversight of third-party licensees may be initiated solely by the Attorney General’s office. The bill in print also seeks to deter third-party licensees using the AI product from violating the bill. Civil actions seeking to enjoin unlawful conduct by third-party licensees may be brought by the Attorney General, as well as local public prosecutors. After discussions with the Committee the author agrees that vesting local public prosecutors with the authority to enforce the full scope of the measure would be prudent. To that end, the author is proposing to the following amendment to the proposed Civil Code Section 22757.4 to read as follows:
22757.4. (a) (1) A covered provider that violates this chapter shall be liable for a civil penalty in the amount of five thousand dollars ($5,000) per violation to be collected in a civil action filed only by the Attorney General, a city attorney, or county counsel.
(2) A prevailing plaintiff in an action brought pursuant to this subdivision shall be entitled to all attorneys’ costs and fees.
(b) Each day that a covered provider is in violation of this chapter shall be deemed a discrete violation.
(c) For a violation by a third-party licensee of paragraph (3) of subdivision (c) of Section 22757.3, the Attorney General, a district attorney, a county counsel, or a city attorney may bring a civil action for both of the following:
(1) Injunctive relief.
(2) Reasonable attorney’s fees and costs.
ARGUMENTS IN SUPPORT: In support of the bill, Protections of the Educational Rights of Kids (PERK) – Advocacy writes:
We support SB-942 because a California AI Transparency Act will allow people to query covered providers about which content they are consuming is generated, in whole or part, by an AI system. The bill will require providing a visible disclosure that includes a clear and conspicuous notice that identifies the content as generated by AI. As technology advances, the government must take steps like this to ensure that people and businesses creating content are responsible. An AI detection tool is a logical, reasonable, and proactive way to establish transparency and accountability.
The Transparency Coalition.AI also in support of the bill writes:
We firmly support SB-942 because the California AI Transparency Act will help bring much needed transparency requirements to generative AI outputs. It will provide the public with critical insight into which content (text, image, video, audio, or multimedia content) was created, in whole or in part by a generative AI system. As the legislature looks to protect the public and enact sensible guardrails as it relates to these systems, ensuring tools for the detection of materials produced by generative AI is among the most important.
ARGUMENTS IN OPPOSITION: The measure is also opposed by the California Chamber of Commerce, Technet, NetChoice, and the Computer & Communications Industry Association who jointly write:
SB-942 imposes technically infeasible and premature requirements on AI developers. While we support the bill’s intent to enhance trust in user-generated content through content provenance verifications and watermarks, the current technologies for content provenance are underdeveloped and unreliable. Research from the University of Maryland shows that existing methods can be easily broken or falsified. We advocate for waiting for federal standards, as proposed in President Biden’s AI Executive Order, to develop science-backed, consistent regulations across the country.
We emphasize the need to clearly differentiate between 1st party and 3rd party content in the bill. While 1st party content, created and distributed using a covered provider’s AI tools, can be reliably watermarked, it is currently infeasible to accurately detect and watermark 3rd party content generated by other platforms. Given the ease with which watermarks can be circumvented, imposing such requirements on 3rd party content would be inappropriate and technically impractical at this stage.
REGISTERED SUPPORT / OPPOSITION:
Support
Protection of the Educational Rights of Kids - Advocacy
Oakland Privacy
Transparency Coalition.AI
Opposition
California Chamber of Commerce
Computer and Communications Industry Association
Netchoice
Technet
Analysis Prepared by: Nicole Larson / JUD. / (916) 319-2334
Date of Hearing: June 18, 2024
ASSEMBLY COMMITTEE ON PRIVACY AND CONSUMER PROTECTION
Rebecca Bauer-Kahan, Chair
SBPCA Bill Id:SB 942
Author:(Becker) – As Amended Ver:May 16, 2024
AS PROPOSED TO BE AMENDED
SENATE VOTE: 32-1
SUBJECT: California AI Transparency Act
As generative artificial intelligence (GenAI) technologies become more accessible, online content that appears real – but is actually false – threatens to flood social media and other large online platforms. The unmitigated spread of synthetic content threatens to harm individual Californians in numerous ways, such as through the proliferation of nonconsensual deepfake pornography, scams, and the distribution of targeted election disinformation.
The issue of GenAI-produced content can be tacked through a three-step plan. First, all artificial content must be labeled as such. Second, all “real” content – such as video and audio recordings – must be labeled as such. Third, large online platforms must be required to prominently display these labels, allowing users to distinguish between artificial and real content.
SB 942 seeks to enact step 1 of this plan. It would require GenAI providers to embed latent, machine-readable disclosures into content produced by their systems. These disclosures would include information about the origin of the content, and would be designed to be difficult to remove. GenAI providers would also be required to develop and make freely available software that identifies content generated by their systems. Finally, the bill requires GenAI providers who license their systems to contractually require licensees to maintain the systems’ disclosure capabilities.
This bill is author-sponsored and supported by PERK Advocacy and Transparency Coalition.ai. It is opposed by a coalition of industry associations including Technet, Netchoice, Computer and Communications Industry Association, and the California Chamber of Commerce.
Proposed Committee amendments revise and clarify the bill. The full text of the amended bill is included at the end of the analysis. If the bill passes this Committee, it will next be heard by the Assembly Judiciary Committee.
SUMMARY: Requires the developers of GenAI systems to both include provenance disclosures in the content their systems produce, and make tools available to identify GenAI-content produced by their systems. Specifically, this bill:
- Defines artificial intelligence (AI) to mean an engineered or machine-based system that varies in its level of autonomy and that can, for explicit or implicit objectives, infer from the input it receives how to generate outputs that can influence physical or virtual environments.
- Defines GenAI to mean artificial intelligence that can generate derived synthetic content, including text, images, video, and audio, that emulates the structure and characteristics of the system’s training data.
- Defines “covered provider” to mean a person that creates, codes, or otherwise produces a GenAI system that has over 1,000,000 monthly visitors or users and is publicly accessible within the geographic boundaries of the state.
- Defines “provenance data” to mean data that is embedded into digital content, or that is included in the digital content’s metadata, for the purpose of verifying the digital content’s authenticity, origin, or history of modification.
- Defines “personal provenance data” to mean provenance data that contains either personal information or unique device, system, or service information that is reasonably capable of being associated with a particular user.
- Defines “system provenance data” to mean provenance data that is not reasonably capable of being associated with a particular user and that contains either information regarding the type of device, system, or service that was used to generate a piece of digital content, or information that provides proof of content authenticity.
- Requires a covered provider to create and make freely available an AI detection tool that can be used to assess whether digital content was created or altered by the covered provider’s GenAI system, and that has various specified characteristics.
- Prohibits a covered provider from collecting or retaining personal information from users of the covered provider’s AI detection tool, or retaining personal provenance data from user-uploaded content, except as specified.
- Requires a covered provider to offer a user of a GenAI tool the option to include a manifest disclosure in generated content, with various specified characteristics.
- Requires a covered provider include a latent disclosure in generated content, with various specified characteristics.
- Requires that if a covered provider licenses their GenAI system to a third party, the covered provider shall require by contract that the licensee maintain the system’s capability to include a latent disclosure.
- Requires that if a covered provider knows a licensee has removed the capability of a licensed GenAI system to include a latent disclosure, the covered provider shall revoke the licensee’s contract.
- Requires that a licensee cease using a licensed GenAI system after its license has been revoked.
- Provides that a covered provider that violates the chapter be liable for a civil penalty of $5000 to be collected in a civil action filed by the Attorney General only.
- Provides that the Attorney General, a district attorney, a county counsel, or a city attorney may bring a civil action for injunctive relief and reasonable attorney’s fees and costs against a third party licensee who violates this chapter.
EXISTING LAW:
- Provides, pursuant to the California Constitution, that all people are by nature free and independent and have inalienable rights. Among these are the fundamental right to privacy. (Cal. Const. art. I, § 1.)
- States that the “right to privacy is a personal and fundamental right protected by Section 1 of Article I of the Constitution of California and by the United States Constitution and that all individuals have a right of privacy in information pertaining to them.” Further states these findings of the Legislature:
- The right to privacy is being threatened by the indiscriminate collection, maintenance, and dissemination of personal information and the lack of effective laws and legal remedies.
- The increasing use of computers and other sophisticated information technology has greatly magnified the potential risk to individual privacy that can occur from the maintenance of personal information.
- In order to protect the privacy of individuals, it is necessary that the maintenance and dissemination of personal information be subject to strict limits. (Civ. Code § 1798.1.)
- Defines “deepfake” to mean audio or visual content that has been generated or manipulated by artificial intelligence (AI) which would falsely appear to be authentic or truthful and which features depictions of people appearing to say or do things they did not say or do without their consent. Requires the Secretary of Government Operations to evaluate the impact of the proliferation of deepfakes on the state. (Gov. Code § 11547.5.)
- Defines “personal information” to mean information that identifies, relates to, describes, is reasonably capable of being associated with, or could reasonably be linked, directly or indirectly, with a particular consumer or household. States that personal information includes, but is not limited to, the following if it identifies, relates to, describes, is reasonably capable of being associated with, or could be reasonably linked, directly or indirectly, with a particular consumer or household (Civ. Code § 1798.140(v).):
- Identifiers such as a real name, alias, postal address, unique personal identifier, online identifier, Internet Protocol address, email address, account name, social security number, driver’s license number, passport number, or other similar identifiers.
- Any personal information described in Section 1798.80(e).
- Characteristics of protected classifications under California or federal law.
- Commercial information, including records of personal property, products or services purchased, obtained, or considered, or other purchasing or consuming histories or tendencies.
- Biometric information.
- Internet or other electronic network activity information, including, but not limited to, browsing history, search history, and information regarding a consumer’s interaction with an internet website application, or advertisement.
- Geolocation data.
- Audio, electronic, visual, thermal, olfactory, or similar information.
- Professional or employment-related information.
- Education information, defined as information that is not publicly available personally identifiable information as defined in the Family Educational Rights and Privacy Act. (20 U.S.C. Sec. 1232g; 34 C.F.R. Part 99).
- Inferences drawn from any of the information identified in this subdivision to create a profile about a consumer reflecting the consumer’s preferences, characteristics, psychological trends, predispositions, behavior, attitudes, intelligence, abilities, and aptitudes.
- Sensitive personal information.
FISCAL EFFECT: As currently in print this bill is keyed fiscal.
COMMENTS:
- AI and GenAI. The development of GenAI is creating exciting opportunities to grow California’s economy and improve the lives of its residents. GenAI can generate compelling text, images and audio in an instant – but with novel technologies come novel safety concerns.
What is AI? In brief, AI is the mimicking of human intelligence by artificial systems such as computers. AI uses algorithms – sets of rules – to transform inputs into outputs. Inputs and outputs can be anything a computer can process: numbers, text, audio, video, or movement. AI is not fundamentally different from other computer functions; its novelty lies in its application. Unlike normal computer functions, AI is able to accomplish tasks that are normally performed by humans.
AI that are trained on small, specific datasets in order to make recommendations and predictions are sometimes referred to as “predictive AI.” This differentiates them from GenAI, which are trained on massive datasets in order to produce detailed text and images. When Netflix suggests a TV show to a viewer, the recommendation is produced by predictive AI that has been trained on the viewing habits of Netflix users. When ChatGPT generates text in clear, concise paragraphs, it uses GenAI that has been trained on the written contents of the internet.
GenAI tools can be released in open-source or closed-source formats by their creators. Open-source tools are publically available; researchers and developers can access their code and parameters. This accessibility increases transparency, but it has downsides: when a tool’s code and parameters can be easily accessed, they can be easily altered, and open-source tools have the potential to be used for nefarious purposes such as generating deepfake pornography and targeted propaganda. By comparison, closed-source tools are opaque with respect to their security features. It is harder for bad actors to generate illicit materials using these tools. But unlike open-source tools, closed-source tools are not subject to collective oversight because their inner workings cannot be examined by independent experts.
- Deepfakes and disinformation. Image manipulation and video doctoring have existed for nearly as long as photography and recording equipment, but they have historically required great effort and talent. In the past few years the rapid development of GenAI has drastically reduced those barriers to entry, allowing a vast quantity of convincing – but ultimately fake – content to be generated in an instant. The creation of imagery, video, and audio by GenAI has the potential to change the world by automating repetitive tasks and fostering creativity. When employed by bad actors, however, these capabilities have the potential to destroy lives and destabilize societies.
Nonconsensual pornography. GenAI has been used to create pornography since its inception. This content is inevitably nonconsensual, and as GenAI technology improves, these products will become harder to distinguish from reality. Women are the primary victims of these efforts; in the run-up to the 2024 Super Bowl, a series of images involving Taylor Swift began to appear on the social media platform X (formerly Twitter). These images were removed over the following days, but the damage had been done:
“We are too little, too late at this point, but we can still try to mitigate the disaster that’s emerging,” says Mary Anne Franks, a professor at George Washington University Law School and president of the Cyber Civil Rights Initiative. Women are “canaries in the coal mine” when it comes to the abuse of artificial intelligence, she adds. “It's not just going to be the 14-year-old girl or Taylor Swift. It’s going to be politicians. It’s going to be world leaders. It’s going to be elections.”[1]
The harms of nonconsensual AI-powered pornography are already being felt in California:
A third school in Southern California has been hit with allegations of digitally manipulated images of students circulating around campus . . . “Sixteen eighth-grade students were identified as being victimized, as well as five egregiously involved eighth-grade students,” Superintendent Michael Bregy wrote. While Bregy acknowledged that children “are still learning and growing, and mistakes are part of the process,” he affirmed disciplinary measures had been taken and noted that the incident was swiftly contained. The district vowed to hold accountable any other students “found to be creating, disseminating, or in possession of AI-generated images of this nature.”[2]
Scams. GenAI-powered speech and video is driving a new era in scamming. These AI tools are often trained on publicly available data – the more data a target has online, the easier it is to develop a passable imitation of them or their loved ones. This is especially true of wealthy clients, whose public appearances, including speeches, are often widely available on the internet.[3] For example, a complicated scam utilizing both deepfake video and false audio was recently performed in Hong Kong. A multinational company lost $25.6 million after employees were fooled by deepfake technology, with one incident involving a digitally recreated version of its chief financial officer ordering money transfers in a video conference call. Everyone present on the video call, except the victim, was a fake representation of real people. The scammers applied deepfake technology to turn publicly available video and other footage into convincing versions of the meeting’s participants.[4]
Political propaganda and disinformation. Deepfake technology is being used around the world to spread disinformation and propaganda. 2024 is a major election year in democracies around the globe: at least 64 countries will hold elections, representing close to 49% of the world’s population.[5] It is also likely to be the first of many election years in which AI plays a pivotal role, as the technology becomes more widely available and easier to use. This has already been observed in Slovakia, where deepfake audio influenced an election in 2023:
Days before a pivotal election in Slovakia to determine who would lead the country, a damning audio recording spread online in which one of the top candidates seemingly boasted about how he’d rigged the election. And if that wasn’t bad enough, his voice could be heard on another recording talking about raising the cost of beer. The recordings immediately went viral on social media, and the candidate, who is pro-NATO and aligned with Western interests, was defeated in September by an opponent who supported closer ties to Moscow and Russian President Vladimir Putin.[6]
Similar deepfakes have now been deployed in the United States in advance of the 2024 presidential election. In late January, between 5000 and 20,000 New Hampshire residents received AI-generated phone calls impersonating President Biden that told them not to vote in the state’s primary. The call told voters: “It’s important that you save your vote for the November election.” Concern about this call has led at least 14 states to introduce legislation targeting AI-powered disinformation.[7] It is still unclear how many people might not have voted based on these calls.
Deepfakes are not only being deployed by third parties; they can be used by the candidates themselves, either to improve their own self-images or to detract from their opponents. In mid-2023, former Republican presidential candidate Governor Ron DeSantis used AI to add fighter jets to one of his campaign videos.[8] Around the same time, Governor DeSantis’ super PAC released an ad containing an AI-generated speech by former president Donald Trump.[9] The Republican National Committee also released a 30-second ad that displayed images of disorder and destruction, with a voiceover that described the “consequences” of re-electing President Biden.[10] None of the images in this ad were real.
- Content provenance. Many of the issues associated with deepfakes could be resolved, if only there were a reliable way to identify GenAI content. While at present no single solution exists, there are ongoing efforts to embed information related to “content provenance” – the verifiable history of a piece of content – into both GenAI products and the products of real-life recorders, such as digital cameras. Under this framework, the users of social media platforms would be able to rely on provenance data to identify trustworthy content.
Metadata. The Merriam-Webster Dictionary defines metadata as “data that provides information about other data.” In practice, metadata is a structured set of descriptors attached to digital content. A photograph’s metadata might include information about the camera used to take the photo, the time and date the photo was taken, and the photo’s precise geolocation. A written document’s metadata may include details about its author, its creation date, and the number of times the document has been edited. Metadata can be used to verify content authenticity, helping to combat fake news by providing a traceable history of content creation and modification. But metadata can also contain personal information about the individual who creates or modifies a piece of content.
Watermarking. Watermarking is the process of embedding an identifiable marker into digital content. This “watermark” can be visible, such as a logo or text overlay, or it can be invisible, data embedded into a file in a manner that does not noticeably alter the content. As a technology, watermarking is in its infancy. Watermarks can be stripped from content relatively easily by common screenshotting tools, file compression software, and image editing programs like Photoshop. They can also be faked by treating a GenAI system like a copy machine: a real image or video can be fed into a GenAI system and spit back out, unaltered except for the addition of a watermark. The system’s user receives a piece of authentic content that has been incorrectly marked as inauthentic, ready to be posted online in order to create confusion and sow discord. Furthermore, while progress has been made towards developing standards for watermarking of images and video, how text should be watermarked is far less clear.
- What this bill would do. This bill has four basic components. First, it requires GenAI system providers include disclosures in content produced by their systems. The bill describes two types of disclosures: manifest and latent. Manifest disclosures are optional, and primarily serve to clearly identify content as AI-generated. Latent disclosures are mandatory, and serve as an imperceptible indicator that content is AI-generated. These disclosures are required to be interpretable by a provider’s AI detection tool, detailed below.
Second, it requires GenAI system providers create and make freely available AI detection tools that can identify content created by the providers’ systems. When a user uploads content to be analyzed, the tool is required to output information related to the system-of-origin of the content, and withhold any personal information included in the disclosure.
Third, it requires that GenAI system providers who license their systems to third parties contractually obligate licensees to retain the systems’ disclosure capabilities. If a provider knows that a licensee has stripped a licensed system of its disclosure capabilities, the provider is required to revoke their license. A licensee who has had their license revoked must stop using the system.
Fourth, the bill outlines enforcement mechanisms. Under the bill’s enforcement scheme, a GenAI system provider who violates the bill’s provisions is liable for a civil penalty of $5000, to be collected in a civil action filed only by the Attorney General. A third party licensee who violates the bill’s provisions may have a civil action brought against them by the Attorney General, a district attorney, a county counsel, or a city attorney, for injunctive relief and reasonable attorney’s costs and fees.
- Author’s statement. According to the author:
Generative Artificial Intelligence (AI) is advancing at an unprecedented pace, revolutionizing industries and everyday life. However, this rapid progress has also sparked a pressing need for regulatory frameworks to govern its use. While AI offers innovative solutions that drive economic growth and efficiency, it simultaneously presents complex challenges related to misinformation, manipulation, bias, and ownership of generated content.
One of the most concerning aspects of unregulated AI models is their potential to produce “deepfake” content, such as manipulated images and videos, which can be used to deceive or manipulate individuals and society at large. Deepfake pornography and fabricated political messages have already emerged as serious threats, highlighting the urgency of implementing effective regulations.
As AI-generated content approaches a level of realism that makes it indistinguishable from genuine content, the need for transparency becomes paramount. It is crucial for industry leaders to take responsibility for clearly labeling AI-generated content and developing tools that can accurately identify such content, especially when there is no visible disclosure. The transparency SB 942 would provide is essential for establishing trust in AI technologies and ensuring that users can make informed decisions about the content they consume.
- Analysis. As noted in this committee’s analysis of AB 3211 (Wicks, 2024), the issue of content authentication can theoretically be solved in three easy steps:
- Require all content produced by GenAI to be labeled “fake.”
- Require all content produced by cameras and other recording devices to be labeled “real.”
- Require all social media platforms to prominently display these labels.
Passing the current bill would represent meaningful progress towards “step 1” of this plan. SB 942 requires that “disclosures” be included in content produced by GenAI systems. Of the two disclosures required by the bill – manifest (visible), and latent (imperceptible to the human eye) – the latent disclosure requirement is more important by far. The bill describes the minimum information that these disclosures must convey:
The name of the covered provider.
The name and version number of the GenAI system used.
The time and date of the content’s creation or alteration.
Which parts of the content were created or altered by the GenAI system.
A unique identifier.
It may not be possible to embed all of these details directly into a single disclosure, especially in content that is small or low-resolution. The bill accounts for this in two ways: first, it allows a GenAI provider to instead embed a link to a permanent internet website into the content. This link, which would be comparatively small, could be repeated many times within a single piece of content. This would impart a degree of stability onto the disclosure – cropping or obscuring large parts of the content would not necessarily affect the disclosure’s ability to be read. Second, the bill does not actually specify a disclosure mechanism for GenAI content. The bill very carefully avoids the term “watermark” – a GenAI provider may choose to use a watermark, but they may also choose to adopt a non-watermark-based approach. The language surrounding this disclosure is left intentionally ambiguous.
Aside from requiring disclosures in GenAI content, the bill requires GenAI providers develop and make freely available tools to identify content generated by their systems. This approach is fraught: as pointed out by Oakland Privacy’s “support if amended” letter, AI detection tools are frequently found to be unreliable:
How bad are AI detection tools? Pretty bad. Content-At-Scale, one of the largest AI detection tools identified sections of the United States Constitution as highly likely to be AI generated. The most developed tools address text and rely on certain indicators including predictability, variety of sentence length and structure, word repetition, unnatural word usage, and inconsistent verb tense. It goes without saying that many of these are also characteristics of poorly written human-generated text. The tools are dealing in probabilities and while they may effectively flag bad writing that *could* be AI-generated, they are simply not accurate enough to be relied on as evidence of anything.
However, the current bill does not require the creation of tools that detect all AI-generated content; instead, this bill only requires GenAI providers be able to identify their own systems’ outputs. Having intimate knowledge of the exact disclosure mechanisms included in their systems’ outputs may allow GenAI developers to succeed here, despite the frequent failures of more “general” tools. This is even truer of GenAI systems that exist entirely online: by keeping a record of all content their systems produce (perhaps in a privacy-protective form, like a cryptographic hash), GenAI providers can quickly and easily determine whether user-uploaded content was generated by their systems.
Furthermore, the examples referenced by Oakland Privacy relate primarily to the detection of AI-generated text. The current bill recognizes that embedding disclosures into text is not yet feasible at scale, and excludes text from the list of digital content that requires disclosure. However, as the inclusion of the phrase “other digital content” could be construed to include text, the authors may wish to edit the language of the bill to clarify whether or not AI-generated text is covered.
Relationship to AB 3211 (Wicks, 2024). AB 3211 would require GenAI providers and the manufacturers of recording devices to include watermarking capabilities in the systems and devices they make available in California, and require social media platforms to label content with provenance info extracted from uploaded content. This bill overlaps with AB 3211 nearly entirely. Where this bill only seeks to address “step 1” in the three-step content authentication plan outlined above, AB 3211 seeks to implement the full plan. Where this bill lacks a particular mechanism for provenance disclosure, AB 3211 specifically requires watermarking. And where this bill’s philosophy is to set a “floor” that GenAI providers are able to innovate around and above, AB 3211 is far more prescriptive in its requirements.
Relationship to AB 1791 (Weber, 2024). AB 1791 would require social media platforms to strip provenance data containing personal information from content uploaded to the platform, while retaining provenance data related to system-of-origin. If this bill represents an attempt to fulfill “step 1” in the three-step plan outlined above, AB 1791 represents an attempt to fulfill “step 3”. The two bills share the language of “provenance data,” “personal provenance data,” and “content provenance data,” and are broadly compatible as a result.
Effect on open source. Software, unlike hardware, cannot be destroyed or contained. This is especially true of open-source software, which is generally free to use, edit, and otherwise manipulate. The concept of open-source is central to the tech industry’s ability to innovate and expand, and the tech industry is central to California’s culture and economy. But the current bill – and many other legislative efforts in California – are predicated on the notion that certain GenAI applications are inherently risky. This bill requires disclosure capabilities be built into GenAI systems. This requirement would seem antithetical to the concept of open source. How can this circle be squared?
The question of how to regulate GenAI applications without trampling on the notion of “open-source” is currently bedeviling lawmakers around the world. Take the current bill: SB 942 requires GenAI systems include disclosure capabilities. How can this be guaranteed? Legislators could specifically require that the disclosure capabilities be permanent and un-removable. But open-source requires software be freely alterable – clearly, these two ideas are not compatible. The current bill instead introduces a novel concept for maintaining disclosure capabilities in GenAI systems, even as those systems are shared and altered.
Under the provisions of this bill, any GenAI provider who licenses their system to a third party must require, as a term of the contract, that the third party not remove their system’s disclosure capability. This is not a physical limitation: the licensee could, if they truly desired, remove the disclosure capability. But if a covered provider discovers that a licensee has done so, the bill requires that the provider revoke the third party’s license. A licensee who continues to use a GenAI system once this license has been revoked (only in response to the removal of the system’s disclosure capabilities, not for other reasons) may have a civil action brought against them for injunctive relief and reasonable costs and attorney’s fees. At every step in this process, the third party is encouraged to stop using the system to create content without disclosures. On the spectrum of actions that could be taken against the users of open-source GenAI software, this approach involves a relatively light touch. Other bills seeking to prevent GenAI systems from being used in dangerous ways might consider adopting something similar.
- Related legislation.
AB 1791 (Weber, 2024) would require social media platforms to strip provenance data containing personal information from content uploaded to the platform, while retaining provenance data related to system-of-origin. The bill is pending in Senate Judiciary Committee.
AB 3050 (Low, 2024) would require CDT to issue regulations to establish standards for watermarks to be included in covered AI-generated material. The bill died in this Committee.
AB 3211 (Wicks, 2024) would require GenAI providers and the manufacturers of recording devices to include watermarking capabilities in the systems and devices they make available in California, and require social media platforms to label content with provenance info extracted from uploaded content. The bill is pending in Senate Judiciary Committee.
- Committee amendments. Several committee amendments clarify the bill’s disclosure requirements and update enforcement language to reflect the responsibilities of both the licensers and licensees of GenAI systems. The bill’s full text is reproduced below, as proposed to be amended:
SECTION 1. Chapter 25 (commencing with Section 22757) is added to Division 8 of the Business and Professions Code, to read:
Chapter 25. AI Transparency Act
22757. This chapter shall be known as the California AI Transparency Act.
22757.1. As used in this chapter:
(a) “AI detection tool” means the tool required by Section 22757.2.
(b)
“Artificial intelligence” or “AI” means a machine-based system that can, for a given set of human-defined objectives, make predictions, recommendations, or decisions influencing real or virtual environments by using machine-based inputs and human-based inputs to perceive real and virtual environments, abstract its perceptions into models through analysis in an automated manner, and use model inference to formulate options for information or action.
(c) “AI-generated content” means any form of digital content that is created with deep learning or machine learning processes.
(d) “Business” has the same meaning as defined in Section 1798.140 of the Civil Code.
(e)
(a) “Artificial intelligence” or “AI” means an engineered or machine-based system that varies in its level of autonomy and that can, for explicit or implicit objectives, infer from the input it receives how to generate outputs that can influence physical or virtual environments.
(b) “Covered provider” is a business means a person that provides creates, codes, or otherwise produces a generativeAI artificial intelligence system that has, on average over the preceding 12 months, has over 1,000,000 monthly visitors or users and is publicly accessible within the geographic boundaries of the state.
(f) “Department” means the Department of Technology.
(g)
(c) “Generative AI system” refers to deep learning models that can generate text, images, and other content based on the data they were trained on. artificial intelligence” or “GenAI” means an artificial intelligence that can generate derived synthetic content, including text, images, video, and audio, that emulates the structure and characteristics of the system’s training data.
(h)
(d) “Metadata” means structural or descriptive information aboutdata, including content, format, source, rights, accuracy, provenance, frequency, periodicity, granularity, publisher or responsible party, contact information, method of collection, and other descriptions. data.
(i) “Model” means a component of an information system that implements artificial intelligence technology and uses computational, statistical, or machine learning techniques to produce outputs from a given set of inputs.
(j) “Person” means a natural person located within the geographic boundaries of the state.
(k)
(e) “Personal information” has the same meaning as defined in Section 1798.140 of the Civil Code.
(f) “Personal provenance data” means provenance data that contains either of the following:
(1) Personal information.
(2) Unique device, system, or service information that is reasonably capable of being associated with a particular user.
(g) “Provenance data” means data that is embedded into digital content, or that is included in the digital content’s metadata, for the purpose of verifying the digital content’s authenticity, origin, or history of modification.
(h) “System provenance data” means provenance data that is not reasonably capable of being associated with a particular user and that contains either of the following:
(1) Information regarding the type of device, system, or service that was used to generate a piece of digital content.
(2) Information that provides proof of content authenticity.
22757.2. (a) A covered provider shall create and make freely available an AI detection tool by which a person can query the covered provider as to the extent to which text, image, video, audio, or multimedia content was created, in whole or in part, by a generative AI system provided by the covered provider that meets all of the following criteria: that meets all of the following criteria:
(1) The tool allows a user to assess whether image, video, audio, or other digital content was created or altered by the covered provider’s GenAI system.
(2) The tool allows a user to determine which parts of the content were created or altered by the covered provider’s GenAI system.
(3) The tool outputs any system provenance data that is detected in the content.
(4) The tool does not output any personal provenance data that is detected in the content.
(1)
(5) The AI detection tool shall be The tool is publicly accessible and available via a uniform resource locator (URL) on through the covered provider’s internet website and through its mobile application, as applicable.
(2)
(6) The AI detection tool shall allow The tool allows a person user to upload content or provide a URL. uniform resource locator (URL) linking to online content.
(3)
(7) The AI detection tool shall support The tool supports an application programming interface (API) that allows a person user to invoke the AI detection tool without visiting the covered provider’s internet website.
(4)
(b) The AI detection tool A covered provider shall allow a person to provide collect user feedback if the person believes the AI detection tool is not properly identifying content that was created by the covered provider. related to the efficacy of the covered provider’s AI detection tool and incorporate that feedback into any attempt to improve the efficacy of the tool.
(b)
(c) In complying with this section, a A covered provider shall not do any of the following:
(1) Reveal personal information that identifies who utilized the covered provider’s generative AI system to create AI-generated content that was submitted to the covered provider’s AI detection tool.
(2)
(1) (A) Subject to Except as provided in subparagraph (B), collect and or retain personal information when a person utilizes from users of the covered provider’s AI detection tool.
(B) (i) A covered provider may collect and retain the contact information of a person user who submitted submits feedback pursuant toparagraph (4) of subdivision (a). subdivision (b) if the user opts in to being contacted by the covered provider.
(ii) User information collected pursuant to clause (i) shall be used only to evaluate and improve the efficacy the covered provider’s AI detection tool.
(3)
(2) Retain any content submitted to the AI detection tool for longer than is necessary to comply with this section.
(3) Retain any personal provenance data from content submitted to the AI detection tool by a user.
22757.3. (a) A covered provider shall offer the user an option to include in AI-generated a manifest disclosure in image, text, video, audio, or multimedia other digital content created or altered by a generative AI the covered provider’s GenAI system it provides a visible disclosure that meets all of the following criteria:
(1) The disclosure shall include a clear and conspicuous notice, as appropriate for the medium of the content, that identifies the content as generated by AI, such that the disclosure is not avoidable, is understandable to a reasonable person, and is not contradicted, mitigated by, or inconsistent with anything else in the communication.
(1) The disclosure identifies content as AI-generated content.
(2) The disclosure is clear, conspicuous, appropriate for the medium of the content, and understandable to a reasonable person.
(2)
(3) The disclosure shall, to the extent technically feasible, shall be permanent or extraordinarily difficult to remove. remove, to the extent it is technically feasible.
(3) The output’s metadata information shall include an identification of the content as being generated by AI, the identity of the tool used to create the content, and the date and time the content was created.
(b) A covered provider shall include in AI-generated image, audio, video, or multimedia content created by a generative AI system an imperceptible disclosure that is machine detectable and is, to the extent technically feasible, permanent or difficult to remove.
(c) A covered provider shall implement reasonable procedures to prevent downstream use of a generative AI system it provides without the disclosure required by this section, including by doing both of the following:
(1) Requiring by contract that third-party licensees of the generative AI system refrain from removing a required disclosure.
(2) Terminating access to the generative AI system when the covered provider has reason to believe that a third-party licensee has removed a required disclosure.
(d) At least once every two years, the department shall review this section and make recommendations to the Legislature regarding any amendments needed to account for changing technology and standards.
(b) A covered provider shall include a latent disclosure in AI-generated image, video, audio, or other digital content created by the covered provider’s GenAI system that meets all of the following criteria:
(1) The disclosure conveys all of the following information, either directly or through a link to a permanent internet website:
(A) The name of the covered provider.
(B) The name and version number of the GenAI system that created or altered the content.
(C) The time and date of the content’s creation or alteration.
(D) Which parts of the content were created or altered by the GenAI system.
(E) A unique identifier.
(2) The disclosure is detectable by the covered provider’s AI detection tool.
(3) The disclosure is consistent with widely accepted industry standards.
(4) The disclosure is permanent or extraordinarily difficult to remove, to the extent it is technically feasible.
(c) (1) If a covered provider licenses its GenAI system to a third party, the covered provider shall require by contract that the licensee maintain the system’s capability to include a disclosure required by subdivision (b) in content the system creates or alters.
(2) If a covered provider knows that a third-party licensee modified a licensed GenAI system such that it is no longer capable of including a disclosure required by subdivision (b) in content the system creates or alters, the covered provider shall revoke the license within 72 hours of discovering the licensee’s action.
(3) A third-party licensee shall cease using a licensed GenAI system after the license for the system has been revoked by the covered provider pursuant to paragraph (2).
22757.4. (a) A covered provider that violates this chapter shall be liable for a civil penalty in the amount of five thousand dollars ($5,000) per violation to be collected in a civil action filed only by the Attorney General.
(b) Each day that a covered provider is in violation of this chapter shall be deemed a discrete violation.
(c) For a violation by a third-party licensee of paragraph (3) of subdivision (c) of Section 22757.3, the Attorney General, a district attorney, a county counsel, or a city attorney may bring a civil action for both of the following:
(1) Injunctive relief.
(2) Reasonable attorney’s fees and costs.
ARGUMENTS IN SUPPORT:
Taking a “support if amended” position on a previous version of the bill Oakland Privacy previously explains the need for disclosure of AI-generated content:
We consider generative AI disclosures to be akin to these measures and to be reasonable measures to protect the integrity of public discussions. A buyer beware absolutism threatens to poison all public discourse by causing massive confusion about what is authentic and what is not. It is likely the higher the stakes in a public debate, the more generative AI content will be created to try to push public opinion in one direction or the other. For the sake of the general welfare, disclosure will at least provide a modicum of accuracy to various kinds of difficult and controversial subjects without banning or removing content.
ARGUMENTS IN OPPOSITION:
A coalition of industry associations including Technet, NetChoice, Computer & Communications Industry Association, and CalChamber writes in opposition, describing their desire for a “federal standard” to be adopted:
While we understand the desire to regulate an emerging technology, this is an area that would benefit from Federal standards and regulation rather than a state by state approach. In President Biden’s AI Executive Order, he tasked the Department of Commerce with “identifying the existing standards, tools, methods, and practices, as well as the potential development of further science-backed standards and techniques, for: (i) authenticating content and tracking its provenance; (ii) labeling synthetic content, such as using watermarking; (iii) detecting synthetic content” and more. We believe in allowing this federal process to advance in order to establish standards that are “science-backed” and can be consistently applied across the country is important.
REGISTERED SUPPORT / OPPOSITION:
Support
Protection of The Educational Rights of Kids - Advocacy (PERK Advocacy)
Transparency Coalition.ai
Support If Amended
Oakland Privacy
Opposition
California Chamber of Commerce
Computer and Communications Industry Association
Netchoice
Technet
Analysis Prepared by: Slater Sharp / P. & C.P. / (916) 319-2200
Brian Contreras, "Tougher AI Policies Could Protect Taylor Swift—And Everyone Else—From Deepfakes," Feb. 8. 2024, www.scientificamerican.com/article/tougher-ai-policies-could-protect-taylor-swift-and-everyone-else-from-deepfakes/. ↑
Mackenzie Tatananni, “'Inappropriate images' circulate at yet another California high school, as officials grapple with how to protect teens from AI porn created by classmates,” Daily Mail, Apr. 11, 2024, https://www.dailymail.co.uk/news/article-13295475/Inappropriate-images-California-Fairfax-High-School-AI-deepfake.html ↑
Emily Flitter and Stacy Cowley, "Voice Deepfakes Are Coming for Your Bank Balance", New York Times, Aug. 30, 2023, www.nytimes.com/2023/08/30/business/voice-deepfakes-bank-scams.html. ↑
Harvey Kong, "‘Everyone looked real’: multinational firm’s Hong Kong office loses HK$200 million after scammers stage deepfake video meeting," South China Morning Post, Feb. 4, 2024, www.scmp.com/news/hong-kong/law-and-crime/article/3250851/everyone-looked-real-multinational-firms-hong-kong-office-loses-hk200-million-after-scammers-stage. ↑
Koh Ewe, "The Ultimate Election Year: All the Elections Around the World in 2024," Time, Dec. 28, 2023, https://time.com/6550920/world-elections-2024/. ↑
Curt Devine, Donie O'Sullivan, Sean Lyngass, "A fake recording of a candidate saying he’d rigged the election went viral. Experts say it’s only the beginning," CNN, Feb. 1, 2024, www.cnn.com/2024/02/01/politics/election-deepfake-threats-invs/index.html. ↑
Adam Edelman, "States turn their attention to regulating AI and deepfakes as 2024 kicks off," NBC News, Jan. 22, 2024, www.nbcnews.com/politics/states-turn-attention-regulating-ai-deepfakes-2024-rcna135122. ↑
Ana Faguy, "New DeSantis Ad Superimposes Fighter Jets In AI-Altered Video Of Speech," Forbes, May 25, 2023, www.forbes.com/sites/anafaguy/2023/05/25/new-desantis-ad-superimposes-fighter-jets-in-ai-altered-video-of-speech/. ↑
Alex Isenstadt, "DeSantis PAC uses AI-generated Trump voice in ad attacking ex-president," Politico, Jul. 17, 2023, www.politico.com/news/2023/07/17/desantis-pac-ai-generated-trump-in-ad-00106695. ↑
GOP, "Beat Biden," Apr. 25, 2023, https://www.youtube.com/watch?v=kLMMxgtxQ1Y. ↑
SENATE RULES COMMITTEE Office of Senate Floor Analyses (916) 651-1520 Fax: (916) 327-4478 | SB 942 |
THIRD READING
Bill No: SB 942
Author: Becker (D)
Amended: 5/16/24
Vote: 21
SENATE JUDICIARY COMMITTEE: 10-1, 4/16/24
AYES: Umberg, Allen, Ashby, Caballero, Durazo, Laird, Min, Niello, Stern, Wahab
NOES: Wilk
SENATE GOVERNMENTAL ORG. COMMITTEE: 11-1, 4/23/24
AYES: Dodd, Alvarado-Gil, Archuleta, Ashby, Bradford, Glazer, Padilla, Portantino, Roth, Rubio, Smallwood-Cuevas
NOES: Wilk
NO VOTE RECORDED: Jones, Nguyen, Ochoa Bogh, Seyarto
SENATE APPROPRIATIONS COMMITTEE: 5-0, 5/16/24
AYES: Caballero, Ashby, Becker, Bradford, Wahab
NO VOTE RECORDED: Jones, Seyarto
SUBJECT: California AI Transparency Act
SOURCE: Author
DIGEST:
This bill places obligations on businesses that provide generative artificial intelligence (AI) systems to develop and make accessible tools to detect whether specified content was generated by those systems. These “covered providers” are required to include visible and imperceptible markings on AI-generated content to identify it as such.
ANALYSIS:
Existing law:
- Defines “deepfake” as audio or visual content that has been generated or manipulated by artificial intelligence which would falsely appear to be authentic or truthful and which features depictions of people appearing to say or do things they did not say or do without their consent. (Gov't Code § 11547.5.)
- Defines “digital content forgery” as the use of technologies, including artificial intelligence and machine learning techniques, to fabricate or manipulate audio, visual, or text content with the intent to mislead. (Gov't Code § 11547.5.)
- Defines “digital content provenance” as the verifiable chronology of the original piece of digital content, such as an image, video, audio recording, or electronic document. (Gov't Code § 11547.5.)
- Requires, upon appropriation by the Legislature, the Secretary of Government Operations Agency (GovOps) to evaluate, among other things, the impact of the proliferation of deepfakes on state government, California-based businesses, and residents of the state.
- The risks, including privacy risks, associated with the deployment of digital content forgery technologies and deepfakes on state and local government, California-based businesses, and residents of the state.
- Potential privacy impacts of technologies allowing public verification of digital content provenance.
- The impact of digital content forgery technologies and deepfakes on civic engagement, including voters.
- The legal implications associated with the use of digital content forgery technologies, deepfakes, and technologies allowing public verification of digital content provenance.
- The best practices for preventing digital content forgery and deepfake technology to benefit the state, California-based businesses, and California residents, including exploring whether and how the adoption of a digital content provenance standard could assist with reducing the proliferation of digital content forgeries and deepfakes. (Gov’t Code § 11547.5(b).)
- Requires the Secretary of GovOps to develop a coordinated plan to accomplish all of the following:
- Investigate the feasibility of, and obstacles to, developing standards and technologies for state departments for determining digital content provenance.
- Increase the ability of internet companies, journalists, watchdog organizations, other relevant entities, and members of the public to meaningfully scrutinize and identify digital content forgeries and relay trust and information about digital content provenance to content consumers.
- Develop or identify mechanisms for content creators to cryptographically certify authenticity of original media and nondeceptive manipulations.
- Develop or identify mechanisms for content creators to enable the public to validate the authenticity of original media and nondeceptive manipulations to establish digital content provenance without materially compromising personal privacy or civil liberties. (Gov’t Code § 11547.5(c).)
- Establishes CDT within GovOps, under the supervision of the Director of Technology (Director), also known as the State Chief Information Officer. (Gov. Code Sec. 11545(a).)
- Provides that the duties of the Director include:
- Advising the Governor on the strategic management and direction of the state’s information technology (IT) resources.
- Establishing and enforcing state IT strategic plans, policies, standards, and enterprise architecture, as specified.
- Minimizing overlap, redundancy, and cost in state IT operations by promoting the efficient and effective use of information technology.
- Providing technology direction to agency and department chief information officers to ensure the integration of statewide technology initiatives, compliance with IT policies and standards, and the promotion of the alignment and effective management of IT services.
- Working to improve organizational maturity and capacity in the effective management of IT; and establishing performance management and improvement processes to ensure state IT systems and services are efficient and effective. (Gov. Code § 11545(b).)
This bill:
- Establishes the California AI Transparency Act.
- Requires a covered provider to create an AI detection tool by which a person can query the covered provider as to the extent to which text, image, video, audio, or multimedia content was created, in whole or in part, by a generative AI system provided by the covered provider that meets all of the following criteria:
- The AI detection tool shall be publicly accessible and available via a uniform resource locator (URL) on the covered provider’s internet website and through its mobile application, as applicable.
- The AI detection tool shall allow a person to upload content or a URL.
- The AI detection tool shall support an application programming interface (API) that allows a person to invoke the AI detection tool without visiting the covered provider’s website.
- The AI detection tool shall allow a person to provide feedback if the person believes the AI detection tool is not properly identifying content that was created by the provider.
- Prohibits a covered provider from doing any of the following in carrying out the duties above:
- Reveal personal information that identifies who utilized the AI system to create AI-generated content that was submitted to the AI detection tool.
- Collect and retain personal information when a person utilizes the covered provider’s AI detection tool, except that it may collect and retain the contact information of a person who submitted feedback.
- Retain any content submitted to the AI detection tool for longer than is necessary to comply with this law.
- Requires a covered provider to include in AI-generated image, text, video, or multimedia content created by its own generative AI system a visible disclosure that meets all of the following criteria:
- The disclosure shall include a clear and conspicuous notice, as appropriate for the medium of the content, that identifies the content as generated by AI, such that the disclosure is not avoidable, is understandable to a reasonable person, and is not contradicted, mitigated by, or inconsistent with anything else in the communication.
- The disclosure shall, to the extent technically feasible, be permanent or difficult to remove.
- The output’s metadata information shall include an identification of the content as being generated by AI, the identity of the tool used to create the content, and the date and time the content was created.
- Requires a covered provider to include in AI-generated image, audio, video, or multimedia content created by its generative AI system an imperceptible disclosure that is machine detectable and is, to the extent technically feasible, permanent or difficult to remove.
- Requires a covered provider to implement reasonable procedures to prevent downstream use of a generative AI system it provides without the disclosure required by this section, including the following:
- Contractually requiring third-party licensees of the generative AI system to refrain from removing a required disclosure.
- Terminating access to the generative AI system when the covered provider has reason to believe that a third-party licensee has removed a required disclosure.
- Requires CDT, at least once every two years, to review these provisions and make recommendations to the Legislature regarding any amendments needed to account for changing technology and standards.
- Provides that a covered provider that violates the above provisions is liable for a civil penalty in the amount of $5,000 per violation to be collected in a civil action filed only by the Attorney General. Each day that a covered provider is in violation shall be deemed a discrete violation.
- Defines the relevant terms, including:
- “Artificial intelligence” or “AI” means a machine-based system that can, for a given set of human-defined objectives, make predictions, recommendations, or decisions influencing real or virtual environments by using machine-based inputs and human-based inputs to perceive real and virtual environments, abstract its perceptions into models through analysis in an automated manner, and use model inference to formulate options for information or action.
- “AI-generated content” means any form of digital content that is created with deep learning or machine learning processes.
- “Covered provider” is a business that provides a generative AI system that has, on average over the preceding 12 months, over 1,000,000 monthly visitors or users and is publicly accessible within the geographic boundaries of the state.
- “Generative AI system” refers to deep learning models that can generate text, images, and other content based on the data they were trained on.
Background
Certain forms of media – audio recordings, video recordings, and still images – can be powerful evidence of the truth. While such media have always been susceptible to some degree of manipulation, fakes were relatively easy to detect. The rapid advancement of AI technology, specifically the wide-scale introduction of generative AI models, has made it drastically cheaper and easier to produce synthetic content created, audio, images, text, and video recordings that are not real, but that are so realistic that they are virtually impossible to distinguish from authentic content, including so-called “deepfakes.”
This bill works to ensure that providers of these generative-AI systems are equipping consumers with a tool to identify when specific content has been generated by their systems. This required tool must be made available to the public, as provided. Covered providers are also required to place markings, both visible and imperceptible, on this generated content to identify it as such and to be able to trace it back to the covered providers’ systems. Providers must work to prevent downstream users from removing the markings. Concerns have been raised that the relevant technology to achieve this is still in its nascent stage and its reliability is insufficient to impose mandates based on it.
This bill is author-sponsored. No support has been received by the Committee. The bill is opposed by a coalition of industry groups, including Netchoice. For a more thorough discussion of the bill, please see the Senate Judiciary Committee analysis, which is incorporated herein by reference.
Comment
The author states:
Generative Artificial Intelligence (AI) is advancing at an unprecedented pace, revolutionizing industries and everyday life. However, this rapid progress has also sparked a pressing need for regulatory frameworks to govern its use. While AI offers innovative solutions that drive economic growth and efficiency, it simultaneously presents complex challenges related to misinformation, manipulation, bias, and ownership of generated content.
One of the most concerning aspects of unregulated AI models is their potential to produce “deepfake” content, such as manipulated images and videos, which can be used to deceive or manipulate individuals and society at large. Deepfake pornography and fabricated political messages have already emerged as serious threats, highlighting the urgency of implementing effective regulations.
As AI-generated content approaches a level of realism that makes it indistinguishable from genuine content, the need for transparency becomes paramount. It is crucial for industry leaders to take responsibility for clearly labeling AI-generated content and developing tools that can accurately identify such content, especially when there is no visible disclosure. This transparency is essential for maintaining trust in AI technologies and ensuring that users can make informed decisions about the content they consume.
SB 942 stands as a critical legislative response in our ongoing regulatory arms race with artificial intelligence.
FISCAL EFFECT: Appropriation: No Fiscal Com.: Yes Local: No
According to the Senate Appropriations Committee:
- Unknown cost for CDT to review covered providers’ disclosures at least once every two years and provide recommendations based on changing technology and standards. To the extent that CDT would need additional or specialized staff to perform oversight activities related to reviewing disclosures, there may be significant General Fund costs.
- The Department of Justice (DOJ) reports costs of $359,000 for 3.0 personnel years (PYs) in Fiscal Year (FY) 2024-25 and $637,000 for 3.0 PYs in FY 2025-26 and ongoing for its Consumer Protection Section within the Public Rights Division to address an anticipated increase in litigation and other potential legal challenges (Unfair Competition Law Fund).
SUPPORT: (Verified 5/16/24)
None received
OPPOSITION: (Verified 5/16/24)
California Chamber of Commerce
Computer and Communications Industry Association
Netchoice
Technet
ARGUMENTS IN OPPOSITION: A coalition, including Technet, writes:
Many of our companies and platforms are at the forefront of developing content provenance and watermarking technology, which is still in its early stages. However, SB 942 enacts requirements for a technology that is still under development and rapidly evolving. For example, there isn’t a program that can watermark text, making the bill’s requirements to do so impossible to comply with. We believe references to text watermarking should be removed to reflect this reality.
Furthermore, content provenance and watermarking is still incredibly unreliable and in many cases easy to break. Researchers at the University of Maryland were able to break all the currently available watermarking methods. Some can be avoided by simple cropping, resizing, or screenshotting an image. More concerning, these researchers were able to insert fake watermarks and credentials into images, creating false positives.
In its standards for large online platforms, SB 942 should more clearly delineate between 1st party and 3rd party content. 1st party content would be images, videos, or audio that is generated using a covered provider’s generative AI tools and is then posted or distributed on that platform. In this instance, a covered provider can control the creation of a content provenance or watermark into the content. As mentioned, many of our companies are already working to incorporate this type of technology to increase transparency around AI-generated content. It is currently technically infeasible to accurately and reliably detect content that is created using a different platform’s AI tools. As noted above, considering the current ease with which current watermarks can be broken, a legal requirement and mandate for 3rd party content isn’t appropriate.
Prepared by: Christian Kurpiewski / JUD. / (916) 651-4113
5/18/24 14:01:47
**** END ****
SENATE COMMITTEE ON APPROPRIATIONS
Senator Anna Caballero, Chair
2023 - 2024 Regular Session
SB 942 (Becker) - California AI Transparency Act
Version: March 20, 2024 | Policy Vote: JUD. 10 - 1, G.O. 11 - 1 |
Urgency: No | Mandate: No |
Hearing Date: May 16, 2024 | Consultant: Janelle Miyashiro |
Bill Summary:
SB 942 establishes the California Artificial Intelligence (AI) Transparency Act, which requires specified providers of generative AI (GenAI) systems to develop and make accessible tools to detect whether specified content was generated by those systems; requires the inclusion of specified visible and machine detectable imperceptible disclosures in AI-generated content; requires the California Department of Technology (CDT) to create a registry of providers of GenAI systems; and establishes a civil penalty for violations of the Act, as specified.
*********** ANALYSIS ADDENDUM – SUSPENSE FILE ***********
The following information is revised to reflect amendments
adopted by the committee on May 16, 2024
Fiscal Impact:
- Unknown cost for CDT to review covered providers’ disclosures at least once every two years and provide recommendations based on changing technology and standards. To the extent that CDT would need additional or specialized staff to perform oversight activities related to reviewing disclosures, there may be significant General Fund costs.
- The Department of Justice (DOJ) reports costs of $359,000 for 3.0 personnel years (PYs) in Fiscal Year (FY) 2024-25 and $637,000 for 3.0 PYs in FY 2025-26 and ongoing for its Consumer Protection Section within the Public Rights Division to address an anticipated increase in litigation and other potential legal challenges (Unfair Competition Law Fund).
Committee Amendments:
Strike provisions related to the CDT Generative AI Registry.
-- END --
SENATE COMMITTEE ON APPROPRIATIONS
Senator Anna Caballero, Chair
2023 - 2024 Regular Session
SB 942 (Becker) - California AI Transparency Act
Version: March 20, 2024 | Policy Vote: JUD. 10 - 1, G.O. 11 - 1 |
Urgency: No | Mandate: No |
Hearing Date: May 6, 2024 | Consultant: Janelle Miyashiro |
Bill Summary:
SB 942 establishes the California Artificial Intelligence (AI) Transparency Act, which requires specified providers of generative AI (GenAI) systems to develop and make accessible tools to detect whether specified content was generated by those systems; requires the inclusion of specified visible and machine detectable imperceptible disclosures in AI-generated content; requires the California Department of Technology (CDT) to create a registry of providers of GenAI systems; and establishes a civil penalty for violations of the Act, as specified.
Fiscal Impact:
- CDT reports total costs of approximately $1,591,000 in the first year of implementation and $3,015,000 ongoing for additional limited-term and permanent staff with the administrative, technical, and research expertise to build new business capabilities and perform the oversight activities required in this bill. Costs also include workload and investments associated with recruitment and staff skill development to attract and retain GenAI talent necessary to meet increased demand for GenAI expertise for the state (General Fund and Generative AI Registry Fund).
CDT will require General Fund resources to stand-up the program and establish the registry. However, the bill establishes a registration fee for covered providers to offset CDT’s ongoing costs to administer the program, maintain the registry, and review disclosures and detection tools as technologies evolve.
- The Department of Justice (DOJ) reports costs of $359,000 for 3.0 personnel years (PYs) in Fiscal Year (FY) 2024-25 and $637,000 for 3.0 PYs in FY 2025-26 and ongoing for its Consumer Protection Section within the Public Rights Division to address an anticipated increase in litigation and other potential legal challenges (Unfair Competition Law Fund).
Background:
Generally, AI is the capability of machines to perform tasks that typically require human intelligence, encompasses technologies that can process information, learn, reason, recognize patterns, and make decisions. GenAI is a type of AI that can create new content, including text, images, code, or music, by learning from existing data. GenAI models can produce realistic and novel artifacts that resemble the data they were trained on, but do not copy it. For example, GenAI can write a poem, draw a picture, or compose a song based on a given prompt or theme. GenAI enables users to quickly generate new content based on a variety of inputs and GenAI models use neural networks to identify the patterns and structures within existing data to generate new and original content.
While the transformative potential of AI-generated content is vast, there are also infinite problematic applications of the technology. There is currently an arms race in techniques for distinguishing between synthetic and authentic content and companies are declaring their commitment to identifying such content. There are various methods for deciphering AI-generated or altered content, although none are foolproof and all require updates as technology advances. In fact, many companies have already voluntarily committed to follow specified guidelines, as described in the White House fact sheet titled “Biden-Harris Administration Secures Voluntary Commitments from Leading Artificial Intelligence Companies to Manage the Risks Posed by AI.”
Proposed Law:
- Establishes the California AI Transparency Act.
- Requires a covered provider to create an AI detection tool by which a person can query the provider as to the extent to which text, image, video, audio, or multimedia content was created, in whole or in part, by a GenAI system that meets all of the following criteria:
- The AI detection tool be publicly accessible and available via a uniform resource locator (URL) on the covered provider’s internet website and through its mobile application, as applicable.
- The AI detection tool allows a person to upload content or a URL.
- The AI detection tool supports an application programming interface that allows a person to invoke the AI detection tool without visiting the covered provider’s internet website.
- The AI detection tool allows a person to provide feedback if the person believes the AI detection tool is not properly identifying content that was created by the covered provider.
- Requires a covered provider to register with CDT and provide the department a URL to any AI detection tool it has created.
- Prohibits a covered provider from doing any of the following:
- Reveal personal information that identifies who utilized the covered provider’s GenAI system to create AI-generated content that was submitted to the covered provider’s AI detection tool.
- Collect and retain personal information when a person utilizes the covered provider’s AI detection tool.
- Retain any content submitted to the AI detection tool for longer than is necessary to comply with this section.
- Requires a covered provider to include a visible disclosure in AI-generated image, text, video, or multimedia content created by a GenAI system that meets all of the following criteria:
- The disclosure includes a clear and conspicuous notice, as appropriate for the medium of the content, that identifies the content as generated by AI, such that the disclosure is not avoidable, is understandable to a reasonable person, and is not contradicted, mitigated by, or inconsistent with anything else in the communication.
- The disclosure, to the extent technically feasible, be permanent or difficult to remove.
- The output’s metadata information includes an identification of the content as being generated by AI, the identity of the tool used to create the content, and the date and time the content was created.
- Requires a covered provider to include an imperceptible disclosure that is machine detectable and permanent or difficult to remove in AI-generated image, audio, video, or multimedia content created by a GenAI system.
- Requires a covered provider to implement reasonable procedures to prevent downstream use of a GenAI system without the disclosures required in the Act by:
- Requiring by contract that third-party licensees of the GenAI system refrain from removing a required disclosure.
- Terminating access to the GenAI system when the covered provider has reason to believe that a third-party licensee has removed a required disclosure.
- Requires CDT to review the disclosure requirements at least once every two years and make recommendations on any amendments needed to account for changing technology and standards.
- Requires CDT to create and display a Generative AI Registry on its website. Requires the registry to display the name of any registered covered provider and a link to the covered provider’s AI detection tool.
- Authorizes CDT to adopt regulations as necessary and to charge a covered provider a registration fee, to be deposited into the Generative AI Registry Fund, to cover the costs of administering the registry.
- Provides that a covered provider that violates the Act is liable for a civil penalty of $5,000 per violation, to be collected in a civil action filed only by the Attorney General.
- Provides that each day a covered provider is in violation of the Act be deemed a discrete violation.
- Establishes definitions.
Related Legislation:
SB 892 (Padilla, 2024) requires CDT to develop and adopt regulations for an AI risk management standard regarding state procurement and use of automated decision systems, as specified. SB 892 is pending on the Suspense File in this committee.
SB 893 (Padilla, 2024) establishes the California Artificial Intelligence Research Hub in the Government Operations Agency to serve as a centralized entity to facilitate collaboration between government agencies, academic institutions, and private sector partners to advance AI research and development, as specified. SB 893 is pending on the Suspense File in this committee.
SB 896 (Dodd, 2024) establishes the Artificial Intelligence Accountability Act, which tasks specified state entities with developing and periodically evaluating general guidelines for the use, regulation, and identification of potential risks of incorporating AI and GenAI within state government operations. SB 896 is pending on the Suspense File in this committee.
SB 1047 (Wiener, 2024) requires developers of powerful AI models, and those providing the computer power to train such models, to put appropriate safeguards and policies into place to prevent critical harms. Additionally, SB 1047 establishes a state entity to oversee the development of these models and calls for the creation of a public cloud computer cluster known as CalCompute, as specified. SB 1047 is pending in this committee.
-- END --
SENATE COMMITTEE ON GOVERNMENTAL ORGANIZATION
Senator Bill Dodd
Chair
2023 - 2024 Regular
Bill No: SB 942 Hearing Date: 4/23/2024
Author: | Becker | ||
Version: | 3/20/2024 Amended | ||
Urgency: | No | Fiscal: | Yes |
Consultant: | Brian Duke | ||
SUBJECT: California AI Transparency Act
DIGEST: This bill, the California Artificial Intelligence (AI) Transparency Act, requires specified providers of generative AI (GenAI) to create an AI detection tool available to the public to determine the extent to which text, image, video, audio, or multimedia content was created by a GenAI system, as specified; requires those providers to include a visible disclosure in AI-generated content including a clear and conspicuous notice that identifies the content as generated by AI; and creates the Generative AI Registry Fund and requires those providers to register with the California Department of Technology (CDT), as specified.
ANALYSIS:
Existing law:
- Requires the Secretary of Government Operations (GovOps) to develop a coordinated plan to, among other things, investigate the feasibility of, and obstacles to, developing standards and technologies for state departments to determine digital content provenance, as specified.
- Requires GovOps, for the purposes of informing that coordinated plan, to evaluate, among other things, the impact of the proliferation of deepfakes, as defined to mean audio or visual content that has been generated or manipulated by AI that would falsely appear to be authentic or truthful and that features depictions of people appearing to say or do things they did not say or do without their consent, on state government, California-based businesses, and residents of the state.
- Establishes CDT, within GovOps, to, among other things, provide technology direction to agency and department chief information officers to ensure the integration of statewide technology initiatives, as specified.
- Defines “deepfake” to mean audio or visual content that has been generated or manipulated by AI which would falsely appear to be authentic or truthful and which features depictions of people appearing to say or do things they did not say or do without their consent.
This bill:
- Requires a covered provider, as defined, to create an AI detection tool by which a person can query the covered provider as to the extent to which text, image, video, audio, or multimedia content was created, in whole or in part, by a GenAI system provided by the covered provider that meets all of the following criteria:
- The AI detection tool shall be publicly accessible and available via a uniform resource locator (URL) on the covered provider’s internet website and through its mobile application, as applicable.
- The AI detection tool shall allow a person to upload content or a URL.
- The AI detection tool shall support an application programming interface that allows a person to invoke the AI detection tool without visiting the covered provider’s Internet website.
- The AI detection tool shall allow a person to provide feedback if the person believes the AI detection tool is not properly identifying content that was created by the covered provider.
- Prohibits a covered provider, in complying with this bill, from doing any of the following:
- Reveal personal information that identifies who utilized the covered provider’s GenAI system to create AI-generated content that was submitted to the covered provider’s AI detection tool.
- Collect and retain personal information when a person utilizes the covered provider’s AI detection tool, as specified.
- Retain any content submitted to the AI detection tool for longer than ins necessary to comply with this bill.
- Requires a covered provider to include AI-generated image, text, video, or multimedia content created by a GenAI system it provides a visible disclosure that meets all of the following criteria:
- The disclosure shall include a clear and conspicuous notice, as appropriate for the medium of the content, that identifies the content as generated by AI, such that the disclosure is not avoidable, is understandable to a reasonably person, and is not contradicted, mitigated by, or inconsistent with anything else in the communication.
- The disclosure shall, to the extent technically feasible, be permanent or difficult to remove.
- The output’s metadata information shall include an identification of the content as being generated by AI, the identity of the tool used to the content, and the date and time the content was created.
- Requires a covered provider to include in AI-generated images, audio, video, or multimedia content created by a GenAI system an imperceptible disclosure that is machine detectable and is, to the extent technically feasible, permanent or difficult to remove.
- Requires a covered provider to implement reasonable procedures to prevent downstream use of a GenAI system it provides without the disclosure required by this bill, including by doing both of the following:
- Requiring by contract that third-party licensees of the GenAI system refrain from removing a required disclosure.
- Terminating access to the GenAI system when the covered provider has reason to believe that a third-party license has removed a required disclosure.
- Requires CDT, at least once every two years, to review this section of law and make recommendations to the Legislature regarding any amendments needed to account for changing technology and standards.
- Requires a covered provider to register with CDT and provide to the department a URL to any AI detection tool it has created.
- Requires CDT to create and display on its Internet website the Generative AI Registry which shall include both of the following:
- The name of any covered provider registered with CDT.
- A link to the covered provider’s AI detection tool.
- Authorizes CDT to charge a registration fee, to be deposited into the Generative AI Registry Fund, to a covered provider, but only to the extent that the registration fee is necessary to cover the costs of administering the registry.
- Establishes the Generative AI Registry Fund, in the State Treasury and requires moneys in the fund be made available, only upon appropriation by the Legislature, to CDT for the purposes of this bill.
- Provides that a covered provider that violates this bill shall be liable for a civil penalty in the amount of $5,000 per violation to be collected in a civil action filed only by the Attorney General (AG).
- Specifies that each day that a covered provider is in violation of this bill shall be deemed a discrete violation.
- Defines “artificial intelligence” or “AI” to mean a machine-based system that can, for a given set of human-defined objectives, make predictions, recommendations, or decisions influencing real or virtual environments by using machine-based inputs and human based inputs to perceive real and virtual environments, abstract its perceptions into models through analysis in an automated manner, and use model inference to formulate options for information or actions.
- Defines “covered provider” to mean a business that provides a GenAI system that has, on average over the preceding 12 months, over one million monthly visitors or users and is publicly accessible within the geographic boundaries of the state.
- Defines “generative AI system” to refer to deep learning models that can generate text, images, and other content based on the data they were trained on.
- Defines “metadata” to mean structural or descriptive information about data, including content, format, source, rights, accuracy, provenance, frequency, periodicity, granularity, publisher or responsible party, contract information, method of collection, and other descriptions.
Background
Author Statement. According to the author’s office, “GenAI is advancing at an unprecedented pace, revolutionizing industries and everyday life. However, this rapid progress has also sparked a pressing need for regulatory frameworks to govern its use. While AI offers innovative solutions that drive economic growth and efficiency, it simultaneously presents complex challenges related to misinformation, manipulation, bias, and ownership of generated content.
Additionally, “one of the most concerning aspects of unregulated AI models is their potential to produce ‘deepfake’ content, such as manipulated images and videos, which can be used to deceive or manipulate individuals and society at large. Deepfake pornography and fabricated political messages have already emerged as serious threats, highlighting the urgency of implementing effective regulations.”
Further, “as AI-generated content approaches a level of realism that makes it indistinguishable from genuine content, the need for transparency becomes paramount. It is crucial for industry leaders to take responsibility for clearly labeling AI-generated content and developing tools that can accurately identify such content, especially when there is no visible disclosure. This transparency is essential for maintaining trust in AI technologies and ensuring that users can make informed decisions about the content they consume. SB 942 stands as a critical legislative response in our ongoing regulatory arms race with artificial intelligence.”
Navigating the AI Landscape. In the rapidly evolving landscape of technology, AI stands at the forefront, bringing a new era of innovation and application across various sectors. AI, defined broadly, is the capability of machines to perform tasks that typically require human intelligence, encompasses technologies that can process information, learn, reason, recognize patterns, and make decisions. The transformative potential of AI is vast, affecting disparate industries from healthcare to transportation, and its development is a subject of both enthusiasm and scrutiny. Given the technology's capacity to revolutionize the way we live and work, governments around the world and across the nation are seeking to nurture its growth while addressing the ethical, privacy, and security concerns it raises.
Blueprint for an AI Bill of Rights. In October 2022, The White House’s Office of Science and Technology Policy (OSTP) released the Blueprint for an AI Bill of Rights: Making Automated Systems Work for the American People (Blueprint) as a framework after a year of public engagement to inform the initiative. The Blueprint is a white paper published by the OSTP intended to support the development of policies and practices that protect civil rights and promote democratic values in the building, deployment, and governance of automated systems. This non-binding document does not constitute U.S. government policy, does not superseded, modify, or direct an interpretation of any existing statute, regulation, policy, or international instrument.
The appropriate application of the principles set forth in the white paper depends on the context in which automated systems are being utilized. The five identified principles in the Blueprint help provide guidance whenever automated systems can meaningfully impact the public’s rights, opportunities, or access to critical needs. Specifically, the Blueprint identifies the following five principles: safe and effective systems; algorithmic discrimination protection; data privacy; notice and explanation; and human alternatives, consideration, and fallback.
SCR 17 (Dodd, Res. Chapter 135, Statutes of 2023) affirmed the California Legislature’s commitment to President Biden’s vision for a safe AI and the principles outlined in the blueprint. Additionally, that resolution expressed the Legislature’s commitment to examining and implementing those principles in its legislation and policies related to the use and deployment of automated systems.
Deepfakes. A “deepfake” is a synthetic media in which a person in an existing image or video is replaced with someone else’s likeness using AI techniques. SB 1216 (Gonzalez, Chapter 885, Statutes of 2022) defined “deepfake” to mean audio or visual content that has been generated or manipulated by AI which would falsely appear to be authentic or truthful and which features depictions of people appearing to say or do things they did not say or do without their consent.
Deepfakes started coming to the public’s attention in the 2010s, as AI development started to evolve faster and faster. The term came to be used for the new type of synthetic multimedia in 2017 when a Reddit moderator created a subreddit called “deepfakes” and began posting videos that used face-swapping technology to insert celebrities’ likenesses into existing pornographic videos.
Examples of deepfakes include a 2020 video on YouTube purporting to be a trailer for Back to the Future 4 featuring deepfakes of Robert Downey Jr and Tom Holland which currently has over 11 million views; 2021 deepfake videos of Tom Cruise which were spread on TikTok racking up tens of millions of views. In May of 2023, a false report of an explosion at the Pentagon, accompanied by an AI-generated image spread on social media and sparked a brief dip in the stock market.
More recently, thousands of New Hampshire residents received deepfake robocalls utilizing AI-generated audio of President Joe Biden’s voice created by a company in Texas to spread disinformation in the lead up to the state’s primary election. The calls falsely stated that citizens’ votes needed to be “saved” to be used in November’s general election, and that casting a ballot in the primary would be ineffectual. In reality, voters were able and encouraged to vote in both the primary and the upcoming November general election.
Infamously, in January of this year, Taylor Swift was the victim of sexually explicit, nonconsensual deepfake images using AI that were widely spread across social media platforms. More disturbingly, a trend has emerged in schools of students creating such images, according to the Los Angeles Times, “[a]t schools across the country, people have used deepfake technology combined with real images of female students to create fraudulent images of nude bodies. The deepfake images can be produced using a cellphone.”
For generations, people have collectively developed technologies like photography, video, and audio recordings to capture and preserve the authenticity of moments in life, creating a reliable record of history and personal memories. Yet, AI technology has now evolved to the point that undermines this very trustworthiness. AI-generated technologies such as deepfakes blur the lines between reality and fabrication, making it increasingly difficult to discern true representations from manipulations. These advancements challenge our ability to trust digital and printed content, confronting society with profound implications for information dissemination, personal security, and democratic integrity. As AI continues to advance, society is navigating a new reality where seeing or hearing is no longer synonymous with believing, requiring new mechanisms for verifying the authenticity of digital multimedia.
One expert at the Copenhagen Institute for Future Studies estimates that should large generative-AI models run amok, up to 99 percent of the internet’s content could be AI-generated by 2025 to 2030. The problematic applications are seemingly infinite, whether it be deepfakes to blackmail or shame victims, misinformation in elections, false impersonations to commit fraud, or other nefarious purposes.
SB 1216 (Gonzalez, Chapter 885, Statutes of 2022) required the Secretary of GovOps, upon appropriation by the Legislature, to evaluate, among other things, the impact the proliferation of deepfakes has and the risks, including privacy risks, associated with the deployment of digital content forgery technologies and deepfakes on government, businesses, and residents of the state. Further, the bill requires GovOps to develop a coordinated plan to accomplish specified objectives, including investigating the feasibility of, and obstacles to, developing standards and technologies for state departments for determining digital content provenance. The legislature has yet to make the necessary appropriation as required by the bill for enactment.
There is currently an arms race in techniques for distinguishing between synthetic and authentic content and companies are declaring their commitment to identifying such content. There are various methods for deciphering AI-generated or altered content, although none are foolproof and all require updates as technology advances. In fact, many companies have already voluntarily committed to follow specified guidelines, as described in the White House fact sheet titled “Biden-Harris Administration Secures Voluntary Commitments from Leading Artificial Intelligence Companies to Manage the Risks Posed by AI.”
This bill largely models legislation introduced at the federal level by Senators Schatz and Kennedy, the AI Labeling Act of 2023. Also at the federal level, President Biden issued an executive order late last year on AI, to set standards for safety and security and to ensure responsible AI development and deployment in the United States. The executive order calls for the development of a risk management framework for GenAI, modeled after the AI Risk Management Framework created by the National Institute for Standards and Technology (NIST). NIST reports that it is contributing to a report on authenticating, labeling, or detecting synthetic content.
European Union AI Act. Last month, the European Parliament signed the European Union AI Act. It highlights these very issues and obligates developers and deployers to assist in ensuring, to the extent feasible, that individuals are able to distinguish between original and AI-generated or manipulated content. The Act states:
A variety of AI systems can generate large quantities of synthetic content that becomes increasingly hard for humans to distinguish from human-generated and authentic content. The wide availability and increasing capabilities of those systems have a significant impact on the integrity and trust in the information ecosystem, raising new risks of misinformation and manipulation at scale, fraud, impersonation and consumer deception. In light of those impacts, the fast technological pace and the need for new methods and techniques to trace origin of information, it is appropriate to require providers of those systems to embed technical solutions that enable marking in a machine readable format and detection that the output has been generated or manipulated by an AI system and not a human. Such techniques and methods should be sufficiently reliable, interoperable, effective and robust as far as this is technically feasible, taking into account available techniques or a combination of such techniques, such as watermarks, metadata identifications, cryptographic methods for proving provenance and authenticity of content, logging methods, fingerprints or other techniques, as may be appropriate. When implementing this obligation, providers should also take into account the specificities and the limitations of the different types of content and the relevant technological and market developments in the field, as reflected in the generally acknowledged state-of-the-art. Such techniques and methods can be implemented at the level of the system or at the level of the model, including general purpose AI models generating content, thereby facilitating fulfilment of this obligation by the downstream provider of the AI system. To remain proportionate, it is appropriate to envisage that this marking obligation should not cover AI systems performing primarily an assistive function for standard editing or AI systems not substantially altering the input data provided by the deployer or the semantics thereof.
It also specifically obligates deployers who use an AI system to generate or manipulate image, audio, or video content that “appreciably resembles existing persons, places or events and would falsely appear to a person to be authentic (deepfakes), should also clearly and distinguishably disclose that the content has been artificially created or manipulated by labelling the AI output accordingly and disclosing its artificial origin.”
The California AI Transparency Act. This bill seeks to address the challenges posed by GenAI and deepfakes by establishing requirements for providers of certain technologies to label, disclose, and register with the state – establishing a precedent for how deepfakes are regulated. Specifically, this bill requires “covered providers,” defined as a business providing GenAI systems that have on average over one million monthly visitors and is publically accessible within the state, to create an AI detection tool by which a person can query the provider as to the extent to which text, image, video, audio, or multimedia content was created, in whole or in part, by a GenAI system created by the provider. This bill requires the AI detection tool to be publicly accessible and available via a URL on the provider’s internet website and through its mobile application.
Detection methods are diverse, ranging from traditional machine learning approaches to more advanced deep learning techniques that include convolutional neural networks, transformers, and systems that analyze biological signals for inconsistencies typical of synthesized media. Despite these advancements, the effectiveness and reliability of deepfake detection tools vary. They are generally effective within controlled environments or when evaluating media generated from similar technologies they were trained on. However, challenges arise when these tools are tested across different datasets or in real-world conditions where they might encounter novel manipulation methods not covered in their training data.
Additionally, this bill requires covered providers to include in any AI-generated image, text, video, or multimedia content created by a GenAI system it provides a visible disclosure that includes a clear and conspicuous notice that identifies the content as generated by AI. The bill requires that the disclosure be, “to the extent technically feasible, permanent or difficult to remove. Further, the bill requires the output to include metadata information identifying the content as being generated by AI, the identity of the tool used to create the content, and the date and time the content was created.
Digital watermarking embeds subtle, often imperceptible, markers into media content that can verify its origins and authenticity. Recent developments in watermarking technologies have shown some promising results in embedding robust, detectable watermarks that are resistant to common manipulations such as cropping, resizing, or editing. Google DeepMind’s SynthID, for example, uses dual neural networks to embed and later identify watermarks in images and audio. The watermark remains detectable even after modifications to the content, aiming to maintain trust and traceability in AI-generated media. However, the effectiveness of these watermarking tools can vary and often depends on the specific technology used, the types of content, and the nature of potential modifications the content might undergo.
Further, this bill requires covered providers to register with CDT and provide to the department a URL to any detection tool it has created. This bill then requires CDT to create and display on its Internet website the Generative AI Registry, which will display the name of any covered provider registered with CDT and a link to the covered provider’s AI detection tool. This bill authorizes CDT to charge a registration fee to a covered provider to the extent that the registration fee is necessary only to cover the costs of administering the registry.
Finally, this bill specifies that any covered provider that violates this bill is liable for a civil penalty in the amount of $5,000 per violation to be collected in a civil action filed by the AG. Each day that a covered provider is in violation of this bill would be considered a discrete violation of the law.
Policy Consideration. This bill includes “text” as one of the forms of content that, when created utilizing a GenAI system, must include a visible disclosure that is, among other things, permanent or difficult to remove, and requires the text content output metadata information to include an identification of the content as being generated by AI, the identity of the tool used to create the content, and the date and time the content was created. Additionally, this bill requires a covered provider’s AI detection tool be available to the public to query the extent to which text content was created, in whole or in part, by a GenAI system provided by the covered provider.
This can raise practical and technological issues regarding the feasibility of embedding content provenance in synthetic text created by AI. Unlike images or videos, where digital watermarks can be subtly integrated into the pixels or audio waveforms, text does not naturally support such embedding without altering the readability or integrity of the content. Moreover, the requirement to embed disclosures directly within AI-generated text could potentially stifle innovation and practical usability in applications where AI-generated text is becoming pivotal. Industries such as journalism, customer service, and creative writing leverage AI to produce content at scale. Requiring intrusive and difficult to remove disclosures within the text could deter the use of these technologies or overly complicate their implementation.
Given these considerations, it may be more practical to focus legislative efforts on ensuring transparency through alternative means, such as mandatory disclosure of AI involvement at the point of content distribution or within the platform’s user interface, rather than embedding it within the text itself. This approach could balance the need for transparency with the technical limitations and practical realities of synthetic text generation.
Prior/Related Legislation
SB 892 (Padilla, 2024) requires CDT to develop and adopt regulations to create an AI risk management standard, consistent with specified publications regarding AI risk management, as specified. (Pending in the Senate Appropriations Committee)
SB 893 (Padilla, 2024) establishes the California AI Research Hub to serve as a centralized entity to facilitate collaboration between government agencies, academic institutions, and private sector partners to advance AI research and development, as specified. (Pending in the Senate Appropriations Committee)
SB 896 (Dodd, 2024) the AI Accountability Act requires state agencies to advance safeguards and consumer protections around AI by incorporating and building upon recent directives from President Joe Biden and Governor Gavin Newsom. (Pending in the Senate Appropriations Committee)
SB 970 (Ashby, 2024) among other things, requires by July 1, 2026, any person or entity that sells or provides access to any AI technology that is designed to create synthetic content to provide a consumer warning that misuse of the technology may result in civil or criminal liability for the user, as specified. (Pending in the Senate Public Safety Committee)
SB 1047 (Wiener, 2024) the Safe and Secure Innovation for Frontier AI Systems Act, among other things, requires the developer of a covered model, as defined, to determine whether it can make a positive safety determination before initiating training of that covered model, as specified, and creates the Frontier Model Division within CDT, as specified. This bill requires CDT to commission consultants, as prescribed, to create a public cloud computing cluster, to be known as CalCompute, with the primary focus of conducting research into the safe and secure deployment of large-scale AI models and fostering equitable innovation that includes, among other things, a fully owned and hosted cloud platform. (Pending in the Senate Governmental Organization Committee)
SB 1235 (Gonzalez, 2024) requires an unspecified public institution of higher education to establish the AI Deepfake Working Group to evaluate and advise the Legislature and the public on the relevant issues and impacts of AI and deepfakes, as specified. (Pending in the Senate Education Committee)
AB 1791 (Weber, 2024) prohibits a social media platform from removing digital content provenance verification from content uploaded to the social media platform by a user, as specified. (Pending in the Assembly Privacy and Consumer Protection Committee)
AB 1856 (Ta & Low, 2024) makes it a crime for a person to intentionally distribute a deepfake of an intimate body part or parts of another identifiable person, as specified. (Pending in the Assembly Public Safety Committee)
AB 2655 (Berman & Pellerin, 2024) the Defending Democracy from Deepfake Deception Act of 2024, among other things, requires large online platforms to block the posting or sending of materially deceptive and digitally modified or created content related to elections, during specified periods before and after an election, as specified. (Pending in the Assembly Judiciary Committee)
AB 2839 (Pellerin & Berman, 2024) prohibits a person, committee, or other entity from knowingly distributing an advertisement or other election communication that contains certain materially deceptive and digitally altered or digitally created images or audio or video files with the intent to influence an election or solicit funds for a candidate or campaign, as specified. (Pending in the Assembly Judiciary Committee)
AB 2885 (Bauer-Kahan, 2024) defines the term “artificial intelligence,” for the purposes of existing applicable provisions of law, to mean an engineered or machine-based system that, for explicit or implicit objectives, infers from the input it receives how to generate outputs that can influence physical or virtual environments. (Pending in the Assembly Appropriations Committee)
AB 3050 (Low, 2024) among other things, requires CDT to issue regulations to establish standards for watermarks to be included in covered AI-generated material, as specified. (Pending in the Assembly Privacy and Consumer Protection Committee)
AB 3211 (Wicks, 2024) the California Provenance, Authenticity and Watermarking Standards Act, among other things, requires a GenAI system provider to take certain actions to assist in the disclosure of provenance data to mitigate harms caused by inauthentic content, as specified. (Pending in the Assembly Appropriations Committee)
SB 313 (Dodd, 2023) the AI-ware Act, among other things, would have required any agency utilizing GenAI to directly communicate with a human to clearly and in a conspicuous way identify as AI, as specified. (Held in the Senate Appropriations Suspense File)
SB 721 (Becker, 2023) creates the California Interagency AI Working Group to deliver a report to the Legislature regarding AI, as specified. (Pending in the Assembly Privacy and Consumer Protection Committee)
SB 1216 (Gonzalez, Chapter 885, Statutes of 2022) requires the Secretary of GovOps to evaluate, among other things, the impact that the proliferation of deepfakes has on government, businesses, and residents of the state, as specified.
FISCAL EFFECT: Appropriation: No Fiscal Com.: Yes Local: No
SUPPORT:
None received
OPPOSITION:
CalChamber
Computer & Communications Industry Association
NetChoice
TechNet
ARGUMENTS IN OPPOSITION: Opponents to the bill write that, “[i]n light of the immense volume of [AI] and AI related bills in California and across the country, our member companies are still reviewing this bill and many others. We hope to be able to provide suggested amendments soon. In the meantime, there are several issues with this bill that we should highlight for consideration.
“Federal Standard. While we understand the desire to regulate an emerging technology, this is an area that would benefit from Federal standards and regulation rather than a state by state approach. In President Biden’s AI Executive Order, he tasked the Department of Commerce with ‘identifying the existing standards, tools, methods, and practices, as well as the potential development of further science-backed standards and techniques, for: (i) authenticating content and tracking its provenance; (ii) labeling synthetic content, such as using watermarking; (iii) detecting ‘synthetic content’ and more. We believe in allowing this federal process to advance in order to establish standards that are ‘science-backed’ and can be consistently applied across the country is important.”
Further, “Prescriptive Requirements on Content Provenance and Watermarking are Technology Premature. Many of our companies and platforms are at the forefront of developing content provenance and watermarking technology, which is still in its early stages. However, SB 942 enacts requirements for a technology that is still under development and rapidly evolving. For example, there isn’t a program that can watermark text, making the bill’s requirements to do so impossible to comply with. We believe references to text watermarking should be removed to reflect this reality.”
Finally, “Generative AI Registry is Redundant. SB 942 creates the Generative AI Registry and requires covered providers to register and pay a fee. The purpose of the registry is to compile URLs for AI detection tools. We believe this requirement is redundant as covered providers are already required to make their AI detection tools publicly accessible and available on their website or mobile application. A registration fee and open ended regulatory authority are unnecessary considering these requirements.”
DUAL REFERRAL: Senate Judiciary Committee (10-1) & Senate Governmental Organization Committee
Senator Thomas Umberg, Chair
2023-2024 Regular Session
Bill No: SB 942 (Author:Becker)
Version: March 20, 2024
Hearing Date: April 16, 2024
Fiscal: Yes
Urgency: No
Consultant: CK
SUBJECT
California AI Transparency Act
DIGEST
This bill places obligations on businesses that provide generative artificial intelligence (AI) systems to develop and make accessible tools to detect whether specified content was generated by those systems. These “covered providers” are required to include visible and imperceptible markings on AI-generated content to identify it as such. The bill requires providers to register with the California Department of Technology (CDT).
EXECUTIVE SUMMARY
Certain forms of media – audio recordings, video recordings, and still images – can be powerful evidence of the truth. While such media have always been susceptible to some degree of manipulation, fakes were relatively easy to detect. The rapid advancement of AI technology, specifically the wide-scale introduction of generative AI models, has made it drastically cheaper and easier to produce synthetic content created, audio, images, text, and video recordings that are not real, but that are so realistic that they are virtually impossible to distinguish from authentic content, including so-called “deepfakes.”
This bill works to ensure that providers of these generative-AI systems are equipping consumers with a tool to identify when specific content has been generated by their systems. This required tool must be made available to the public, as provided. Covered providers are also required to place markings, both visible and imperceptible, on this generated content to identify it as such and to be able to trace it back to the covered providers systems. Providers must register with CDT and work to prevent downstream users from removing the markings. This bill is author-sponsored. No support has been received by the Committee. The bill is opposed by a coalition of industry groups, including Netchoice. Should this pass out of this Committee, it will be referred to the Senate Governmental Organization Committee.
PROPOSED CHANGES TO THE LAW
Existing law:
- Defines “deepfake” as audio or visual content that has been generated or manipulated by artificial intelligence which would falsely appear to be authentic or truthful and which features depictions of people appearing to say or do things they did not say or do without their consent. (Gov't Code § 11547.5.)
- Defines “digital content forgery” as the use of technologies, including artificial intelligence and machine learning techniques, to fabricate or manipulate audio, visual, or text content with the intent to mislead. (Gov't Code § 11547.5.)
- Defines “digital content provenance” as the verifiable chronology of the original piece of digital content, such as an image, video, audio recording, or electronic document. (Gov't Code § 11547.5.)
- Requires, upon appropriation by the Legislature, the Secretary of Government Operations Agency (GovOps) to evaluate the following:
- The impact of the proliferation of deepfakes on state government, California-based businesses, and residents of the state.
- The risks, including privacy risks, associated with the deployment of digital content forgery technologies and deepfakes on state and local government, California-based businesses, and residents of the state.
- Potential privacy impacts of technologies allowing public verification of digital content provenance.
- The impact of digital content forgery technologies and deepfakes on civic engagement, including voters.
- The legal implications associated with the use of digital content forgery technologies, deepfakes, and technologies allowing public verification of digital content provenance.
- The best practices for preventing digital content forgery and deepfake technology to benefit the state, California-based businesses, and California residents, including exploring whether and how the adoption of a digital content provenance standard could assist with reducing the proliferation of digital content forgeries and deepfakes. (Gov’t Code § 11547.5(b).)
- Requires the Secretary of GovOps to develop a coordinated plan to accomplish all of the following:
- Investigate the feasibility of, and obstacles to, developing standards and technologies for state departments for determining digital content provenance.
- Increase the ability of internet companies, journalists, watchdog organizations, other relevant entities, and members of the public to meaningfully scrutinize and identify digital content forgeries and relay trust and information about digital content provenance to content consumers.
- Develop or identify mechanisms for content creators to cryptographically certify authenticity of original media and nondeceptive manipulations.
- Develop or identify mechanisms for content creators to enable the public to validate the authenticity of original media and nondeceptive manipulations to establish digital content provenance without materially compromising personal privacy or civil liberties. (Gov’t Code § 11547.5(c).)
- Establishes CDT within GovOps, under the supervision of the Director of Technology (Director), also known as the State Chief Information Officer. (Gov. Code Sec. 11545(a).)
- Provides that the duties of the Director include:
- Advising the Governor on the strategic management and direction of the state’s information technology (IT) resources.
- Establishing and enforcing state IT strategic plans, policies, standards, and enterprise architecture, as specified.
- Minimizing overlap, redundancy, and cost in state IT operations by promoting the efficient and effective use of information technology.
- Providing technology direction to agency and department chief information officers to ensure the integration of statewide technology initiatives, compliance with IT policies and standards, and the promotion of the alignment and effective management of IT services.
- Working to improve organizational maturity and capacity in the effective management of IT; and establishing performance management and improvement processes to ensure state IT systems and services are efficient and effective. (Gov. Code § 11545(b).)
This bill:
- Establishes the California AI Transparency Act.
- Requires a covered provider to create an AI detection tool by which a person can query the covered provider as to the extent to which text, image, video, audio, or multimedia content was created, in whole or in part, by a generative AI system provided by the covered provider that meets all of the following criteria:
- The AI detection tool shall be publicly accessible and available via a uniform resource locator (URL) on the covered provider’s internet website and through its mobile application, as applicable.
- The AI detection tool shall allow a person to upload content or a URL.
- The AI detection tool shall support an application programming interface (API) that allows a person to invoke the AI detection tool without visiting the covered provider’s website.
- The AI detection tool shall allow a person to provide feedback if the person believes the AI detection tool is not properly identifying content that was created by the provider.
- Prohibits a covered provider from doing any of the following in carrying out the duties above:
- Reveal personal information that identifies who utilized the AI system to create AI-generated content that was submitted to the AI detection tool.
- Collect and retain personal information when a person utilizes the covered provider’s AI detection tool, except that it may collect and retain the contact information of a person who submitted feedback.
- Retain any content submitted to the AI detection tool for longer than is necessary to comply with this law.
- Requires a covered provider to include in AI-generated image, text, video, or multimedia content created by its own generative AI system a visible disclosure that meets all of the following criteria:
- The disclosure shall include a clear and conspicuous notice, as appropriate for the medium of the content, that identifies the content as generated by AI, such that the disclosure is not avoidable, is understandable to a reasonable person, and is not contradicted, mitigated by, or inconsistent with anything else in the communication.
- The disclosure shall, to the extent technically feasible, be permanent or difficult to remove.
- The output’s metadata information shall include an identification of the content as being generated by AI, the identity of the tool used to create the content, and the date and time the content was created.
- Requires a covered provider to include in AI-generated image, audio, video, or multimedia content created by its generative AI system an imperceptible disclosure that is machine detectable and is, to the extent technically feasible, permanent or difficult to remove.
- Requires a covered provider to implement reasonable procedures to prevent downstream use of a generative AI system it provides without the disclosure required by this section, including the following:
- Contractually requiring third-party licensees of the generative AI system to refrain from removing a required disclosure.
- Terminating access to the generative AI system when the covered provider has reason to believe that a third-party licensee has removed a required disclosure.
- Requires CDT, at least once every two years, to review these provisions and make recommendations to the Legislature regarding any amendments needed to account for changing technology and standards.
- Requires covered providers to register with CDT and provide a URL to any AI detection tool it has created. Requires CDT to create and display on its website a Generative AI Registry that displays the names of registered covered providers and a link to the covered provider’s AI detection tool. CDT is authorized to charge a registry fee to covered providers to cover administrative costs.
- Authorizes CDT to adopt regulations necessary to perform its duties hereunder.
- Provides that a covered provider that violates the above provisions is liable for a civil penalty in the amount of $5,000 per violation to be collected in a civil action filed only by the Attorney General. Each day that a covered provider is in violation shall be deemed a discrete violation.
- Defines the relevant terms, including:
- “Artificial intelligence” or “AI” means a machine-based system that can, for a given set of human-defined objectives, make predictions, recommendations, or decisions influencing real or virtual environments by using machine-based inputs and human-based inputs to perceive real and virtual environments, abstract its perceptions into models through analysis in an automated manner, and use model inference to formulate options for information or action.
- “AI-generated content” means any form of digital content that is created with deep learning or machine learning processes.
- “Covered provider” is a business that provides a generative AI system that has, on average over the preceding 12 months, over 1,000,000 monthly visitors or users and is publicly accessible within the geographic boundaries of the state.
- “Generative AI system” refers to deep learning models that can generate text, images, and other content based on the data they were trained on.
COMMENTS
- Blurring reality: AI-generated content
Generative AI is a type of artificial intelligence that can create new content, including text, images, code, or music, by learning from existing data. Generative AI models can produce realistic and novel artifacts that resemble the data they were trained on, but do not copy it. For example, generative AI can write a poem, draw a picture, or compose a song based on a given prompt or theme. Generative AI enables users to quickly generate new content based on a variety of inputs. Generative AI models use neural networks to identify the patterns and structures within existing data to generate new and original content.
The world has been in awe of the powers of this generative AI since the widespread introduction of AI systems such as ChatGPT. However, the capabilities of these advanced systems leads to a blurring between reality and fiction. The Brookings Institution lays out the issue:
Over the last year, generative AI tools have made the jump from research prototype to commercial product. Generative AI models like OpenAI’s ChatGPT and Google’s Gemini can now generate realistic text and images that are often indistinguishable from human-authored content, with generative AI for audio and video not far behind. Given these advances, it’s no longer surprising to see AI-generated images of public figures go viral or AI-generated reviews and comments on digital platforms. As such, generative AI models are raising concerns about the credibility of digital content and the ease of producing harmful content going forward.
Against the backdrop of such technological advances, civil society and policymakers have taken increasing interest in ways to distinguish AI-generated content from human-authored content.[1]
One expert at the Copenhagen Institute for Future Studies estimates that should large generative-AI models run amok, up to 99 percent of the internet’s content could be AI-generated by 2025 to 2030.[2] The problematic applications are seemingly infinite, whether it be deepfakes to blackmail or shame victims, misinformation in elections, false impersonations to commit fraud, or other nefarious purposes. Infamously, in January of this year, Taylor Swift was the victim of sexually explicit, nonconsensual deepfake images using AI that were widely spread across social media platforms.[3] Perhaps more disturbingly, a trend has emerged in schools of students creating such images: “At schools across the country, people have used deepfake technology combined with real images of female students to create fraudulent images of nude bodies. The deepfake images can be produced using a cellphone.”[4] In February of this year, voters in New Hampshire received robocalls that are purported to have used an AI voice resembling President Joe Biden advising them against voting in the presidential primary and saving their vote for the November general election.[5] Recently, a former federal judge urged the federal judiciary’s Advisory Committee on Evidence Rules to update evidentiary rules regarding the admissibility of evidence believed to be AI generated.[6] But, in addition to concerns about the potential for AI-generated evidence to be admitted is the reverse, false claims that real evidence is synthetic. As more of the population becomes aware of the potential to realistically fake images, video, and text, some will use the skepticism that creates to challenge the authenticity of real content, a phenomena coined the “liar’s dividend.”[7]
- Taking action to identify synthetic content and address its usage
Last month, the European Parliament signed the European Union AI Act. It highlights these very issues and obligates developers and deployers to assist in ensuring, to the extent feasible, that individuals are able to distinguish between original and AI-generated or manipulated content. The Act states:
A variety of AI systems can generate large quantities of synthetic content that becomes increasingly hard for humans to distinguish from human-generated and authentic content. The wide availability and increasing capabilities of those systems have a significant impact on the integrity and trust in the information ecosystem, raising new risks of misinformation and manipulation at scale, fraud, impersonation and consumer deception. In light of those impacts, the fast technological pace and the need for new methods and techniques to trace origin of information, it is appropriate to require providers of those systems to embed technical solutions that enable marking in a machine readable format and detection that the output has been generated or manipulated by an AI system and not a human. Such techniques and methods should be sufficiently reliable, interoperable, effective and robust as far as this is technically feasible, taking into account available techniques or a combination of such techniques, such as watermarks, metadata identifications, cryptographic methods for proving provenance and authenticity of content, logging methods, fingerprints or other techniques, as may be appropriate. When implementing this obligation, providers should also take into account the specificities and the limitations of the different types of content and the relevant technological and market developments in the field, as reflected in the generally acknowledged state-of-the-art. Such techniques and methods can be implemented at the level of the system or at the level of the model, including general purpose AI models generating content, thereby facilitating fulfilment of this obligation by the downstream provider of the AI system. To remain proportionate, it is appropriate to envisage that this marking obligation should not cover AI systems performing primarily an assistive function for standard editing or AI systems not substantially altering the input data provided by the deployer or the semantics thereof.
It also specifically obligates deployers who use an AI system to generate or manipulate image, audio, or video content that “appreciably resembles existing persons, places or events and would falsely appear to a person to be authentic (deep fakes), should also clearly and distinguishably disclose that the content has been artificially created or manipulated by labelling the artificial intelligence output accordingly and disclosing its artificial origin.”
There is currently an arms race in techniques for distinguishing between synthetic and authentic content and companies are declaring their commitment to identifying such content. There are various methods for deciphering AI-generated or altered content, although none are foolproof and all require updates as technology advances:
There are several approaches that have been proposed for detecting AI-generated content. The four most prominent approaches are watermarking (in its various forms), which is the embedding of an identifiable pattern in a piece of content to track its origin; content provenance, which securely embeds and maintains information about the origin of the content in its metadata; retrieval-based detectors, where all AI-generated content is stored in a database that can be queried to check the origin of content; and post-hoc detectors, which rely on machine learning models to identify subtle but systematic patterns in AI-generated content that distinguish it from human-authored content.[8]
Recently, Meta has committed to “label images that users post to Facebook, Instagram and Threads when we can detect industry standard indicators that they are AI-generated.”[9] A group of tech companies, including Adobe, Google, and Microsoft, has established the Coalition for Content Provenance and Authenticity (C2PA) to address “the prevalence of misleading information online through the development of technical standards for certifying the source and history (or provenance) of media content.”[10] OpenAI announced that it will add C2PA metadata to images created with ChatGPT and the API for the DALL-E 3 model.
In fact, many companies have already voluntarily committed to follow specified guidelines. As described in the White House fact sheet:
President Biden [convened] seven leading AI companies at the White House [] – Amazon, Anthropic, Google, Inflection, Meta, Microsoft, and OpenAI – to announce that the Biden-Harris Administration has secured voluntary commitments from these companies to help move toward safe, secure, and transparent development of AI technology.
Companies that are developing these emerging technologies have a responsibility to ensure their products are safe. To make the most of AI’s potential, the Biden-Harris Administration is encouraging this industry to uphold the highest standards to ensure that innovation doesn’t come at the expense of Americans’ rights and safety.
These commitments, which the companies have chosen to undertake immediately, underscore three principles that must be fundamental to the future of AI – safety, security, and trust – and mark a critical step toward developing responsible AI. As the pace of innovation continues to accelerate, the Biden-Harris Administration will continue to remind these companies of their responsibilities and take decisive action to keep Americans safe.[11]
The most relevant commitment is focused on earning the public’s trust by ensuring individuals are aware of when content is AI generated:
Develop and deploy mechanisms that enable users to understand if audio or visual content is AI-generated, including robust provenance, watermarking, or both, for AI-generated audio or visual content
Companies making this commitment recognize that it is important for people to be able to understand when audio or visual content is AI-generated. To further this goal, they agree to develop robust mechanisms, including provenance and/or watermarking systems for audio or visual content created by any of their publicly available systems within scope introduced after the watermarking system is developed. They will also develop tools or APIs to determine if a particular piece of content was created with their system. Audiovisual content that is readily distinguishable from reality or that is designed to be readily recognizable as generated by a company’s AI system—such as the default voices of AI assistants—is outside the scope of this commitment. The watermark or provenance data should include an identifier of the service or model that created the content, but it need not include any identifying user information. More generally, companies making this commitment pledge to work with industry peers and standards-setting bodies as appropriate towards developing a technical framework to help users distinguish audio or visual content generated by users from audio or visual content generated by AI.
- Creating tools to identify synthetic content
This bill looks to implement some of these approaches to ensuring individuals can identify when content is AI generated by placing a series of obligations on “covered providers,” businesses that provide a generative AI system that has met certain thresholds.
These covered providers are required to create AI detection tools that individuals can use to assess whether certain content was created by the provider’s generative AI system. Individuals will be able to upload content to the tool, which must be publicly accessible through a URL on the provider’s website. The bill places strict privacy limitations both for those that created the AI-generated content and those running queries on the tool, except as provided.
Using methods like those discussed in detail above, covered providers are required to include both clear and conspicuous and imperceptible disclosures in content generated by their systems to identify it as synthetic content. Despite the identified limitations for such markings, the author argues this process will help to create a digital paper trail for AI-generated content. Specifically, the bill requires the systems’ outputs metadata to include not only identification of the content as synthetic, but also the date and time it was created and the system used to create it. These markings must be permanent or difficult to remove, to the extent technically feasible. To deter tampering with these disclosures, the bill requires providers to implement reasonable procedures to prevent downstream interference with the required disclosures, including requirements to contractually obligate third-party licensees not to remove them and to terminate access when there is a reasonable basis to believe they have.
The bill creates an oversight role for CDT. Covered providers must register with and pay a fee to CDT and provide a URL to their detection tools. CDT is to create a registry that displays the providers with basic identifying information and links to their detection tools. CDT is authorized to issue regulations as necessary.
The Attorney General is charged with enforcement, authorizing civil penalties of $5,000 per violation per day.
The author states:
Generative Artificial Intelligence (AI) is advancing at an unprecedented pace, revolutionizing industries and everyday life. However, this rapid progress has also sparked a pressing need for regulatory frameworks to govern its use. While AI offers innovative solutions that drive economic growth and efficiency, it simultaneously presents complex challenges related to misinformation, manipulation, bias, and ownership of generated content.
One of the most concerning aspects of unregulated AI models is their potential to produce “deepfake” content, such as manipulated images and videos, which can be used to deceive or manipulate individuals and society at large. Deepfake pornography and fabricated political messages have already emerged as serious threats, highlighting the urgency of implementing effective regulations.
As AI-generated content approaches a level of realism that makes it indistinguishable from genuine content, the need for transparency becomes paramount. It is crucial for industry leaders to take responsibility for clearly labeling AI-generated content and developing tools that can accurately identify such content, especially when there is no visible disclosure. This transparency is essential for maintaining trust in AI technologies and ensuring that users can make informed decisions about the content they consume.
SB 942 stands as a critical legislative response in our ongoing regulatory arms race with artificial intelligence.
Concerns have been raised in response to this and other measures attempting to implement identification protocols or detection tools. Many existing tools suffer from inaccuracies and false positives. Various technical methods for synthetic content transparency, such as watermarking, fingerprinting, or metadata, are being developed and are evolving but are far from foolproof, as individuals can, for instance, sometimes find ways to remove or obscure these disclosures from content. A coalition in opposition, including Technet, writes:
Many of our companies and platforms are at the forefront of developing content provenance and watermarking technology, which is still in its early stages. However, SB 942 enacts requirements for a technology that is still under development and rapidly evolving. For example, there isn’t a program that can watermark text, making the bill’s requirements to do so impossible to comply with. We believe references to text watermarking should be removed to reflect this reality.
Furthermore, content provenance and watermarking is still incredibly unreliable and in many cases easy to break. Researchers at the University of Maryland were able to break all the currently available watermarking methods. Some can be avoided by simple cropping, resizing, or screenshotting an image. More concerning, these researchers were able to insert fake watermarks and credentials into images, creating false positives.
In its standards for large online platforms, SB 942 should more clearly delineate between 1st party and 3rd party content. 1st party content would be images, videos, or audio that is generated using a covered provider’s generative AI tools and is then posted or distributed on that platform. In this instance, a covered provider can control the creation of a content provenance or watermark into the content. As mentioned, many of our companies are already working to incorporate this type of technology to increase transparency around AI-generated content. It is currently technically infeasible to accurately and reliably detect content that is created using a different platform’s AI tools. As noted above, considering the current ease with which current watermarks can be broken, a legal requirement and mandate for 3rd party content isn’t appropriate.
The author responds:
While studies have shown the vulnerability of existing third-party AI-detection tools and industry watermarking efforts to bad actors, we don’t believe it to be crucial enough to abandon transparency efforts. We recognize that watermarking efforts will not prevent all forms of abuse, but it will provide a measure of protection against harm. Our bill also recognizes the need for these safeguards to be continually adapted and refined as adversaries find novel ways to weaponize the latest technologies.
Writing in a support if amended position, Oakland Privacy also focuses in on the nascent stage of digital provenance tools and raises issues with this portion of the bill:
The second part of the bill, and the part which we do not support, relates to a requirement for the providers to place an AI detection tool on their websites and requires the Department of Government Operations to set up a registry of providers and the URLs for their detection tools. We acknowledge that the bill is meant to be forward-thinking and that AI detection tools will improve over time, but the current state of development of these tools is so nascent and so currently flawed that state sanction of these tools as accurate and helpful tools is misguided and inappropriate.
How bad are AI detection tools? Pretty bad. Content-At-Scale, one of the largest AI detection tools identified sections of the United States Constitution as highly likely to be AI generated. The most developed tools address text and rely on certain indicators including predictability, variety of sentence length and structure, word repetition, unnatural word usage, and inconsistent verb tense. It goes without saying that many of these are also characteristics of poorly written human-generated text. The tools are dealing in probabilities and while they may effectively flag bad writing that *could* be AI-generated, they are simply not accurate enough to be relied on as evidence of anything. We would also express a fairly high level of certainty based on these indicators that they would frequently target text written by non-native speakers of English, introducing yet another possible ramification of bias in AI tools.
The bill largely models legislation introduced at the federal level by Senators Schatz and Kennedy, the AI Labeling Act of 2023. Also at the federal level, President Biden issued an executive order late last year on AI, to set standards for safety and security and to ensure responsible AI development and deployment in the United States. The order defined generative AI as “the class of AI models that emulate the structure and characteristics of input data in order to generate derived synthetic content. This can include images, videos, audio, text, and other digital content.” The author may wish to incorporate this definition into the bill to ensure greater cohesion with federal efforts and to create efficiencies for compliance in the industry.
The EO calls for the development of a risk management framework for generative AI, modeled after the AI Risk Management Framework created by the National Institute for Standards and Technology (NIST). NIST reports that it is contributing to a report on authenticating, labeling, or detecting synthetic content. The report will identify existing, and potential future, standards, tools, methods, and practices for:
- Authenticating content and tracking its provenance.
- Detecting and labeling synthetic content, with techniques such as watermarking.
- Preventing generative AI from producing child sexual abuse material or any material containing non-consensual intimate imagery of real individuals.
- Testing software used for the above purposes.
- Auditing and maintaining synthetic content.[12]
Such guidance will assist covered providers in identifying best practices in carrying out their obligations pursuant to this bill.
- Defining AI
Given the immense potential but attendant challenges and dangers of advancing AI technologies, the Legislature is currently considering dozens of bills on the subject of regulating and fostering AI. The first challenge is determining exactly what we mean by the term. A more thorough discussion of the various definitions that have been crafted by national and international entities for AI can be found in this Committee’s analysis of SB 1047 (Wiener, 2024). In order to gain both the benefit of the expertise and compromise that went into formulating those definitions and the efficiencies that come with harmonization, the Committee, with a variety of stakeholders, including the author, have come up with the following definition to begin this process and to amend into the bill:
“Artificial intelligence” means an engineered or machine-based system that, for explicit or implicit objectives, infers, from the input it receives, how to generate outputs that can influence physical or virtual environments and that may operate with varying levels of autonomy.
SUPPORT
None received
OPPOSITION
California Chamber of Commerce
Computer and Communications Industry Association
Netchoice
Technet
RELATED LEGISLATION
Pending Legislation:
SB 970 (Ashby, 2024) ensures that media manipulated or generated by artificial intelligence (AI) technology is incorporated into the right of publicity law and criminal false impersonation statutes. The bill requires those providing access to such technology to provide a warning to consumers about liability for misuse. The bill also requires Judicial Council to create screening procedures to identify written evidence altered or created by AI technology and to provide educational materials to court users and personnel on identifying such materials. SB 970 is currently in the Senate Public Safety Committee.
SCR 17 (Dodd, 2023) affirms the California Legislature’s commitment to President Biden’s vision for a safe AI and the principles outlined in the “Blueprint for an AI Bill of Rights” and expresses the Legislature’s commitment to examining and implementing those principles in its legislation and policies related to the use and deployment of automated systems. SCR 17 is currently in the Assembly Privacy and Consumer Protection Committee.
AB 2930 (Bauer-Kahan, 2024) requires, among other things, a deployer and a developer of an automated decision tool to, on or before January 1, 2026, and annually thereafter, perform an impact assessment for any automated decision tool the deployer uses that includes, among other things, a statement of the purpose of the automated decision tool and its intended benefits, uses, and deployment contexts. The assessments must be provided to the Civil Rights Department within 7 days of a request. AB 2930 requires a deployer to, at or before the time an automated decision tool is used to make a consequential decision, notify any natural person that is the subject of the consequential decision that an automated decision tool is being used to make, or be a controlling factor in making, the consequential decision and to provide that person with, among other things, a statement of the purpose of the automated decision tool. AB 2930 is currently in the Assembly Privacy and Consumer Protection Committee.
AB 3211 (Wicks, 2024) establishes the California Provenance, Authenticity and Watermarking Standards Act, which requires a generative AI system provider, as defined, to take certain actions to assist in the disclosure of provenance data to mitigate harms caused by inauthentic content, including placing imperceptible and maximally indelible watermarks containing provenance data into content created by an AI system that the generative AI system provider makes available. AB 3211 also requires a large online platform, as defined, to, among other things, use labels to prominently disclose the provenance data found in watermarks or digital signatures in content distributed to users on its platforms, as specified. The bill would require a large online platform to use state-of-the-art techniques, including, but not limited to, analysis of user behavioral signals indicating usage of synthetic content, to detect and label inauthentic text content that is uploaded or distributed by individual users or networks of users. AB 3211 is currently in the Assembly Privacy and Consumer Protection Committee.
Prior Legislation:
SB 444 (Umberg, 2019) would have requested the Regents of the University of California (UC) to enact a resolution authorizing the law schools at UC Berkeley and UC Irvine to participate in a pilot project to develop AI or machine-learning solutions to address access to justice issues faced by self-representing litigants in their respective courts. The bill died in the Assembly Higher Education Committee.
AB 1576 (Calderon, 2019) would have required the Secretary of GovOps to appoint participants to an AI working group to evaluate the uses, risks, benefits, and legal implications associated with the development and deployment of AI by California-based businesses. The bill was held on the Senate Appropriations Committee suspense file.
SJR 6 (Chang, Res. Ch. 112, Stats. 2019) urged the President and the Congress of the United States to develop a comprehensive AI Advisory Committee and to adopt a comprehensive AI policy.
**************
Siddarth Srinivasan, Detecting AI fingerprints: A guide to watermarking and beyond (January 4, 2024) Brookings Institution, https://www.brookings.edu/articles/detecting-ai-fingerprints-a-guide-to-watermarking-and-beyond/#:~:text=Google%20also%20recently%20announced%20SynthID,model%20to%20detect%20the%20watermark. All internet citations are current as of April 2, 2024. ↑
Lonnie Lee Hood, Experts Say That Soon, Almost The Entire Internet Could Be Generated by AI (March 4, 2022) The Byte, https://futurism.com/the-byte/ai-internet-generation. ↑
Brian Contreras, Tougher AI Policies Could Protect Taylor Swift—And Everyone Else—From Deepfakes (February 8, 2024) Scientific American, https://www.scientificamerican.com/article/tougher-ai-policies-could-protect-taylor-swift-and-everyone-else-from-deepfakes/. ↑
Hannah Fry, Laguna Beach High School investigates ‘inappropriate’ AI-generated images of students (April 2, 2024) Los Angeles Times, https://www.latimes.com/california/story/2024-04-02/laguna-beach-high-school-investigating-creation-of-ai-generated-images-of-students. ↑
Em Steck & Andrew Kaczynski, Fake Joe Biden robocall urges New Hampshire voters not to vote in Tuesday’s Democratic primary (January 22, 2024) CNN, https://www.cnn.com/2024/01/22/politics/fake-joe-biden-robocall/index.html. ↑
Avalon Zoppo, Threat of AI-Generated ‘Deepfake’ Evidence Needs Judiciary’s Attention, Former Judge Says (October 27, 2023) The National Law Journal, https://www.law.com/nationallawjournal/2023/10/27/threat-of-ai-generated-deepfake-evidence-needs-judiciarys-attention-former-judge-says/?slreturn=20240303000917. ↑
Bobby Chesney & Danielle Citron, Deep Fakes: A Looming Challenge for Privacy, Democracy, and National Security (July 14, 2018) 107 California Law Review 1753 (2019), https://ssrn.com/abstract=3213954. ↑
See footnote 1. ↑
Nick Clegg, Labeling AI-Generated Images on Facebook, Instagram and Threads (February 6, 2024) Meta, https://about.fb.com/news/2024/02/labeling-ai-generated-images-on-facebook-instagram-and-threads/. ↑
Overview, Coalition for Content Provenance and Authenticity, https://c2pa.org/. ↑
FACT SHEET: Biden-Harris Administration Secures Voluntary Commitments from Leading Artificial Intelligence Companies to Manage the Risks Posed by AI (July 21, 2023) The White House, https://www.whitehouse.gov/briefing-room/statements-releases/2023/07/21/fact-sheet-biden-harris-administration-secures-voluntary-commitments-from-leading-artificial-intelligence-companies-to-manage-the-risks-posed-by-ai/. ↑
Executive Order on Safe, Secure, and Trustworthy Artificial Intelligence: Synthetic Content, NIST, https://www.nist.gov/artificial-intelligence/executive-order-safe-secure-and-trustworthy-artificial-intelligence-2. ↑
Discussed in Hearing
Senate Floor
Assembly Floor
Assembly Standing Committee on Judiciary
Assembly Standing Committee on Privacy and Consumer Protection
Senate Floor
Senate Standing Committee on Governmental Organization
Senate Standing Committee on Governmental Organization
Senate Standing Committee on Governmental Organization
Bill Author
Bill Co-Author(s):