AB 2655: Defending Democracy from Deepfake Deception Act of 2024.
- Session Year: 2023-2024
- House: Assembly
- Latest Version Date: 2024-09-17
Current Status:
Passed
(2024-09-17: Chaptered by Secretary of State - Chapter 261, Statutes of 2024.)
Introduced
In Committee
First Chamber
In Committee
Second Chamber
Enacted
What does this bill do?
The bill requires large online platforms (sites and apps with at least 1,000,000 California users) to block materially deceptive media (deepfakes) that falsely show or quote candidates or election officials during specified months before and after California elections, and to label manipulated images, audio, or video as inauthentic during a broader pre‑election period. Platforms must give Californians an easy way to report such content and respond quickly, and candidates, election officials, the attorney general, or local prosecutors may ask a court to order removal or labeling; the bill exempts broadcasters, routinely published online newspapers, and satire or parody.
The summary is provided by generative AI which can make mistakes. For original language, please view the details in the table below.
Existing law establishes requirements for the conduct of election campaigns, including requirements regarding the endorsement of candidates, political corporations, campaign funds, fair campaign practices, and libel and slander. Existing law, until January 1, 2027, prohibits any person, committee, or other entity from distributing, with actual malice, materially deceptive audio or visual media of a candidate for elective office with the intent to injure the candidates reputation or to deceive a voter into voting for or against the candidate, within 60 days of the election. Existing law requires specified actions pertaining to elections to be given precedence when they are filed in court, including actions involving the registration of voters, the certification of candidates and measures, and election contests, and, until January 1, 2027, actions involving the foregoing prohibition against materially deceptive media.
This bill, to be known as the Defending Democracy from Deepfake Deception Act of 2024, would require a large online platform, as defined, to block the posting of materially deceptive content related to elections in California, during specified periods before and after an election. The bill would require a large online platform to label certain additional content inauthentic, fake, or false during specified periods before and after an election in California.
The bill would require a large online platform to develop procedures for California residents to report content that has not been blocked or labeled in compliance with the act. The bill would also authorize candidates for elected office, elected officials, elections officials, the Attorney General, and a district attorney or city attorney to seek injunctive relief against a large online platform for noncompliance with the act, as specified, and would assign precedence to such actions when they are filed in court.
The bill would exempt from its provisions a broadcasting station and a regularly published online newspaper, magazine, or other periodical of general circulation that satisfy specified requirements. The bill would also exempt content that is satire or parody.
The bill would incorporate additional changes to Section 35 of the Code of Civil Procedure proposed by AB 2839 to be operative only if this bill and AB 2839 are enacted and this bill is enacted last.
Existing law establishes requirements for the conduct of election campaigns, including requirements regarding the endorsement of candidates, political corporations, campaign funds, fair campaign practices, and libel and slander. Existing law, until January 1, 2027, prohibits any person, committee, or other entity from distributing, with actual malice, materially deceptive audio or visual media of a candidate for elective office with the intent to injure the candidates reputation or to deceive a voter into voting for or against the candidate, within 60 days of the election. Existing law requires specified actions pertaining to elections to be given precedence when they are filed in court, including actions involving the registration of voters, the certification of candidates and measures, and election contests, and, until January 1, 2027, actions involving the foregoing prohibition against materially deceptive media.
This bill, to be known as the Defending Democracy from Deepfake Deception Act of 2024, would require a large online platform, as defined, to block the posting of materially deceptive content related to elections in California, during specified periods before and after an election. The bill would require a large online platform to label certain additional content inauthentic, fake, or false during specified periods before and after an election in California.
The bill would require a large online platform to develop procedures for California residents to report content that has not been blocked or labeled in compliance with the act. The bill would also authorize candidates for elected office, elected officials, elections officials, the Attorney General, and a district attorney or city attorney to seek injunctive relief against a large online platform for noncompliance with the act, as specified, and would assign precedence to such actions when they are filed in court.
The bill would exempt from its provisions a broadcasting station and a regularly published online newspaper, magazine, or other periodical of general circulation that satisfy specified requirements. The bill would also exempt content that is satire or parody.
The bill would incorporate additional changes to Section 35 of the Code of Civil Procedure proposed by AB 2839 to be operative only if this bill and AB 2839 are enacted and this bill is enacted last.
Existing law establishes requirements for the conduct of election campaigns, including requirements regarding the endorsement of candidates, political corporations, campaign funds, fair campaign practices, and libel and slander. Existing law, until January 1, 2027, prohibits any person, committee, or other entity from distributing, with actual malice, materially deceptive audio or visual media of a candidate for elective office with the intent to injure the candidates reputation or to deceive a voter into voting for or against the candidate, within 60 days of the election. Existing law requires specified actions pertaining to elections to be given precedence when they are filed in court, including actions involving the registration of voters, the certification of candidates and measures, and election contests, and, until January 1, 2027, actions involving the foregoing prohibition against materially deceptive media.
This bill, to be known as the Defending Democracy from Deepfake Deception Act of 2024, would require a large online platform, as defined, to block the posting of materially deceptive content related to elections in California, during specified periods before and after an election. The bill would require a large online platform to label certain additional content inauthentic, fake, or false during specified periods before and after an election in California.
The bill would require a large online platform to develop procedures for California residents to report content that has not been blocked or labeled in compliance with the act. The bill would also authorize candidates for elected office, elected officials, elections officials, the Attorney General, and a district attorney or city attorney to seek injunctive relief against a large online platform for noncompliance with the act, as specified, and would assign precedence to such actions when they are filed in court.
The bill would exempt from its provisions a broadcasting station and a regularly published online newspaper, magazine, or other periodical of general circulation that routinely carries news and commentary of general interest, if the publication complies with specified disclosure satisfy specified requirements. The bill would also exempt content that is satire or parody.
The bill would incorporate additional changes to Section 35 of the Code of Civil Procedure proposed by AB 2839 to be operative only if this bill and AB 2839 are enacted and this bill is enacted last.
Existing law establishes requirements for the conduct of election campaigns, including requirements regarding the endorsement of candidates, political corporations, campaign funds, fair campaign practices, and libel and slander. Existing law, until January 1, 2027, prohibits any person, committee, or other entity from distributing, with actual malice, materially deceptive audio or visual media of a candidate for elective office with the intent to injure the candidates reputation or to deceive a voter into voting for or against the candidate, within 60 days of the election. Existing law requires specified actions pertaining to elections to be given precedence when they are filed in court, including actions involving the registration of voters, the certification of candidates and measures, and election contests, and, until January 1, 2027, actions involving the foregoing prohibition against materially deceptive media.
This bill, to be known as the Defending Democracy from Deepfake Deception Act of 2024, would require a large online platform, as defined, to block the posting of materially deceptive content related to elections, elections in California, during specified periods before and after an election. The bill would require a large online platform to label certain additional content inauthentic, fake, or false during specified periods before and after an election. election in California.
The bill would require a large online platform to develop procedures for California residents to report content that has not been blocked or labeled in compliance with the act. The bill would also authorize candidates for elected office, elected officials, elections officials, the Attorney General, and a district attorney or city attorney to seek injunctive relief against a large online platform for noncompliance with the act, as specified, and would assign precedence to such actions when they are filed in court.
The bill would exempt from its provisions a regularly published online newspaper, magazine, or other periodical of general circulation that routinely carries news and commentary of general interest, if the publication complies with specified disclosure requirements. The bill would also exempt content that is satire or parody.
Existing law establishes requirements for the conduct of election campaigns, including requirements regarding the endorsement of candidates, political corporations, campaign funds, fair campaign practices, and libel and slander. Existing law, until January 1, 2027, prohibits any person, committee, or other entity from distributing, with actual malice, materially deceptive audio or visual media of a candidate for elective office with the intent to injure the candidates reputation or to deceive a voter into voting for or against the candidate, within 60 days of the election. Existing law requires specified actions pertaining to elections to be given precedence when they are filed in court, including actions involving the registration of voters, the certification of candidates and measures, and election contests, and, until January 1, 2027, actions involving the foregoing prohibition against materially deceptive media.
This bill, to be known as the Defending Democracy from Deepfake Deception Act of 2024, would require a large online platform, as defined, to block the posting or sending of materially deceptive content related to elections, during specified periods before and after an election. The bill would require a large online platform to label certain additional content inauthentic, fake, or false during specified periods before and after an election.
The bill would require a large online platform to develop procedures for California residents to report content that has not been blocked or labeled in compliance with the act. The bill would also authorize candidates for elected office, elected officials, elections officials, the Attorney General, and a district attorney or city attorney to seek injunctive relief against a large online platform for noncompliance with the act, as specified, and would assign precedence to such actions when they are filed in court.
The bill would exempt from its provisions a regularly published online newspaper, magazine, or other periodical of general circulation that routinely carries news and commentary of general interest, if the publication complies with specified disclosure requirements. The bill would also exempt content that is satire or parody.
Existing law establishes requirements for the conduct of election campaigns, including requirements regarding the endorsement of candidates, political corporations, campaign funds, fair campaign practices, and libel and slander. Existing law, until January 1, 2027, prohibits any person, committee, or other entity from distributing, with actual malice, materially deceptive audio or visual media of a candidate for elective office with the intent to injure the candidates reputation or to deceive a voter into voting for or against the candidate, within 60 days of the election. Existing law requires specified actions pertaining to elections to be given precedence when they are filed in court, including actions involving the registration of voters, the certification of candidates and measures, and election contests, and, until January 1, 2027, actions involving the foregoing prohibition against materially deceptive media.
This bill, to be known as the Defending Democracy from Deepfake Deception Act of 2024, would require a large online platform, as defined, to block the posting or sending of materially deceptive and digitally modified or created content related to elections, during specified periods before and after an election. The bill would require a large online platform to label certain additional content inauthentic, fake, or false during specified periods before and after an election.
The bill would require a large online platform to develop procedures for California residents to report content that has not been blocked or labeled in compliance with the act. The bill would also authorize candidates for elected office, elected officials,elections officials, elections officials, the Attorney General, and a district attorney or city attorney to seek injunctive relief against a large online platform for noncompliance with the act, as specified, and would assign precedence to such actions when they are filed in court.
The bill would exempt from its provisions a regularly published online newspaper, magazine, or other periodical of general circulation that routinely carries news and commentary of general interest, if the publication complies with specified disclosure requirements. The bill would also exempt content that is satire or parody.
Existing law establishes requirements for the conduct of election campaigns, including requirements regarding the endorsement of candidates, political corporations, campaign funds, fair campaign practices, and libel and slander. Existing law, until January 1, 2027, prohibits any person, committee, or other entity from distributing, with actual malice, materially deceptive audio or visual media of a candidate for elective office with the intent to injure the candidates reputation or to deceive a voter into voting for or against the candidate, within 60 days of the election. Existing law requires specified actions pertaining to elections to be given precedence when they are filed in court, including actions involving the registration of voters, the certification of candidates and measures, and election contests, and, until January 1, 2027, actions involving the foregoing prohibition against materially deceptive media.
This bill would establish bill, to be known as the Defending Democracy from Deepfake Deception Act of 2024 for the purpose of preventing the online dissemination of manipulated media and disinformation meant to deceive voters and to prevent them from voting. The bill 2024, would require a large online platform, as defined, to block the posting or sending of materially deceptive and digitally modified or created content related to elections, during specified periods before and after an election. The bill would require a large online platform to label certain additional content inauthentic, fake, or false during specified periods before and after an election.
The bill would require a large online platform to develop procedures for California residents to report content that has not been blocked or labeled in compliance with the act. The bill would also authorize California residents, candidates for elected office, elected officials,elections officials, the Attorney General, and a district attorney or city attorney to seek injunctive relief against a large online platform for noncompliance with the act, as specified, and would assign precedence to such actions when they are filed in court.
The bill would exempt from its provisions a regularly published online newspaper, magazine, or other periodical of general circulation that routinely carries news and commentary of general interest, if the publication complies with specified disclosure requirements. The bill would also exempt content that is satire or parody.
Existing law establishes requirements for the conduct of election campaigns, including requirements regarding the endorsement of candidates, political corporations, campaign funds, fair campaign practices, and libel and slander. Existing law, until January 1, 2027, prohibits any person, committee, or other entity from distributing, with actual malice, materially deceptive audio or visual media of a candidate for elective office with the intent to injure the candidates reputation or to deceive a voter into voting for or against the candidate, within 60 days of the election. Existing law requires specified actions pertaining to elections to be given precedence when they are filed in court, including actions involving the registration of voters, the certification of candidates and measures, and election contests, and, until January 1, 2027, actions involving the foregoing prohibition against materially deceptive media.
This bill would establish the Defending Democracy from Deepfake Deception Act of 2024 for the purpose of preventing the online dissemination of manipulated media and disinformation meant to deceive voters and to prevent them from voting. The bill would require a large online platform, as defined, to block the posting or sending of materially deceptive and digitally modified or created content related to elections, during specified periods before and after an election. The bill would require a large online platform to label certain additional content inauthentic, fake, or false during specified periods before and after an election.
The bill would require a large online platform to develop procedures for California residents to report content that has not been blocked or labeled in compliance with the act. The bill would also authorize California residents, the Attorney General, and a district attorney or city attorney to seek injunctive relief against a large online platform for noncompliance with the act, as specified, and would assign precedence to such actions when they are filed in court.
The bill would exempt from its provisions a regularly published online newspaper, magazine, or other periodical of general circulation that routinely carries news and commentary of general interest, if the publication complies with specified disclosure requirements. The bill would also exempt content that is satire or parody.
Existing law establishes requirements for the conduct of election campaigns, including requirements regarding the endorsement of candidates, political corporations, campaign funds, fair campaign practices, and libel and slander. Existing law, until January 1, 2027, prohibits any person, committee, or other entity from distributing, with actual malice, materially deceptive audio or visual media of a candidate for elective office with the intent to injure the candidates reputation or to deceive a voter into voting for or against the candidate, within 60 days of the election. Existing law requires specified actions pertaining to elections to be given precedence when they are filed in court, including actions involving the registration of voters, the certification of candidates and measures, and election contests, and, until January 1, 2027, actions involving the foregoing prohibition against materially deceptive media.
This bill would establish the Defending Democracy from Deepfake Deception Act of 2024 for the purpose of preventing the online dissemination of manipulated media and disinformation meant to deceive voters and to prevent them from voting. The bill would require a large online platform, as defined, to block the posting or sending of materially deceptive and digitally modified or created content related to elections, during specified periods before and after an election. The bill would require a large online platform to label certain additional content inauthentic, fake, or false during specified periods before and after an election.
The bill would require a large online platform to develop procedures for California residents to report content that has not been blocked or labeled in compliance with the act. The bill would also authorize California residents, the Attorney General, and a district attorney or city attorney to seek injunctive relief against a large online platform for noncompliance with the act, as specified, and would assign precedence to such actions when they are filed in court.
The bill would exempt from its provisions a regularly published online newspaper, magazine, or other periodical of general circulation that routinely carries news and commentary of general interest, if the publication complies with specified disclosure requirements. The bill would also exempt content that is satire or parody.
Existing law establishes requirements for the conduct of election campaigns, including requirements regarding the endorsement of candidates, political corporations, campaign funds, fair campaign practices, and libel and slander. Existing law, until January 1, 2027, prohibits any person, committee, or other entity from distributing, with actual malice, materially deceptive audio or visual media of a candidate for elective office with the intent to injure the candidates reputation or to deceive a voter into voting for or against the candidate, within 60 days of the election. Existing law requires specified actions pertaining to elections to be given precedence when they are filed in court, including actions involving the registration of voters, the certification of candidates and measures, and election contests, and, until January 1, 2027, actions involving the foregoing prohibition against materially deceptive media.
This bill would establish the Defending Democracy from Deepfake Deception Act of 2024 for the purpose of preventing the online dissemination of manipulated media and disinformation meant to deceive voters and to prevent them from voting. The bill would require a large online platform, as defined, to block the posting or sending of materially deceptive and digitally modified or created content related to elections, during specified periods before and after an election. The bill would require a large online platform to label certain additional content inauthentic, fake, or false during specified periods before and after an election.
The bill would require a large online platform to develop procedures for California residents to report content that has not been blocked or labeled in compliance with the act. The bill would also authorize California residents, the Attorney General, and a district attorney or city attorney to seek injunctive relief against a large online platform for noncompliance with the act, as specified, and would assign precedence to such actions when they are filed in court.
The bill would exempt from its provisions a regularly published online newspaper, magazine, or other periodical of general circulation that routinely carries news and commentary of general interest, if the publication complies with specified disclosure requirements. The bill would also exempt content that is satire or parody.
Existing law, until January 1, 2027, prohibits a person, committee, or other entity, within 60 days of an election at which a candidate for elective office will appear on the ballot, from distributing with actual malice materially deceptive audio or visual media of the candidate with the intent to injure the candidates reputation or to deceive a voter into voting for or against the candidate, unless the media includes a disclosure stating that the media has been manipulated, subject to specified exemptions.Existing law, until January 1, 2027, creates a civil cause of action for violations of that prohibition, as specified, and requires a court to give precedence to such proceedings, among others.This bill would extend the operation of these provisions indefinitely.Existing law, until January 1, 2027, prohibits a person, committee, or other entity, within 60 days of an election at which a candidate for elective office will appear on the ballot, from distributing with actual malice materially deceptive audio or visual media of the candidate with the intent to injure the candidates reputation or to deceive a voter into voting for or against the candidate, unless the media includes a disclosure stating that the media has been manipulated, subject to specified exemptions.
Existing law, until January 1, 2027, creates a civil cause of action for violations of that prohibition, as specified, and requires a court to give precedence to such proceedings, among others.
This bill would extend the operation of these provisions indefinitely.
(a)Proceedings in cases involving the registration or denial of registration of voters, the certification or denial of certification of candidates, the certification or denial of certification of ballot measures, election contests, actions under Section 20010 of the Elections Code, actions under Chapter 7 (commencing with Section 20510) of Division 20 of the Elections Code, and actions under Chapter 2 (commencing with Section 21100) of Division 21 of the Elections Code shall be placed on the calendar in the order of their date of filing and shall be given precedence.
(b)This section shall remain in effect only until January 1, 2027, and as of that date is repealed, unless a later enacted statute, that is enacted before January 1, 2027, deletes or extends that date.
SEC. 1.5.Section 35 of the Code of Civil Procedure, as amended by Section 1 of Chapter 343 of the Statutes of 2023, is amended to read:35.(a)Proceedings in cases involving the registration or denial of registration of voters, the certification or denial of certification of candidates, the certification or denial of certification of ballot measures, election contests, actions under Section 20010 or 20012 of the Elections Code, actions under Chapter 7 (commencing with Section 20510) of Division 20 of the Elections Code, and actions under Chapter 2 (commencing with Section 21100) of Division 21 of the Elections Code shall be placed on the calendar in the order of their date of filing and shall be given precedence.
(b)This section shall remain in effect only until January 1, 2027, and as of that date is repealed, unless a later enacted statute, that is enacted before January 1, 2027, deletes or extends that date.
SEC. 2.Section 35 of the Code of Civil Procedure, as amended by Section 2 of Chapter 343 of the Statutes of 2023, is amended to read:35.(a)Proceedings in cases involving the registration or denial of registration of voters, the certification or denial of certification of candidates, the certification or denial of certification of ballot measures, election contests, actions under Chapter 7 (commencing with Section 20510) of Division 20 of the Elections Code, and actions under Chapter 2 (commencing with Section 21100) of Division 21 of the Elections Code shall be placed on the calendar in the order of their date of filing and shall be given precedence.
(b)This section shall become operative January 1, 2027.
SEC. 2.5.Section 35 of the Code of Civil Procedure, as amended by Section 2 of Chapter 343 of the Statutes of 2023, is amended to read:35.(a)Proceedings in cases involving the registration or denial of registration of voters, the certification or denial of certification of candidates, the certification or denial of certification of ballot measures, election contests, actions under Section 20012 of the Elections Code, actions under Chapter 7 (commencing with Section 20510) of Division 20 of the Elections Code, and actions under Chapter 2 (commencing with Section 21100) of Division 21 of the Elections Code shall be placed on the calendar in the order of their date of filing and shall be given precedence.
(b)This section shall become operative January 1, 2027.
SEC. 3.Chapter 7 (commencing with Section 20510) is added to Division 20 of the Elections Code, to read:7.Defending Democracy from Deepfake Deception Act of 202420510.This chapter shall be known and may be cited as the Defending Democracy from Deepfake Deception Act of 2024.
20511.The Legislature finds and declares all of the following:
(a)California is entering its first-ever generative artificial intelligence (AI) election, in which disinformation powered by generative AI will pollute our information ecosystems like never before. Voters will not know what images, audio, or video they can trust.
(b)In a few clicks, using current technology, bad actors now have the power to create a false image of a candidate accepting a bribe or a fake video of an elections official caught on tape saying that voting machines are not secure, or to generate the Governors voice telling millions of Californians their voting site has changed.
(c)In the lead-up to the 2024 presidential elections, candidates and parties are already creating and distributing deepfake images and audio and video content. These fake images or files can spread to millions of Californians in seconds and skew election results or undermine trust in the ballot counting process.
(d)The labeling information required by this bill is narrowly tailored to provide consumers with factual information about the inauthenticity of particular images, audio, video, or text content in order to prevent consumer deception.
(e)In order to ensure California elections are free and fair, California must, for a limited time before and after elections, prevent the use of deepfakes and disinformation meant to prevent voters from voting and to deceive voters based on fraudulent content. Accordingly, the provisions of this chapter are narrowly tailored to support California's compelling interest in protecting its free and fair elections.
20512.For purposes of this chapter, the following terms have the following meanings:
(a)Advertisement means any general or public communication that a large online platform knows is authorized or paid for with the purpose of supporting or opposing a candidate for elective office.
(b)Broadcasting station means a radio or television broadcasting station, including any of the following:
(1)Cable operator, programmer, or producer.
(2)Streaming service operator, programmer, or producer.
(3)Direct-to-home satellite television operator, programmer, or producer.
(c)Candidate means any person running for a voter-nominated office as defined in Section 359.5, any person running for the office of President or Vice President of the United States, and any person running for the office of Superintendent of Public Instruction.
(d)Deepfake means audio or visual media that is digitally created or modified such that it would falsely appear to a reasonable person to be an authentic record of the actual speech or conduct of the individual depicted in the media.
(e)Election communication means a general or public communication that is not an advertisement and that concerns any of the following:
(1)A candidate for elective office.
(2)Voting or refraining from voting in an election in California.
(3)The canvass of the vote for an election in California, as defined in subdivision (f).
(4)Voting machines, ballots, voting sites, or other property or equipment related to an election in California.
(5)Proceedings or processes of the electoral college in California.
(f)Election in California means any election where a candidate, as defined in this section, is on the ballot, and any election where a statewide initiative or statewide referendum measure is on the ballot.
(g)Elections official means either of the following persons acting in their official capacity:
(1)An elections official as defined in Section 320.
(2)The Secretary of State.
(h)Large online platform means a public-facing internet website, web application, or digital application, including a social media platform as defined in Section 22675 of the Business and Professions Code, video sharing platform, advertising network, or search engine that had at least 1,000,000 California users during the preceding 12 months.
(i)(1)Materially deceptive content means audio or visual media that is digitally created or modified, and that includes, but is not limited to, deepfakes and the output of chatbots, such that it would falsely appear to a reasonable person to be an authentic record of the content depicted in the media.
(2)Materially deceptive content does not include any audio or visual media that contains only minor modifications that do not significantly change the perceived contents or meaning of the content. Minor changes include changes to the brightness or contrast of images, removal of background noise in audio, and other minor changes that do not impact the content of the image or audio or visual media.
20513.(a)Any large online platform shall develop and implement procedures for the use of state-of-the-art techniques to identify and remove materially deceptive content if all of the following conditions are met:
(1)The content is reported pursuant to subdivision (a) of Section 20515.
(2)The materially deceptive content is any of the following:
(A)A candidate for elective office portrayed as doing or saying something that the candidate did not do or say and that is reasonably likely to harm the reputation or electoral prospects of a candidate.
(B)An elections official portrayed as doing or saying something in connection with the performance of their elections-related duties that the elections official did not do or say and that is reasonably likely to falsely undermine confidence in the outcome of one or more election contests.
(C)An elected official portrayed as doing or saying something that influences an election in California that the elected official did not do or say and that is reasonably likely to falsely undermine confidence in the outcome of one or more election contests.
(3)The content is posted during the applicable time period or periods set forth in subdivision (e).
(4)The large online platform knows or acts with reckless disregard for the fact that the content meets the requirements of this section.
(b)If a post is determined to meet the requirements for removal pursuant to subdivision (a), any large online platform shall remove the post upon that determination, but no later than 72 hours after a report is made pursuant to subdivision (a) of Section 20515 in order to be in compliance with this chapter.
(c)Any large online platform shall identify, using state-of-the-art techniques, and remove, upon discovering or being alerted to the posting or reposting of, any identical or substantially similar materially deceptive content that the platform had previously removed pursuant to this chapter, provided that this removal occurs during the applicable time period or periods set forth in subdivision (e).
(d)(1)Notwithstanding subparagraph (A) of paragraph (2) of subdivision (a), this section does not apply to a candidate for elective office who, during the time period set forth in subdivision (e), portrays themself as doing or saying something that the candidate did not do or say, if the digital content includes a disclosure stating the following: This _____ has been manipulated. The blank in this disclosure shall be filled in with whichever of the following terms most accurately describes the media:
(A)Image.
(B)Audio.
(C)Video.
(2)(A)For visual media, the text of the disclosure shall appear in a size that is easily readable by the average viewer and no smaller than the largest font size of other text appearing in the visual media. If the visual media does not include any other text, the disclosure shall appear in a size that is easily readable by the average viewer. For visual media that is video, the disclosure shall appear for the duration of the video.
(B)If the media consists of audio only, the disclosure shall be read in a clearly spoken manner and in a pitch that can be easily heard by the average listener, at the beginning of the audio, at the end of the audio, and, if the audio is greater than two minutes in length, interspersed within the audio at intervals of not greater than two minutes each.
(e)(1)Except as provided in paragraph (2), any large online platform shall remove the content to the extent required by subdivisions (a) to (c), inclusive, and any candidate for elective office shall include the disclosure required by subdivision (d), during a period beginning 120 days before an election in California and through the day of the election.
(2)If the content described in subdivision (a) depicts or pertains to elections officials, any large online platform shall remove the content to the extent required by subdivisions (a) to (c), inclusive during a period beginning 120 days before an election in California and ending on the 60th day after the election.
20514.(a)Any large online platform shall develop and implement procedures for the use of state-of-the-art techniques to identify materially deceptive content and for labeling such content as provided in subdivision (c) if all of the following conditions are met:
(1)The content is reported pursuant to subdivision (a) of Section 20515.
(2)The materially deceptive content is either of the following:
(A)Included within subdivision (a) of Section 20513, but is posted outside the applicable time period described in subdivision (e) of Section 20513.
(B)Appears within an advertisement or election communication and is not subject to Section 20513.
(3)The large online platform knows or acts with reckless disregard for the fact that the materially deceptive content meets the requirements of this section.
(b)If a post is determined to meet the requirements for labeling pursuant to subdivision (a), any large online platform shall label the post upon that determination, but no later than 72 hours after a report is made pursuant to subdivision (a) of Section 20515 in order to be in compliance with this chapter.
(c)The label required by subdivision (a) shall state: This _____ has been manipulated and is not authentic. The blank in this disclosure shall be filled in with whichever of the following terms most accurately describes the media:
(1)Image.
(2)Audio.
(3)Video.
(d)The label required by subdivision (a) shall permit users to click or tap on it for additional explanation about the materially deceptive content in an easy-to-understand format.
(e)The labeling requirement set forth in subdivision (a) applies during any of the following time periods, to the extent applicable:
(1)The period beginning six months before an election in California and through the day of the election.
(2)The period beginning six months before an election in California and ending on the 60th day after the election, if the content depicts or pertains to elections officials, the electoral college process, a voting machine, ballot, voting site, or other equipment related to an election, or the canvass of the vote.
20515.(a)A large online platform shall provide an easily accessible way for California residents to report to that platform content that should be removed pursuant to Section 20513 or labeled pursuant to Section 20514. The large online platform shall respond to the person who made the report within 36 hours of the report, describing any action taken or not taken by the large online platform with respect to the content.
(b)A candidate for elective office, elected official, or elections official who has made a report to a large online platform under subdivision (a) and who either has not received a response within 36 hours or disagrees with the response, action taken, or failure by the large online platform to take action within 72 hours, may seek injunctive or other equitable relief against the large online platform to compel the removal of specific content as required by Section 20513, labeling of specific content as required by Section 20514, or compliance with the reporting process required by subdivision (a). The plaintiff shall bear the burden of establishing the violation through clear and convincing evidence. An action under this subdivision shall be entitled to precedence in accordance with Section 35 of the Code of Civil Procedure.
20516.The Attorney General or any district attorney or city attorney may seek injunctive or other equitable relief against any large online platform to compel the removal of specific content as required by Section 20513, labeling of specific content as required by Section 20514, or compliance with the reporting process required by subdivision (a) of Section 20515. The plaintiff shall bear the burden of establishing the violation through clear and convincing evidence. An action under this section shall be entitled to precedence in accordance with Section 35 of the Code of Civil Procedure.
20517.This chapter applies to materially deceptive content, regardless of the language used in the content. If the language used is not English, the disclosure required by subdivision (d) of Section 20513 and the label required by Section 20514 must appear in the language used as well as in English.
20518.(a)This chapter does not preclude a large online platform from blocking, removing, or labeling any materially deceptive content outside of the time periods specified in Sections 20513 and 20514.
(b)This chapter does not preclude any online platform not subject to this chapter from blocking, removing, or labeling any materially deceptive content.
20519.This chapter does not apply to any of the following:
(a)A regularly published online newspaper, magazine, or other periodical of general circulation that routinely carries news and commentary of general interest, and that publishes any materially deceptive content that an online platform is required to block or label based on this chapter, if the publication contains a clear disclosure that the materially deceptive content does not accurately represent any actual event, occurrence, appearance, speech, or expressive conduct.
(b)(1)A broadcasting station that broadcasts any materially deceptive content prohibited by this chapter as part of a bona fide newscast, news interview, news documentary, commentary of general interest, or on-the-spot coverage of bona fide news events, if the broadcast clearly acknowledges through content or a disclosure, in a manner that can be easily heard or read by the average listener or viewer, that the materially deceptive content does not accurately represent any actual event, occurrence, appearance, speech, or expressive conduct.
(2)A broadcasting station when it is paid to broadcast materially deceptive content and either of the following circumstances exist:
(A)The broadcasting station can show that it has prohibition and disclaimer requirements that are consistent with the requirements in this chapter and that it has provided those prohibition and disclaimer requirements to each person or entity that purchased the advertisement.
(B)Federal law requires the broadcasting station to air advertisements from legally qualified candidates or prohibits the broadcasting station from censoring or altering the message.
(c)Materially deceptive content that constitutes satire or parody.
20520.The provisions of this chapter are severable. If any provision of this chapter or its application is held invalid, that invalidity shall not affect other provisions or applications that can be given effect without the invalid provision or application.
SEC. 4.Section 1.5 of this bill incorporates amendments to Section 35 of the Code of Civil Procedure that are proposed by this bill and Assembly Bill 2839. Section 1.5 of this bill shall only become operative if (1) both bills are enacted and become effective on or before January 1, 2025, (2) each bill amends Section 35 of the Code of Civil Procedure, as amended by Section 1 of Chapter 343 of the Statutes of 2023, and (3) this bill is enacted after Assembly Bill 2839, in which case Section 1 of this bill shall not become operative. Additionally, if the above conditions are met and Assembly Bill 2839 becomes operative before January 1, 2025, Section 35 of the Code of Civil Procedure, as amended by Assembly Bill 2839, shall remain operative only until January 1, 2025, at which time Section 1.5 of this bill shall become operative, and Section 1 of this bill shall not become operative.
SEC. 5.Section 2.5 of this bill incorporates amendments to Section 35 of the Code of Civil Procedure that are proposed by this bill and Assembly Bill 2839. Section 2.5 of this bill shall only become operative if (1) both bills are enacted and become effective on or before January 1, 2025, (2) each bill amends Section 35 of the Code of Civil Procedure, as amended by Section 2 of Chapter 343 of the Statutes of 2023, and (3) this bill is enacted after Assembly Bill 2839, in which case Section 2 of this bill shall not become operative. Additionally, if the above conditions are met and Assembly Bill 2839 becomes operative before January 1, 2025, Section 35 of the Code of Civil Procedure, as amended by Assembly Bill 2839, shall remain operative only until January 1, 2025, at which time Section 2.5 of this bill shall become operative, and Section 2 of this bill shall not become operative.
(a)Proceedings in cases involving the registration or denial of registration of voters, the certification or denial of certification of candidates, the certification or denial of certification of ballot measures, election contests, actions under Section 20010 of the Elections Code, actions under Chapter 7 (commencing with Section 20510) of Division 20 of the Elections Code, and actions under Chapter 2 (commencing with Section 21100) of Division 21 of the Elections Code shall be placed on the calendar in the order of their date of filing and shall be given precedence.
(b)This section shall remain in effect only until January 1, 2027, and as of that date is repealed, unless a later enacted statute, that is enacted before January 1, 2027, deletes or extends that date.
SEC. 1.5.Section 35 of the Code of Civil Procedure, as amended by Section 1 of Chapter 343 of the Statutes of 2023, is amended to read:35.(a)Proceedings in cases involving the registration or denial of registration of voters, the certification or denial of certification of candidates, the certification or denial of certification of ballot measures, election contests, actions under Section 20010 or 20012 of the Elections Code, actions under Chapter 7 (commencing with Section 20510) of Division 20 of the Elections Code, and actions under Chapter 2 (commencing with Section 21100) of Division 21 of the Elections Code shall be placed on the calendar in the order of their date of filing and shall be given precedence.
(b)This section shall remain in effect only until January 1, 2027, and as of that date is repealed, unless a later enacted statute, that is enacted before January 1, 2027, deletes or extends that date.
SEC. 2.Section 35 of the Code of Civil Procedure, as amended by Section 2 of Chapter 343 of the Statutes of 2023, is amended to read:35.(a)Proceedings in cases involving the registration or denial of registration of voters, the certification or denial of certification of candidates, the certification or denial of certification of ballot measures, election contests, actions under Chapter 7 (commencing with Section 20510) of Division 20 of the Elections Code, and actions under Chapter 2 (commencing with Section 21100) of Division 21 of the Elections Code shall be placed on the calendar in the order of their date of filing and shall be given precedence.
(b)This section shall become operative January 1, 2027.
SEC. 2.5.Section 35 of the Code of Civil Procedure, as amended by Section 2 of Chapter 343 of the Statutes of 2023, is amended to read:35.(a)Proceedings in cases involving the registration or denial of registration of voters, the certification or denial of certification of candidates, the certification or denial of certification of ballot measures, election contests, actions under Section 20012 of the Elections Code, actions under Chapter 7 (commencing with Section 20510) of Division 20 of the Elections Code, and actions under Chapter 2 (commencing with Section 21100) of Division 21 of the Elections Code shall be placed on the calendar in the order of their date of filing and shall be given precedence.
(b)This section shall become operative January 1, 2027.
SEC. 3.Chapter 7 (commencing with Section 20510) is added to Division 20 of the Elections Code, to read:7.Defending Democracy from Deepfake Deception Act of 202420510.This chapter shall be known and may be cited as the Defending Democracy from Deepfake Deception Act of 2024.
20511.The Legislature finds and declares all of the following:
(a)California is entering its first-ever generative artificial intelligence (AI) election, in which disinformation powered by generative AI will pollute our information ecosystems like never before. Voters will not know what images, audio, or video they can trust.
(b)In a few clicks, using current technology, bad actors now have the power to create a false image of a candidate accepting a bribe or a fake video of an elections official caught on tape saying that voting machines are not secure, or to generate the Governors voice telling millions of Californians their voting site has changed.
(c)In the lead-up to the 2024 presidential elections, candidates and parties are already creating and distributing deepfake images and audio and video content. These fake images or files can spread to millions of Californians in seconds and skew election results or undermine trust in the ballot counting process.
(d)The labeling information required by this bill is narrowly tailored to provide consumers with factual information about the inauthenticity of particular images, audio, video, or text content in order to prevent consumer deception.
(e)In order to ensure California elections are free and fair, California must, for a limited time before and after elections, prevent the use of deepfakes and disinformation meant to prevent voters from voting and to deceive voters based on fraudulent content. Accordingly, the provisions of this chapter are narrowly tailored to support California's compelling interest in protecting its free and fair elections.
20512.For purposes of this chapter, the following terms have the following meanings:
(a)Advertisement means any general or public communication that a large online platform knows is authorized or paid for with the purpose of supporting or opposing a candidate for elective office.
(b)Broadcasting station means a radio or television broadcasting station, including any of the following:
(1)Cable operator, programmer, or producer.
(2)Streaming service operator, programmer, or producer.
(3)Direct-to-home satellite television operator, programmer, or producer.
(c)Candidate means any person running for a voter-nominated office as defined in Section 359.5, any person running for the office of President or Vice President of the United States, and any person running for the office of Superintendent of Public Instruction.
(d)Deepfake means audio or visual media that is digitally created or modified such that it would falsely appear to a reasonable person to be an authentic record of the actual speech or conduct of the individual depicted in the media.
(e)Election communication means a general or public communication that is not an advertisement and that concerns any of the following:
(1)A candidate for elective office.
(2)Voting or refraining from voting in an election in California.
(3)The canvass of the vote for an election in California, as defined in subdivision (f).
(4)Voting machines, ballots, voting sites, or other property or equipment related to an election in California.
(5)Proceedings or processes of the electoral college in California.
(f)Election in California means any election where a candidate, as defined in this section, is on the ballot, and any election where a statewide initiative or statewide referendum measure is on the ballot.
(g)Elections official means either of the following persons acting in their official capacity:
(1)An elections official as defined in Section 320.
(2)The Secretary of State.
(h)Large online platform means a public-facing internet website, web application, or digital application, including a social media platform as defined in Section 22675 of the Business and Professions Code, video sharing platform, advertising network, or search engine that had at least 1,000,000 California users during the preceding 12 months.
(i)(1)Materially deceptive content means audio or visual media that is digitally created or modified, and that includes, but is not limited to, deepfakes and the output of chatbots, such that it would falsely appear to a reasonable person to be an authentic record of the content depicted in the media.
(2)Materially deceptive content does not include any audio or visual media that contains only minor modifications that do not significantly change the perceived contents or meaning of the content. Minor changes include changes to the brightness or contrast of images, removal of background noise in audio, and other minor changes that do not impact the content of the image or audio or visual media.
20513.(a)Any large online platform shall develop and implement procedures for the use of state-of-the-art techniques to identify and remove materially deceptive content if all of the following conditions are met:
(1)The content is reported pursuant to subdivision (a) of Section 20515.
(2)The materially deceptive content is any of the following:
(A)A candidate for elective office portrayed as doing or saying something that the candidate did not do or say and that is reasonably likely to harm the reputation or electoral prospects of a candidate.
(B)An elections official portrayed as doing or saying something in connection with the performance of their elections-related duties that the elections official did not do or say and that is reasonably likely to falsely undermine confidence in the outcome of one or more election contests.
(C)An elected official portrayed as doing or saying something that influences an election in California that the elected official did not do or say and that is reasonably likely to falsely undermine confidence in the outcome of one or more election contests.
(3)The content is posted during the applicable time period or periods set forth in subdivision (e).
(4)The large online platform knows or acts with reckless disregard for the fact that the content meets the requirements of this section.
(b)If a post is determined to meet the requirements for removal pursuant to subdivision (a), any large online platform shall remove the post upon that determination, but no later than 72 hours after a report is made pursuant to subdivision (a) of Section 20515 in order to be in compliance with this chapter.
(c)Any large online platform shall identify, using state-of-the-art techniques, and remove, upon discovering or being alerted to the posting or reposting of, any identical or substantially similar materially deceptive content that the platform had previously removed pursuant to this chapter, provided that this removal occurs during the applicable time period or periods set forth in subdivision (e).
(d)(1)Notwithstanding subparagraph (A) of paragraph (2) of subdivision (a), this section does not apply to a candidate for elective office who, during the time period set forth in subdivision (e), portrays themself as doing or saying something that the candidate did not do or say, if the digital content includes a disclosure stating the following: This _____ has been manipulated. The blank in this disclosure shall be filled in with whichever of the following terms most accurately describes the media:
(A)Image.
(B)Audio.
(C)Video.
(2)(A)For visual media, the text of the disclosure shall appear in a size that is easily readable by the average viewer and no smaller than the largest font size of other text appearing in the visual media. If the visual media does not include any other text, the disclosure shall appear in a size that is easily readable by the average viewer. For visual media that is video, the disclosure shall appear for the duration of the video.
(B)If the media consists of audio only, the disclosure shall be read in a clearly spoken manner and in a pitch that can be easily heard by the average listener, at the beginning of the audio, at the end of the audio, and, if the audio is greater than two minutes in length, interspersed within the audio at intervals of not greater than two minutes each.
(e)(1)Except as provided in paragraph (2), any large online platform shall remove the content to the extent required by subdivisions (a) to (c), inclusive, and any candidate for elective office shall include the disclosure required by subdivision (d), during a period beginning 120 days before an election in California and through the day of the election.
(2)If the content described in subdivision (a) depicts or pertains to elections officials, any large online platform shall remove the content to the extent required by subdivisions (a) to (c), inclusive during a period beginning 120 days before an election in California and ending on the 60th day after the election.
20514.(a)Any large online platform shall develop and implement procedures for the use of state-of-the-art techniques to identify materially deceptive content and for labeling such content as provided in subdivision (c) if all of the following conditions are met:
(1)The content is reported pursuant to subdivision (a) of Section 20515.
(2)The materially deceptive content is either of the following:
(A)Included within subdivision (a) of Section 20513, but is posted outside the applicable time period described in subdivision (e) of Section 20513.
(B)Appears within an advertisement or election communication and is not subject to Section 20513.
(3)The large online platform knows or acts with reckless disregard for the fact that the materially deceptive content meets the requirements of this section.
(b)If a post is determined to meet the requirements for labeling pursuant to subdivision (a), any large online platform shall label the post upon that determination, but no later than 72 hours after a report is made pursuant to subdivision (a) of Section 20515 in order to be in compliance with this chapter.
(c)The label required by subdivision (a) shall state: This _____ has been manipulated and is not authentic. The blank in this disclosure shall be filled in with whichever of the following terms most accurately describes the media:
(1)Image.
(2)Audio.
(3)Video.
(d)The label required by subdivision (a) shall permit users to click or tap on it for additional explanation about the materially deceptive content in an easy-to-understand format.
(e)The labeling requirement set forth in subdivision (a) applies during any of the following time periods, to the extent applicable:
(1)The period beginning six months before an election in California and through the day of the election.
(2)The period beginning six months before an election in California and ending on the 60th day after the election, if the content depicts or pertains to elections officials, the electoral college process, a voting machine, ballot, voting site, or other equipment related to an election, or the canvass of the vote.
20515.(a)A large online platform shall provide an easily accessible way for California residents to report to that platform content that should be removed pursuant to Section 20513 or labeled pursuant to Section 20514. The large online platform shall respond to the person who made the report within 36 hours of the report, describing any action taken or not taken by the large online platform with respect to the content.
(b)A candidate for elective office, elected official, or elections official who has made a report to a large online platform under subdivision (a) and who either has not received a response within 36 hours or disagrees with the response, action taken, or failure by the large online platform to take action within 72 hours, may seek injunctive or other equitable relief against the large online platform to compel the removal of specific content as required by Section 20513, labeling of specific content as required by Section 20514, or compliance with the reporting process required by subdivision (a). The plaintiff shall bear the burden of establishing the violation through clear and convincing evidence. An action under this subdivision shall be entitled to precedence in accordance with Section 35 of the Code of Civil Procedure.
20516.The Attorney General or any district attorney or city attorney may seek injunctive or other equitable relief against any large online platform to compel the removal of specific content as required by Section 20513, labeling of specific content as required by Section 20514, or compliance with the reporting process required by subdivision (a) of Section 20515. The plaintiff shall bear the burden of establishing the violation through clear and convincing evidence. An action under this section shall be entitled to precedence in accordance with Section 35 of the Code of Civil Procedure.
20517.This chapter applies to materially deceptive content, regardless of the language used in the content. If the language used is not English, the disclosure required by subdivision (d) of Section 20513 and the label required by Section 20514 must appear in the language used as well as in English.
20518.(a)This chapter does not preclude a large online platform from blocking, removing, or labeling any materially deceptive content outside of the time periods specified in Sections 20513 and 20514.
(b)This chapter does not preclude any online platform not subject to this chapter from blocking, removing, or labeling any materially deceptive content.
20519.This chapter does not apply to any of the following:
(a)A regularly published online newspaper, magazine, or other periodical of general circulation that routinely carries news and commentary of general interest, and that publishes any materially deceptive content that an online platform is required to block or label based on this chapter, if the publication contains a clear disclosure that the materially deceptive content does not accurately represent any actual event, occurrence, appearance, speech, or expressive conduct.
(b)(1)A broadcasting station that broadcasts any materially deceptive content prohibited by this chapter as part of a bona fide newscast, news interview, news documentary, commentary of general interest, or on-the-spot coverage of bona fide news events, if the broadcast clearly acknowledges through content or a disclosure, in a manner that can be easily heard or read by the average listener or viewer, that the materially deceptive content does not accurately represent any actual event, occurrence, appearance, speech, or expressive conduct.
(2)A broadcasting station when it is paid to broadcast materially deceptive content and either of the following circumstances exist:
(A)The broadcasting station can show that it has prohibition and disclaimer requirements that are consistent with the requirements in this chapter and that it has provided those prohibition and disclaimer requirements to each person or entity that purchased the advertisement.
(B)Federal law requires the broadcasting station to air advertisements from legally qualified candidates or prohibits the broadcasting station from censoring or altering the message.
(c)Materially deceptive content that constitutes satire or parody.
20520.The provisions of this chapter are severable. If any provision of this chapter or its application is held invalid, that invalidity shall not affect other provisions or applications that can be given effect without the invalid provision or application.
SEC. 4.Section 1.5 of this bill incorporates amendments to Section 35 of the Code of Civil Procedure that are proposed by this bill and Assembly Bill 2839. Section 1.5 of this bill shall only become operative if (1) both bills are enacted and become effective on or before January 1, 2025, (2) each bill amends Section 35 of the Code of Civil Procedure, as amended by Section 1 of Chapter 343 of the Statutes of 2023, and (3) this bill is enacted after Assembly Bill 2839, in which case Section 1 of this bill shall not become operative. Additionally, if the above conditions are met and Assembly Bill 2839 becomes operative before January 1, 2025, Section 35 of the Code of Civil Procedure, as amended by Assembly Bill 2839, shall remain operative only until January 1, 2025, at which time Section 1.5 of this bill shall become operative, and Section 1 of this bill shall not become operative.
SEC. 5.Section 2.5 of this bill incorporates amendments to Section 35 of the Code of Civil Procedure that are proposed by this bill and Assembly Bill 2839. Section 2.5 of this bill shall only become operative if (1) both bills are enacted and become effective on or before January 1, 2025, (2) each bill amends Section 35 of the Code of Civil Procedure, as amended by Section 2 of Chapter 343 of the Statutes of 2023, and (3) this bill is enacted after Assembly Bill 2839, in which case Section 2 of this bill shall not become operative. Additionally, if the above conditions are met and Assembly Bill 2839 becomes operative before January 1, 2025, Section 35 of the Code of Civil Procedure, as amended by Assembly Bill 2839, shall remain operative only until January 1, 2025, at which time Section 2.5 of this bill shall become operative, and Section 2 of this bill shall not become operative.
(a)Proceedings in cases involving the registration or denial of registration of voters, the certification or denial of certification of candidates, the certification or denial of certification of ballot measures, election contests, actions under Section 20010 of the Elections Code, actions under Chapter 7 (commencing with Section 20510) of Division 20 of the Elections Code, and actions under Chapter 2 (commencing with Section 21100) of Division 21 of the Elections Code shall be placed on the calendar in the order of their date of filing and shall be given precedence.
(b)This section shall remain in effect only until January 1, 2027, and as of that date is repealed, unless a later enacted statute, that is enacted before January 1, 2027, deletes or extends that date.
SEC. 1.5.Section 35 of the Code of Civil Procedure, as amended by Section 1 of Chapter 343 of the Statutes of 2023, is amended to read:35.(a)Proceedings in cases involving the registration or denial of registration of voters, the certification or denial of certification of candidates, the certification or denial of certification of ballot measures, election contests, actions under Section 20010 or 20012 of the Elections Code, actions under Chapter 7 (commencing with Section 20510) of Division 20 of the Elections Code, and actions under Chapter 2 (commencing with Section 21100) of Division 21 of the Elections Code shall be placed on the calendar in the order of their date of filing and shall be given precedence.
(b)This section shall remain in effect only until January 1, 2027, and as of that date is repealed, unless a later enacted statute, that is enacted before January 1, 2027, deletes or extends that date.
SEC. 2.Section 35 of the Code of Civil Procedure, as amended by Section 2 of Chapter 343 of the Statutes of 2023, is amended to read:35.(a)Proceedings in cases involving the registration or denial of registration of voters, the certification or denial of certification of candidates, the certification or denial of certification of ballot measures, election contests, actions under Chapter 7 (commencing with Section 20510) of Division 20 of the Elections Code, and actions under Chapter 2 (commencing with Section 21100) of Division 21 of the Elections Code shall be placed on the calendar in the order of their date of filing and shall be given precedence.
(b)This section shall become operative January 1, 2027.
SEC. 2.5.Section 35 of the Code of Civil Procedure, as amended by Section 2 of Chapter 343 of the Statutes of 2023, is amended to read:35.(a)Proceedings in cases involving the registration or denial of registration of voters, the certification or denial of certification of candidates, the certification or denial of certification of ballot measures, election contests, actions under Section 20012 of the Elections Code, actions under Chapter 7 (commencing with Section 20510) of Division 20 of the Elections Code, and actions under Chapter 2 (commencing with Section 21100) of Division 21 of the Elections Code shall be placed on the calendar in the order of their date of filing and shall be given precedence.
(b)This section shall become operative January 1, 2027.
SEC. 3.Chapter 7 (commencing with Section 20510) is added to Division 20 of the Elections Code, to read:7.Defending Democracy from Deepfake Deception Act of 202420510.This chapter shall be known and may be cited as the Defending Democracy from Deepfake Deception Act of 2024.
20511.The Legislature finds and declares all of the following:
(a)California is entering its first-ever generative artificial intelligence (AI) election, in which disinformation powered by generative AI will pollute our information ecosystems like never before. Voters will not know what images, audio, or video they can trust.
(b)In a few clicks, using current technology, bad actors now have the power to create a false image of a candidate accepting a bribe or a fake video of an elections official caught on tape saying that voting machines are not secure, or to generate the Governors voice telling millions of Californians their voting site has changed.
(c)In the lead-up to the 2024 presidential elections, candidates and parties are already creating and distributing deepfake images and audio and video content. These fake images or files can spread to millions of Californians in seconds and skew election results or undermine trust in the ballot counting process.
(d)The labeling information required by this bill is narrowly tailored to provide consumers with factual information about the inauthenticity of particular images, audio, video, or text content in order to prevent consumer deception.
(e)In order to ensure California elections are free and fair, California must, for a limited time before and after elections, prevent the use of deepfakes and disinformation meant to prevent voters from voting and to deceive voters based on fraudulent content. Accordingly, the provisions of this chapter are narrowly tailored to support California's compelling interest in protecting its free and fair elections.
20512.For purposes of this chapter, the following terms have the following meanings:
(a)Advertisement means any general or public communication that a large online platform knows is authorized or paid for with the purpose of supporting or opposing a candidate for elective office.
(b)Broadcasting station means a radio or television broadcasting station, including any of the following:
(1)Cable operator, programmer, or producer.
(2)Streaming service operator, programmer, or producer.
(3)Direct-to-home satellite television operator, programmer, or producer.
(b)(c)Candidate means any person running for federal, state, or local office in California. The term includes persons a voter-nominated office as defined in Section 359.5, any person running for the office of President or Vice President of the United Stateswho will seek to or will appear on a ballot issued in California. States, and any person running for the office of Superintendent of Public Instruction.
(c)(d)Deepfake means audio or visual media that is digitally created or modified such that it would falsely appear to a reasonable person to be an authentic record of the actual speech or conduct of the individual depicted in the media.
(d)(e)Election communication means a general or public communication that is not an advertisement and that concerns any of the following:
(1)A candidate for elective office.
(2)Voting or refraining from voting in an election in California.
(3)The canvass of the vote for an election in California. California, as defined in subdivision (f).
(4)Voting machines, ballots, voting sites, or other property or equipment related to an election in California.
(5)Proceedings or processes of the electoral college in California.
(f)Election in California means any election where a candidate, as defined in this section, is on the ballot, and any election where a statewide initiative or statewide referendum measure is on the ballot.
(e)(g)(1)Elections official means any of the following persons, but only in their capacity as a person charged with holding or conducting an election in California, conducting a canvass in California, assisting with the holding or conducting of an election or a canvass in California, or performing another duty related to administering the provisions of this code: either of the following persons acting in their official capacity:
(A)(1)An elections official as defined in Section 320.
(B)(2)The Secretary of State and their staff. State.
(C)A temporary worker, poll worker, or member of a precinct board.(D)Any other person charged with holding or conducting an election, conducting a canvass, assisting with the holding or conducting of an election or a canvas, or performing another duty related to administering the provisions of this code.(2)The requirements of this chapter relating to content portraying an elections official apply only if the large online platform knows or should know that the person is an elections official. (f)(h)Large online platform means a public-facing internet website, web application, or digital application, including a social media platform as defined in Section 22675 of the Business and Professions Code, video sharing platform, advertising network, or search engine that had at least 1,000,000 California users during the preceding 12 months.
(g)(i)(1)Materially deceptive content means audio or visual media that is digitally created or modified, and that includes, but is not limited to, deepfakes and the output of chatbots, such that it would falsely appear to a reasonable person to be an authentic record of the content depicted in the media.
(2)Materially deceptive content does not include any audio or visual media that contains only minor modifications that do not significantly change the perceived contents or meaning of the content. Minor changes include changes to the brightness or contrast of images, removal of background noise in audio, and other minor changes that do not impact the content of the image or audio or visual media.
20513.(a)Any large online platform, using state-of-the-art, best-available tools to detect materially deceptive content, platform shall develop and implement procedures for blocking and preventing, and shall, if the large online platform knows or acts with reckless disregard for the fact that the the use of state-of-the-art techniques to identify and remove materially deceptive content meets the requirements of this section, block and prevent the posting of any materially deceptive content, if both if all of the following conditions are met:
(1)The content is posted during the applicable time period or periods set forth in subdivision (c).(2)The materially deceptive content is any of the following:(1)The content is reported pursuant to subdivision (a) of Section 20515.
(2)The materially deceptive content is any of the following:
(A)A candidate for elective office portrayed as doing or saying something that the candidate did not do or say and that is reasonably likely to harm the reputation or electoral prospects of a candidate.
(B)An elections official portrayed as doing or saying something in connection with the performance of their elections-related duties that the elections official did not do or say and that is reasonably likely to falsely undermine confidence in the outcome of one or more election contests.
(C)An elected official portrayed as doing or saying something that influences an election in California that the elected official did not do or say and that is reasonably likely to falsely undermine confidence in the outcome of one or more election contests.
(3)The content is posted during the applicable time period or periods set forth in subdivision (e).
(4)The large online platform knows or acts with reckless disregard for the fact that the content meets the requirements of this section.
(b)If a post is determined to meet the requirements for removal pursuant to subdivision (a), any large online platform shall remove the post upon that determination, but no later than 72 hours after a report is made pursuant to subdivision (a) of Section 20515 in order to be in compliance with this chapter.
(c)Any large online platform shall identify, using state-of-the-art techniques, and remove, upon discovering or being alerted to the posting or reposting of, any identical or substantially similar materially deceptive content that the platform had previously removed pursuant to this chapter, provided that this removal occurs during the applicable time period or periods set forth in subdivision (e).
(b)(d)(1)Notwithstanding subparagraph (A) of paragraph (1) (2) of subdivision (a), this section shall does not apply to a candidate for elective office who, during the time period set forth in subdivision (c), (e), portrays themself as doing or saying something that the candidate did not do or say, if the digital content includes a disclosure stating the following: This _____ has been manipulated. The blank in this disclosure shall be filled in with whichever of the following terms most accurately describes the media:
(A)Image.
(B)Audio.
(C)Video.
(2)(A)For visual media, the text of the disclosure shall appear in a size that is easily readable by the average viewer and no smaller than the largest font size of other text appearing in the visual media. If the visual media does not include any other text, the disclosure shall appear in a size that is easily readable by the average viewer. For visual media that is video, the disclosure shall appear for the duration of the video.
(B)If the media consists of audio only, the disclosure shall be read in a clearly spoken manner and in a pitch that can be easily heard by the average listener, at the beginning of the audio, at the end of the audio, and, if the audio is greater than two minutes in length, interspersed within the audio at intervals of not greater than two minutes each.
(c)(e)(1)Except as provided in paragraph (2), any large online platform shall block and prevent remove the content described in subdivision (a), to the extent required by subdivisions (a) to (c), inclusive, and any candidate for elective office shall include the disclosure required by subdivision (b), (d), during a period beginning 120 days before an election in California and through the day of the election.
(2)If the content described in subdivision (a) depicts or pertains to elections officials, any large online platform shall block and prevent remove the content to the extent required by subdivisions (a) to (c), inclusive during a period beginning 120 days before an election in California and ending on the 60th day after the election.
20514.(a)Any large online platform, using state-of-the-art, best-available tools to detect materially deceptive content platform shall develop and implement procedures procedures for the use of state-of-the-art techniques to identify materially deceptive content and for labeling such content as inauthentic, fake, or false provided in subdivision (c) if both all of the following conditions are met:
(1)The content is reported pursuant to subdivision (a) of Section 20515.
(1)(2)The materially deceptive content is either of the following:
(A)Included within subdivision (a) of Section 20513, but is posted outside the applicable time period described in subdivision (c) (e) of Section 20513.
(B)Appears within an advertisement or election communication and is not subject to Section 20513.
(2)(3)The large online platform knows or acts with reckless disregard for the fact that the materially deceptive content meets the requirements of this section.
(b)If a post is determined to meet the requirements for labeling pursuant to subdivision (a), any large online platform shall label the post upon that determination, but no later than 72 hours after a report is made pursuant to subdivision (a) of Section 20515 in order to be in compliance with this chapter.
(c)The label required by subdivision (a) shall state: This _____ has been manipulated and is not authentic. The blank in this disclosure shall be filled in with whichever of the following terms most accurately describes the media:
(1)Image.
(2)Audio.
(3)Video.
(b)(d)The label required by subdivision (a) shall permit users to click or tap on it and to inspect all available provenance data for additional explanation about the materially deceptive content in an easy-to-understand format.
(c)(e)The labeling requirement set forth in subdivision (a) applies during any of the following time periods, to the extent applicable:
(1)The period beginning six months before an election in California and through the day of the election.
(2)The period beginning six months before an election in California and ending on the 60th day after the election, if the content depicts or pertains to elections officials, the electoral college process, a voting machine, ballot, voting site, or other property or equipment related to an election, or the canvass of the vote.
20515.(a)A large online platform shall provide an easily accessible way for California residents to report to that platform content subject that should be removed pursuant to Section 20513 or 20514 that was not blocked or labeled as required. labeled pursuant to Section 20514. The large online platform shall respond to the person who made the report within 36 hours of the report, describing any action taken or not taken by the large online platform with respect to the content.
(b)A candidate for elective office, elected official, or elections official who has made a report to a large online platform under subdivision (a) and who either has not received a response within 36 hours or disagrees with the response, action taken, or failure by the large online platform to take action within 72 hours, may seek injunctive or other equitable relief against the large online platform to compel compliance with this chapter. the removal of specific content as required by Section 20513, labeling of specific content as required by Section 20514, or compliance with the reporting process required by subdivision (a). The plaintiff shall bear the burden of establishing the violation through clear and convincing evidence. An action under this subdivision shall be entitled to precedence in accordance with Section 35 of the Code of Civil Procedure.
20516.The Attorney General or any district attorney or city attorney may seek injunctive or other equitable relief against any large online platform to compel compliance with this chapter. the removal of specific content as required by Section 20513, labeling of specific content as required by Section 20514, or compliance with the reporting process required by subdivision (a) of Section 20515. The plaintiff shall bear the burden of establishing the violation through clear and convincing evidence. An action under this section shall be entitled to precedence in accordance with Section 35 of the Code of Civil Procedure.
20517.This chapter applies to materially deceptive content, regardless of the language used in the content. If the language used is not English, the disclosure required by subdivision (b) (d) of Section 20513 and the label required by Section 20514 must appear in the language used as well as in English.
20518.(a)This chapter does not preclude a large online platform from blocking blocking, removing, or labeling any materially deceptive content outside of the time periods specified in Sections 20513 and 20514.
(b)This chapter does not preclude any online platform not subject to this chapter from blocking blocking, removing, or labeling any materially deceptive content.
20519.This chapter does not apply to either any of the following:
(a)A regularly published online newspaper, magazine, or other periodical of general circulation that routinely carries news and commentary of general interest, and that publishes any materially deceptive content that an online platform is required to block or label based on this chapter, if the publication contains a clear disclosure that the materially deceptive content does not accurately represent any actual event, occurrence, appearance, speech, or expressive conduct.
(b)(1)A broadcasting station that broadcasts any materially deceptive content prohibited by this chapter as part of a bona fide newscast, news interview, news documentary, commentary of general interest, or on-the-spot coverage of bona fide news events, if the broadcast clearly acknowledges through content or a disclosure, in a manner that can be easily heard or read by the average listener or viewer, that the materially deceptive content does not accurately represent any actual event, occurrence, appearance, speech, or expressive conduct.
(2)A broadcasting station when it is paid to broadcast materially deceptive content and either of the following circumstances exist:
(A)The broadcasting station can show that it has prohibition and disclaimer requirements that are consistent with the requirements in this chapter and that it has provided those prohibition and disclaimer requirements to each person or entity that purchased the advertisement.
(B)Federal law requires the broadcasting station to air advertisements from legally qualified candidates or prohibits the broadcasting station from censoring or altering the message.
(b)(c)Materially deceptive content that constitutes satire or parody.
20520.The provisions of this chapter are severable. If any provision of this chapter or its application is held invalid, that invalidity shall not affect other provisions or applications that can be given effect without the invalid provision or application.
SEC. 4.Section 1.5 of this bill incorporates amendments to Section 35 of the Code of Civil Procedure that are proposed by this bill and Assembly Bill 2839. Section 1.5 of this bill shall only become operative if (1) both bills are enacted and become effective on or before January 1, 2025, (2) each bill amends Section 35 of the Code of Civil Procedure, as amended by Section 1 of Chapter 343 of the Statutes of 2023, and (3) this bill is enacted after Assembly Bill 2839, in which case Section 1 of this bill shall not become operative. Additionally, if the above conditions are met and Assembly Bill 2839 becomes operative before January 1, 2025, Section 35 of the Code of Civil Procedure, as amended by Assembly Bill 2839, shall remain operative only until January 1, 2025, at which time Section 1.5 of this bill shall become operative, and Section 1 of this bill shall not become operative.
SEC. 5.Section 2.5 of this bill incorporates amendments to Section 35 of the Code of Civil Procedure that are proposed by this bill and Assembly Bill 2839. Section 2.5 of this bill shall only become operative if (1) both bills are enacted and become effective on or before January 1, 2025, (2) each bill amends Section 35 of the Code of Civil Procedure, as amended by Section 2 of Chapter 343 of the Statutes of 2023, and (3) this bill is enacted after Assembly Bill 2839, in which case Section 2 of this bill shall not become operative. Additionally, if the above conditions are met and Assembly Bill 2839 becomes operative before January 1, 2025, Section 35 of the Code of Civil Procedure, as amended by Assembly Bill 2839, shall remain operative only until January 1, 2025, at which time Section 2.5 of this bill shall become operative, and Section 2 of this bill shall not become operative.
(a)Proceedings in cases involving the registration or denial of registration of voters, the certification or denial of certification of candidates, the certification or denial of certification of ballot measures, election contests, actions under Section 20010 of the Elections Code, actions under Chapter 7 (commencing with Section 20510) of Division 20 of the Elections Code, and actions under Chapter 2 (commencing with Section 21100) of Division 21 of the Elections Code shall be placed on the calendar in the order of their date of filing and shall be given precedence.
(b)This section shall remain in effect only until January 1, 2027, and as of that date is repealed, unless a later enacted statute, that is enacted before January 1, 2027, deletes or extends that date.
SEC. 2.Section 35 of the Code of Civil Procedure, as amended by Section 2 of Chapter 343 of the Statutes of 2023, is amended to read:35.(a)Proceedings in cases involving the registration or denial of registration of voters, the certification or denial of certification of candidates, the certification or denial of certification of ballot measures, election contests, actions under Chapter 7 (commencing with Section 20510) of Division 20 of the Elections Code, and actions under Chapter 2 (commencing with Section 21100) of Division 21 of the Elections Code shall be placed on the calendar in the order of their date of filing and shall be given precedence.
(b)This section shall become operative January 1, 2027.
SEC. 3.Chapter 7 (commencing with Section 20510) is added to Division 20 of the Elections Code, to read:7.Defending Democracy from Deepfake Deception Act of 202420510.This chapter shall be known and may be cited as the Defending Democracy from Deepfake Deception Act of 2024.
20511.The Legislature finds and declares all of the following:
(a)California is entering its first-ever generative artificial intelligence (AI) election, in which disinformation powered by generative AI will pollute our information ecosystems like never before. Voters will not know what images, audio, or video they can trust.
(b)In a few clicks, using current technology, bad actors now have the power to create a false image of a candidate accepting a bribe or a fake video of an elections official caught on tape saying that voting machines are not secure, or to generate the Governors voice telling millions of Californians their voting site has changed.
(c)In the lead-up to the 2024 presidential elections, candidates and parties are already creating and distributing deepfake images and audio and video content. These fake images or files can spread to millions of Californians in seconds and skew election results or undermine trust in the ballot counting process.
(d)The labeling information required by this bill is narrowly tailored to provide consumers with factual information about the inauthenticity of particular images, audio, video, or text content in order to prevent consumer deception.
(e)In order to ensure California elections are free and fair, California must, for a limited time before and after elections, prevent the use of deepfakes and disinformation meant to prevent voters from voting and to deceive voters based on fraudulent content. Accordingly, the provisions of this chapter are narrowly tailored to support California's compelling interest in protecting its free and fair elections.
20512.For purposes of this chapter, the following terms have the following meanings:
(a)Advertisement means any general or public communication that is authorized or paid for with the purpose of supporting or opposing a candidate for elective office.
(b)Candidate means any person running for federal, state, or local office in California. The term includes persons running for the office of President or Vice President of the United States who will seek to or will appear on a ballot issued in California.
(b)(c)Deepfake means audio or visual media that is digitally created or modified such that it would falsely appear to a reasonable person to be an authentic record of the actual speech or conduct of the individual depicted in the media.
(c)(d)Election communication means a general or public communication that is not an advertisement and that concerns any of the following:
(1)A candidate for elective office.
(2)Voting or refraining from voting in an election. election in California.
(3)The canvass of the vote. vote in California.
(4)Voting machines, ballots, voting sites, or other property or equipment related to an election. election in California.
(5)Proceedings or processes of redistricting or the electoral college. college in California.
(d)(e)(1)Elections official means any of the following persons, but only in their capacity as a person charged with holding or conducting an election, election in California, conducting a canvass, canvass in California, assisting with the holding or conducting of an election or a canvas, canvass in California, or performing another duty related to administering the provisions of this code:
(A)An elections official as defined in Section 320.
(B)The Secretary of State and their staff.
(C)A temporary worker, poll worker, or member of a precinct board.
(D)Any other person charged with holding or conducting an election, conducting a canvass, assisting with the holding or conducting of an election or a canvas, or performing another duty related to administering the provisions of this code.
(2)The requirements of this chapter relating to content portraying an elections official apply only if the large online platform knows or should know that the person is an elections official.
(f)Large online platform means a public-facing internet website, web application, or digital application, including a social media platform as defined in Section 22675 of the Business and Professions Code, video sharing platform, advertising network, or search engine that had at least 1,000,000 California users during the preceding 12 months.
(e)(g)(1)Materially deceptive content means audio or visual media that is digitally created or modified, and that includes, but is not limited to, deepfakes and chatbots, such that it would falsely appear to a reasonable person to be an authentic record of the content depicted in the media.
(2)Materially deceptive content does not include any audio or visual media that contains only minor modifications that do not significantly change the perceived contents or meaning of the content. Minor changes include changes to the brightness or contrast of images, removal of background noise in audio, and other minor changes that do not impact the content of the image or audio or visual media.
(f)Large online platform means a public-facing internet website, web application, or digital application, including a social media platform as defined in Section 22675 of the Business and Professions Code, video sharing platform, advertising network, or search engine that had at least 1,000,000 California users during the preceding 12 months.20513.(a)Any large online platform, using state-of-the-art, best available best-available tools to detect materially deceptive content, shall develop and implement procedures for blocking and preventing, and shall, if the large online platform knows or acts with reckless disregard for the fact that the materially deceptive content meets the requirements of this section, block and prevent the posting of any materially deceptive content, if both of the following conditions are met:
(1)The content is posted during the applicable time period or periods set forth in subdivision (c).
(2)The materially deceptive content is any of the following:
(A)A candidate for elective office portrayed as doing or saying something that the candidate did not do or say and that is reasonably likely to harm the reputation or electoral prospects of a candidate.
(B)An elections official portrayed as doing or saying something in connection with the performance of their elections-related duties that the elections official did not do or say and that is reasonably likely to falsely undermine confidence in the outcome of one or more election contests.
(C)An elected official portrayed as doing or saying something that influences the an election in California that the elected official did not do or say and that is reasonably likely to falsely undermine confidence in the outcome of one or more election contests.
(b)(1)Notwithstanding paragraph (1) of subdivision (a), this section shall not apply to a candidate for elective office who, during the time period set forth in subdivision (c), portrays themself as doing or saying something that the candidate did not do or say, if the digital content includes a disclosure stating the following: This _____ has been manipulated. The blank in this disclosure shall be filled in with whichever of the following terms most accurately describes the media:
(A)Image.
(B)Audio.
(C)Video.
(2)(A)For visual media, the text of the disclosure shall appear in a size that is easily readable by the average viewer and no smaller than the largest font size of other text appearing in the visual media. If the visual media does not include any other text, the disclosure shall appear in a size that is easily readable by the average viewer. For visual media that is video, the disclosure shall appear for the duration of the video.
(B)If the media consists of audio only, the disclosure shall be read in a clearly spoken manner and in a pitch that can be easily heard by the average listener, at the beginning of the audio, at the end of the audio, and, if the audio is greater than two minutes in length, interspersed within the audio at intervals of not greater than two minutes each.
(c)(1)Except as provided in paragraph (2), any large online platform shall block and prevent the content described in subdivision (a), and any candidate for elective office shall include the disclosure required by subdivision (b), during a period beginning 120 days before the an election in California and through the day of the election.
(2)If the content described in subdivision (a) depicts or pertains to elections officials, any large online platform shall block and prevent the content during a period beginning 120 days before the an election in California and ending on the 60th day after the election.
20514.(a)Any large online platform, using state-of-the-art, best available best-available tools to detect materially deceptive content shall develop and implement procedures for labeling such content as inauthentic, fake, or false if both of the following conditions are met:
(1)The materially deceptive content is either of the following:
(A)Included within subdivision (a) of Section 20513, but is posted outside the applicable time period described in subdivision (c) of Section 20513.
(B)Appears within an advertisement or election communication and is not subject to Section 20513.
(2)The large online platform knows or acts with reckless disregard for the fact that the materially deceptive content meets the requirements of this section.
(b)The label required by subdivision (a) shall permit users to click or tap on it and to inspect all available provenance data about the materially deceptive content in an easy-to-understand format.
(c)The labeling requirement set forth in subdivision (a) applies during any of the following time periods, to the extent applicable:
(1)The period beginning one year six months before the an election in California and through the day of the election.
(2)The period beginning one year six months before the an election in California and ending on the 60th day after the election, if the content depicts or pertains to elections officials, the electoral college process, a voting machine, ballot, voting site, or other property or equipment related to an election, or the canvass of the vote.
(3)During a governmental process related to redistricting that is required or authorized by state or local law.20515.(a)A large online platform shall provide an easily accessible way for California residents to report to that platform content subject to Section 20513 or 20514 that was not blocked or labeled as required. The online platform shall respond to the person who made the report, report within 36 hours of the report, describing any action taken or not taken by the online platform with respect to the content.
(b)A candidate for elective office, elected official, or elections official who has made a report to a large online platform under subdivision (a) and who either has not received a response within 36 hours or disagrees with the response, may seek injunctive or other equitable relief against the online platform to compel compliance with this chapter. The plaintiff shall bear the burden of establishing the violation through clear and convincing evidence. An action under this subdivision shall be entitled to precedence in accordance with Section 35 of the Code of Civil Procedure.
20516.The Attorney General or any district attorney or city attorney may seek injunctive or other equitable relief against any large online platform to compel compliance with this chapter. The plaintiff shall bear the burden of establishing the violation through clear and convincing evidence. An action under this section shall be entitled to precedence in accordance with Section 35 of the Code of Civil Procedure.
20517.This chapter applies to materially deceptive content, regardless of the language used in the content. If the language used is not English, the disclosure required by subdivision (b) of Section 20513 and the label required by Section 20514 must appear in the language used as well as in English.
20518.(a)This chapter does not preclude a large online platform from blocking or labeling any materially deceptive content outside of the time periods specified in Sections 20513 and 20514.
(b)This chapter does not preclude any online platform not subject to this chapter from blocking or labeling any materially deceptive content.
20519.This chapter does not apply to either of the following:
(a)A regularly published online newspaper, magazine, or other periodical of general circulation that routinely carries news and commentary of general interest, and that publishes any materially deceptive content that an online platform is required to block or label based on this chapter, if the publication contains a clear disclosure that the materially deceptive content does not accurately represent any actual event, occurrence, appearance, speech, or expressive conduct.
(b)Materially deceptive content that constitutes satire or parody.
20520.The provisions of this chapter are severable. If any provision of this chapter or its application is held invalid, that invalidity shall not affect other provisions or applications that can be given effect without the invalid provision or application.
(a)Proceedings in cases involving the registration or denial of registration of voters, the certification or denial of certification of candidates, the certification or denial of certification of ballot measures, election contests, actions under Section 20010 of the Elections Code, actions under Chapter 7 (commencing with Section 20510) of Division 20 of the Elections Code, and actions under Chapter 2 (commencing with Section 21100) of Division 21 of the Elections Code shall be placed on the calendar in the order of their date of filing and shall be given precedence.
(b)This section shall remain in effect only until January 1, 2027, and as of that date is repealed, unless a later enacted statute, that is enacted before January 1, 2027, deletes or extends that date.
SEC. 2.Section 35 of the Code of Civil Procedure, as amended by Section 2 of Chapter 343 of the Statutes of 2023, is amended to read:35.(a)Proceedings in cases involving the registration or denial of registration of voters, the certification or denial of certification of candidates, the certification or denial of certification of ballot measures, election contests, actions under Chapter 7 (commencing with Section 20510) of Division 20 of the Elections Code, and actions under Chapter 2 (commencing with Section 21100) of Division 21 of the Elections Code shall be placed on the calendar in the order of their date of filing and shall be given precedence.
(b)This section shall become operative January 1, 2027.
SEC. 3.Chapter 7 (commencing with Section 20510) is added to Division 20 of the Elections Code, to read:7.Defending Democracy from Deepfake Deception Act of 202420510.This chapter shall be known and may be cited as the Defending Democracy from Deepfake Deception Act of 2024.
20511.The Legislature finds and declares all of the following:
(a)California is entering its first-ever generative artificial intelligence (AI) election, in which disinformation powered by generative AI will pollute our information ecosystems like never before. Voters will not know what images, audio, or video they can trust.
(b)In a few clicks, using current technology, bad actors now have the power to create a false image of a candidate accepting a bribe or a fake video of an elections official caught on tape saying that voting machines are not secure, or to generate the Governors voice telling millions of Californians their voting site has changed.
(c)In the lead-up to the 2024 presidential elections, candidates and parties are already creating and distributing deepfake images and audio and video content. These fake images or files can spread to millions of Californians in seconds and skew election results or undermine trust in the ballot counting process.
(d)The labeling information required by this bill is narrowly tailored to provide consumers with factual information about the inauthenticity of particular images, audio, video, or text content in order to prevent consumer deception.
(e)In order to ensure California elections are free and fair, California must, for a limited time before and after elections, prevent the use of deepfakes and disinformation meant to prevent voters from voting and to deceive voters based on fraudulent content. Accordingly, the provisions of this chapter are narrowly tailored to support California's compelling interest in protecting its free and fair elections.
20512.For purposes of this chapter, the following terms have the following meanings:
(a)Advertisement means any general or public communication that is authorized or paid for with the purpose of supporting or opposing a candidate for elective office.
(b)Deepfake means audio or visual media that is digitally created or modified such that it would falsely appear to a reasonable person to be an authentic record of the actual speech or conduct of the individual depicted in the media.
(c)Election communication means a general or public communication that is not an advertisement and that concerns any of the following:
(1)A candidate for elective office.
(2)Voting or refraining from voting in an election.
(3)The canvass of the vote.
(4)Voting machines, ballots, voting sites, or other property or equipment related to an election.
(5)Proceedings or processes of redistricting or the electoral college.
(d)(1)Elections official means any of the following persons, but only in their capacity as a person charged with holding or conducting an election, conducting a canvass, assisting with the holding or conducting of an election or a canvas, or performing another duty related to administering the provisions of this code:
(A)An elections official as defined in Section 320.
(B)The Secretary of State and their staff.
(C)A temporary worker, poll worker, or member of a precinct board.
(D)Any other person charged with holding or conducting an election, conducting a canvass, assisting with the holding or conducting of an election or a canvas, or performing another duty related to administering the provisions of this code.
(2)The requirements of this chapter relating to content portraying an elections official apply only if the large online platform knows or should know that the person is an elections official.
(e)(1)Materially deceptive content means audio or visual media that is digitally created or modified, and that includes, but is not limited to, deepfakes and chatbots, such that it would falsely appear to a reasonable person to be an authentic record of the content depicted in the media.
(2)Materially deceptive content does not include any audio or visual media that contains only minor modifications that do not significantly change the perceived contents or meaning of the content. Minor changes include changes to the brightness or contrast of images, removal of background noise in audio, and other minor changes that do not impact the content of the image or audio or visual media.
(f)Large online platform means a public-facing internet website, web application, or digital application, including a social network, media platform as defined in Section 22675 of the Business and Professions Code, video sharing platform, advertising network, or search engine that had at least 1,000,000 California users during the preceding 12 months.
20513.(a)Any large online platform, using state-of-the-art, best available tools to detect materially deceptive content, shall develop and implement procedures for blocking and preventing, and shall, if the large online platform knows or should know or acts with reckless disregard for the fact that the materially deceptive content meets the requirements of this section, section, block and prevent, prevent the posting or sending of any materially deceptive content, if all both of the following conditions are met:
(1)The content is posted or sent during the applicable time period or periods set forth in subdivision (c).
(2)The materially deceptive content is any of the following:
(A)A candidate for elective office portrayed as doing or saying something that the candidate did not do or say and that is reasonably likely to harm the reputation or electoral prospects of a candidate.
(B)An elections official portrayed as doing or saying something in connection with the performance of their elections-related duties that the elections official did not do or say and that is reasonably likely to falsely undermine confidence in the outcome of one or more election contests.
(C)An elected official portrayed as doing or saying something that influences the election that the elected official did not do or say and that is reasonably likely to falsely undermine confidence in the outcome of one or more election contests.
(3)The person or entity who created the materially deceptive content did so knowing it was false or with reckless disregard for the truth. There shall be a rebuttable presumption that the person or entity knew the materially deceptive content was false or acted with reckless disregard for the truth if the content would cause a reasonable person to have a fundamentally different understanding or impression of the content than the person would have if hearing or seeing an authentic version of the content.(b)(1)Notwithstanding paragraph (1) of subdivision (a), a large online platform shall not prevent this section shall not apply to a candidate for elective office, office who, during the time period set forth in subdivision (c), from portraying (c), portrays themself as doing or saying something that the candidate did not do or say, but only if the digital content includes a disclosure stating the following: This _____ has been manipulated. The blank in this disclosure shall be filled in with whichever of the following terms most accurately describes the media:
(A)Image.
(B)Audio.
(C)Video.
(2)(A)For visual media, the text of the disclosure shall appear in a size that is easily readable by the average viewer and no smaller than the largest font size of other text appearing in the visual media. If the visual media does not include any other text, the disclosure shall appear in a size that is easily readable by the average viewer. For visual media that is video, the disclosure shall appear for the duration of the video.
(B)If the media consists of audio only, the disclosure shall be read in a clearly spoken manner and in a pitch that can be easily heard by the average listener, at the beginning of the audio, at the end of the audio, and, if the audio is greater than two minutes in length, interspersed within the audio at intervals of not greater than two minutes each.
(c)(1)Except as provided in paragraph (2), any large online platform shall block and prevent the content described in subdivision (a), and any candidate for elective office shall include the disclosure required by subdivision (b), during a period beginning 120 days before the election and through the day of the election.
(2)If the content described in subdivision (a) depicts or pertains to elections officials, any large online platform shall block and prevent the content during a period beginning 120 days before the election and ending on the 60th day after the election.
20514.(a)(1)Any large online platform, using state-of-the-art, best available tools to detect materially deceptive content shall develop and implement procedures for labeling such content as inauthentic, fake, or false if all both of the following conditions are met:
(A)(1)The materially deceptive content is either of the following:
(i)(A)Included within subdivision (a) of Section 20513, but is posted or sent outside the applicable time period described in subdivision (c) of Section 20513.
(ii)(B)Appears within an advertisement or election communication and is not subject to Section 20513.
(B)The person or entity who created the materially deceptive content did so knowing it was false or with reckless disregard for the truth. There shall be a rebuttable presumption that the person or entity knew the materially deceptive content was false or acted with reckless disregard for the truth if the content would cause a reasonable person to have a fundamentally different understanding or impression of the content than the person would have if hearing or seeing an authentic version of the content.(C)(2)The large online platform knows or should know or acts with reckless disregard for the fact that the materially deceptive content meets the requirements of this section.
(b)The label required by subdivision (a) shall permit users to click or tap on it and to inspect all available provenance data about the materially deceptive content in an easy-to-understand format.
(c)The labeling requirement set forth in subdivision (a) applies during any of the following time periods, to the extent applicable:
(1)The period beginning one year before the election and through the day of the election.
(2)The period beginning one year before the election and ending on the 60th day after the election, if the content depicts or pertains to elections officials, the electoral college process, a voting machine, ballot, voting site, or other property or equipment related to an election, or the canvass of the vote.
(3)During a governmental process related to redistricting that is required or authorized by state or local law.
20515.(a)A large online platform shall provide an easily accessible way for California residents to report to that platform content subject to Section 20513 or 20514 that was not blocked or labeled as required. The online platform shall respond to the person who made the report, within 36 hours of the report, describing any action taken or not taken by the online platform with respect to the content.
(b)A candidate for elective office, elected official, or elections official who has made a report to a large online platform under subdivision (a) and who either has not received a response within 36 hours or disagrees with the response, may seek injunctive or other equitable relief against the online platform to compel compliance with this chapter. The plaintiff shall bear the burden of establishing the violation through clear and convincing evidence. The court shall award a prevailing plaintiff reasonable attorneys fees and costs. An action under this subdivision shall be entitled to precedence in accordance with Section 35 of the Code of Civil Procedure.
20516.The Attorney General or any district attorney or city attorney may seek injunctive or other equitable relief against any large online platform to compel compliance with this chapter. The plaintiff shall bear the burden of establishing the violation through clear and convincing evidence. The court shall award a prevailing plaintiff reasonable attorneys fees and costs. An action under this section shall be entitled to precedence in accordance with Section 35 of the Code of Civil Procedure.
20517.This chapter applies to materially deceptive content, regardless of the language used in the content. If the language used is not English, the disclosure required by subdivision (b) of Section 20513 and the label required by Section 20514 must appear in the language used as well as in English.
20518.A large online platform that blocks or labels any materially deceptive content shall maintain a copy of the digital content for a period of not less than five years from the election and shall make such digital content available to the Secretary of State, the Fair Political Practices Commission, and researchers, if requested.20519.20518.(a)This chapter does not preclude a large online platform from blocking or labeling any materially deceptive content outside of the time periods specified in Sections 20513 and 20514.
(b)This chapter does not preclude any online platform not subject to this chapter from blocking or labeling any materially deceptive content.
20520.20519.This chapter does not apply to either of the following:
(a)A regularly published online newspaper, magazine, or other periodical of general circulation that routinely carries news and commentary of general interest, and that publishes any materially deceptive content that an online platform is required to block or label based on this chapter, if the publication contains a clear disclosure that the materially deceptive content does not accurately represent any actual event, occurrence, appearance, speech, or expressive conduct.
(b)Materially deceptive content that constitutes satire or parody.
20521.20520.The provisions of this chapter are severable. If any provision of this chapter or its application is held invalid, that invalidity shall not affect other provisions or applications that can be given effect without the invalid provision or application.
(a)Proceedings in cases involving the registration or denial of registration of voters, the certification or denial of certification of candidates, the certification or denial of certification of ballot measures, election contests, actions under Section 20010 of the Elections Code, actions under Chapter 7 (commencing with Section 20510) of Division 20 of the Elections Code, and actions under Chapter 2 (commencing with Section 21100) of Division 21 of the Elections Code shall be placed on the calendar in the order of their date of filing and shall be given precedence.
(b)This section shall remain in effect only until January 1, 2027, and as of that date is repealed, unless a later enacted statute, that is enacted before January 1, 2027, deletes or extends that date.
SEC. 2.Section 35 of the Code of Civil Procedure, as amended by Section 2 of Chapter 343 of the Statutes of 2023, is amended to read:35.(a)Proceedings in cases involving the registration or denial of registration of voters, the certification or denial of certification of candidates, the certification or denial of certification of ballot measures, election contests, actions under Chapter 7 (commencing with Section 20510) of Division 20 of the Elections Code, and actions under Chapter 2 (commencing with Section 21100) of Division 21 of the Elections Code shall be placed on the calendar in the order of their date of filing and shall be given precedence.
(b)This section shall become operative January 1, 2027.
SEC. 3.Chapter 7 (commencing with Section 20510) is added to Division 20 of the Elections Code, to read:7.Defending Democracy from Deepfake Deception Act of 202420510.This chapter shall be known and may be cited as the Defending Democracy from Deepfake Deception Act of 2024.
20511.The Legislature finds and declares all of the following:
(a)California is entering its first-ever generative artificial intelligence (AI) election, in which disinformation powered by generative AI will pollute our information ecosystems like never before. Voters will not know what images, audio, or video they can trust.
(b)In a few clicks, using current technology, bad actors now have the power to create a false image of a candidate accepting a bribe or a fake video of an elections official caught on tape saying that voting machines are not secure, or to generate the Governors voice telling millions of Californians their voting site has changed.
(c)In the lead-up to the 2024 presidential elections, candidates and parties are already creating and distributing deepfake images and audio and video content. These fake images or files can spread to millions of Californians in seconds and skew election results or undermine trust in the ballot counting process.
(d)The labeling information required by this bill is narrowly tailored to provide consumers with factual information about the inauthenticity of particular images, audio, video, or text content in order to prevent consumer deception.
(e)In order to ensure California elections are free and fair, California must, for a limited time before and after elections, prevent the use of deepfakes and disinformation meant to prevent voters from voting and to deceive voters based on fraudulent content. Accordingly, the provisions of this chapter are narrowly tailored to support California's compelling interest in protecting its free and fair elections.
20512.For purposes of this chapter, the following terms have the following meanings:
(a)Advertisement means any general or public communication that is authorized or paid for with the purpose of supporting or opposing a candidate for elective office.
(b)Artificial intelligence means an engineered or machine-based system that varies in its level of autonomy and that can, for explicit or implicit objectives, infer from the input it receives how to generate outputs that can influence physical or virtual environments.(b)Deepfake means audio or visual media that is digitally created or modified such that it would falsely appear to a reasonable person to be an authentic record of the actual speech or conduct of the individual depicted in the media.
(c)Election communication means a general or public communication that is not an advertisement and that concerns any of the following:
(1)A candidate for elective office.
(2)Voting or refraining from voting in an election.
(3)The canvass of the vote.
(4)Voting machines, ballots, voting sites, or other property or equipment related to an election.
(5)Proceedings or processes of redistricting or the electoral college.
(d)(1)Elections official means any of the following persons, but only in their capacity as a person charged with holding or conducting an election, conducting a canvass, assisting with the holding or conducting of an election or a canvas, or performing another duty related to administering the provisions of this code:
(A)An elections official as defined in Section 320.
(B)The Secretary of State and their staff.
(C)A temporary worker, poll worker, or member of a precinct board.
(D)Any other person charged with holding or conducting an election, conducting a canvass, assisting with the holding or conducting of an election or a canvas, or performing another duty related to administering the provisions of this code.
(2)The requirements of this chapter relating to content portraying an elections official apply only if the large online platform knows or should know that the person is an elections official.
(e)(1)Materially deceptive and digitally modified or created content means an image or an audio or video recording or other digital content, including a chatbot, that has been intentionally manipulated such that all of the following conditions are met:(A)(i)The digital content is the product of digital manipulation, including, but not limited to, artificial intelligence, such that it so closely resembles authentic content that a reasonable person would not be able to readily distinguish the digitally manipulated or created content from the authentic content.(ii)For purposes of this subdivision, false portrayal means the content would cause a reasonable person to have a fundamentally different understanding or impression of the content than the person would have if they were hearing or seeing an authentic version of the content.(B)The person or entity who attempted to post or send, or who did post or send, the content did so knowing the portrayal was false, or did so with reckless disregard for whether the portrayal was false. If the content is intentionally manipulated and contains a false portrayal as specified in subparagraph (A), there shall be a rebuttable presumption that the person or entity knew the portrayal was false or that they acted with reckless disregard for whether the portrayal was false.(e)(1)Materially deceptive content means audio or visual media that is digitally created or modified, and that includes, but is not limited to, deepfakes and chatbots, such that it would falsely appear to a reasonable person to be an authentic record of the content depicted in the media.
(2)Materially deceptive and digitally modified or created content does not include any image or audio or video recording visual media that contains only minor modifications that do not lead to significant changes to significantly change the perceived contents or meaning of the content. Minor changes include changes to the brightness or contrast of images, removal of background noise in audio, and other minor changes that do not impact the content of the image or audio or video recording. visual media.
(f)Large online platform means a public-facing internet website, web application, or digital application, including a social network, video sharing platform, advertising network, or search engine that had at least 1,000,000 California users during the preceding 12 months.
20513.(a)Any large online platform, using state-of-the-art, best available tools to detect digitally modified or created materially deceptive content, shall develop and implement procedures for blocking and preventing, and shall, if the large online platform knows or should know that the digitally modified or created materially deceptive content meets the requirements of this section, block and prevent, the posting or sending of any materially deceptive and digitally modified or created content, during the applicable time period or periods set forth in subdivision (c), of any of the following: content, if all of the following conditions are met:
(1)The content is posted or sent during the applicable time period or periods set forth in subdivision (c).
(2)The materially deceptive content is any of the following:
(1)
(A)A candidate for elective office portrayed as doing or saying something that the candidate did not do or say. say and that is reasonably likely to harm the reputation or electoral prospects of a candidate.
(2)
(B)An elections official portrayed as doing or saying something in connection with the performance of their elections-related duties that the elections official did not do or say. say and that is reasonably likely to falsely undermine confidence in the outcome of one or more election contests.
(3)
(C)An elected official portrayed as doing or saying something that influences the election that the elected official did not do or say. say and that is reasonably likely to falsely undermine confidence in the outcome of one or more election contests.
(4)A voting machine, ballot, voting site, or other property or equipment related to an election that is portrayed in a materially false way.(3)The person or entity who created the materially deceptive content did so knowing it was false or with reckless disregard for the truth. There shall be a rebuttable presumption that the person or entity knew the materially deceptive content was false or acted with reckless disregard for the truth if the content would cause a reasonable person to have a fundamentally different understanding or impression of the content than the person would have if hearing or seeing an authentic version of the content.
(b)(1)Notwithstanding paragraph (1) of subdivision (a), a large online platform shall not prevent a candidate for elective office, during the time period set forth in subdivision (c), from portraying themself as doing or saying something that the candidate did not do or say, but only if the digital content includes a disclosure stating the following: This _____ has been manipulated. The blank in this disclosure shall be filled in with whichever of the following terms most accurately describes the media:
(A)Image.
(B)Audio.
(C)Video.
(2)(A)For visual media, the text of the disclosure shall appear in a size that is easily readable by the average viewer and no smaller than the largest font size of other text appearing in the visual media. If the visual media does not include any other text, the disclosure shall appear in a size that is easily readable by the average viewer. For visual media that is video, the disclosure shall appear for the duration of the video.
(B)If the media consists of audio only, the disclosure shall be read in a clearly spoken manner and in a pitch that can be easily heard by the average listener, at the beginning of the audio, at the end of the audio, and, if the audio is greater than two minutes in length, interspersed within the audio at intervals of not greater than two minutes each.
(c)(1)Except as provided in paragraph (2), any large online platform shall block and prevent the content described in subdivision (a), and any candidate for elective office shall include the disclosure required by subdivision (b), during a period beginning 120 days before the election and through the day of the election.
(2)If the content described in subdivision (a) depicts or pertains to elections officials, or depicts or pertains to a voting machine, ballot, voting site, or other property or equipment related to an election, a any large online platform shall block and prevent the content during a period beginning 120 days before the election and ending on the 60th day after the election.
20514.(a)(1)This section applies to any Any large online platform, using state-of-the-art, best available tools to detect materially deceptive and digitally modified or created content that satisfies either of the following requirements: content shall develop and implement procedures for labeling such content as inauthentic, fake, or false if all of the following conditions are met:
(A)The materially deceptive content is either of the following:
(A)The content is included within(i)Included within paragraphs (1) to (4), inclusive, of subdivision (a) of Section 20513, but the content is posted or sent outside the applicable time period described in subdivision (c) of Section 20513.
(B)The content is, or appears within,(ii)Appears within an advertisement or election communication that and is not subject to Section 20513.
(2)With respect to any content described in paragraph (1), a large online platform, using state-of-the-art, best available tools to detect digitally modified or created content, shall develop and implement procedures for labeling such content as inauthentic, fake, or false, and shall, if the large online platform knows or should know that the digitally modified or created content meets the requirements of this section, label such content in this manner, during the applicable time period or periods set forth in subdivision (c).(B)The person or entity who created the materially deceptive content did so knowing it was false or with reckless disregard for the truth. There shall be a rebuttable presumption that the person or entity knew the materially deceptive content was false or acted with reckless disregard for the truth if the content would cause a reasonable person to have a fundamentally different understanding or impression of the content than the person would have if hearing or seeing an authentic version of the content.
(C)The large online platform knows or should know that the materially deceptive content meets the requirements of this section.
(b)The label required by subdivision (a) shall permit users to click or tap on it and to inspect all available provenance data about the digitally modified or created materially deceptive content in an easy-to-understand format.
(c)The labeling requirement set forth in subdivision (a) applies during any of the following time periods, to the extent applicable:
(1)The period beginning one year before the election and through the day of the election.
(2)The period beginning one year before the election and ending on the 60th day after the election, if the content depicts or pertains to elections officials, the electoral college process, a voting machine, ballot, voting site, or other property or equipment related to an election, or the canvass of the vote.
(3)During a governmental process related to redistricting that is required or authorized by state or local law.
20515.(a)A large online platform shall provide an easily accessible way for California residents to report to that platform content subject to Section 20513 or 20514 that was not blocked or labeled as required. The online platform shall respond to the person who made the report, within 36 hours of the report, describing any action taken or not taken by the online platform with respect to the content.
(b)A candidate for elective office, elected official, or elections official who has made a report to a large online platform under subdivision (a) and who either has not received a response within 36 hours or disagrees with the response, may seek injunctive or other equitable relief against the online platform to compel compliance with this chapter. The plaintiff shall bear the burden of establishing the violation through clear and convincing evidence. The court shall award a prevailing plaintiff reasonable attorneys fees and costs. An action under this subdivision shall be entitled to precedence in accordance with Section 35 of the Code of Civil Procedure.
20516.The Attorney General or any district attorney or city attorney may seek injunctive or other equitable relief against any large online platform to compel compliance with this chapter. The plaintiff shall bear the burden of establishing the violation through clear and convincing evidence. The court shall award a prevailing plaintiff reasonable attorneys fees and costs. An action under this section shall be entitled to precedence in accordance with Section 35 of the Code of Civil Procedure.
20517.This chapter applies to materially deceptive and digitally modified or created content, regardless of the language used in the content. If the language used is not English, the disclosure required by subdivision (b) of Section 20513 and the label required by Section 20514 must appear in the language used as well as in English.
20518.A large online platform that blocks or labels any materially deceptive and digitally modified or created content shall maintain a copy of the digital content for a period of not less than five years from the election and shall make such digital content available to the Secretary of State, the Fair Political Practices Commission, and researchers, if requested.
20519.(a)This chapter does not preclude a large online platform from blocking or labeling any materially deceptive and digitally modified or created content outside of the time periods specified in Sections 20513 and 20514.
(b)This chapter does not preclude any online platform not subject to this chapter from blocking or labeling any materially deceptive and digitally modified or created content.
20520.This chapter does not apply to either of the following:
(a)A regularly published online newspaper, magazine, or other periodical of general circulation that routinely carries news and commentary of general interest, and that publishes any materially deceptive and digitally altered or digitally created image, audio, or video recording content that an online platform is required to block or label based on this chapter, if the publication contains a clear disclosure that the materially deceptive and digitally altered or digitally created image or audio or video recording content does not accurately represent any actual event, occurrence, appearance, speech, or expressive conduct.
(b)Materially deceptive and digitally altered or digitally created content that constitutes satire or parody.
20521.The provisions of this chapter are severable. If any provision of this chapter or its application is held invalid, that invalidity shall not affect other provisions or applications that can be given effect without the invalid provision or application.
(a)Proceedings in cases involving the registration or denial of registration of voters, the certification or denial of certification of candidates, the certification or denial of certification of ballot measures, election contests, actions under Section 20010 of the Elections Code, actions under Chapter 7 (commencing with Section 20510) of Division 20 of the Elections Code, and actions under Chapter 2 (commencing with Section 21100) of Division 21 of the Elections Code shall be placed on the calendar in the order of their date of filing and shall be given precedence.
(b)This section shall remain in effect only until January 1, 2027, and as of that date is repealed, unless a later enacted statute, that is enacted before January 1, 2027, deletes or extends that date.
SEC. 2.Section 35 of the Code of Civil Procedure, as amended by Section 2 of Chapter 343 of the Statutes of 2023, is amended to read:35.(a)Proceedings in cases involving the registration or denial of registration of voters, the certification or denial of certification of candidates, the certification or denial of certification of ballot measures, election contests, actions under Chapter 7 (commencing with Section 20510) of Division 20 of the Elections Code, and actions under Chapter 2 (commencing with Section 21100) of Division 21 of the Elections Code shall be placed on the calendar in the order of their date of filing and shall be given precedence.
(b)This section shall become operative January 1, 2027.
SEC. 3.Chapter 7 (commencing with Section 20510) is added to Division 20 of the Elections Code, to read:7.Defending Democracy from Deepfake Deception Act of 202420510.This chapter shall be known and may be cited as the Defending Democracy from Deepfake Deception Act of 2024.
20511.The Legislature finds and declares all of the following:
(a)California is entering its first-ever generative artificial intelligence (AI) election, in which disinformation powered by generative AI will pollute our information ecosystems like never before. Voters will not know what images, audio, or video they can trust.
(b)In a few clicks, using current technology, bad actors now have the power to create a false image of a candidate accepting a bribe or a fake video of an elections official caught on tape saying that voting machines are not secure, or to generate the Governors voice telling millions of Californians their voting site has changed.
(c)In the lead-up to the 2024 presidential elections, candidates and parties are already creating and distributing deepfake images and audio and video content. These fake images or files can spread to millions of Californians in seconds and skew election results or undermine trust in the ballot counting process.
(d)The labeling information required by this bill is narrowly tailored to provide consumers with factual information about the inauthenticity of particular images, audio, video, or text content in order to prevent consumer deception.
(e)In order to ensure California elections are free and fair, California must, for a limited time before and after elections, prevent the use of deepfakes and disinformation meant to prevent voters from voting and to deceive voters based on fraudulent content. Accordingly, the provisions of this chapter are narrowly tailored to support California's compelling interest in protecting its free and fair elections.
20512.For purposes of this chapter, the following terms have the following meanings:
(a)Advertisement means any general or public communication that is authorized or paid for with the purpose of supporting or opposing a candidate for elective office.
(a)(b)Artificial intelligence means an engineered or machine-based system that varies in its level of autonomy and that can, for explicit or implicit objectives, infer from the input it receives how to generate outputs that can influence physical or virtual environments.
(c)Election communication means a general or public communication that is not an advertisement and that concerns any of the following:
(1)A candidate for elective office.
(2)Voting or refraining from voting in an election.
(3)The canvass of the vote.
(4)Voting machines, ballots, voting sites, or other property or equipment related to an election.
(5)Proceedings or processes of redistricting or the electoral college.
(b)(d)(1)Elections official means any of the following persons, but only in their capacity as a person charged with holding or conducting an election, conducting a canvass, assisting with the holding or conducting of an election or a canvas, or performing another duty related to administering the provisions of this code:
(A)An elections official as defined in Section 320.
(B)The Secretary of State and their staff.
(C)A temporary worker, poll worker, or member of a precinct board.
(D)Any other person charged with holding or conducting an election, conducting a canvass, assisting with the holding or conducting of an election or a canvas, or performing another duty related to administering the provisions of this code.
(2)The requirements of this chapter relating to content portraying an elections official apply only if the large online platform knows or should know that the person is an elections official.
(c)Election processes means any government process related to an election, including, but not limited to, elections, candidates, vote counting, redistricting, and proceedings or processes of the electoral college.(d)(e)(1)Materially deceptive and digitally modified or created content means an image or an audio or video recording or other digital content, including a chatbot, that has been intentionally manipulated such that all of the following conditions are met:
(A)(i)The digital content is the product of digital manipulation, including, but not limited to, artificial intelligence, that appears authentic, but that contains a false portrayal of any of the following: a candidate for elective office, elected official, elections official, voting machine, ballot, voting site, other property or equipment related to an election, or elections process. such that it so closely resembles authentic content that a reasonable person would not be able to readily distinguish the digitally manipulated or created content from the authentic content.
(ii)For purposes of this subdivision, false portrayal means the content would cause a reasonable person to have a fundamentally different understanding or impression of the content than the person would have if they were hearing or seeing an authentic version of the content.
(B)The person or entity who attempted to post or send, or who did post or send, the content did so knowing the portrayal was false, or did so with reckless disregard for whether the portrayal was false. If the content is intentionally manipulated and contains a false portrayal as specified in subparagraph (A), there shall be a rebuttable presumption that the person or entity knew the portrayal was false or that they acted with reckless disregard for whether the portrayal was false.
(2)Materially deceptive and digitally modified or created content does not include any image or audio or video recording that contains only minor modifications that do not lead to significant changes to the perceived contents or meaning of the content. Minor changes include changes to the brightness or contrast of images, removal of background noise in audio, and other minor changes that do not impact the content of the image or audio or video recording.
(e)(f)Large online platform means a public-facing internet website, web application, or digital application, including a social network, video sharing platform, advertising network, or search engine that had at least 1,000,000 California users during the preceding 12 months.
20513.(a)Any large online platform, using state-of-the-art, best available tools to detect digitally modified or created content, shall develop and implement procedures for blocking and preventing, and shall, if the large online platform knows or should know that the digitally modified or created content meets the requirements of this section, block and prevent, the posting or sending of any materially deceptive and digitally modified or created content, during the applicable time period or periods set forth in subdivision (c), of any of the following:
(1)A candidate for election elective office portrayed as doing or saying something that the candidate did not do or say.
(2)An elections official portrayed as doing or saying something in connection with the performance of their elections-related duties that the elections official did not do or say.
(3)An elected official portrayed as doing or saying something that influences the election that the elected official did not do or say.
(4)A voting machine, ballot, voting site, or other property or equipment related to an election that is portrayed in a materially false way.
(b)(1)Notwithstanding paragraph (1) of subdivision (a), a large online platform shall not prevent a candidate for an election, elective office, during the time period set forth in subdivision (c), from portraying themself as doing or saying something that the candidate did not do or say, but only if the digital content includes a disclosure stating the following: This _____ has been manipulated. The blank in this disclosure shall be filled in with whichever of the following terms most accurately describes the media:
(A)Image.
(B)Audio.
(C)Video.
(2)(A)For visual media, the text of the disclosure shall appear in a size that is easily readable by the average viewer and no smaller than the largest font size of other text appearing in the visual media. If the visual media does not include any other text, the disclosure shall appear in a size that is easily readable by the average viewer. For visual media that is video, the disclosure shall appear for the duration of the video.
(B)If the media consists of audio only, the disclosure shall be read in a clearly spoken manner and in a pitch that can be easily heard by the average listener, at the beginning of the audio, at the end of the audio, and, if the audio is greater than two minutes in length, interspersed within the audio at intervals of not greater than two minutes each.
(c)(1)Except as provided in paragraph (2), any large online platform shall block and prevent the content described in subdivision (a), and any candidate for elective office shall include the disclosure required by subdivision (b), during a period beginning 120 days before the election and through the day of the election.
(2)If the content described in subdivision (a) depicts or pertains to elections officials, or depicts or pertains to a voting machine, ballot, voting site, or other property or equipment related to an election, a large online platform shall block and prevent the content during a period beginning 120 days before the election and ending on the 60th day after the election.
20514.(a)With respect to any materially deceptive and digitally modified or created content that pertains to election processes and that is not subject to Section 20513, a large online platform, using state-of-the-art, best available tools to detect digitally modified or created content, shall develop and implement procedures for labeling such content as inauthentic, fake, or false, and shall, if the large online platform knows or should know that the digitally modified or created content meets the requirements of this section, label such content in this manner, during the applicable time period or periods set forth in subdivision (c).20514.(a)(1)This section applies to any materially deceptive and digitally modified or created content that satisfies either of the following requirements:
(A)The content is included within paragraphs (1) to (4), inclusive, of subdivision (a) of Section 20513, but the content is posted or sent outside the applicable time period described in subdivision (c) of Section 20513.
(B)The content is, or appears within, an advertisement or election communication that is not subject to Section 20513.
(2)With respect to any content described in paragraph (1), a large online platform, using state-of-the-art, best available tools to detect digitally modified or created content, shall develop and implement procedures for labeling such content as inauthentic, fake, or false, and shall, if the large online platform knows or should know that the digitally modified or created content meets the requirements of this section, label such content in this manner, during the applicable time period or periods set forth in subdivision (c).
(b)The label required by subdivision (a) shall permit users to click or tap on it and to inspect all available provenance data about the digitally modified or created content in an easy-to-understand format.
(c)The labeling requirement set forth in subdivision (a) applies during any of the following time periods, to the extent applicable:
(1)The period beginning one year before the election and through the day of the election that is specified in or implicated by the content. election.
(2)The period beginning one year before the election process and through the final day of the election process that is specified in or implicated by the content. election and ending on the 60th day after the election, if the content depicts or pertains to elections officials, the electoral college process, a voting machine, ballot, voting site, or other property or equipment related to an election, or the canvass of the vote.
(3)If the content depicts or pertains to elections officials, the period beginning one year before the election or election process that is specified in or implicated by the content and ending on the 60th day after that election or the 60th day after the final day of that election process, as applicable.(3)During a governmental process related to redistricting that is required or authorized by state or local law.
20515.(a)A large online platform shall provide an easily accessible way for California residents to report to that platform content subject to Section 20513 or 20514 that was not blocked or labeled as required. The online platform shall respond to the person who made the report, within 36 hours of the report, describing any action taken or not taken by the online platform with respect to the content.
(b)Any California resident A candidate for elective office, elected official, or elections official who has made a report to a large online platform under subdivision (a) and who either has not received a response within 36 hours or disagrees with the response, may seek injunctive or other equitable relief against the online platform to compel compliance with this chapter. The plaintiff shall bear the burden of establishing the violation through clear and convincing evidence. The court shall award a prevailing plaintiff reasonable attorneys fees and costs. An action under this subdivision shall be entitled to precedence in accordance with Section 35 of the Code of Civil Procedure.
20516.The Attorney General or any district attorney or city attorney may seek injunctive or other equitable relief against any large online platform to compel compliance with this chapter. The plaintiff shall bear the burden of establishing the violation through clear and convincing evidence. The court shall award a prevailing plaintiff reasonable attorneys fees and costs. An action under this section shall be entitled to precedence in accordance with Section 35 of the Code of Civil Procedure.
20517.This chapter applies to materially deceptive and digitally modified or created content, regardless of the language used in the content. If the language used is not English, the disclosure required by subdivision (b) of Section 20513 and the label required by Section 20514 must appear in the language used as well as in English.
20518.A large online platform that blocks or labels any materially deceptive and digitally modified or created content shall maintain a copy of the digital content for a period of not less than five years from the election or election process specified or implicated in the content and shall make such digital content available to the Secretary of State, the Fair Political Practices Commission, and researchers, if requested.
20519.(a)This chapter does not preclude a large online platform from blocking or labeling any materially deceptive and digitally modified or created content outside of the time periods specified in Sections 20513 and 20514.
(b)This chapter does not preclude any online platform not subject to this chapter from blocking or labeling any materially deceptive and digitally modified or created content.
20520.This chapter does not apply to either of the following:
(a)A regularly published online newspaper, magazine, or other periodical of general circulation that routinely carries news and commentary of general interest, and that publishes any materially deceptive and digitally altered or digitally created image, audio, or video recording that an online platform is required to block or label based on this chapter, if the publication contains a clear disclosure that the materially deceptive and digitally altered or digitally created image or audio or video recording does not accurately represent any actual event, occurrence, appearance, speech, or expressive conduct.
(b)Materially deceptive and digitally altered or digitally created content that constitutes satire or parody.
20521.The provisions of this chapter are severable. If any provision of this chapter or its application is held invalid, that invalidity shall not affect other provisions or applications that can be given effect without the invalid provision or application.
(a)Proceedings in cases involving the registration or denial of registration of voters, the certification or denial of certification of candidates, the certification or denial of certification of ballot measures, election contests, actions under Section 20010 of the Elections Code, actions under Chapter 7 (commencing with Section 20510) of Division 20 of the Elections Code, and actions under Chapter 2 (commencing with Section 21100) of Division 21 of the Elections Code shall be placed on the calendar in the order of their date of filing and shall be given precedence.
(b)This section shall remain in effect only until January 1, 2027, and as of that date is repealed, unless a later enacted statute, that is enacted before January 1, 2027, deletes or extends that date.
SEC. 2.Section 35 of the Code of Civil Procedure, as amended by Section 2 of Chapter 343 of the Statutes of 2023, is amended to read:35.(a)Proceedings in cases involving the registration or denial of registration of voters, the certification or denial of certification of candidates, the certification or denial of certification of ballot measures, election contests, actions under Chapter 7 (commencing with Section 20510) of Division 20 of the Elections Code, and actions under Chapter 2 (commencing with Section 21100) of Division 21 of the Elections Code shall be placed on the calendar in the order of their date of filing and shall be given precedence.
(b)This section shall become operative January 1, 2027.
SEC. 3.Chapter 7 (commencing with Section 20510) is added to Division 20 of the Elections Code, to read:7.Defending Democracy from Deepfake Deception Act of 202420510.This chapter shall be known and may be cited as the Defending Democracy from Deepfake Deception Act of 2024.
20511.The Legislature finds and declares all of the following:
(a)California is entering its first-ever generative artificial intelligence (AI) election, in which disinformation powered by generative AI will pollute our information ecosystems like never before. Voters will not know what images, audio, or video they can trust.
(b)In a few clicks, using current technology, bad actors now have the power to create a false image of a candidate accepting a bribe or a fake video of an elections official caught on tape saying that voting machines are not secure, or to generate the Governors voice telling millions of Californians their voting site has changed.
(c)In the lead-up to the 2024 presidential elections, candidates and parties are already creating and distributing deepfake images and audio and video content. These fake images or files can spread to millions of Californians in seconds and skew election results or undermine trust in the ballot counting process.
(d)The labeling information required by this bill is narrowly tailored to provide consumers with factual information about the inauthenticity of particular images, audio, video, or text content in order to prevent consumer deception.
(e)In order to ensure California elections are free and fair, California must, for a limited time before and after elections, prevent the use of deepfakes and disinformation meant to prevent voters from voting and to deceive voters based on fraudulent content.
20512.For purposes of this chapter, the following terms have the following meanings:
(a)Artificial intelligence means an engineered or machine-based system that varies in its level of autonomy and that can, for explicit or implicit objectives, infer from the input it receives how to generate outputs that can influence physical or virtual environments.
(a)(b)(1)Elections official means any of the following persons, but only in their capacity as a person charged with holding or conducting an election, conducting a canvass, assisting with the holding or conducting of an election or a canvas, or performing another duty related to administering the provisions of this code:
(A)An elections official as defined in Section 320.
(B)The Secretary of State and their staff.
(C)A temporary worker, poll worker, or member of a precinct board.
(D)Any other person charged with holding or conducting an election, conducting a canvass, assisting with the holding or conducting of an election or a canvas, or performing another duty related to administering the provisions of this code.
(2)The requirements of this chapter relating to content portraying an elections official apply only if the large online platform knows or should know that the person is an elections official.
(b)(c)Election processes means any government process related to an election, including, but not limited to, elections, candidates, vote counting, redistricting, and proceedings or processes of the electoral college.
(c)(d)(1)Materially deceptive and digitally modified or created content means an image or an audio or video recording or other digital content, including a chatbot, that has been intentionally manipulated such that all of the following conditions are met:
(A)(i)The digital content is the product of digital manipulation, including, but not limited to, artificial intelligence, or machine learning, including deep learning techniques, that merges, combines, replaces, or superimposes content onto an image or an audio or video recording, creating an image or an audio or video recording that appears authentic, or that otherwise generates an inauthentic image or an audio or video recording that appears authentic, and but that contains a false portrayal of any of the following: a candidate for elective office, elected official, elections official, voting machine, ballot, voting site, other property or equipment related to an election, or elections process.
(ii)For purposes of this subdivision, false portrayal means the content would cause a reasonable person to have a fundamentally different understanding or impression of the content than the person would have if they were hearing or seeing the unaltered, original an authentic version of the content.
(B)The person or entity who attempted to post or send, or who did post or send, the content did so knowing the portrayal was false, or did so with reckless disregard for whether the portrayal was false. If the content is intentionally manipulated and contains a false portrayal as specified in subparagraph (A), there shall be a rebuttable presumption that the person or entity knew the portrayal was false or that they acted with reckless disregard for whether the portrayal was false.
(2)Materially deceptive and digitally modified or created content does not include any image or audio or video recording that contains only minor modifications that do not lead to significant changes to the perceived contents or meaning of the content. Minor changes include changes to the brightness or contrast of images, removal of background noise in audio, and other minor changes that do not impact the content of the image or audio or video recording.
(d)(e)Large online platform means a public-facing internet website, web application, or digital application, including a social network, video sharing platform, advertising network, or search engine that had at least 1,000,000 California users during the preceding 12 months.
20513.(a)Any large online platform, using state-of-the-art, best available tools to detect digitally modified or created content, shall develop and implement procedures for blocking and preventing, and shall, if the large online platform knows or should know that the digitally modified or created content meets the requirements of this section, block and prevent, the posting or sending of any materially deceptive and digitally modified or created content, during the applicable time period or periods set forth in subdivision (c), of any of the following:
(1)A candidate for election portrayed as doing or saying something that the candidate did not do or say.
(2)An elections official portrayed as doing or saying something in connection with the performance of their elections-related duties that the elections official did not do or say.
(3)An elected official portrayed as doing or saying something that influences the election that the elected official did not do or say.
(4)A voting machine, ballot, voting site, or other property or equipment related to an election that is portrayed in a materially false way.
(b)(1)Notwithstanding paragraph (1) of subdivision (a), a large online platform shall not prevent a candidate for an election, during the time period set forth in subdivision (c), from portraying themself as doing or saying something that the candidate did not do or say, but only if the digital content includes a disclosure stating the following: This _____ has been manipulated. The blank in this disclosure shall be filled in with whichever of the following terms most accurately describes the media:
(A)Image.
(B)Audio.
(C)Video.
(2)(A)For visual media, the text of the disclosure shall appear in a size that is easily readable by the average viewer and no smaller than the largest font size of other text appearing in the visual media. If the visual media does not include any other text, the disclosure shall appear in a size that is easily readable by the average viewer. For visual media that is video, the disclosure shall appear for the duration of the video.
(B)If the media consists of audio only, the disclosure shall be read in a clearly spoken manner and in a pitch that can be easily heard by the average listener, at the beginning of the audio, at the end of the audio, and, if the audio is greater than two minutes in length, interspersed within the audio at intervals of not greater than two minutes each.
(c)(1)Except as provided in paragraph (2), any large online platform shall block and prevent the content described in subdivision (a), and any candidate shall include the disclosure required by subdivision (b), during a period beginning 120 days before the election and through the day of the election.
(2)If the content described in subdivision (a) depicts or pertains to elections officials, or depicts or pertains to a voting machine, ballot, voting site, or other property or equipment related to an election, a large online platform shall block and prevent the content during a period beginning 120 days before the election and ending on the 60th day after the election.
20514.(a)With respect to any materially deceptive and digitally modified or created content that pertains to election processes and that is not subject to Section 20513, a large online platform, using state-of-the-art, best available tools to detect digitally modified or created content, shall develop and implement procedures for labeling such content as inauthentic, fake, or false, and shall, if the large online platform knows or should know that the digitally modified or created content meets the requirements of this section, label such content in this manner, during the applicable time period or periods set forth in subdivision (c).
(b)The label required by subdivision (a) shall permit users to click or tap on it and to inspect all available provenance data about the digitally modified or created content in an easy-to-understand format.
(c)The labeling requirement set forth in subdivision (a) applies during any of the following time periods, to the extent applicable:
(1)The period beginning one year before the election and through the day of the election that is specified in or implicated by the content.
(2)The period beginning one year before the election process and through the final day of the election process that is specified in or implicated by the content.
(3)If the content depicts or pertains to elections officials, the period beginning one year before the election or election process that is specified in or implicated by the content and ending on the 60th day after that election or the 60th day after the final day of that election process, as applicable.
20515.(a)A large online platform shall provide an easily accessible way for California residents to report to that platform content subject to Section 20513 or 20514 that was not blocked or labeled as required. The online platform shall respond to the person who made the report, within 36 hours of the report, describing any action taken or not taken by the online platform with respect to the content.
(b)Any California resident who has made a report to a large online platform under subdivision (a) and who either has not received a response within 36 hours or disagrees with the response, may seek injunctive or other equitable relief against the online platform to compel compliance with this chapter. The court shall award a prevailing plaintiff reasonable attorneys fees and costs. An action under this subdivision shall be entitled to precedence in accordance with Section 35 of the Code of Civil Procedure.
20516.The Attorney General or any district attorney or city attorney may seek injunctive or other equitable relief against any large online platform to compel compliance with this chapter. The court shall award a prevailing plaintiff reasonable attorneys fees and costs. An action under this section shall be entitled to precedence in accordance with Section 35 of the Code of Civil Procedure.
20517.This chapter applies to materially deceptive and digitally modified or created content, regardless of the language used in the content. If the language used is not English, the disclosure required by subdivision (b) of Section 20513 and the label required by Section 20514 must appear in the language used as well as in English.
20518.A large online platform that blocks or labels any materially deceptive and digitally modified or created content shall maintain a copy of the digital content for a period of not less than five years from the election or election process specified or implicated in the content and shall make such digital content available to the Secretary of State, the Fair Political Practices Commission, and researchers, if requested.
20519.(a)This chapter does not preclude a large online platform from blocking or labeling any materially deceptive and digitally modified or created content outside of the time periods specified in Sections 20513 and 20514.
(b)This chapter does not preclude any online platform not subject to this chapter from blocking or labeling any materially deceptive and digitally modified or created content.
20520.This chapter does not apply to either of the following:
(a)A regularly published online newspaper, magazine, or other periodical of general circulation that routinely carries news and commentary of general interest, and that publishes any materially deceptive and digitally altered or digitally created image, audio, or video recording that an online platform is required to block or label based on this chapter, if the publication contains a clear disclosure that the materially deceptive and digitally altered or digitally created image or audio or video recording does not accurately represent any actual event, occurrence, appearance, speech, or expressive conduct.
(b)Materially deceptive and digitally altered or digitally created content that constitutes satire or parody.
20521.The provisions of this chapter are severable. If any provision of this chapter or its application is held invalid, that invalidity shall not affect other provisions or applications that can be given effect without the invalid provision or application.
(a)Proceedings in cases involving the registration or denial of registration of voters, the certification or denial of certification of candidates, the certification or denial of certification of ballot measures, election contests, actions under Section 20010 of the Elections Code, actions under Chapter 7 (commencing with Section 20510) of Division 20 of the Elections Code, and actions under Chapter 2 (commencing with Section 21100) of Division 21 of the Elections Code shall be placed on the calendar in the order of their date of filing and shall be given precedence.
(b)This section shall remain in effect only until January 1, 2027, and as of that date is repealed, unless a later enacted statute, that is enacted before January 1, 2027, deletes or extends that date.
SEC. 2.Section 35 of the Code of Civil Procedure, as amended by Section 2 of Chapter 343 of the Statutes of 2023, is amended to read:35.(a)Proceedings in cases involving the registration or denial of registration of voters, the certification or denial of certification of candidates, the certification or denial of certification of ballot measures, election contests, actions under Chapter 7 (commencing with Section 20510) of Division 20 of the Elections Code, and actions under Chapter 2 (commencing with Section 21100) of Division 21 of the Elections Code shall be placed on the calendar in the order of their date of filing and shall be given precedence.
(b)This section shall become operative January 1, 2027.
SEC. 3.Chapter 7 (commencing with Section 20510) is added to Division 20 of the Elections Code, to read:7.Defending Democracy from Deepfake Deception Act of 202420510.This chapter shall be known and may be cited as the Defending Democracy from Deepfake Deception Act of 2024.
20511.The Legislature finds and declares all of the following:
(a)California is entering its first-ever generative artificial intelligence (AI) election, in which disinformation powered by generative AI will pollute our information ecosystems like never before. Voters will not know what images, audio, or video they can trust.
(b)In a few clicks, using current technology, bad actors now have the power to create a false image of a candidate accepting a bribe or a fake video of an elections official caught on tape saying that voting machines are not secure, or to generate the Governors voice telling millions of Californians their voting site has changed.
(c)In the lead-up to the 2024 presidential elections, candidates and parties are already creating and distributing deepfake images and audio and video content. These fake images or files can spread to millions of Californians in seconds and skew election results or undermine trust in the ballot counting process.
(d)The labeling information required by this bill is narrowly tailored to provide consumers with factual information about the inauthenticity of particular images, audio, video, or text content in order to prevent consumer deception.
(e)In order to ensure California elections are free and fair, California must, for a limited time before and after elections, prevent the use of deepfakes and disinformation meant to prevent voters from voting and to deceive voters based on fraudulent content.
20512.For purposes of this chapter, the following terms have the following meanings:
(a)(1)Elections official means any of the following persons, but only in their capacity as a person charged with holding or conducting an election, conducting a canvass, assisting with the holding or conducting of an election or a canvas, or performing another duty related to administering the provisions of this code:
(1)(A)An elections official as defined in Section 320.
(2)(B)The Secretary of State and their staff.
(3)(C)A temporary worker, poll worker, or member of a precinct board.
(4)(D)Any other person charged with holding or conducting an election, conducting a canvass, assisting with the holding or conducting of an election or a canvas, or performing another duty related to administering the provisions of this code.
(2)The requirements of this chapter relating to content portraying an elections official apply only if the large online platform knows or should know that the person is an elections official.
(b)Election processes means any government process related to an election, including, but not limited to, elections, candidates, vote counting, redistricting, and proceedings or processes of the electoral college.
(c)(1)Materially deceptive and digitally modified or created content means an image or an audio or video recording or other digital content, including a chatbot, that has been intentionally manipulated such that all of the following conditions are met:
(A)(i)The digital content is the product of digital manipulation, artificial intelligence, or machine learning, including deep learning techniques, that merges, combines, replaces, or superimposes content onto an image or an audio or video recording, creating an image or an audio or video recording that appears authentic, or that otherwise generates an inauthentic image or an audio or video recording that appears authentic, and that contains a false portrayal of any of the following: a candidate for elective office, elected official, elections official, voting machine, ballot, voting site, other property or equipment related to an election, or elections process.
(ii)For purposes of this subdivision, false portrayal means the content would cause a reasonable person to have a fundamentally different understanding or impression of the content than the person would have if they were hearing or seeing the unaltered, original version of the content.
(B)The person or entity who attempted to post or send, or who did post or send, the content did so knowing the portrayal was false, or did so with reckless disregard for whether the portrayal was false. If the content is intentionally manipulated and contains a false portrayal as specified in subparagraph (A), there shall be a rebuttable presumption that the person or entity knew the portrayal was false or that they acted with reckless disregard for whether the portrayal was false.
(2)Materially deceptive and digitally modified or created content does not include any image or audio or video recording that contains only minor modifications that do not lead to significant changes to the perceived contents or meaning of the content. Minor changes include changes to the brightness or contrast of images, removal of background noise in audio, and other minor changes that do not impact the content of the image or audio or video recording.
(d)Large online platform means a public-facing internet website, web application, or digital application, including a social network, video sharing platform, messaging platform, advertising network, or search engine that had at least 1,000,000 California users during the preceding 12 months.
20513.(a)Any large online platform platform, using state-of-the-art, best available tools to detect digitally modified or created content, shall develop and implement procedures for blocking and preventing, and shall shall, if the large online platform knows or should know that the digitally modified or created content meets the requirements of this section, block and prevent, the posting or sending of any materially deceptive and digitally modified or created content, during the applicable time period or periods set forth in subdivision (c), of any of the following:
(1)A candidate for election portrayed as doing or saying something that the candidate did not do or say.
(2)An elections official portrayed as doing or saying something in connection with the performance of their elections-related duties that the elections official did not do or say.
(3)An elected official portrayed as doing or saying something that influences the election that the elected official did not do or say.
(4)A voting machine, ballot, voting site, or other property or equipment related to an election that is portrayed in a materially false way.
(b)(1)Notwithstanding paragraph (1) of subdivision (a), a large online platform shall not prevent a candidate for an election, during the time period set forth in subdivision (c), from portraying themself as doing or saying something that the candidate did not do or say, but only if the digital content includes a disclosure stating the following: This _____ has been manipulated. The blank in this disclosure shall be filled in with whichever of the following terms most accurately describes the media:
(A)Image.
(B)Audio.
(C)Video.
(2)(A)For visual media, the text of the disclosure shall appear in a size that is easily readable by the average viewer and no smaller than the largest font size of other text appearing in the visual media. If the visual media does not include any other text, the disclosure shall appear in a size that is easily readable by the average viewer. For visual media that is video, the disclosure shall appear for the duration of the video.
(B)If the media consists of audio only, the disclosure shall be read in a clearly spoken manner and in a pitch that can be easily heard by the average listener, at the beginning of the audio, at the end of the audio, and, if the audio is greater than two minutes in length, interspersed within the audio at intervals of not greater than two minutes each.
(c)(1)Except as provided in paragraph (2), any large online platform shall block and prevent the content described in subdivision (a), and any candidate shall include the disclosure required by subdivision (b), during a period beginning 120 days before the election and through the day of the election.
(2)If the content described in subdivision (a) depicts or pertains to elections officials, or depicts or pertains to a voting machine, ballot, voting site, or other property or equipment related to an election, a large online platform shall block and prevent the content during a period beginning 120 days before the election and ending on the 60th day after the election.
20514.(a)With respect to any materially deceptive and digitally modified or created content that pertains to election processes and that is not subject to Section 20513, a large online platform platform, using state-of-the-art, best available tools to detect digitally modified or created content, shall develop and implement procedures for labeling such content as inauthentic, fake, or false, and shall shall, if the large online platform knows or should know that the digitally modified or created content meets the requirements of this section, label such content in this manner, during the applicable time period or periods set forth in subdivision (c).
(b)The label required by subdivision (a) shall permit users to click or tap on it and to inspect all available provenance data about the digitally modified or created content in an easy-to-understand format.
(c)The labeling requirement set forth in subdivision (a) applies during any of the following time periods, to the extent applicable:
(1)The period beginning one year before the election and through the day of the election that is specified in or implicated by the content.
(2)The period beginning one year before the election process and through the final day of the election process that is specified in or implicated by the content.
(3)If the content depicts or pertains to elections officials, the period beginning one year before the election or election process that is specified in or implicated by the content and ending on the 60th day after that election or the 60th day after the final day of that election process, as applicable.
20515.(a)A large online platform shall provide an easily accessible way for California residents to report to that platform content subject to Section 20513 or 20514 that was not blocked or labeled as required. The online platform shall respond to the person who made the report, within 36 hours of the report, describing any action taken or not taken by the online platform with respect to the content.
(b)Any California resident who has made a report to a large online platform under subdivision (a) and who either has not received a response within 36 hours or disagrees with the response, may seek injunctive or other equitable relief against the online platform to compel compliance with this chapter. The court shall award a prevailing plaintiff reasonable attorneys fees and costs. An action under this subdivision shall be entitled to precedence in accordance with Section 35 of the Code of Civil Procedure.
20516.The Attorney General or any district attorney or city attorney may seek injunctive or other equitable relief against any large online platform to compel compliance with this chapter. The court shall award a prevailing plaintiff reasonable attorneys fees and costs. An action under this section shall be entitled to precedence in accordance with Section 35 of the Code of Civil Procedure.
20517.This chapter applies to materially deceptive and digitally modified or created content, regardless of the language used in the content. If the language used is not English, the disclosure required by subdivision (b) of Section 20513 and the label required by Section 20514 must appear in the language used. used as well as in English.
20518.A large online platform that blocks or labels any materially deceptive and digitally modified or created content shall maintain a copy of the digital content for a period of not less than five years from the election or election process specified or implicated in the content and shall make such digital content available to the Secretary of State, the Fair Political Practices Commission, and researchers, if requested.
20519.(a)This chapter does not preclude a large online platform from blocking or labeling any materially deceptive and digitally modified or created content outside of the time periods specified in Sections 20513 and 20514.
(b)This chapter does not preclude any online platform not subject to this chapter from blocking or labeling any materially deceptive and digitally modified or created content.
20520.This chapter does not apply to either of the following:
(a)A regularly published online newspaper, magazine, or other periodical of general circulation that routinely carries news and commentary of general interest, and that publishes any materially deceptive and digitally altered or digitally created image, audio, or video recording that an online platform is required to block or label based on this chapter, if the publication contains a clear disclosure that the materially deceptive and digitally altered or digitally created image or audio or video recording does not accurately represent any actual event, occurrence, appearance, speech, or expressive conduct.
(b)Materially deceptive and digitally altered or digitally created content that constitutes satire or parody.
20521.The provisions of this chapter are severable. If any provision of this chapter or its application is held invalid, that invalidity shall not affect other provisions or applications that can be given effect without the invalid provision or application.
REVISIONS: HeadingLine 1.(a)Proceedings in cases involving the registration or denial of registration of voters, the certification or denial of certification of candidates, the certification or denial of certification of ballot measures, election contests, actions under Section 20010 of the Elections Code, actions under Chapter 7 (commencing with Section 20510) of Division 20 of the Elections Code, and actions under Chapter 2 (commencing with Section 21100) of Division 21 of the Elections Code shall be placed on the calendar in the order of their date of filing and shall be given precedence.
(b)This section shall remain in effect only until January 1, 2027, and as of that date is repealed, unless a later enacted statute, that is enacted before January 1, 2027, deletes or extends that date.
SEC. 2.Section 35 of the Code of Civil Procedure, as amended by Section 2 of Chapter 343 of the Statutes of 2023, is amended to read:35.(a)Proceedings in cases involving the registration or denial of registration of voters, the certification or denial of certification of candidates, the certification or denial of certification of ballot measures, election contests, actions under Chapter 7 (commencing with Section 20510) of Division 20 of the Elections Code, and actions under Chapter 2 (commencing with Section 21100) of Division 21 of the Elections Code shall be placed on the calendar in the order of their date of filing and shall be given precedence.
(b)This section shall become operative January 1, 2027.
SEC. 3.Chapter 7 (commencing with Section 20510) is added to Division 20 of the Elections Code, to read:7.Defending Democracy from Deepfake Deception Act of 202420510.This chapter shall be known and may be cited as the Defending Democracy from Deepfake Deception Act of 2024.
20511.The Legislature finds and declares all of the following:
(a)California is entering its first-ever generative artificial intelligence (AI) election, in which disinformation powered by generative AI will pollute our information ecosystems like never before. Voters will not know what images, audio, or video they can trust.
(b)In a few clicks, using current technology, bad actors now have the power to create a false image of a candidate accepting a bribe or a fake video of an elections official caught on tape saying that voting machines are not secure, or to generate the Governors voice telling millions of Californians their voting site has changed.
(c)In the lead-up to the 2024 presidential elections, candidates and parties are already creating and distributing deepfake images and audio and video content. These fake images or files can spread to millions of Californians in seconds and skew election results or undermine trust in the ballot counting process.
(d)The labeling information required by this bill is narrowly tailored to provide consumers with factual information about the inauthenticity of particular images, audio, video, or text content in order to prevent consumer deception.
(e)In order to ensure California elections are free and fair, California must, for a limited time before and after elections, prevent the use of deepfakes and disinformation meant to prevent voters from voting and to deceive voters based on fraudulent content.
20512.For purposes of this chapter, the following terms have the following meanings:
(a)Elections official means any of the following persons, but only in their capacity as a person charged with holding or conducting an election, conducting a canvass, assisting with the holding or conducting of an election or a canvas, or performing another duty related to administering the provisions of this code:
(1)An elections official as defined in Section 320.
(2)The Secretary of State and their staff.
(3)A temporary worker, poll worker, or member of a precinct board.
(4)Any other person charged with holding or conducting an election, conducting a canvass, assisting with the holding or conducting of an election or a canvas, or performing another duty related to administering the provisions of this code.
(b)Election processes means any government process related to an election, including, but not limited to, elections, candidates, vote counting, redistricting, and proceedings or processes of the electoral college.
(c)(1)Materially deceptive and digitally modified or created content means an image or an audio or video recording or other digital content, including a chatbot, that has been intentionally manipulated such that all of the following conditions are met:
(A)(i)The digital content is the product of digital manipulation, artificial intelligence, or machine learning, including deep learning techniques, that merges, combines, replaces, or superimposes content onto an image or an audio or video recording, creating an image or an audio or video recording that appears authentic, or that otherwise generates an inauthentic image or an audio or video recording that appears authentic, and that contains a false portrayal of any of the following: a candidate for elective office, elected official, elections official, voting machine, ballot, voting site, other property or equipment related to an election, or elections process.
(ii)For purposes of this subdivision, false portrayal means the content would cause a reasonable person to have a fundamentally different understanding or impression of the content than the person would have if they were hearing or seeing the unaltered, original version of the content.
(B)The person or entity who attempted to post or send, or who did post or send, the content did so knowing the portrayal was false, or did so with reckless disregard for whether the portrayal was false. If the content is intentionally manipulated and contains a false portrayal as specified in subparagraph (A), there shall be a rebuttable presumption that the person or entity knew the portrayal was false or that they acted with reckless disregard for whether the portrayal was false.
(2)Materially deceptive and digitally modified or created content does not include any image or audio or video recording that contains only minor modifications that do not lead to significant changes to the perceived contents or meaning of the content. Minor changes include changes to the brightness or contrast of images, removal of background noise in audio, and other minor changes that do not impact the content of the image or audio or video recording.
(d)Large online platform means a public-facing internet website, web application, or digital application, including a social network, video sharing platform, messaging platform, advertising network, or search engine that had at least 1,000,000 California users during the preceding 12 months.
20513.(a)Any large online platform shall develop and implement procedures for blocking and preventing, and shall block and prevent, the posting or sending of any materially deceptive and digitally modified or created content, during the applicable time period or periods set forth in subdivision (c), of any of the following:
(1)A candidate for election portrayed as doing or saying something that the candidate did not do or say.
(2)An elections official portrayed as doing or saying something in connection with the performance of their elections-related duties that the elections official did not do or say.
(3)An elected official portrayed as doing or saying something that influences the election that the elected official did not do or say.
(4)A voting machine, ballot, voting site, or other property or equipment related to an election that is portrayed in a materially false way.
(b)(1)Notwithstanding paragraph (1) of subdivision (a), a large online platform shall not prevent a candidate for an election, during the time period set forth in subdivision (c), from portraying themself as doing or saying something that the candidate did not do or say, but only if the digital content includes a disclosure stating the following: This _____ has been manipulated. The blank in this disclosure shall be filled in with whichever of the following terms most accurately describes the media:
(A)Image.
(B)Audio.
(C)Video.
(2)(A)For visual media, the text of the disclosure shall appear in a size that is easily readable by the average viewer and no smaller than the largest font size of other text appearing in the visual media. If the visual media does not include any other text, the disclosure shall appear in a size that is easily readable by the average viewer. For visual media that is video, the disclosure shall appear for the duration of the video.
(B)If the media consists of audio only, the disclosure shall be read in a clearly spoken manner and in a pitch that can be easily heard by the average listener, at the beginning of the audio, at the end of the audio, and, if the audio is greater than two minutes in length, interspersed within the audio at intervals of not greater than two minutes each.
(c)(1)Except as provided in paragraph (2), any large online platform shall block and prevent the content described in subdivision (a), and any candidate shall include the disclosure required by subdivision (b), during a period beginning 120 days before the election and through the day of the election.
(2)If the content described in subdivision (a) depicts or pertains to elections officials, or depicts or pertains to a voting machine, ballot, voting site, or other property or equipment related to an election, a large online platform shall block and prevent the content during a period beginning 120 days before the election and ending on the 60th day after the election.
20514.(a)With respect to any materially deceptive and digitally modified or created content that pertains to election processes and that is not subject to Section 20513, a large online platform shall develop and implement procedures for labeling such content as inauthentic, fake, or false, and shall label such content in this manner, during the applicable time period or periods set forth in subdivision (c).
(b)The label required by subdivision (a) shall permit users to click or tap on it and to inspect all available provenance data about the digitally modified or created content in an easy-to-understand format.
(c)The labeling requirement set forth in subdivision (a) applies during any of the following time periods, to the extent applicable:
(1)The period beginning one year before the election and through the day of the election that is specified in or implicated by the content.
(2)The period beginning one year before the election process and through the final day of the election process that is specified in or implicated by the content.
(3)If the content depicts or pertains to elections officials, the period beginning one year before the election or election process that is specified in or implicated by the content and ending on the 60th day after that election or the 60th day after the final day of that election process, as applicable.
20515.(a)A large online platform shall provide an easily accessible way for California residents to report to that platform content subject to Section 20513 or 20514 that was not blocked or labeled as required. The online platform shall respond to the person who made the report, within 36 hours of the report, describing any action taken or not taken by the online platform with respect to the content.
(b)Any California resident who has made a report to a large online platform under subdivision (a) and who either has not received a response within 36 hours or disagrees with the response, may seek injunctive or other equitable relief against the online platform to compel compliance with this chapter. The court shall award a prevailing plaintiff reasonable attorneys fees and costs. An action under this subdivision shall be entitled to precedence in accordance with Section 35 of the Code of Civil Procedure.
20516.The Attorney General or any district attorney or city attorney may seek injunctive or other equitable relief against any large online platform to compel compliance with this chapter. The court shall award a prevailing plaintiff reasonable attorneys fees and costs. An action under this section shall be entitled to precedence in accordance with Section 35 of the Code of Civil Procedure.
20517.This chapter applies to materially deceptive and digitally modified or created content, regardless of the language used in the content. If the language used is not English, the disclosure required by subdivision (b) of Section 20513 and the label required by Section 20514 must appear in the language used.
20518.A large online platform that blocks or labels any materially deceptive and digitally modified or created content shall maintain a copy of the digital content for a period of not less than five years from the election or election process specified or implicated in the content and shall make such digital content available to the Secretary of State, the Fair Political Practices Commission, and researchers, if requested.
20519.(a)This chapter does not preclude a large online platform from blocking or labeling any materially deceptive and digitally modified or created content outside of the time periods specified in Sections 20513 and 20514.
(b)This chapter does not preclude any online platform not subject to this chapter from blocking or labeling any materially deceptive and digitally modified or created content.
20520.This chapter does not apply to either of the following:
(a)A regularly published online newspaper, magazine, or other periodical of general circulation that routinely carries news and commentary of general interest, and that publishes any materially deceptive and digitally altered or digitally created image, audio, or video recording that an online platform is required to block or label based on this chapter, if the publication contains a clear disclosure that the materially deceptive and digitally altered or digitally created image or audio or video recording does not accurately represent any actual event, occurrence, appearance, speech, or expressive conduct.
(b)Materially deceptive and digitally altered or digitally created content that constitutes satire or parody.
20521.The provisions of this chapter are severable. If any provision of this chapter or its application is held invalid, that invalidity shall not affect other provisions or applications that can be given effect without the invalid provision or application.
SECTION 1.Section 35 of the Code of Civil Procedure, as amended by Section 1 of Chapter 343 of the Statutes of 2023, is amended to read:35.Proceedings in cases involving the registration or denial of registration of voters, the certification or denial of certification of candidates, the certification or denial of certification of ballot measures, election contests, actions under Section 20010 of the Elections Code, and actions under Chapter 2 (commencing with Section 21100) of Division 21 of the Elections Code shall be placed on the calendar in the order of their date of filing and shall be given precedence.SEC. 2.Section 35 of the Code of Civil Procedure, as amended by Section 2 of Chapter 343 of the Statutes of 2023, is repealed.SEC. 3.Section 20010 of the Elections Code, as amended by Section 3 of Chapter 745 of the Statutes of 2022, is amended to read:20010.(a)Except as provided in subdivision (b), a person, committee, as defined in Section 82013 of the Government Code, or other entity shall not, within 60 days of an election at which a candidate for elective office will appear on the ballot, distribute, with actual malice, materially deceptive audio or visual media, as defined in subdivision (e), of the candidate with the intent to injure the candidates reputation or to deceive a voter into voting for or against the candidate.(b)(1)The prohibition in subdivision (a) does not apply if the audio or visual media includes a disclosure stating: This _____ has been manipulated.(2)The blank in the disclosure required by paragraph (1) shall be filled with whichever of the following terms most accurately describes the media:(A)Image.(B)Video.(C)Audio.(3)(A)For visual media, the text of the disclosure shall appear in a size that is easily readable by the average viewer and no smaller than the largest font size of other text appearing in the visual media. If the visual media does not include any other text, the disclosure shall appear in a size that is easily readable by the average viewer. For visual media that is video, the disclosure shall appear for the duration of the video.(B)If the media consists of audio only, the disclosure shall be read in a clearly spoken manner and in a pitch that can be easily heard by the average listener, at the beginning of the audio, at the end of the audio, and, if the audio is greater than two minutes in length, interspersed within the audio at intervals of not greater than two minutes each.(c)(1)A candidate for elective office whose voice or likeness appears in a materially deceptive audio or visual media distributed in violation of this section may seek injunctive or other equitable relief prohibiting the distribution of audio or visual media in violation of this section. An action under this paragraph shall be entitled to precedence in accordance with Section 35 of the Code of Civil Procedure.(2)A candidate for elective office whose voice or likeness appears in a materially deceptive audio or visual media distributed in violation of this section may bring an action for general or special damages against the person, committee, or other entity that distributed the materially deceptive audio or visual media. The court may also award a prevailing party reasonable attorneys fees and costs. This subdivision shall not be construed to limit or preclude a plaintiff from securing or recovering any other available remedy.(3)In any civil action alleging a violation of this section, the plaintiff shall bear the burden of establishing the violation through clear and convincing evidence.(d)(1)This section shall not be construed to alter or negate any rights, obligations, or immunities of an interactive service provider under Section 230 of Title 47 of the United States Code.(2)This section does not apply to a radio or television broadcasting station, including a cable or satellite television operator, programmer, or producer, that broadcasts materially deceptive audio or visual media prohibited by this section as part of a bona fide newscast, news interview, news documentary, or on-the-spot coverage of bona fide news events, if the broadcast clearly acknowledges through content or a disclosure, in a manner that can be easily heard or read by the average listener or viewer, that there are questions about the authenticity of the materially deceptive audio or visual media.(3)This section does not apply to a radio or television broadcasting station, including a cable or satellite television operator, programmer, or producer, when it is paid to broadcast materially deceptive audio or visual media.(4)This section does not apply to an internet website, or a regularly published newspaper, magazine, or other periodical of general circulation, including an internet or electronic publication, that routinely carries news and commentary of general interest, and that publishes materially deceptive audio or visual media prohibited by this section, if the publication clearly states that the materially deceptive audio or visual media does not accurately represent the speech or conduct of the candidate.(5)This section does not apply to materially deceptive audio or visual media that constitutes satire or parody.(e)As used in this section, materially deceptive audio or visual media means an image or an audio or video recording of a candidates appearance, speech, or conduct that has been intentionally manipulated in a manner such that both of the following conditions are met:(1)The image or audio or video recording would falsely appear to a reasonable person to be authentic.(2)The image or audio or video recording would cause a reasonable person to have a fundamentally different understanding or impression of the expressive content of the image or audio or video recording than that person would have if the person were hearing or seeing the unaltered, original version of the image or audio or video recording.(f)The provisions of this section are severable. If any provision of this section or its application is held invalid, that invalidity shall not affect other provisions or applications that can be given effect without the invalid provision or application.SEC. 4.Section 20010 of the Elections Code, as amended by Section 4 of Chapter 745 of the Statutes of 2022, is repealed.(a)Proceedings in cases involving the registration or denial of registration of voters, the certification or denial of certification of candidates, the certification or denial of certification of ballot measures, election contests, actions under Section 20010 of the Elections Code, and actions under Chapter 2 (commencing with Section 21100) of Division 21 of the Elections Code shall be placed on the calendar in the order of their date of filing and shall be given precedence.
(b)This section shall remain in effect only until January 1, 2027, and as of that date is repealed, unless a later enacted statute, that is enacted before January 1, 2027, deletes or extends that date.SEC. 2.Section 35 of the Code of Civil Procedure, as amended by Section 2 of Chapter 343 of the Statutes of 2023, is repealed.35.(a)Proceedings in cases involving the registration or denial of registration of voters, the certification or denial of certification of candidates, the certification or denial of certification of ballot measures, election contests, and actions under Chapter 2 (commencing with Section 21100) of Division 21 of the Elections Code shall be placed on the calendar in the order of their date of filing and shall be given precedence.(b)This section shall become operative January 1, 2027.SEC. 3.Section 20010 of the Elections Code, as amended by Section 3 of Chapter 745 of the Statutes of 2022, is amended to read:20010.(a)Except as provided in subdivision (b), a person, committee, as defined in Section 82013 of the Government Code, or other entity shall not, within 60 days of an election at which a candidate for elective office will appear on the ballot, distribute, with actual malice, materially deceptive audio or visual media, as defined in subdivision (e), of the candidate with the intent to injure the candidates reputation or to deceive a voter into voting for or against the candidate.
(b)(1)The prohibition in subdivision (a) does not apply if the audio or visual media includes a disclosure stating: This _____ has been manipulated.
(2)The blank in the disclosure required by paragraph (1) shall be filled with whichever of the following terms most accurately describes the media:
(A)Image.
(B)Video.
(C)Audio.
(3)(A)For visual media, the text of the disclosure shall appear in a size that is easily readable by the average viewer and no smaller than the largest font size of other text appearing in the visual media. If the visual media does not include any other text, the disclosure shall appear in a size that is easily readable by the average viewer. For visual media that is video, the disclosure shall appear for the duration of the video.
(B)If the media consists of audio only, the disclosure shall be read in a clearly spoken manner and in a pitch that can be easily heard by the average listener, at the beginning of the audio, at the end of the audio, and, if the audio is greater than two minutes in length, interspersed within the audio at intervals of not greater than two minutes each.
(c)(1)A candidate for elective office whose voice or likeness appears in a materially deceptive audio or visual media distributed in violation of this section may seek injunctive or other equitable relief prohibiting the distribution of audio or visual media in violation of this section. An action under this paragraph shall be entitled to precedence in accordance with Section 35 of the Code of Civil Procedure.
(2)A candidate for elective office whose voice or likeness appears in a materially deceptive audio or visual media distributed in violation of this section may bring an action for general or special damages against the person, committee, or other entity that distributed the materially deceptive audio or visual media. The court may also award a prevailing party reasonable attorneys fees and costs. This subdivision shall not be construed to limit or preclude a plaintiff from securing or recovering any other available remedy.
(3)In any civil action alleging a violation of this section, the plaintiff shall bear the burden of establishing the violation through clear and convincing evidence.
(d)(1)This section shall not be construed to alter or negate any rights, obligations, or immunities of an interactive service provider under Section 230 of Title 47 of the United States Code.
(2)This section does not apply to a radio or television broadcasting station, including a cable or satellite television operator, programmer, or producer, that broadcasts materially deceptive audio or visual media prohibited by this section as part of a bona fide newscast, news interview, news documentary, or on-the-spot coverage of bona fide news events, if the broadcast clearly acknowledges through content or a disclosure, in a manner that can be easily heard or read by the average listener or viewer, that there are questions about the authenticity of the materially deceptive audio or visual media.
(3)This section does not apply to a radio or television broadcasting station, including a cable or satellite television operator, programmer, or producer, when it is paid to broadcast materially deceptive audio or visual media.
(4)This section does not apply to an internet website, or a regularly published newspaper, magazine, or other periodical of general circulation, including an internet or electronic publication, that routinely carries news and commentary of general interest, and that publishes materially deceptive audio or visual media prohibited by this section, if the publication clearly states that the materially deceptive audio or visual media does not accurately represent the speech or conduct of the candidate.
(5)This section does not apply to materially deceptive audio or visual media that constitutes satire or parody.
(e)As used in this section, materially deceptive audio or visual media means an image or an audio or video recording of a candidates appearance, speech, or conduct that has been intentionally manipulated in a manner such that both of the following conditions are met:
(1)The image or audio or video recording would falsely appear to a reasonable person to be authentic.
(2)The image or audio or video recording would cause a reasonable person to have a fundamentally different understanding or impression of the expressive content of the image or audio or video recording than that person would have if the person were hearing or seeing the unaltered, original version of the image or audio or video recording.
(f)The provisions of this section are severable. If any provision of this section or its application is held invalid, that invalidity shall not affect other provisions or applications that can be given effect without the invalid provision or application.
(g)This section shall remain in effect only until January 1, 2027, and as of that date is repealed, unless a later enacted statute, that is enacted before January 1, 2027, deletes or extends that date.SEC. 4.Section 20010 of the Elections Code, as amended by Section 4 of Chapter 745 of the Statutes of 2022, is repealed.20010.(a)Except as provided in subdivision (b), a person, firm, association, corporation, campaign committee, or organization shall not, with actual malice, produce, distribute, publish, or broadcast campaign material that contains (1) a picture or photograph of a person or persons into which the image of a candidate for public office is superimposed or (2) a picture or photograph of a candidate for public office into which the image of another person or persons is superimposed. Campaign material includes, but is not limited to, any printed matter, advertisement in a newspaper or other periodical, television commercial, or computer image. For purposes of this section, actual malice means the knowledge that the image of a person has been superimposed on a picture or photograph to create a false representation, or a reckless disregard of whether or not the image of a person has been superimposed on a picture or photograph to create a false representation.(b)A person, firm, association, corporation, campaign committee, or organization may produce, distribute, publish, or broadcast campaign material that contains a picture or photograph prohibited by subdivision (a) only if each picture or photograph in the campaign material includes the following statement in the same point size type as the largest point size type used elsewhere in the campaign material: This picture is not an accurate representation of fact. The statement shall be immediately adjacent to each picture or photograph prohibited by subdivision (a).(c)(1)Any registered voter may seek a temporary restraining order and an injunction prohibiting the publication, distribution, or broadcasting of any campaign material in violation of this section. Upon filing a petition under this section, the plaintiff may obtain a temporary restraining order in accordance with Section 527 of the Code of Civil Procedure.(2)A candidate for public office whose likeness appears in a picture or photograph prohibited by subdivision (a) may bring a civil action against any person, firm, association, corporation, campaign committee, or organization that produced, distributed, published, or broadcast the picture or photograph prohibited by subdivision (a). The court may award damages in an amount equal to the cost of producing, distributing, publishing, or broadcasting the campaign material that violated this section, in addition to reasonable attorneys fees and costs.(d)(1)This section does not apply to a holder of a license granted pursuant to the federal Communications Act of 1934 (47 U.S.C. Sec. 151 et seq.) in the performance of the functions for which the license is granted.(2)This section does not apply to the publisher or an employee of a newspaper, magazine, or other periodical that is published on a regular basis for any material published in that newspaper, magazine, or other periodical. For purposes of this subdivision, a newspaper, magazine, or other periodical that is published on a regular basis does not include any newspaper, magazine, or other periodical that has as its primary purpose the publication of campaign advertising or communication, as defined by Section 304.(e)This section shall become operative on January 1, 2027.|
Date
|
Status
|
|---|---|
| Sep 17, 2024 | Chaptered by Secretary of State - Chapter 261, Statutes of 2024. |
| Sep 17, 2024 | Approved by the Governor. |
| Sep 9, 2024 | Enrolled and presented to the Governor at 3:30 p.m. |
| Aug 28, 2024 | Senate amendments concurred in. To Engrossing and Enrolling. (Ayes 59. Noes 8.). |
| Aug 28, 2024 | In Assembly. Concurrence in Senate amendments pending. May be considered on or after August 30 pursuant to Assembly Rule 77. |
| Aug 28, 2024 | Assembly Rule 77 suspended. |
| Aug 27, 2024 | Read third time. Passed. Ordered to the Assembly. (Ayes 31. Noes 9.). |
| Aug 26, 2024 | Read second time. Ordered to third reading. |
| Aug 23, 2024 | Read third time and amended. Ordered to second reading. |
| Aug 19, 2024 | Read second time. Ordered to third reading. |
| Aug 15, 2024 | Read second time and amended. Ordered returned to second reading. |
| Aug 15, 2024 | From committee: Amend, and do pass as amended. (Ayes 5. Noes 2.) (August 15). |
| Aug 5, 2024 | In committee: Referred to APPR suspense file. |
| Jul 3, 2024 | Read second time and amended. Re-referred to Com. on APPR. |
| Jul 3, 2024 | From committee: Amend, and do pass as amended and re-refer to Com. on APPR. (Ayes 9. Noes 2.) (July 2). |
| Jun 18, 2024 | From committee: Do pass and re-refer to Com. on JUD. (Ayes 6. Noes 1.) (June 18). Re-referred to Com. on JUD. |
| Jun 11, 2024 | From committee chair, with author's amendments: Amend, and re-refer to committee. Read second time, amended, and re-referred to Com. on E. & C.A. |
| Jun 5, 2024 | Referred to Coms. on E. & C.A. and JUD. |
| May 23, 2024 | In Senate. Read first time. To Com. on RLS. for assignment. |
| May 22, 2024 | Read third time. Passed. Ordered to the Senate. (Ayes 56. Noes 1. Page 5544.) |
| May 21, 2024 | Read second time. Ordered to third reading. |
| May 20, 2024 | Read second time and amended. Ordered returned to second reading. |
| May 20, 2024 | From committee: Amend, and do pass as amended. (Ayes 11. Noes 1.) (May 16). |
| May 8, 2024 | In committee: Set, first hearing. Referred to APPR. suspense file. |
| Apr 25, 2024 | Re-referred to Com. on APPR. |
| Apr 24, 2024 | Read second time and amended. |
| Apr 23, 2024 | From committee: Amend, and do pass as amended and re-refer to Com. on APPR. (Ayes 9. Noes 0.) (April 23). |
| Apr 10, 2024 | From committee: Do pass and re-refer to Com. on JUD. (Ayes 6. Noes 1.) (April 10). Re-referred to Com. on JUD. |
| Apr 10, 2024 | Coauthors revised. |
| Apr 2, 2024 | Re-referred to Com. on ELECTIONS. |
| Apr 1, 2024 | Re-referred to Com. on ELECTIONS. |
| Apr 1, 2024 | From committee chair, with author's amendments: Amend, and re-refer to Com. on ELECTIONS. Read second time and amended. |
| Mar 21, 2024 | Referred to Coms. on ELECTIONS and JUD. |
| Mar 21, 2024 | From committee chair, with author's amendments: Amend, and re-refer to Com. on ELECTIONS. Read second time and amended. |
| Feb 15, 2024 | From printer. May be heard in committee March 16. |
| Feb 14, 2024 | Read first time. To print. |
|
Date
|
Description
|
Location
|
Votes
|
|
|---|---|---|---|---|
| Aug 28, 2024 | AB 2655 Berman Concurrence in Senate Amendments | Assembly Floor |
Passed |
|
| Aug 27, 2024 | Assembly 3rd Reading AB2655 Berman et al. By Umberg | Senate Floor |
Passed |
|
| Aug 15, 2024 | Do pass as amended | Senate Standing Committee on Appropriations |
Passed |
|
| Aug 5, 2024 | Placed on suspense file | Senate Standing Committee on Appropriations |
Passed |
|
| Jul 2, 2024 | Do pass as amended, but first amend, and re-refer to the Committee on [Appropriations] | Senate Standing Committee on Judiciary |
Passed |
|
| Jun 18, 2024 | Do pass, but first be re-referred to the Committee on [Judiciary] | Senate Standing Committee on Elections and Constitutional Amendments |
Passed |
|
| May 22, 2024 | AB 2655 Berman Assembly Third Reading | Assembly Floor |
Passed |
|
| May 16, 2024 | Do pass as amended. | Assembly Standing Committee on Appropriations |
Passed |
|
| Apr 23, 2024 | Do pass as amended and be re-referred to the Committee on [Appropriations] | Assembly Standing Committee on Judiciary |
Passed |
|
| Apr 10, 2024 | Do pass and be re-referred to the Committee on [Judiciary] | Assembly Standing Committee on Elections |
Passed |
This list shows positions taken on this bill by organizations on the date displayed, which is linked to the original source.
Positions by individuals are not included.
|
Organization
|
Position
|
Date
|
|---|---|---|
| Aclu California Action | ** Against | |
| American Federation of State, County and Municipal Employees | ** Support | |
| Asian Americans Advancing Justice - Asian Law Caucus | ** Support | |
| Asian Americans and Pacific Islanders for Civic Empowerment | ** Support | |
| Asian Law Alliance | ** Support | |
| Bay Rising | ** Support | |
| Board of Supervisors for the City and County of San Francisco | ** Support | |
| California Chamber Of Commerce | ** Against | |
| California Clean Money Campaign | ** Support | |
| California Common Cause | * Support | Apr 23, 2024 |
| California Environmental Voters | ** Support | |
| California Initiative for Technology & Democracy | ** Support | |
| California State Sheriffs Association | ** Support | |
| California Voter Foundation | ** Support | |
| Center for Countering Digital Hate | ** Support | |
| Chamber Of Progress | ** Against | |
| Chinese Progressive Association | ** Support | |
| City and County of San Francisco Board of Supervisors | ** Support | |
| Computer and Communications Industry | ** Against | |
| Courage California | ** Support | |
| Disability Rights California | ** Support | |
| Electronic Frontier Foundation | ** Against | |
| First Amendment Coalition | ** Against | |
| Hmong Innovating Politics | ** Support | |
| Indivisible Ca Statestrong | ** Support | |
| Inland Empire United | ** Support | |
| Internet Works | ** Against | |
| League of Women Voters in California | ** Support | |
| Move the Valley | ** Support | |
| NetChoice | ** Against | |
| NextGen California | ** Support | |
| Northern California Recycling Association | ** Support | |
| Oakland Privacy | ** Against | |
| Partnership for the Advancement of New Americans | ** Support | |
| SEIU California | ** Support | |
| Software And Information Industry Association | ** Against | |
| Techequity Action | ** Support | |
| Technet | ** Against | |
| UC Berkeley | * Support | Apr 23, 2024 |
| Verified Voting | ** Support | |
| Voices for Progress | * Support | Apr 10, 2024 |
| Young People’s Alliance | ** Support | |
| Youth Power Project | ** Support |
CONCURRENCE IN SENATE AMENDMENTS
CSA1 Bill Id:AB 2655
Author:(Berman and Pellerin)
As Amended Ver:August 23, 2024
Majority vote
SUMMARY
Requires large online platforms, as defined, to remove materially deceptive and digitally modified or created content related to elections, or to label that content, during specified periods before and after an election, if the content is reported to the platform, as specified.
Senate Amendments
- Provide that a large online platform is required to remove or label content only if that content is first reported to the platform by a California resident as being content that is covered by the provisions of this bill. Require the platform to remove or label the content no later than 72 hours after it is reported.
- Limit the bill's applicability to content related to elections in California, and to candidates for President and Vice President, statewide office, Board of Equalization, state Legislature, and United States (US) House of Representatives.
- Reduce the period of time during which materially deceptive content must be labeled such that the period begins six months before an election in California, instead of beginning one year before an election as was provided in the Assembly-approved version of this bill.
- Specify that the bill does not apply to a broadcasting station in either of the following circumstances:
- When it distributes deceptively-altered media as part of its news coverage, as specified, if the media includes a clear acknowledgement that it is not accurately representative.
- When it is paid to broadcast materially deceptive content if federal law requires the station to air the advertisement or if the station has its own prohibition and disclaimer requirements that are generally consistent with the requirements of this bill, as specified.
- Delete provisions of the bill that would have allowed a court to award reasonable attorney's fees and costs to a prevailing plaintiff party in an action brought under this bill.
- Delete a provision of the bill that would have required a platform to maintain a copy of any content that it blocks or labels under this bill for at least five years.
- Delete provisions of the bill that would have made it applicable to deceptive and digitally modified or created content related to redistricting.
- Recast various provisions of the bill to improve clarity, and make other clarifying, technical, and conforming changes.
- Add double-jointing language to avoid chaptering problems with AB 2839 (Pellerin) of the current legislative session.
COMMENTS
The use of false and deceptive information in campaigns to influence election outcomes is not a new phenomenon. Laws aimed at curbing such practices and preserving the integrity of elections have a long history in California. In 1850, the First Session of the California State Legislature created penalties for election misconduct, including for "deceiving [an elector] and causing him to vote for a different person for any office than such elector desired or intended to vote for" (Chapter 38, Statutes of 1850).
Advancements in technology have made it increasingly simple to produce false and misleading media that closely resembles authentic content. Moreover, platforms like social media have facilitated the rapid dissemination of deceptive media to large audiences at minimal cost. Given these developments, the potential threat posed by manipulated media to future elections' integrity may be more significant than in the past.
Past legislative efforts have addressed concerns about manipulated media's use to deceive voters during elections. Those laws, however, are limited, and are designed primarily to target the harms to candidates that may result from the distribution of manipulated media of those candidates. In contrast, this bill aims to regulate materially deceptive and digitally altered media depicting not only candidates, but also elections officials and elected officials who are not candidates. Additionally, this bill targets media that portrays elections materials and equipment in materially deceptive ways. The author and supporters of this bill believe that these provisions will safeguard voters against deceitful media that could undermine trust in the electoral process.
The Legislature is considering a number of bills this year that seek to address deceptive and digitally altered elections-related content in an effort to protect the integrity of elections in California. While other legislation related to this topic applies broadly to the distribution of such content through various mediums, this bill specifically targets the distribution of deceptive content through online platforms, including social media. Recognizing that those online platforms can facilitate the rapid spread of deceptive content, this bill seeks to minimize that potential by obligating large online platforms to remove or label offending content.
In recognition that the regulation of the distribution of content can create free speech concerns, this bill contains various provisions that tailor the content to which it applies, such that it targets content that has the highest likelihood of deceiving voters and undermining electoral integrity. While that tailoring does limit the content that online platforms would be required to remove or label, it also adds additional factors that platforms must consider in order to identify content that is required to be removed or labeled under this bill.
Along with other limitations, this bill applies only to content that 1) is distributed during specified time periods around elections and election processes, 2) includes media relating to elections or the electoral process in specified ways, 3) that was intentionally manipulated digitally to be materially deceptive, and 4) that is not satire or parody. Each of these limitations adds additional factors that online platforms would need to consider when determining whether a specific communication must be removed or labeled by this bill. Given the number of elections and candidates in California at any given time, making the determinations at scale about which content must be removed or labeled likely will be considerably more challenging than making those determinations on a case-by-case basis in a court of law. Senate amendments, however, eliminated the requirements for platforms to proactively remove or label such content, and instead impose an obligation on platforms to act only after a report has been made. Those amendments likely will reduce the burden that this bill creates on platforms to some degree.
A question could be raised about whether this bill is consistent with the right to freedom of speech that is guaranteed by the US and California constitutions. The US Supreme Court has ruled that even false statements are protected by the First Amendment (United States v. Alvarez (2012), 567 U.S. 709). When a law burdens core political speech, the restrictions on speech generally must be "narrowly tailored to serve an overriding state interest," McIntyre v. Ohio Elections Commission (1995), 514 US 334.
This bill targets deceptive content that could undermine trust in elections, prevent voters from voting, and distort the electoral process. The US Supreme Court generally has found that the protection of the integrity of elections is an overriding (or compelling) government interest (Id. at 349; Burson v. Freeman (1992) 504 U.S. 191, 199). A challenge of this bill on First Amendment grounds, then, likely would hinge on whether the court found this bill's provisions to be narrowly tailored. This bill includes provisions to limit its scope to communications posing the greatest threat to election integrity in an effort to tailor its provisions. Whether these limitations adequately protect this bill from a potential constitutional challenge is unclear. However, while these limitations may help protect the bill against a constitutional challenge, they may also make it harder for the bill to achieve its aims of limiting the spread of materially deceptive communications that have the potential to undermine election integrity.
The Senate amendments eliminate the requirement for platforms to proactively block or label deceptive content, and instead require platforms to address deceptive content only upon receiving reports from users about that content. The Senate amendments additionally make various changes in response to opposition concerns including narrowing the bill's applicability to broadcast stations and limiting the types of elections and candidates to which the bill applies, among other changes.
Please see the policy committee analysis for a full discussion of this bill.
According to the Author
"AB 2655 will ensure that online platforms restrict the spread of election-related deceptive deepfakes meant to prevent voters from voting or to deceive them based on fraudulent content. Deepfakes are a powerful and dangerous tool in the arsenal of those that want to wage disinformation campaigns, and they have the potential to wreak havoc on our democracy by attributing speech and conduct to a person that is false or that never happened. Advances in AI make it easy for practically anyone to generate this deceptive content, making it that much more important that we identify and restrict its spread before it has the chance to deceive voters and undermine our democracy."
Arguments in Support
The sponsor of this bill, the California Initiative for Technology & Democracy, writes in support, "Those trying to influence campaigns – conspiracy theorists, foreign states, online trolls, and candidates themselves – are already creating and distributing election-threatening deepfake images, audio, and video content in the US and around the world… AB 2655 strikes the right balance by seeking to ban, for a strictly limited time before and after elections, the online spread of the worst deepfakes and disinformation maliciously intended to prevent voters from voting or getting them to vote erroneously based on fraudulent content. The bill also requires that other fake online content related to elections and elections processes…which is also designed to undermine election procedures and democratic institutions, must be labeled as fake, again just for a limited time. The bill only applies to the largest online platforms with the greatest reach of potential election disinformation, and we believe it is fully implementable today based on tools these companies already possess… AB 2655's approach is narrowly tailored and does not extend the law to hot button controversies or inflammatory claims…It simply stops the use of obviously, demonstrably untrue and provably false content meant to impermissibly influence our elections at peak election times. It is therefore respectful of the protections of the First Amendment and avoids concerns based on Section 230 of the Communications Decency Act."
Arguments in Opposition
A coalition of business and technology industry associations writes in opposition, "AB 2655 appears to be based on the false assumption that online platforms definitively know whether any particular piece of content has been manipulated in such a way that is defined under the bill. While digital services may employ tools to identify and detect these materials with some degree of certainty, it is an evolving and imperfect science in its current form. AB 2655 also presumes that online platforms are an appropriate arbiter of deciding what constitutes accurate election information. However, most digital services are not equipped with the tools or expertise to make such judgments. Because covered platforms are not privy to the intent and context behind each piece of content, they may inadvertently over-block or over-label material. This could lead to user frustration and the suppression of political speech. Political speech is fundamental to the First Amendment's purpose. Therefore, AB 2655 raises concerns about how its provisions could have a chilling effect on online speech and whether it can withstand constitutional scrutiny."
FISCAL COMMENTS
According to the Senate Appropriations Committee:
- The Department of Justice (DOJ) indicates that it would incur costs of $911,000 in 2024-25, $1.4 million each in 2025-26 and 2026-27, $1.2 million in 2027-28, and $1 million annually thereafter, to implement the provisions of the bill (General Fund).
- By authorizing a claim by specified parties against a large online platform for failing to block or label specified content, this bill could result in an increased number of civil actions. Consequently, the bill could result in potentially significant cost pressures to the courts; the magnitude is unknown (Trial Court Trust Fund (TCTF)). The specific number of new actions that could be filed under the bill also is unknown; however, it generally costs about $1,000 to operate a courtroom for one hour. Courts are not funded on the basis of workload, and increased pressure on TCTF may create a need for increased funding for courts from the General Fund. The enacted 2024-25 budget includes $37 million in ongoing support from the General Fund to continue to backfill TCTF for revenue declines.
VOTES:
ASM ELECTIONS: 6-1-1
YES: Pellerin, Bennett, Berman, Cervantes, Low, Weber
NO: Essayli
ABS, ABST OR NV: Lackey
ASM JUDICIARY: 9-0-3
YES: Kalra, Bauer-Kahan, Bryan, Connolly, Haney, Maienschein, McKinnor, Pacheco, Reyes
ABS, ABST OR NV: Dixon, Sanchez, Waldron
ASM APPROPRIATIONS: 11-1-3
YES: Wicks, Arambula, Bryan, Calderon, Wendy Carrillo, Mike Fong, Grayson, Haney, Hart, Pellerin, Villapudua
NO: Dixon
ABS, ABST OR NV: Sanchez, Jim Patterson, Ta
ASSEMBLY FLOOR: 56-1-23
YES: Addis, Aguiar-Curry, Alvarez, Arambula, Bains, Bauer-Kahan, Bennett, Berman, Boerner, Bonta, Bryan, Juan Carrillo, Wendy Carrillo, Connolly, Mike Fong, Gabriel, Garcia, Gipson, Grayson, Haney, Hart, Irwin, Jackson, Jones-Sawyer, Kalra, Lee, Low, Lowenthal, Maienschein, McCarty, McKinnor, Muratsuchi, Stephanie Nguyen, Ortega, Pacheco, Papan, Pellerin, Petrie-Norris, Quirk-Silva, Ramos, Rendon, Reyes, Rodriguez, Blanca Rubio, Santiago, Schiavo, Soria, Ting, Valencia, Ward, Weber, Wicks, Wilson, Wood, Zbur, Robert Rivas
NO: Dixon
ABS, ABST OR NV: Alanis, Calderon, Cervantes, Chen, Megan Dahle, Davies, Essayli, Flora, Vince Fong, Friedman, Gallagher, Holden, Hoover, Lackey, Mathis, Jim Patterson, Joe Patterson, Luz Rivas, Sanchez, Ta, Villapudua, Waldron, Wallis
SENATE FLOOR: 31-9-0
YES: Allen, Archuleta, Ashby, Atkins, Becker, Blakespear, Bradford, Caballero, Cortese, Dodd, Durazo, Eggman, Glazer, Gonzalez, Hurtado, Laird, Limón, McGuire, Menjivar, Min, Newman, Padilla, Portantino, Roth, Rubio, Skinner, Smallwood-Cuevas, Stern, Umberg, Wahab, Wiener
NO: Alvarado-Gil, Dahle, Grove, Jones, Nguyen, Niello, Ochoa Bogh, Seyarto, Wilk
UPDATED
VERSION: August 23, 2024
CONSULTANT: Ethan Jones / ELECTIONS / (916) 319-2094 FN: 0004820
SENATE RULES COMMITTEE Office of Senate Floor Analyses (916) 651-1520 Fax: (916) 327-4478 | AB 2655 |
THIRD READING
Bill No: AB 2655
Author: Berman (D) and Pellerin (D), et al.
Amended: 8/23/24 in Senate
Vote: 21
SENATE ELECTIONS & C.A. COMMITTEE: 6-1, 6/18/24
AYES: Blakespear, Allen, Menjivar, Newman, Portantino, Umberg
NOES: Nguyen
SENATE JUDICIARY COMMITTEE: 9-2, 7/2/24
AYES: Umberg, Allen, Ashby, Caballero, Durazo, Laird, Roth, Stern, Wahab
NOES: Wilk, Niello
SENATE APPROPRIATIONS COMMITTEE: 5-2, 8/15/24
AYES: Caballero, Ashby, Becker, Bradford, Wahab
NOES: Jones, Seyarto
ASSEMBLY FLOOR: 56-1, 5/22/24 - See last page for vote
SUBJECT: Defending Democracy from Deepfake Deception Act of 2024
SOURCE: California Initiative for Technology & Democracy
DIGEST:
This bill requires an online platform with at least one million California users to develop and implement procedures to identify and remove materially deceptive content if certain conditions are met.
Senate Floor Amendments of 8/23/24 remove local offices from the provisions of the bill, narrow the definition of an “elections official,” modify the duties of large online platforms to identify and remove materially deceptive content, and specify that broadcasting stations are exempt from the provisions of the bill if there is an acknowledgement that the materially deceptive content does not represent any actual event. The amendments also contain technical and other clarifications. Finally, the amendments address chaptering issues with AB 2839 (Pellerin, 2024).
ANALYSIS:
Existing law:
- Prohibits anyone from, until January 1, 2027, distributing within 60 days of an election materially deceptive audio or visual media of a candidate with the intent to injure the candidate’s reputation or to deceive a voter into voting for or against the candidate.
- Prohibits anyone from, beginning January 1, 2027, producing, distributing, publishing, or broadcasting campaign material that contains a superimposed image of a candidate unless the campaign material includes a disclaimer that the picture is an inaccurate representation of fact.
This bill:
- Requires any large online platform (platform) using state-of-the-art techniques to identify and remove materially deceptive content related to elections in California if:
- The content is posted between 120 days before the election and through Election Day (or through the 60th day after the election in the case of content that depicts or pertains to elections officials), reported, and the materially deceptive content is any of the following:
- A candidate for elective office portrayed as doing or saying something they did not do or say and is reasonably likely to harm the reputation or electoral prospects of a candidate.
- An elections official portrayed as doing or saying something in connection with their elections-related duties they did not do or say and is reasonably likely to falsely undermine confidence in the outcome of one or more election contests.
- An elected official portrayed as doing or saying something that influences an election in California that they did not do or say and is reasonably likely to falsely undermine confidence in the outcome of one or more election contests.
- The platform knows or acts with reckless disregard for the fact that the content meets the requirements prescribed by this bill.
- Does not contain a disclosure stating that the media has been manipulated.
- The content is posted between 120 days before the election and through Election Day (or through the 60th day after the election in the case of content that depicts or pertains to elections officials), reported, and the materially deceptive content is any of the following:
- Requires platforms to develop and implement procedures for identifying and labeling content if the content is reported, posted outside of the time period described in (1) or appears within an advertisement or election communication not subject to (1), and the platform knows or acts with reckless disregard for that fact that the content meets the requirements of the bill. Requires platforms, upon determining the content meets the requirements of labelling and during a specified time period, to label the post, but no later than 72 hours after a report is made.
- Requires platforms to provide a way for Californians to report content that was not properly blocked or labeled on the platform. Allows a candidate, elected official, or elections official who reported content to the platform but does not receive a response in 36 hours or disagrees with the response – as well as the Attorney General (AG), a district attorney, or a city attorney – to seek injunctive or other equitable relief against a platform to compel compliance with this bill.
- Provides this bill does not apply to a regularly-published online periodical or broadcast that publishes materially deceptive content that contains a clear disclosure that the content is not an accurate representation of reality or when specified conditions are met. The bill does not apply to materially deceptive content that constitutes as satire or parody.
- Defines the following terms for the purposes of this bill:
- “Deepfake” to mean audio or visual media that is digitally created or modified such that it would falsely appear to a reasonable person to be an authentic record of the actual speech or conduct of the individual depicted in the media;
- “Materially deceptive content” to mean audio or visual media that is digitally created or modified, and that includes, but is not limited to, deepfakes and the output of chatbots, such that it would falsely appear to a reasonable person to be an authentic record of the content depicted in the media; and
- “Large online platform” to mean a public-facing internet website, web application, digital application, video sharing platform, advertising network, or search engine that had at least one million California users in the preceding 12 months.
- Addresses chaptering issues between this bill and AB 2839.
Background
Artificial Intelligence (AI) and Elections. On June 4, 2024, the Senate Committee on Elections and Constitutional Amendments and the Assembly Committee on Elections held a joint information hearing focusing on AI and elections.
The purpose of the hearing was to inform and assist the Legislature in making informed decisions on legislation related to AI-generated and altered content. It became evident that the ease with which people can create and spread mis- and disinformation creates a world where many people may have trouble determining what is fact and what is fiction. The development of increasingly advanced AI tools has made once time-consuming activities much easier to complete, while also enabling the completion of tasks that are otherwise too complex for humans to tackle alone.
State Action. In 2018, the Legislature approved and Governor Brown signed AB 3075 (Berman, Chapter 241, Statutes of 2018) to establish the Office of Elections Cybersecurity (OEC) in the Secretary of State’s (SOS) office. The OEC has two primary missions. First, it is responsible for coordinating efforts between the SOS and local elections officials to reduce the likelihood and severity of cyber incidents that could interfere with the security or integrity of elections in California. The OEC is also tasked with monitoring and counteracting false or misleading information regarding the electoral process that is published online or on other platforms that may suppress voter participation, cause confusion, or disrupt the ability to ensure a secure election. According to the OEC’s website, the office serves California with the sole purpose of keeping every Californian’s vote safe from online interference, especially the spread of mis- and disinformation.
In 2019, the Legislature approved and Governor Newsom signed AB 730 (Berman, Chapter 493, Statutes of 2019). AB 730 sought to address concerns that deepfake technology could be used to spread misinformation in political campaigns. Legislative analyses of AB 730 described “deepfake technology” as software capable of producing a realistic looking video of someone saying or doing something they did not actually say or do.
AB 730 prohibits anyone from distributing deceptive audio or visual media with actual malice and the intent to injure a candidate’s reputation or to deceive a voter, unless the media includes a disclaimer that it has been manipulated. AB 730 does not apply exclusively to deepfakes, but also to any intentional manipulation of audio or visual images where a reasonable person would be misled into believing it was authentic. Notably, AB 730 focused on materially deceptive representations of candidates, and not on deceptive media of other aspects of the electoral process. AB 730 included a January 1, 2023 sunset date, but in 2022, the Legislature approved AB 972 (Berman, Chapter 745, Statutes of 2022) to extend the sunset date to January 1, 2027.
Comments
- According to the Author: “AB 2655 will ensure that online platforms restrict the spread of election-related deceptive deepfakes meant to prevent voters from voting or to deceive them based on fraudulent content. Deepfakes are a powerful and dangerous tool in the arsenal of those that want to wage disinformation campaigns, and they have the potential to wreak havoc on our democracy by attributing speech and conduct to a person that is false or that never happened. Advances in technology make it easy for practically anyone to generate this deceptive content, making it that much more important that we identify and restrict its spread before it has the chance to deceive voters and undermine our democracy.”
- First Amendment Considerations. The First Amendment to the United States (US) Constitution provides in relevant part “Congress shall make no law…abridging the freedom of speech…” Similarly, Section 2 of Article I of the California Constitution provides in relevant part “Every person may freely speak, write, and publish his or her sentiments on all subjects, being responsible for the abuse of this right. A law may not restrain or abridge liberty of speech or press.”
This bill seeks to regulate the distribution by online platforms of media containing intentionally manipulated images, audio, or video related to candidates, elections officials, elected officials, and election materials and equipment under certain circumstances. A question could be raised about whether this bill is consistent with the right to freedom of speech that is guaranteed by the US and California Constitutions. The US Supreme Court has ruled that even false statements are protected by the First Amendment (United States v. Alvarez (2012), 567 U.S. 709). When a law burdens core political speech, the restrictions on speech generally must be “narrowly tailored to serve an overriding state interest,” McIntyre v. Ohio Elections Commission (1995, 514 US 334).
This bill targets deceptive content that could undermine the public’s trust in elections, prevent voters from voting, and distort the electoral process. The US Supreme Court generally has found the protection of the integrity of elections is an overriding (or compelling) government interest (Id. at 349; Burson v. Freeman (1992) 504 U.S. 191, 199). A challenge of this bill on First Amendment grounds would likely hinge on whether the court finds this bill’s provisions to be narrowly tailored.
Related/Prior Legislation
AB 2355 (W. Carrillo, 2024) requires a campaign committee that creates, originally publishes, or originally distributes a political advertisement to include a disclosure stating that the audio, image, or video was generated or substantially altered using AI.
AB 2839 (Pellerin, 2024) prohibits the distribution of campaign advertisements and other election communications that contain media that has been digitally altered in a deceptive way.
FISCAL EFFECT: Appropriation: No Fiscal Com.: Yes Local: No
According to the Senate Appropriations Committee:
- The Department of Justice (DOJ) indicates that it would incur costs of $911,000 in 2024-25, $1.4 million each in 2025-26 and 2026-27, $1.2 million in 2027-28, and $1 million annually thereafter, to implement the provisions of the bill (General Fund).
- By authorizing a claim by specified parties against a large online platform for failing to block or label specified content, this bill could result in an increased number of civil actions. Consequently, the bill could result in potentially significant cost pressures to the courts; the magnitude is unknown (Trial Court Trust Fund (TCTF)). The specific number of new actions that could be filed under the bill also is unknown; however, it generally costs about $1,000 to operate a courtroom for one hour. Courts are not funded on the basis of workload, and increased pressure on TCTF may create a need for increased funding for courts from the General Fund. The enacted 2024-25 budget includes $37 million in ongoing support from the General Fund to continue to backfill TCTF for revenue declines.
SUPPORT: (Verified 8/26/24)
California Initiative for Technology & Democracy (source)
American Federation of State, County and Municipal Employees
Bay Rising
California Environmental Voters
California Voter Foundation
Center for Countering Digital Hate
Chinese Progressive Association
Disability Rights California
Hmong Innovating Politics
League of Women Voters in California
MOVE the Valley
NextGen California
Partnership for the Advancement of New Americans
SEIU California
TechEquity Action
Youth Power Project
OPPOSITION: (Verified 8/26/24)
ACLU California Action
California Chamber of Commerce
Computer and Communications Industry
Electronic Frontier Foundation
First Amendment Coalition
NetChoice
Oakland Privacy
Software and Information Industry Association
TECHNET
ASSEMBLY FLOOR: 56-1, 5/22/24
AYES: Addis, Aguiar-Curry, Alvarez, Arambula, Bains, Bauer-Kahan, Bennett, Berman, Boerner, Bonta, Bryan, Juan Carrillo, Wendy Carrillo, Connolly, Mike Fong, Gabriel, Garcia, Gipson, Grayson, Haney, Hart, Irwin, Jackson, Jones-Sawyer, Kalra, Lee, Low, Lowenthal, Maienschein, McCarty, McKinnor, Muratsuchi, Stephanie Nguyen, Ortega, Pacheco, Papan, Pellerin, Petrie-Norris, Quirk-Silva, Ramos, Rendon, Reyes, Rodriguez, Blanca Rubio, Santiago, Schiavo, Soria, Ting, Valencia, Ward, Weber, Wicks, Wilson, Wood, Zbur, Robert Rivas
NOES: Dixon
NO VOTE RECORDED: Alanis, Calderon, Cervantes, Chen, Megan Dahle, Davies, Essayli, Flora, Vince Fong, Friedman, Gallagher, Holden, Hoover, Lackey, Mathis, Jim Patterson, Joe Patterson, Luz Rivas, Sanchez, Ta, Villapudua, Waldron, Wallis
Prepared by: Scott Matsumoto / E. & C.A. / (916) 651-4106
8/26/24 17:01:21
**** END ****
SENATE RULES COMMITTEE Office of Senate Floor Analyses (916) 651-1520 Fax: (916) 327-4478 | AB 2655 |
THIRD READING
Bill No: AB 2655
Author: Berman (D) and Pellerin (D), et al.
Amended: 8/15/24 in Senate
Vote: 21
SENATE ELECTIONS & C.A. COMMITTEE: 6-1, 6/18/24
AYES: Blakespear, Allen, Menjivar, Newman, Portantino, Umberg
NOES: Nguyen
SENATE JUDICIARY COMMITTEE: 9-2, 7/2/24
AYES: Umberg, Allen, Ashby, Caballero, Durazo, Laird, Roth, Stern, Wahab
NOES: Wilk, Niello
SENATE APPROPRIATIONS COMMITTEE: 5-2, 8/15/24
AYES: Caballero, Ashby, Becker, Bradford, Wahab
NOES: Jones, Seyarto
ASSEMBLY FLOOR: 56-1, 5/22/24 - See last page for vote
SUBJECT: Defending Democracy from Deepfake Deception Act of 2024
SOURCE: California Initiative for Technology & Democracy
DIGEST:
This bill requires an online platform with at least one million California users to block, prevent the posting or spread of, or to label any elections-related content deemed to be materially deceptive if certain conditions are met.
ANALYSIS:
Existing law:
- Prohibits anyone from, until January 1, 2027, distributing within 60 days of an election materially deceptive audio or visual media of a candidate with the intent to injure the candidate’s reputation or to deceive a voter into voting for or against the candidate.
- Prohibits anyone from, beginning January 1, 2027, producing, distributing, publishing, or broadcasting campaign material that contains a superimposed image of a candidate unless the campaign material includes a disclaimer that the picture is an inaccurate representation of fact.
This bill:
- Requires any large online platform (platform) using state-of-the-art, best available tools to detect materially deceptive content, to block and prevent the posting or sending of materially deceptive content related to elections in California if:
- The content is posted between 120 days before the election and through Election Day (or through the 60th day after the election in the case of content that depicts or pertains to elections officials) if the platform knows or should know the content portrays any of the following:
- A candidate for elective office portrayed as doing or saying something they did not do or say and is reasonably likely to harm the reputation or electoral prospects of a candidate.
- An elections official portrayed as doing or saying something in connection with their elections-related duties they did not do or say and is reasonably likely to falsely undermine confidence in the outcome of one or more election contests.
- An elected official portrayed as doing or saying something that influences an election in California that they did not do or say and is reasonably likely to falsely undermine confidence in the outcome of one or more election contests.
- The person or entity who created the materially deceptive content did so knowing it was false or with reckless disregard for the truth. There shall be a rebuttable presumption the person or entity knew the materially deceptive content was false or acted with reckless disregard for the truth if the content would cause a reasonable person to have a fundamentally different understanding or impression of the content than the person would have if hearing or seeing an authentic version of the content.
- The content is posted between 120 days before the election and through Election Day (or through the 60th day after the election in the case of content that depicts or pertains to elections officials) if the platform knows or should know the content portrays any of the following:
- Requires platforms to develop and implement procedures for labeling content as inauthentic, fake, or false if the content is posted outside of the time period described in (1) or appears within an advertisement or election communication not subject to (1). This would apply in cases where the person or entity created the content knowing it was false, and the platform knew or should have known that the content was suppose to meet the labeling requirements prescribed by this bill.
- Requires platforms to provide a way for Californians to report content that was not properly blocked or labeled to the platform. Allows a candidate, elected official, or elections official who reported content to the platform but does not receive a response in 36 hours or disagrees with the response – as well as the Attorney General (AG), a district attorney, or a city attorney – to seek injunctive or other equitable relief against a platform to compel compliance with this bill.
- Provides this bill does not apply to satire or parody or to a regularly-published online periodical that publishes materially deceptive content that contains a clear disclosure that the content is not an accurate representation of reality.
- Defines the following terms for the purposes of this bill:
- “Deepfake” to mean audio or visual media that is digitally created or modified such that it would falsely appear to a reasonable person to be an authentic record of the actual speech or conduct of the individual depicted in the media;
- “Materially deceptive content” to mean audio or visual media that is digitally created or modified, and that includes, but is not limited to, deepfakes and chatbots, such that it would falsely appear to a reasonable person to be an authentic record of the content depicted in the media; and
- “Large online platform” to mean a public-facing internet website, web application, digital application, video sharing platform, advertising network, or search engine that had at least one million California users in the preceding 12 months.
Background
Artificial Intelligence (AI) and Elections. On June 4, 2024, the Senate Elections and Constitutional Amendments Committee and the Assembly Committee on Elections held a joint information hearing focusing on AI and elections.
The purpose of the hearing was to inform and assist the Legislature in making informed decisions on legislation related to AI-generated and altered content. It became evident that the ease with which people can create and spread mis- and disinformation creates a world where many people may have trouble determining what is fact and what is fiction. The development of increasingly advanced AI tools has made once time-consuming activities much easier to complete, while also enabling the completion of tasks that are otherwise too complex for humans to tackle alone.
State Action. In 2018, the Legislature approved and Governor Brown signed AB 3075 (Berman, Chapter 241, Statutes of 2018) to establish the Office of Elections Cybersecurity (OEC) in the Secretary of State’s (SOS) office. The OEC has two primary missions. First, it is responsible for coordinating efforts between the SOS and local elections officials to reduce the likelihood and severity of cyber incidents that could interfere with the security or integrity of elections in California. The OEC is also tasked with monitoring and counteracting false or misleading information regarding the electoral process that is published online or on other platforms that may suppress voter participation, cause confusion, or disrupt the ability to ensure a secure election. According to the OEC’s website, the office serves California with the sole purpose of keeping every Californian’s vote safe from online interference, especially the spread of mis- and disinformation.
In 2019, the Legislature approved and Governor Newsom signed AB 730 (Berman, Chapter 493, Statutes of 2019). AB 730 sought to address concerns that deepfake technology could be used to spread misinformation in political campaigns. Legislative analyses of AB 730 described “deepfake technology” as software capable of producing a realistic looking video of someone saying or doing something they did not actually say or do.
AB 730 prohibits anyone from distributing deceptive audio or visual media with actual malice and the intent to injure a candidate’s reputation or to deceive a voter, unless the media includes a disclaimer that it has been manipulated. AB 730 does not apply exclusively to deepfakes, but also to any intentional manipulation of audio or visual images where a reasonable person would be misled into believing it was authentic. Notably, AB 730 focused on materially deceptive representations of candidates, and not on deceptive media of other aspects of the electoral process.
AB 730 included a January 1, 2023 sunset date, but in 2022, the Legislature approved AB 972 (Berman, Chapter 745, Statutes of 2022) to extend the sunset date to January 1, 2027.
Tech Accord to Combat Deceptive Use of AI in 2024. In February 2024, 20 technology companies signed the “Tech Accord to Combat Deceptive Use of AI in 2024 Elections.” This set of commitments seeks to combat harmful AI-generated content meant to deceive voters. The signatories included Adobe, Amazon, Anthropic, Arm, ElevenLabs, Google, IBM, Inflection AI, LinkedIn, McAfee, Meta, Microsoft, Nota, OpenAI, Snap Inc., Stability AI, TikTok, Trend Micro, Truepic, and X.
The signatories committed to taking the following steps through this year:
- Develop and implement technology to mitigate risks related to deceptive AI content.
- Assess and better understand the risks presented by deceptive AI election content.
- Seek ways to detect the distribution of deceptive AI election content.
- Seek to address deceptive AI election content.
- Share best practices and explore pathways to share tools throughout the industry.
- Provide transparency to the public.
- Continue to engage with stakeholders to better understand the global risk landscape.
- Support efforts to raise public awareness regarding deceptive AI election content.
Comments
- Author’s statement. According to the author, “AB 2655 will ensure that online platforms restrict the spread of election-related deceptive deepfakes meant to prevent voters from voting or to deceive them based on fraudulent content. Deepfakes are a powerful and dangerous tool in the arsenal of those that want to wage disinformation campaigns, and they have the potential to wreak havoc on our democracy by attributing speech and conduct to a person that is false or that never happened. Advances in technology make it easy for practically anyone to generate this deceptive content, making it that much more important that we identify and restrict its spread before it has the chance to deceive voters and undermine our democracy.”
- First Amendment Considerations. The First Amendment to the United States (US) Constitution provides in relevant part “Congress shall make no law…abridging the freedom of speech…” Similarly, Section 2 of Article I of the California Constitution provides in relevant part “Every person may freely speak, write, and publish his or her sentiments on all subjects, being responsible for the abuse of this right. A law may not restrain or abridge liberty of speech or press.”
This bill seeks to regulate the distribution by online platforms of media containing intentionally manipulated images, audio, or video related to candidates, elections officials, elected officials, and election materials and equipment under certain circumstances. A question could be raised about whether this bill is consistent with the right to freedom of speech that is guaranteed by the US and California Constitutions. The US Supreme Court has ruled that even false statements are protected by the First Amendment (United States v. Alvarez (2012), 567 U.S. 709). When a law burdens core political speech, the restrictions on speech generally must be “narrowly tailored to serve an overriding state interest,” McIntyre v. Ohio Elections Commission (1995), 514 US 334.
This bill targets deceptive content that could undermine the public’s trust in elections, prevent voters from voting, and distort the electoral process. The US Supreme Court generally has found the protection of the integrity of elections is an overriding (or compelling) government interest (Id. at 349; Burson v. Freeman (1992) 504 U.S. 191, 199). A challenge of this bill on First Amendment grounds would likely hinge on whether the court finds this bill’s provisions to be narrowly tailored.
- Election Timing. This bill requires platforms to block content beginning 120 days before an election and ending as many as 60 days after the election. As a result, in counties where there are large numbers of elections (statewide, legislative, and congressional elections are held in every even-numbered year and many cities hold local elections in odd-numbered years), platforms may be blocking content almost constantly in order to ensure compliance with the provisions of this bill.
Related/Prior Legislation
AB 2355 (W. Carrillo, 2024) required a campaign committee that creates, originally publishes, or originally distributes a political advertisement to include a disclosure stating that the audio, image, or video was generated or substantially altered using AI.
AB 2839 (Pellerin, 2024) prohibited the distribution of campaign advertisements and other election communications containing materially deceptive and digitally altered or created images or audio or video files with the intent to influence an election or solicit funds for a candidate or campaign.
FISCAL EFFECT: Appropriation: No Fiscal Com.: Yes Local: No
According to the Senate Appropriations Committee:
- The Department of Justice (DOJ) indicates that it would incur costs of $911,000 in 2024-25, $1.4 million each in 2025-26 and 2026-27, $1.2 million in 2027-28, and $1 million annually thereafter, to implement the provisions of the bill (General Fund).
- By authorizing a claim by specified parties against a large online platform for failing to block or label specified content, this bill could result in an increased number of civil actions. Consequently, the bill could result in potentially significant cost pressures to the courts; the magnitude is unknown (Trial Court Trust Fund (TCTF)). The specific number of new actions that could be filed under the bill also is unknown; however, it generally costs about $1,000 to operate a courtroom for one hour. Courts are not funded on the basis of workload, and increased pressure on TCTF may create a need for increased funding for courts from the General Fund. The enacted 2024-25 budget includes $37 million in ongoing support from the General Fund to continue to backfill TCTF for revenue declines.
SUPPORT: (Verified 8/16/24)
California Initiative for Technology & Democracy (source)
American Federation of State, County and Municipal Employees
Bay Rising
California Environmental Voters
California Voter Foundation
Center for Countering Digital Hate
Chinese Progressive Association
Disability Rights California
Hmong Innovating Politics
League of Women Voters in California
MOVE (Mobilize, Organize, Empower, Vote) the Valley
NextGen California
Partnership for the Advancement of New Americans
SEIU California
TechEquity Action
Youth Power Project
OPPOSITION: (Verified 8/16/24)
ACLU California Action
California Chamber of Commerce
Computer and Communications Industry
Electronic Frontier Foundation
First Amendment Coalition
NetChoice
Oakland Privacy
Software and Information Industry Association
TECHNET
ASSEMBLY FLOOR: 56-1, 5/22/24
AYES: Addis, Aguiar-Curry, Alvarez, Arambula, Bains, Bauer-Kahan, Bennett, Berman, Boerner, Bonta, Bryan, Juan Carrillo, Wendy Carrillo, Connolly, Mike Fong, Gabriel, Garcia, Gipson, Grayson, Haney, Hart, Irwin, Jackson, Jones-Sawyer, Kalra, Lee, Low, Lowenthal, Maienschein, McCarty, McKinnor, Muratsuchi, Stephanie Nguyen, Ortega, Pacheco, Papan, Pellerin, Petrie-Norris, Quirk-Silva, Ramos, Rendon, Reyes, Rodriguez, Blanca Rubio, Santiago, Schiavo, Soria, Ting, Valencia, Ward, Weber, Wicks, Wilson, Wood, Zbur, Robert Rivas
NOES: Dixon
NO VOTE RECORDED: Alanis, Calderon, Cervantes, Chen, Megan Dahle, Davies, Essayli, Flora, Vince Fong, Friedman, Gallagher, Holden, Hoover, Lackey, Mathis, Jim Patterson, Joe Patterson, Luz Rivas, Sanchez, Ta, Villapudua, Waldron, Wallis
Prepared by: Scott Matsumoto / E. & C.A. / (916) 651-4106
8/19/24 10:55:48
**** END ****
SENATE COMMITTEE ON APPROPRIATIONS
Senator Anna Caballero, Chair
2023 - 2024 Regular Session
AB 2655 (Berman) - Defending Democracy from Deepfake Deception Act of 2024
Version: July 3, 2024 | Policy Vote: E. & C.A. 6 - 1, JUD. 9 - 2 |
Urgency: No | Mandate: No |
Hearing Date: August 15, 2024 | Consultant: Robert Ingenito |
Bill Summary:
AB 2655 would bill require an online platform with at least one million California users to block, prevent the posting or spread of, or to label any elections-related content deemed to be materially deceptive if specified conditions are met.
*********** ANALYSIS ADDENDUM – SUSPENSE FILE ***********
The following information is revised to reflect amendments
adopted by the committee on August 15, 2024
Fiscal Impact:
- The Department of Justice (DOJ) indicates that it would incur costs of $911,000 in 2024-25, $1.4 million each in 2025-26 and 2026-27, $1.2 million in 2027-28, and $1 million annually thereafter, to implement the provisions of the bill (General Fund).
- By authorizing a claim by specified parties against a large online platform for failing to block or label specified content, this bill could result in an increased number of civil actions. Consequently, the bill could result in potentially significant cost pressures to the courts; the magnitude is unknown (Trial Court Trust Fund (TCTF)). The specific number of new actions that could be filed under the bill also is unknown; however, it generally costs about $1,000 to operate a courtroom for one hour. Courts are not funded on the basis of workload, and increased pressure on TCTF may create a need for increased funding for courts from the General Fund. The enacted 2024-25 budget includes $37 million in ongoing support from the General Fund to continue to backfill TCTF for revenue declines.
Author Amendments:
Clarify that the bill would only apply to California elections and candidates seeking to be on a California ballot, remove redistricting as a covered activity, and reduce the timeframe for the labeling provision from one year to six months.
-- END --
SENATE COMMITTEE ON
Senator Anna Caballero, Chair
2023 - 2024 Regular Session
AB 2655 (Berman) - Defending Democracy from Deepfake Deception Act of 2024
Version: | Policy Vote: |
Urgency: | Mandate: |
Hearing Date: | Consultant: |
Bill Summary:
AB 2655 would bill require an online platform with at least one million California users to block, prevent the posting or spread of, or to label any elections-related content deemed to be materially deceptive if specified conditions are met.
Fiscal Impact:
- The Department of Justice (DOJ) indicates that it would incur costs of $911,000 in 2024-25, $1.4 million each in 2025-26 and 2026-27, $1.2 million in 2027-28, and $1 million annually thereafter, to implement the provisions of the bill (General Fund).
- By authorizing a claim by specified parties against a large online platform for failing to block or label specified content, this bill could result in an increased number of civil actions. Consequently, the bill could result in potentially significant cost pressures to the courts; the magnitude is unknown (Trial Court Trust Fund (TCTF)). The specific number of new actions that could be filed under the bill also is unknown; however, it generally costs about $1,000 to operate a courtroom for one hour. Courts are not funded on the basis of workload, and increased pressure on TCTF may create a need for increased funding for courts from the General Fund. The enacted 2024-25 budget includes $37 million in ongoing support from the General Fund to continue to backfill TCTF for revenue declines.
Background:
Generative AI is a type of artificial intelligence that can create new content, including text, images, code, or music, by learning from existing data. Generative AI models can produce realistic and novel artifacts that resemble the data they were trained on, but do not copy it. For example, generative AI can write a poem, draw a picture, or compose a song based on a given prompt or theme. Generative AI enables users to quickly generate new content based on a variety of inputs. Generative AI models use neural networks to identify the patterns and structures within existing data to generate new and original content.
Current law includes various provisions criminalizing deceptive tactics that undermine election integrity or interfere with voters’ ability to participate in elections. This includes laws that prohibit distribution of false and misleading information about qualifications to vote or about the days, dates, times, and places where voting may occur; prohibit the misleading use of government seals in campaign literature; and prohibit coercing or deceiving people into voting in a way that was inconsistent with the person’s intent.
In June 2024, a joint hearing was conducted by the Legislature’s two elections committees focusing on artificial intelligence (AI) and elections. The hearing’s purpose was to assist the Legislature in making informed decisions on legislation related to AI-generated and altered content. The ease with which people can create and spread misinformation and disinformation creates a world where many people may have trouble determining what is fact and what is fiction. The development of increasingly advanced AI tools has made once time-consuming activities much easier to complete, while also enabling the completion of tasks that are otherwise too complex for humans to complete on their own.
In 2018, the Legislature and Governor established the Office of Elections Cybersecurity (OEC) within SOS. OEC has two primary missions. First, it is responsible for coordinating efforts between SOS and local elections officials to reduce the likelihood and severity of cyber incidents that could interfere with the security or integrity of state elections. Second, OEC is tasked with monitoring and counteracting false or misleading information regarding the electoral process that is published online or on other platforms that may suppress voter participation, cause confusion, or disrupt the ability to ensure a secure election.
In 2019, the Legislature enacted AB 730 (Berman), which sought to address concerns that deepfake technology could be used to spread misinformation in political campaigns. AB 730 prohibits anyone from distributing deceptive audio or visual media with actual malice and the intent to injure a candidate’s reputation or to deceive a voter, unless the media includes a disclaimer that it has been manipulated. AB 730 does not apply exclusively to deepfakes; it also applies to any intentional manipulation of audio or visual images where a reasonable person would be misled into believing it was authentic. The bill focused on materially deceptive representations of candidates, and not on deceptive media of other aspects of the electoral process. AB 730’s provisions have been extended, and currently sunset on January 1, 2027.
Proposed Law:
This bill, to be known as the Defending Democracy from Deepfake Deception Act of 2024, would do the following:
- Require a large online platform, as defined, to block the posting of materially deceptive content related to elections, during specified periods before and after an election.
- Require a large online platform to label certain additional content inauthentic, fake, or false during specified periods before and after an election.
- Require a large online platform to develop procedures for California residents to report content that has not been blocked or labeled in compliance with the Act.
- Authorize candidates for elected office, elected officials, elections officials, the Attorney General, and a district attorney or city attorney to seek injunctive relief against a large online platform for noncompliance with the Act, as specified, and would assign precedence to such actions when they are filed in court.
- Exempt from its provisions a regularly published online newspaper, magazine, or other periodical of general circulation that routinely carries news and commentary of general interest, if the publication complies with specified disclosure requirements. The bill would also exempt content that is satire or parody.
Related Legislation:
- AB 2355 (W. Carrillo) would require a campaign committee that creates, originally publishes, or originally distributes a political advertisement to include a disclosure stating that the audio, image, or video was generated or substantially altered using AI. The bill is currently pending in this Committee.
- AB 2839 (Pellerin) would prohibit the distribution of campaign advertisements and other election communications containing materially deceptive and digitally altered or created images or audio or video files with the intent to influence an election or solicit funds for a candidate or campaign. The bill is currently pending in this Committee.
Staff Comments:
The $37 million in General Fund support for TCTF in 2024-25 is down sharply (over 50 percent) from previous years. The comparative figure was $74 million in 2023-24, $72 million in 2022-23, and $84 million in 2021-22.
-- END --
Senator Thomas Umberg, Chair
2023-2024 Regular Session
Bill No: AB 2655 (Author:Berman)
Version: June 11, 2024
Hearing Date: July 2, 2024
Fiscal: Yes
Urgency: No
Consultant: CK
SUBJECT
Defending Democracy from Deepfake Deception Act of 2024
DIGEST
This bill establishes the Defending Democracy from Deepfake Deception Act of 2024, which requires a large online platform to block the posting or sending of materially deceptive and digitally modified or created content related to elections, during specified periods before and after an election. It requires these platforms to label certain additional content inauthentic, fake, or false during specified periods before and after an election and to provide mechanisms to report such content.
EXECUTIVE SUMMARY
The rapid advancement of AI technology, specifically the wide-scale introduction of generative AI models, has made it drastically cheaper and easier to produce synthetic content – audio, images, text, and video recordings that are not real, but that are so realistic that they are virtually impossible to distinguish from authentic content, including so-called “deepfakes.” In the context of election campaigns, such deepfakes can be weaponized to deceive voters into thinking that a candidate said or did something which the candidate did not, or otherwise falsely call election results into question. A series of bills currently pending before this Committee attempt to address these issues by restricting or labeling AI-altered or –generated content. However, this bill specifically targets social media platforms and such materially deceptive content on their platforms, requiring platforms to block and prevent it, label it, and provide mechanisms for reporting it.
The bill is sponsored by the California Initiative for Technology & Democracy. It is supported by various organizations, including the League of Women Voters of California and Disability Rights California. It is opposed by Oakland Privacy and various industry associations, including TechNet. The bill passed out of the Senate Elections and Constitutional Amendments Committee on a 6 to 1 vote.
PROPOSED CHANGES TO THE LAW
Existing law:
- Provides that “Congress shall make no law… abridging the freedom of speech...” (U.S. Const., amend. 1.)
- Applies the First Amendment to the states through operation of the Fourteenth Amendment. (Gitlow v. New York (1925) 268 U.S. 652; NAACP v. Alabama (1925) 357 U.S. 449.)
- Provides, in federal law, that a provider or user of an interactive computer service shall not be treated as the publisher or speaker of any information provided by another information content provider. (47 U.S.C. § 230(c)(1).)
- Provides that a provider or user of an interactive computer service shall not be held liable on account of:
- any action voluntarily taken in good faith to restrict access to or availability of material that the provider or user considers to be obscene, lewd, lascivious, filthy, excessively violent, harassing, or otherwise objectionable, whether or not such material is constitutionally protected; or
- any action taken to enable or make available to information content providers or others the technical means to restrict access to such material. (47 U.S.C. § 230(c)(2).)
- Provides that no provider or user of an interactive computer service shall be treated for liability purposes as the publisher or speaker of any information provided by another information content provider. (47 U.S.C. § 230.)
- Defines “materially deceptive audio or visual media” as an image or an audio or video recording of a candidate’s appearance, speech, or conduct that has been intentionally manipulated in a manner such that both of the following conditions are met:
- The image or audio or video recording would falsely appear to a reasonable person to be authentic.
- The image or audio or video recording would cause a reasonable person to have a fundamentally different understanding or impression of the expressive content of the image or audio or video recording than that person would have if the person were hearing or seeing the unaltered, original version of the image or audio or video recording. (Elec. Code § 20010(e).)
- Prohibits a person, committee, or other entity from distributing with actual malice materially deceptive audio or visual media of a candidate with the intent to injure the candidate’s reputation or to deceive a voter into voting for or against the candidate within 60 days of an election at which a candidate for elective office will appear on the ballot, as specified and unless specified conditions are met. (Elec. Code § 20010(a).)
- Exempts audio or visual media that includes a disclosure stating: “This _____ has been manipulated.” Requires the blank in the disclosure to be filled with a term that most accurately describes the media, as specified. Requires the following disclosures for visual and audio-only media:
- For visual media, the text of the disclosure shall appear in a size that is easily readable by the average viewer and no smaller than the largest font size of other text appearing in the visual media. If the visual media does not include any other text, then the disclosure shall appear in a size that is easily readable by the average viewer. Requires, for visual media that is video, the disclosure to be displayed throughout the duration of the video.
- For audio-only media, the disclosure shall be read in the clearly spoken manner and in a pitch that can be easily heard by the average listener, at the beginning of the audio, at the end of the audio, and, if the audio is greater than two minutes in length, interspersed within the audio at intervals of not greater than two minutes each. (Elec. Code § 20010(b).)
- Permits a candidate for elective office whose voice or likeness appears in a materially deceptive audio or visual media distributed in violation of the above provisions, to seek injunctive or other equitable relief prohibiting the distribution of audio or visual media in violation. (Elec. Code § 20010(c)(1).)
- Permits a candidate for elective office whose voice or likeness appears in materially deceptive audio or visual media distributed in violation of the above provisions to bring an action for general or special damages against the person, committee, or other entity that distributed the materially deceptive audio or visual media, as specified. Requires the plaintiff to bear the burden of establishing the violation through clear and convincing evidence in any civil action alleging a violation, as specified. (Elec. Code § 21101(c)(2).)
This bill:
- Establishes the Defending Democracy from Deepfake Deception Act of 2024.
- Requires a large online platform, using state-of-the-art, best available tools to detect materially deceptive content, to develop and implement procedures for blocking and preventing, and, if the platform knows or should know that the materially deceptive content meets the requirements hereof, to block and prevent the posting or sending of any materially deceptive content, if all of the following conditions are met:
- The content is posted or sent during a period beginning 120 days before the election and through the day of the election. For content that depicts or pertains to elections officials, this period shall extend to the 60th day after the election.
- The materially deceptive content is any of the following:
- A candidate for elective office portrayed as doing or saying something that the candidate did not do or say and that is reasonably likely to harm the reputation or electoral prospects of a candidate.
- An elections official portrayed as doing or saying something in connection with the performance of their elections-related duties that the elections official did not do or say and that is reasonably likely to falsely undermine confidence in the outcome of one or more election contests.
- An elected official portrayed as doing or saying something that influences the election that the elected official did not do or say and that is reasonably likely to falsely undermine confidence in the outcome of one or more election contests.
- The person or entity who created the materially deceptive content did so knowing it was false or with reckless disregard for the truth. There shall be a rebuttable presumption that the person or entity knew the materially deceptive content was false or acted with reckless disregard for the truth if the content would cause a reasonable person to have a fundamentally different understanding or impression of the content than the person would have if hearing or seeing an authentic version of the content.
- Requires, notwithstanding the above, a large online platform to allow a candidate for elective office, during a period beginning 120 days before the election and through the day of the election, to portray themself as doing or saying something that the candidate did not do or say, but only if the digital content includes a disclosure meeting specified conditions and states the following: “This [category of content] has been manipulated.”
- Requires a large online platform, using state-of-the-art, best available tools to detect materially deceptive content to develop and implement procedures for labeling such content as inauthentic, fake, or false if all of the following conditions are met:
- The materially deceptive content is either of the following:
- Meets the standards set above, but is posted or sent outside the applicable time period.
- Appears within an advertisement or election communication and is not subject to the above.
- The person or entity who created the materially deceptive content did so knowing it was false or with reckless disregard for the truth. There shall be a rebuttable presumption that the person or entity knew the materially deceptive content was false or acted with reckless disregard for the truth if the content would cause a reasonable person to have a fundamentally different understanding or impression of the content than the person would have if hearing or seeing an authentic version of the content.
- The large online platform knows or should know that the materially deceptive content meets the requirements of this section.
- The materially deceptive content is either of the following:
- Specifies required functionality of the label above and states the labeling requirement applies during redistricting and during a period from one year before the election through election day. If the content involves elections officials, the electoral college process, the canvass of the vote, or election-related equipment or property, the time period is extended 60 days beyond the election.
- Requires a large online platform to provide an easily accessible way for California residents to report to that platform content subject to the above provisions that was not blocked or labeled as required. The online platform shall respond to the person who made the report, within 36 hours of the report, describing any action taken or not taken by the online platform with respect to the content.
- Authorizes a candidate for elective office, elected official, or elections official who has made a report and who either has not received a response within 36 hours or disagrees with the response, as well as the Attorney General or any district attorney or city attorney, to seek injunctive or other equitable relief against the online platform to compel compliance. The plaintiff shall bear the burden of establishing the violation through clear and convincing evidence. The court is required to award a prevailing plaintiff reasonable attorney’s fees and costs. Such actions are given precedence in accordance with Section 35 of the Code of Civil Procedure.
- Clarifies that it applies to materially deceptive content, regardless of the language used in the content. If the language used is not English, the required disclosure and label must appear in the language used as well as in English.
- Requires a large online platform that blocks or labels any materially deceptive content to maintain a copy of the digital content for a period of not less than five years from the election and shall make such digital content available to the Secretary of State, the Fair Political Practices Commission, and researchers, if requested.
- Exempts from the scope of the bill the following:
- A regularly published online newspaper, magazine, or other periodical of general circulation that routinely carries news and commentary of general interest, and that publishes any materially deceptive content that an online platform is required to block or label based on this chapter, if the publication contains a clear disclosure that the materially deceptive content does not accurately represent any actual event, occurrence, appearance, speech, or expressive conduct.
- Materially deceptive content that constitutes satire or parody.
- Includes findings and declarations and a severability clause.
- Defines the relevant terms, including:
- “Materially deceptive content” means audio or visual media that is digitally created or modified, and that includes, but is not limited to, deepfakes and chatbots, such that it would falsely appear to a reasonable person to be an authentic record of the content depicted in the media.
- “Large online platform” means a public-facing internet website, web application, or digital application, including a social network, video sharing platform, advertising network, or search engine that had at least 1,000,000 California users during the preceding 12 months.
COMMENTS
- Blurring reality: AI-generated content
Generative AI is a type of artificial intelligence that can create new content, including text, images, code, or music, by learning from existing data. Generative AI models can produce realistic and novel artifacts that resemble the data they were trained on, but do not copy it. For example, generative AI can write a poem, draw a picture, or compose a song based on a given prompt or theme. Generative AI enables users to quickly generate new content based on a variety of inputs. Generative AI models use neural networks to identify the patterns and structures within existing data to generate new and original content.
The world has been in awe of the powers of this generative AI since the widespread introduction of AI systems such as ChatGPT. However, the capabilities of these advanced systems leads to a blurring between reality and fiction. The Brookings Institution lays out the issue:
Over the last year, generative AI tools have made the jump from research prototype to commercial product. Generative AI models like OpenAI’s ChatGPT and Google’s Gemini can now generate realistic text and images that are often indistinguishable from human-authored content, with generative AI for audio and video not far behind. Given these advances, it’s no longer surprising to see AI-generated images of public figures go viral or AI-generated reviews and comments on digital platforms. As such, generative AI models are raising concerns about the credibility of digital content and the ease of producing harmful content going forward.
Against the backdrop of such technological advances, civil society and policymakers have taken increasing interest in ways to distinguish AI-generated content from human-authored content.[1]
One expert at the Copenhagen Institute for Future Studies estimates that should large generative-AI models run amok, up to 99 percent of the internet’s content could be AI-generated by 2025 to 2030.[2] The problematic applications are seemingly infinite, whether it be deepfakes to blackmail or shame victims, false impersonations to commit fraud, or other nefarious purposes. Infamously, in January of this year, Taylor Swift was the victim of sexually explicit, nonconsensual deepfake images using AI that were widely spread across social media platforms.[3] Perhaps more disturbingly, a trend has emerged in schools of students creating such images: “At schools across the country, people have used deepfake technology combined with real images of female students to create fraudulent images of nude bodies. The deepfake images can be produced using a cellphone.”[4] As more of the population becomes aware of the potential to realistically fake images, video, and text, some will use the skepticism that creates to challenge the authenticity of real content, a phenomena coined the “liar’s dividend.”[5]
Relevant here, AI and specifically generative AI can spread misinformation regarding elections with ease, both in California and across the world:
Artificial intelligence is supercharging the threat of election disinformation worldwide, making it easy for anyone with a smartphone and a devious imagination to create fake – but convincing – content aimed at fooling voters.
It marks a quantum leap from a few years ago, when creating phony photos, videos or audio clips required teams of people with time, technical skill and money. Now, using free and low-cost generative artificial intelligence services from companies like Google and OpenAI, anyone can create high-quality “deepfakes” with just a simple text prompt.
A wave of AI deepfakes tied to elections in Europe and Asia has coursed through social media for months, serving as a warning for more than 50 countries heading to the polls this year.
“You don’t need to look far to see some people ... being clearly confused as to whether something is real or not,” said Henry Ajder, a leading expert in generative AI based in Cambridge, England.
The question is no longer whether AI deepfakes could affect elections, but how influential they will be, said Ajder, who runs a consulting firm called Latent Space Advisory.
As the U.S. presidential race heats up, FBI Director Christopher Wray recently warned about the growing threat, saying generative AI makes it easy for “foreign adversaries to engage in malign influence.”[6]
On that last note, in February of this year, voters in New Hampshire received robocalls that are purported to have used an AI voice resembling President Joe Biden advising them against voting in the presidential primary and saving their vote for the November general election.[7] The examples are endless:
Former President Donald Trump, who is running in 2024, has shared AI-generated content with his followers on social media. A manipulated video of CNN host Anderson Cooper that Trump shared on his Truth Social platform on Friday, which distorted Cooper’s reaction to the CNN town hall this past week with Trump, was created using an AI voice-cloning tool.
A dystopian campaign ad released last month by the Republican National Committee offers another glimpse of this digitally manipulated future. The online ad, which came after President Joe Biden announced his reelection campaign, and starts with a strange, slightly warped image of Biden and the text “What if the weakest president we’ve ever had was re-elected?”
A series of AI-generated images follows: Taiwan under attack; boarded up storefronts in the United States as the economy crumbles; soldiers and armored military vehicles patrolling local streets as tattooed criminals and waves of immigrants create panic.
“An AI-generated look into the country’s possible future if Joe Biden is re-elected in 2024,” reads the ad’s description from the RNC.
The RNC acknowledged its use of AI, but others, including nefarious political campaigns and foreign adversaries, will not, said Petko Stoyanov, global chief technology officer at Forcepoint, a cybersecurity company based in Austin, Texas. Stoyanov predicted that groups looking to meddle with U.S. democracy will employ AI and synthetic media as a way to erode trust.[8]
Legislatures across the country are pushing legislation that would address this looming threat.
- Materially deceptive content in political advertisements
According to the author:
AB 2655 will ensure that online platforms restrict the spread of election-related deceptive deepfakes meant to prevent voters from voting or to deceive them based on fraudulent content. Deepfakes are a powerful and dangerous tool in the arsenal of those that want to wage disinformation campaigns, and they have the potential to wreak havoc on our democracy by attributing speech and conduct to a person that is false or that never happened. Advances in technology make it easy for practically anyone to generate this deceptive content, making it that much more important that we identify and restrict its spread before it has the chance to deceive voters and undermine our democracy.
Unlike existing law or other bills pending before this Committee in this area, this bill seeks to place responsibility on large online platforms with regard to “materially deceptive content” regarding elections, placing a series of obligations on them. The bill defines “materially deceptive content” as audio or visual media that is digitally created or modified, and that includes, but is not limited to, deepfakes and chatbots, such that it would falsely appear to a reasonable person to be an authentic record of the content depicted in the media. “Large online platform” means a public-facing internet website, web application, or digital application, including a social network, video sharing platform, advertising network, or search engine that had at least 1,000,000 California users during the preceding 12 months.[9]
- Preventing and blocking materially deceptive content
The bill requires platforms to develop and implement procedures for blocking and preventing, and, to block and prevent the posting or sending of, materially deceptive content, if the platform knows or should know that the materially deceptive content meets the requirements of the bill and certain conditions are met.
The materially deceptive content must portray one of the following. First is content portraying a candidate for elective office as doing or saying something they did not do or say and that is reasonably likely to harm the reputation or electoral prospects of a candidate. Or it must portray an elected official as doing or saying something that influences the election or an elections official as doing or saying something in connection with the performance of their elections-related duties that the official did not do or say and that is reasonably likely to falsely undermine confidence in the outcome of one or more election contests.
Second, the content must be posted or sent during a period beginning 120 days before the election and through the day of the election. For content that depicts or pertains to elections officials, this period shall extend to the 60th day after the election.
Finally, to trigger the requirement for platforms to block and prevent the content, the person or entity who created the content must have done so knowing it was false or with reckless disregard for the truth.
In any ensuing litigation, the bill establishes a rebuttable presumption that the person or entity knew the materially deceptive content was false or acted with reckless disregard for the truth if the content would cause a reasonable person to have a fundamentally different understanding or impression of the content than the person would have if hearing or seeing an authentic version of the content.
Carved out of this obligation is digital content that a candidate for elective office posts or shares that portrays themself as doing or saying something that they did not do, so long as there is a disclosure indicating that the content has been manipulated that meets certain specifications. However, not only is this content not subject to the requirement for a platform to block and prevent, but platforms are required to host such content and cannot prevent such material, with no exception for whether it violates the platforms terms and services. This provision similarly applies during the period starting 120 days before an election through election day.
- Labeling materially deceptive content
Large online platforms are also required to develop and implement procedures for labeling materially deceptive content as inauthentic, fake, or false. This applies to such content that meets the requirements from the above section but falls outside of the specified time range or that does not meet the requirements but that appears within an advertisement or election communication. The person or entity who created the materially deceptive content must have also done so knowing it was false or with reckless disregard for the truth. The rebuttable presumption again applies. And, for this obligation to trigger, the large online platform must have known or should have known that the materially deceptive content meets these requirements.
The label must allow users to click or tap on it and to inspect all available provenance data about the content in an easy-to-understand format. The labeling requirement applies during specified time periods: (1) the period starting one year before the election through election day; (2) that period through the 60th day after the election, if it depicts or pertains to elections officials, the electoral college process, a voting machine, ballot, voting site, or other property or equipment related to an election, or the canvass of the vote; and (3) during a governmental process related to redistricting, as provided.
- Retention requirement
Content that either requires such a label or that must be blocked and prevented from being posted and shared must be retained by the platform for not less than five years from the relevant election. Platforms must share the content, upon request, with the Secretary of State, the Fair Political Practices Commission, and researchers.
- Reporting mechanism
Lastly, the bill requires a large online platform to provide an easily accessible way for California residents to report to that platform content subject to the above provisions that was not blocked or labeled as required. The online platform shall respond to the person who made the report, within 36 hours, describing any action taken or not taken by the online platform.
- Enforcement
The bill provides standing to candidates, elected officials, or elections officials who have made reports but who have either not received a timely response or who disagree with it to bring an action for injunctive and other equitable relief. The Attorney General, district attorneys, and city attorneys are also so authorized. A prevailing plaintiff is entitled to attorneys’ fees and costs. Such actions are given precedence in the courts.
However, plaintiffs in such actions are required to establish a violation by clear and convincing evidence.
- Legal concerns
Concerns have been raised about whether the bill runs afoul of federal statutory and constitutional law. Namely, whether the bill is preempted by Section 230 of the Communications Decency Act, 47 U.S.C. § 230 and the First Amendment to the United States Constitution.
- Section 230
Section 230 does not apply to the users of social media (or the internet generally), but rather applies to the platforms themselves. In the early 1990s, prior to the enactment of Section 230, two trial court orders—one in the United States District Court for the Southern District of New York, and New York state court—suggested that internet platforms could be held liable for allegedly defamatory statements made by the platforms’ users if the platforms engaged in any sort of content moderation (e.g., filtering out offensive material).[10] In response, two federal legislators and members of the burgeoning internet industry crafted a law that would give internet platforms immunity from liability for users’ statements, even if they might have reason to know that statements might be false, defamatory, or otherwise actionable.[11] The result—Section 230—was relatively uncontroversial at the time, in part because of the relative novelty of the internet and in part because Section 230 was incorporated into a much more controversial internet regulation scheme that was the subject of greater debate.[12]
The crux of Section 230 is laid out in two parts. The first provides that “[n]o provider or user of an interactive computer service shall be treated as the publisher or speaker of any information provided by another information content provider.”[13] The second provides a safe harbor for content moderation, by stating that no provider or user shall be held liable because of good-faith efforts to restrict access to material that is “obscene, lewd, lascivious, filthy, excessively violent, harassing, or otherwise objectionable, whether or not such material is constitutionally protected.”[14]
Together, these two provisions give platforms immunity from any civil or criminal liability that could be incurred by user statements, while explicitly authorizing platforms to engage in their own content moderation without risking that immunity. Section 230 specifies that “[n]o cause of action may be brought and no liability may be imposed under any State law that is inconsistent with this section.”[15] Courts have applied Section 230 in a vast range of cases to immunize internet platforms from “virtually all suits arising from third-party content.”[16]
This bill provides for the potential liability of platforms for failing to block and prevent certain content from being posted or shared by users. If a user’s content qualifies as materially deceptive, and other conditions are met, then the platform can be held liable for it.
Supporters point to the fact that monetary damages are not available and injunctive relief is essentially the only remedy available. The bill does allow for attorneys’ fees and costs, which could be considered the type of liability that triggers Section 230’s preemptive effect. The author has agreed to amendments that remove these remedies, leaving only injunctive relief. While courts, including the California Supreme Court, have found Section 230 immunity can extend to liability for solely injunctive relief, it is far from settled law in the country.[17]
In addition, the bill provide that if the platform engages in content moderation that restricts access to a candidate’s deceptive portrayal of themselves (with the required disclosure and during the applicable time period), the platform can be held liable for that content moderation decision, regardless of the justification. As discussed below, the author has agreed to an amendment that removes this provision.
Ultimately, the bill is likely to face challenge on these grounds but these amendments work toward insulating the bill from such a challenge.
- First Amendment
The First Amendment, as applied to the states through the Fourteenth Amendment, prohibits Congress or the states from passing any law “abridging the freedom of speech.”[18] “[A]s a general matter, the First Amendment means that government has no power to restrict expression because of its message, its ideas, its subject matter, or its content.”[19] However, while the amendment is written in absolute terms, the courts have created a handful of narrow exceptions to the First Amendment’s protections, including “true threats,”[20] “fighting words,”[21] incitement to imminent lawless action,[22] defamation,[23] and obscenity.[24] Moreover, the First Amendment not only protects the right to speak, as a logical corollary it protects the “right to receive information and ideas.”[25] Expression on the internet is given the same measure of protection granted to in-person speech or statements published in a physical medium.[26]
“Laws that burden political speech are subject to strict scrutiny, which requires the Government to prove that the restriction furthers a compelling interest and is narrowly tailored to achieve that interest.”[27] Content-based restrictions subject to strict scrutiny are “presumptively unconstitutional.”[28] California courts have been clear that political expression in the context of campaigns of any manner should be given wide latitude:
Hyperbole, distortion, invective, and tirades are as much a part of American politics as kissing babies and distributing bumper stickers and pot holders. Political mischief has been part of the American political scene since, at least, 1800.
In any election, public calumny of candidates is all too common. “Once an individual decides to enter the political wars, he subjects himself to this kind of treatment. . . . [D]eeply ingrained in our political history is a tradition of free-wheeling, irresponsible, bare knuckled, Pier 6, political brawls.” To endure the animadversion, brickbats and skullduggery of a given campaign, a politician must be possessed with the skin of a rhinoceros. Harry Truman cautioned would-be solons with sage advice about the heat in the kitchen.
Nevertheless, political campaigns are one of the most exhilarating phenomena of our democracy. They bring out the best and the worst in us. They allow candidates and their supporters to express the most noble and, lamentably, the most vile sentiments. They can be fractious and unruly, but what they yield is invaluable: an opportunity to criticize and comment upon government and the issues of the day.
The candidate who finds himself or herself the victim of misconduct is not without a remedy. Those campaign tactics which go beyond the pale are sanctionable under FPPC laws.
It is abhorrent that many political campaigns are mean-spirited affairs that shower the voters with invective instead of insight. The elimination from political campaigns of opprobrium, deception and exaggeration would shed more light on the substantive issues, resulting in a more informed electorate. It would encourage more able people to seek public office. But to ensure the preservation of a citizen’s right of free expression, we must allow wide latitude.[29]
The United States Supreme Court has emphasized the extraordinary protection afforded to political speech:
Discussion of public issues and debate on the qualifications of candidates are integral to the operation of the system of government established by our Constitution. The First Amendment affords the broadest protection to such political expression in order “to assure [the] unfettered interchange of ideas for the bringing about of political and social changes desired by the people.” Although First Amendment protections are not confined to “the exposition of ideas,” “there is practically universal agreement that a major purpose of that Amendment was to protect the free discussion of governmental affairs,… of course includ[ing] discussions of candidates….” This no more than reflects our “profound national commitment to the principle that debate on public issues should be uninhibited, robust, and wide-open.” In a republic where the people are sovereign, the ability of the citizenry to make informed choices among candidates for office is essential, for the identities of those who are elected will inevitably shape the course that we follow as a nation. As the Court observed in Monitor Patriot Co. v. Roy, 401 U.S. 265, 272 (1971), “it can hardly be doubted that the constitutional guarantee has its fullest and most urgent application precisely to the conduct of campaigns for political office.”[30]
This protection does not end where the truth of the speech does. “Although false statements of fact, by themselves, have no constitutional value, constitutional protection is not withheld from all such statements.”[31] For instance, in the seminal opinion in New York Times Co. v. Sullivan (1964) 376 U.S. 254, 279-80, the court found the Constitution requires a rule that “prohibits a public official from recovering damages for a defamatory falsehood relating to his official conduct unless he proves that the statement was made with ‘actual malice’ -- that is, with knowledge that it was false or with reckless disregard of whether it was false or not. The Supreme Court has expounded on this principle, providing nuance based on the knowledge of the speaker:
Truth may not be the subject of either civil or criminal sanctions where discussion of public affairs is concerned. And since “. . . erroneous statement is inevitable in free debate, and . . . it must be protected if the freedoms of expression are to have the ‘breathing space’ that they ‘need . . . to survive’ . . . ,” only those false statements made with the high degree of awareness of their probable falsity demanded by New York Times may be the subject of either civil or criminal sanctions. For speech concerning public affairs is more than self-expression; it is the essence of self-government. The First and Fourteenth Amendments embody our “profound national commitment to the principle that debate on public issues should be uninhibited, robust, and wide-open, and that it may well include vehement, caustic, and sometimes unpleasantly sharp attacks on government and public officials.”
The use of calculated falsehood, however, would put a different cast on the constitutional question. Although honest utterance, even if inaccurate, may further the fruitful exercise of the right of free speech, it does not follow that the lie, knowingly and deliberately published about a public official, should enjoy a like immunity. At the time the First Amendment was adopted, as today, there were those unscrupulous enough and skillful enough to use the deliberate or reckless falsehood as an effective political tool to unseat the public servant or even topple an administration. That speech is used as a tool for political ends does not automatically bring it under the protective mantle of the Constitution. For the use of the known lie as a tool is at once at odds with the premises of democratic government and with the orderly manner in which economic, social, or political change is to be effected. Calculated falsehood falls into that class of utterances which “are no essential part of any exposition of ideas, and are of such slight social value as a step to truth that any benefit that may be derived from them is clearly outweighed by the social interest in order and morality. . . .” Hence the knowingly false statement and the false statement made with reckless disregard of the truth, do not enjoy constitutional protection.[32]
As stated, a restriction can survive strict scrutiny only if it uses the least-restrictive means available to achieve a compelling government purpose.[33] This bill implicates both the right to speak about elections, as well as the right to receive information regarding them. The bill is aimed at protecting the integrity of our elections, arguably a clearly compelling governmental interest. The question is whether the bill sufficiently tailors its provisions to effectuating that goal.
The bill seeks to prevent “materially deceptive content,” which is audio or visual media that is digitally created or modified, and that includes, but is not limited to, deepfakes and chatbots, such that it would falsely appear to a reasonable person to be an authentic record of the content depicted in the media, when it portrays candidates or elections officials doing or saying something they did not do or say. However, it does not target the person creating, posting, or sharing such content, but the platforms that host it. The bill attempts to tailor itself to the boundaries sketched out above. For instance, it imposes liability on the platforms only where they knew or should have known the content qualified as “materially deceptive content.” However, this falls short of the malice standard set forth in Sullivan, establishing something akin to a negligence standard instead.
The bill does impose a malice requirement but on the person or entity who created the content, requiring that they created it knowing it was false or with reckless disregard for the truth. However, liability is not imposed on the creator, nor even the one posting or sharing the content, but the social media platform allowing it on their platform. Further undercutting this element, there is a rebuttable presumption that the person who created it acted with malice if the content causes “a reasonable person to have a fundamentally different understanding or impression of the content than the person would have if hearing or seeing an authentic version of the content.” Therefore, the bill puts the onus on the platform to establish that the creator of the content, a person or entity the platform may not even have a relationship with or know, did not act with malice. Many of the relevant cases stress that the level of burden placed on a defendant to defend their political speech is a factor to consider. For instance, the following was stated in Sullivan:
A rule compelling the critic of official conduct to guarantee the truth of all his factual assertions -- and to do so on pain of libel judgments virtually unlimited in amount -- leads to a comparable “self-censorship.” Allowance of the defense of truth, with the burden of proving it on the defendant, does not mean that only false speech will be deterred. Even courts accepting this defense as an adequate safeguard have recognized the difficulties of adducing legal proofs that the alleged libel was true in all its factual particulars.[34]
While the plaintiff is required to prove their case by clear and convincing evidence, the standards above place a burden on platforms to establish facts potentially well outside their bounds of knowing.
The California Initiative for Technology & Democracy (CITED), the sponsor of the bill, argues the case:
AB 2655's approach is narrowly tailored and does not extend the law to hot button controversies or inflammatory claims – it does not ask social media platforms to adjudicate controversial opinions post by post. It simply stops the use of obviously, demonstrably untrue and provably false content meant to impermissibly influence our elections at peak election times. It is therefore respectful of the protections of the First Amendment and avoids concerns based on Section 230 of the Communications Decency Act.
Writing in opposition, ACLU California Action assesses the issue:
Digitally modified content or content created using artificial intelligence (AI) tools is also entitled to [First Amendment] protections, unless the content falls within recognized First Amendment exceptions such as libel or fraud. The “novelty of deepfake technology and the speed with which it is improving” do not justify relaxing the stringent protections afforded to political speech by the First Amendment. The Supreme Court has held that “whatever the challenges of applying the Constitution to ever-advancing technology, ‘the basic principles of freedom of speech and the press, like the First Amendment’s command, do not vary’ when a new and different medium for communication appears.”
The law has long made clear that the First Amendment was intended to create a wide berth for political speech because it is the core of our democracy. The First Amendment provides robust protection for speech of all kinds. Speech that is false, confusing, or which presents content that some find abhorrent, nevertheless maintains its constitutional protections as a driver of free discourse. This remains so no matter what the technology used to speak. Unfortunately, the provisions of AB 2655 as currently drafted threaten to intrude on those rights and deter that vital speech.
In response to these concerns, the author has agreed to amendments that remove the provision that applies this malice standard to the creators of the content and instead more closely hews the platform’s basis for liability to the malice standard, holding the large online platform liable only if it knows that the materially deceptive content meets the requirements of the bill or acts with a reckless disregard for the truth.[35]
As the bill also requires platforms to allow certain potentially misleading content to be posted, the bill could be found to implicate the First Amendment rights of platforms in their editorial discretion. Two laws in Florida and Texas that similarly seek to prevent platforms from taking down certain content have been challenged. The consolidated case has been argued before the United States Supreme Court and an opinion is forthcoming. The 11th Circuit Court of Appeals laid out its assessment of the First Amendment implications of such laws:
Social-media platforms like Facebook, Twitter, YouTube, and TikTok are private companies with First Amendment rights and when they (like other entities) "disclos[e]," "publish[]," or "disseminat[e]" information, they engage in "speech within the meaning of the First Amendment." More particularly, when a platform removes or deprioritizes a user or post, it makes a judgment about whether and to what extent it will publish information to its users—a judgment rooted in the platform's own views about the sorts of content and viewpoints that are valuable and appropriate for dissemination on its site. As the officials who sponsored and signed S.B. 7072 [the challenged Florida law] recognized when alleging that "Big Tech" companies harbor a "leftist" bias against "conservative" perspectives, the companies that operate social-media platforms express themselves (for better or worse) through their content-moderation decisions. When a platform selectively removes what it perceives to be incendiary political rhetoric, pornographic content, or public-health misinformation, it conveys a message and thereby engages in "speech" within the meaning of the First Amendment.
Laws that restrict platforms' ability to speak through content moderation therefore trigger First Amendment scrutiny.[36]
As constitutional analysis is subject to changing norms and interpretations, especially in the more political charged federal judiciary of the day, it is inherently difficult to predict whether this law will be struck down for violating the protections of the First Amendment. However, it is safe to say it will likely face legal challenge and arguably be vulnerable thereto.
In order to insulate the bill from such challenge, the author has agreed to an amendment that simply provides that the bill does not apply to a candidate’s portrayal of themselves doing or saying something that the candidate did not do or say, where it includes the required disclosure.
- Additional concerns
The bill raises a few additional concerns. First, the bill requires platforms to retain all content they have prevented or blocked or labeled pursuant to the bill. This forced retention of information raises some thorny legal issues and may interfere with existing consumer rights. For instance, the CCPA, as amended by the CPRA, grants a series of rights to consumers, including the right to delete information held by businesses. In addition, given that the retention provision is essentially a government mandate on private businesses to seize certain information of private individuals, Fourth Amendment issues arguably arise. Furthermore, the bill requires platforms to hand over the content to specified government entities and even “researchers,” upon request. There is no limitation that there be evidence of a crime or some other justification and no probable cause necessary to be provided the information. In response, the author has agreed to an amendment to remove this retention requirement.
In addition, it is unclear what exactly is required by the bill’s requirement to block or prevent the sending of materially deceptive content. This could be read to apply to private messaging features of these platforms, essentially requiring platforms to scan private communications. This would raise serious privacy concerns. In response, the author has agreed to amendments that remove the “sending” element of the bill.
In addition, groups in opposition raise concerns that the bill presupposes a level of sophistication for technology that can detect AI-generated or manipulated content that simply does not exist. A coalition of industry associations, including NetChoice writes in opposition:
AB 2655 appears to be based on the false assumption that online platforms definitively know whether any particular piece of content has been manipulated in such a way that is defined under the bill. While digital services may employ tools to identify and detect these materials with some degree of certainty, it is an evolving and imperfect science in its current form. AB 2655 also presumes that online platforms are an appropriate arbiter of deciding what constitutes accurate election information. However, most digital services are not equipped with the tools or expertise to make such judgments.
Oakland Privacy writes in opposition:
The bill language offers that a technology company should be the judge, jury and executioner, although it may be unclear if the content is or is not generative AI-created and what role generative AI played in the content. It is unclear to us how any technology platform can be expected to know everything that every candidate in every city, county, state and federal election said and everywhere they went. Not to mention every other elected official in the state. If this is the basis for the removal of content by a technology platform, it is highly speculative and largely dependent on reports to the platform, which may be inaccurate, politically motivated, or malicious.
We appreciate amendments to raise the bar for the knowledge level of online platforms. But we continue to have concerns on the other side of the spectrum: the removal of content that should not be removed and may well impact election results.
In other words, the bill language is relying on two imprecise measures: technically scanning content for synthetic material with highly inaccurate tools, and real-life reports from the public, candidates and election officials and campaigns or chaos actors to power a broad censorship regime of blocking content. We cannot support that, even under the guise of defending democracy.
The opposition coalition also takes issue with the enforcement mechanism:
[B]ecause AB 2655 is focused on enforcement against covered platforms and not the actors who are intentionally seeking to materially deceive other consumers, it is unlikely to meaningfully reduce the amount of election mis- and disinformation hosted online. While the June 11 amendments appear to attempt to address this issue, we do not believe the new language effectively resolves our concerns. For example, the bill now allows for a "rebuttable presumption" but still fails to effectively address and hold accountable the purveyors of deceptive content.
- Support
CITED, the sponsor of the bill, writes:
Those trying to influence campaigns – conspiracy theorists, foreign states, online trolls, and candidates themselves – are already creating and distributing election-threatening deepfake images, audio, and video content in the US and around the world. This threat is not imaginary: generative AI has been used in various ways – most of them deeply deceptive – to influence the national elections in Slovakia, Bangladesh, Argentina, Pakistan, and elsewhere, including in our own country. Examples of this occurring in U.S. elections include Ron Desantis using AI-generated images to attack his opponent in his presidential run, foreign states caught attempting to influence American politics through social media, and just this month, a supporter of former President Trump creating a deepfake image of Trump with Black Americans designed to persuade Black voters to support Trump.
These examples demonstrate the power of generative AI-fueled disinformation to skew election results and weaken our faith in our democracy. We cannot let it undermine our elections here in California, and we are grateful you are leading the effort to try to stop it.
AB 2655 strikes the right balance by seeking to ban, for a strictly limited time before and after elections, the online spread of the worst deepfakes and disinformation maliciously intended to prevent voters from voting or getting them to vote erroneously based on fraudulent content. The bill also requires that other fake online content related to elections and elections processes (such as redistricting), which is also designed to undermine election procedures and democratic institutions, must be labeled as fake, again just for a limited time. The bill only applies to the largest online platforms with the greatest reach of potential election disinformation, and we believe it is fully implementable today based on tools these companies already possess. The companies covered by the bill’s requirements are all already subject to similar requirements under the European Union's Digital Services Act, which is designed to, among other things, crack down on election interference.
A coalition of groups in support, including the American Federation of State, County, and Municipal Employees (AFSCME) and NextGen CA, write:
AB 2655 seeks to solve these problems by, for a limited time before and after elections, banning the online spread of the worst of the deepfakes and disinformation meant to prevent voters from voting or to deceive them based on fraudulent content, and requiring that other fake content to be labeled as such. The approach leans heavily on increasing transparency, with bans used at only the highest-leverage moments, making it narrowly tailored. Additionally, it does not extend the law to hot button controversies or inflammatory claims – just the depiction of demonstrably untrue and provably false content meant to impermissibly influence our elections, at peak times – and is therefore implementable and respectful of the protections of the First Amendment.
Writing in support, the Northern California Recycling Association explains the need for the bill:
California is entering its first-ever generative Artificial Intelligence (AI) election, in which disinformation powered by generative AI would and will pollute our information ecosystems like never before. Deepfakes are a powerful and dangerous tool in the arsenal of those that want to wage disinformation campaigns, and they have the potential to wreak havoc on our democracy by attributing speech and conduct to a person that is false or that never happened. Advances in AI make it easy for practically anyone to generate this deceptive content, making it that much more important that we identify and restrict its spread before it has the chance to deceive voters and undermine our democracy.
SUPPORT
California Initiative on Technology and Democracy (sponsor)
AFSCME California
Asian Americans Advancing Justice - Asian Law Caucus
Asian Americans and Pacific Islanders for Civic Empowerment
Asian Law Alliance
Bay Rising
Board of Supervisors for the City and County of San Francisco
California Clean Money Campaign
California Environmental Voters
California State Sheriff’s Association
California Voter Foundation
Center for Countering Digital Hate
Chinese Progressive Association
City and County of San Francisco Board of Supervisors
Courage California
Disability Rights California
Hmong Innovating Politics
Inland Empire United
League of Women Voters of California
Move (mobilize, Organize, Vote, Empower) the Valley
Nextgen California
Northern California Recycling Association
Partnership for the Advancement of New Americans
SEIU California
Techequity Action
Verified Voting
Young People’s Alliance
Youth Power Project
OPPOSITION
ACLU California Action
California Chamber of Commerce
Computer & Communications Industry Association
Electronic Frontier Foundation
Internet Works
Netchoice
Oakland Privacy
Software & Information Industry Association
Technet
RELATED LEGISLATION
Pending Legislation:
SB 942 (Becker, 2024) establishes the California AI Transparency Act, requiring covered providers to create and make freely available an AI detection tool to detect content as AI-generated and to include disclosures in content generated by the provider’s system. SB 942 is currently in the Assembly Judiciary Committee.
SB 970 (Ashby, 2024) ensures that media manipulated or generated by artificial intelligence (AI) technology is incorporated into the right of publicity law and criminal false impersonation statutes. The bill requires those providing access to such technology to provide a warning to consumers about liability for misuse. SB 970 was held on suspense in the Senate Appropriations Committee.
AB 2355 (Wendy Carrillo, 2024) requires committees that create, publish, or distribute a political advertisement that contains any image, audio, or video that is generated or substantially altered using artificial intelligence to include a disclosure in the advertisement disclosing that the content has been so altered. AB 2355 is currently in this Committee.
AB 2839 (Pellerin, 2024) prohibits a person, committee, or other entity from knowingly distributing an advertisement or other election communication that contains materially deceptive content, as defined and specified, with malice, except as provided, within 120 days of a California election, and in specified cases, 60 days thereafter. AB 2839 is currently in this Committee.
AB 2930 (Bauer-Kahan, 2024) requires, among other things, a deployer and a developer of an automated decision tool to perform an impact assessment for any automated decision tool the deployer uses that includes, among other things, a statement of the purpose of the automated decision tool and its intended benefits, uses, and deployment contexts. AB 2930 requires a deployer to, at or before the time an automated decision tool is used to make a consequential decision, notify any natural person that is the subject of the consequential decision that an automated decision tool is being used to make, or be a substantial factor in making, the consequential decision and to provide that person with, among other things, a statement of the purpose of the automated decision tool. AB 2930 is currently in this Committee.
AB 3211 (Wicks, 2024) establishes the California Provenance, Authenticity and Watermarking Standards Act, which requires a generative AI system provider to take certain actions to assist in the disclosure of provenance data to mitigate harms caused by inauthentic content, including placing imperceptible and maximally indelible watermarks containing provenance data into content created by an AI system that the generative AI system provider makes available. AB 3211 also requires a large online platform, as defined, to, among other things, use labels to prominently disclose the provenance data found in watermarks or digital signatures in content distributed to users on its platforms, as specified. AB 3211 is currently in the Senate Appropriations Committee.
Prior Legislation: AB 730 (Berman, Ch. 493, Stats. 2019) prohibited the use of deepfakes depicting a candidate for office within 60 days of the election unless the deepfake is accompanied by a prominent notice that the content of the audio, video, or image has been manipulated. Additionally, AB 730 authorized a candidate who was falsely depicted in a deepfake to seek rapid injunctive relief against further publication and distribution of the deepfake.
PRIOR VOTES:
Senate Elections and Constitutional Amendments Committee (Ayes 6, Noes 1)
Assembly Floor (Ayes 56, Noes 1)
Assembly Appropriations Committee (Ayes 11, Noes 1)
Assembly Judiciary Committee (Ayes 9, Noes 0)
Assembly Elections Committee (Ayes 6, Noes 1)
**************
Siddarth Srinivasan, Detecting AI fingerprints: A guide to watermarking and beyond (January 4, 2024) Brookings Institution, https://www.brookings.edu/articles/detecting-ai-fingerprints-a-guide-to-watermarking-and-beyond/#:~:text=Google%20also%20recently%20announced%20SynthID,model%20to%20detect%20the%20watermark. All internet citations are current as of June 23, 2024. ↑
Lonnie Lee Hood, Experts Say That Soon, Almost The Entire Internet Could Be Generated by AI (March 4, 2022) The Byte, https://futurism.com/the-byte/ai-internet-generation. ↑
Brian Contreras, Tougher AI Policies Could Protect Taylor Swift—And Everyone Else—From Deepfakes (February 8, 2024) Scientific American, https://www.scientificamerican.com/article/tougher-ai-policies-could-protect-taylor-swift-and-everyone-else-from-deepfakes/. ↑
Hannah Fry, Laguna Beach High School investigates ‘inappropriate’ AI-generated images of students (April 2, 2024) Los Angeles Times, https://www.latimes.com/california/story/2024-04-02/laguna-beach-high-school-investigating-creation-of-ai-generated-images-of-students. ↑
Bobby Chesney & Danielle Citron, Deep Fakes: A Looming Challenge for Privacy, Democracy, and National Security (July 14, 2018) 107 California Law Review 1753 (2019), https://ssrn.com/abstract=3213954. ↑
Ali Swenson & Kelvin Chan, Election disinformation takes a big leap with AI being used to deceive worldwide (March 14, 2024) Associated Press, https://apnews.com/article/artificial-intelligence-elections-disinformation-chatgpt-bc283e7426402f0b4baa7df280a4c3fd. ↑
Em Steck & Andrew Kaczynski, Fake Joe Biden robocall urges New Hampshire voters not to vote in Tuesday’s Democratic primary (January 22, 2024) CNN, https://www.cnn.com/2024/01/22/politics/fake-joe-biden-robocall/index.html. ↑
David Klepper & Ali Swenson, AI-generated disinformation poses threat of misleading voters in 2024 election (May 14, 2023) PBS News, https://www.pbs.org/newshour/politics/ai-generated-disinformation-poses-threat-of-misleading-voters-in-2024-election. ↑
The author has agreed to an amendment that cross-references the existing definition for “social media platform, to replace the reference in the bill to “social network.” ↑
See Cubby, Inc. v. Compuserve, Inc. (S.D.N.Y. 1991) 776 F.Supp. 135, 141; Stratton Oakmont v. Prodigy Servs. Co. (N.Y. Sup. Ct., May 26, 1995) 1995 N.Y. Misc. LEXIS 229, *10-14. These opinions relied on case law developed in the context of other media, such as whether bookstores and libraries could be held liable for distributing defamatory material when they had no reason to know the material was defamatory. (See Cubby, Inc., 776 F. Supp. at p. 139; Smith v. California (1959) 361 U.S. 147, 152-153.) ↑
Kosseff, The Twenty-Six Words That Created The Internet (2019) pp. 57-65. ↑
Id. at pp. 68-73. Section 230 was added to the Communications Decency Act of 1996 (title 5 of the Telecommunications Act of 1996, Pub. L. 104-104, 110 Stat. 56), which would have imposed criminal liability on internet platforms if they did not take steps to prevent minors from obtaining “obscene or indecent” material online. The Supreme Court invalidated the CDA, except for Section 230, on the basis that it violated the First Amendment. (See Reno v. ACLU (1997) 521 U.S. 844, 874.) ↑
Id., § 230(c)(1). ↑
Id., § 230(c)(1) & (2). ↑
Id., § 230(e)(1) & (3). ↑
Kosseff, supra, fn. 13, at pp. 94-95; see, e.g., Doe v. MySpace Inc. (5th Cir. 2008) 528 F.3d 413, 421-422; Carfano v. Metrospalsh.com, Inc. (9th Cir. 2003) 339 F.3d 1119, 1125; Zeran v. America Online, Inc. (4th Cir. 1997) 129 F.3d 327, 333-334. ↑
Hassell v. Bird (2018) 5 Cal. 5th 522, 547. ↑
U.S. Const., 1st & 14th amends. ↑
Ashcroft v. American Civil Liberties Union (2002) 535 U.S. 564, 573. ↑
Snyder v. Phelps (2011) 562 U.S. 443, 452. ↑
Cohen v. California (1971) 403 U.S. 15, 20. ↑
Virginia v. Black (2003) 538 U.S. 343, 359. ↑
R.A.V. v. St. Paul (1992) 505 U.S. 377, 383. ↑
Ibid. ↑
Stanley v Georgia (1969) 394 U.S. 557, 564. Internal citations omitted ↑
Reno v. ACLU (1997) 521 U.S. 844, 870. ↑
Citizens United v. FEC (2010) 558 U.S. 310, 340. Internal citations omitted. It should be noted that while not controversial for the principle cited herein, this opinion is widely criticized for further tilting political influence toward wealthy donors and corporations. ↑
Reed v. Town of Gilbert (2015) 135 S.Ct. 2218, 2226 (Reed). ↑
Beilenson v. Superior Court (1996) 44 Cal. App. 4th 944, 954-55. Internal citations omitted. ↑
Buckley v. Valeo (1976) 424 U.S. 1, 14-15. Internal citations omitted. ↑
People v. Stanistreet (2002) 29 Cal. 4th 497, 505. ↑
Garrison v. Louisiana (1964) 379 U.S. 64, 74-75. Internal citations omitted. ↑
United States v. Playboy Entertainment Group (2000) 529 U.S. 803, 813. ↑
N.Y. Times Co. v. Sullivan, at 279. ↑
This amendment includes corresponding changes in the labeling section of the bill. ↑
NetChoice, LLC v. AG, Fla. (11th Cir. 2022) 34 F.4th 1196, 1210. Internal citations and quotations omitted. ↑
SENATE COMMITTEE ON
ELECTIONS AND CONSTITUTIONAL AMENDMENTS
Senator Catherine Blakespear, Chair
2023 - 2024 Regular
Bill No: 2655 Hearing Date: 6/18/24
Author: | Berman | ||
Version: |
| ||
Urgency: | Fiscal: | ||
Consultant: | |||
Subject:
DIGEST
This bill requires an online platform with at least one million California users to block, prevent the posting or spread of, or to label any elections-related content deemed to be materially deceptive if certain conditions are met.
ANALYSIS
Existing law:
- Prohibits anyone from, until January 1, 2027, distributing within 60 days of an election materially deceptive audio or visual media of a candidate with the intent to injure the candidate’s reputation or to deceive a voter into voting for or against the candidate.
- Prohibits anyone from, beginning January 1, 2027, producing, distributing, publishing, or broadcasting campaign material that contains a superimposed image of a candidate unless the campaign material includes a disclaimer that the picture is an inaccurate representation of fact.
This bill:
- Requires a large online platform (platform) using state-of-the-art, best available tools to detect materially deceptive content, to block and prevent the posting or sending of materially deceptive content related to elections if:
- The content is posted between 120 days before the election and through Election Day (or through the 60th day after the election in the case of content that depicts or pertains to elections officials) if the platform knows or should know the content portrays any of the following:
- A candidate for elective office portrayed as doing or saying something they did not do or say and is reasonably likely to harm the reputation or electoral prospects of a candidate.
- An elections official portrayed as doing or saying something in connection with their elections-related duties they did not do or say and is reasonably likely to falsely undermine confidence in the outcome of one or more election contests.
- An elected official portrayed as doing or saying something that influences the election that they did not do or say and is reasonably likely to falsely undermine confidence in the outcome of one or more election contests.
- The person or entity who created the materially deceptive content did so knowing it was false or with reckless disregard for the truth. There shall be a rebuttable presumption the person or entity knew the materially deceptive content was false or acted with reckless disregard for the truth if the content would cause a reasonable person to have a fundamentally different understanding or impression of the content than the person would have if hearing or seeing an authentic version of the content.
- The content is posted between 120 days before the election and through Election Day (or through the 60th day after the election in the case of content that depicts or pertains to elections officials) if the platform knows or should know the content portrays any of the following:
- Requires platforms to develop and implement procedures for labeling content as inauthentic, fake, or false if the content is posted outside of the time period described in (1) or appears within an advertisement or election communication not subject to (1). This would apply in cases where the person or entity created the content knowing it was false, and the platform knew or should have known that the content was suppose to meet the labeling requirements prescribed by this bill.
- Requires platforms to provide a way for Californians to report content that was not properly blocked or labeled to the platform. Allows a candidate, elected official, or elections official who reported content to the platform but does not receive a response in 36 hours or disagrees with the response – as well as the Attorney General (AG), a district attorney, or a city attorney – to seek injunctive or other equitable relief against a platform to compel compliance with this bill.
- Provides this bill does not apply to satire or parody or to a regularly-published online periodical that publishes materially deceptive content that contains a clear disclosure that the content is not an accurate representation of reality.
- Defines the following terms for the purposes of this bill:
- “Deepfake” to mean audio or visual media that is digitally created or modified such that it would falsely appear to a reasonable person to be an authentic record of the actual speech or conduct of the individual depicted in the media;
- “Materially deceptive content” to mean audio or visual media that is digitally created or modified, and that includes, but is not limited to, deepfakes and chatbots, such that it would falsely appear to a reasonable person to be an authentic record of the content depicted in the media; and
- “Large online platform” to mean a public-facing internet website, web application, or digital application that had at least one million California users in the preceding 12 months.
BACKGROUND
Manipulated Media in Campaign Communications. The use of false and deceptive information in campaigns to influence election outcomes is not a new phenomenon. Laws aimed at curbing such practices and preserving the integrity of elections have a long history in California. During its inaugural session in 1850, the California State Legislature created penalties for election misconduct, including for “deceiving [an elector] and causing him to vote for a different person for any office than such elector desired or intended to vote for.”
California law today includes various provisions criminalizing deceptive tactics that undermine election integrity or interfere with voters’ ability to participate in elections. This includes laws that prohibit the distribution of false and misleading information about qualifications to vote or about the days, dates, times, and places where voting may occur; prohibit the misleading use of government seals in campaign literature; and prohibit coercing or deceiving people into voting in a way that was inconsistent with the person’s intent.
Artificial Intelligence (AI) and Elections. On June 4, 2024, the Senate Committee on Elections and Constitutional Amendments and the Assembly Committee on Elections held a joint information hearing focusing on AI and elections.
The purpose of the hearing was to inform and assist the Legislature in making informed decisions on legislation related to AI-generated and altered content. It became evident that the ease with which people can create and spread mis- and disinformation creates a world where many people may have trouble determining what is fact and what is fiction. The development of increasingly advanced AI tools has made once time-consuming activities much easier to complete, while also enabling the completion of tasks that are otherwise too complex for humans to tackle alone.
State Action. In 2018, the Legislature approved and Governor Brown signed AB 3075 (Berman), Chapter 241, Statutes of 2018 to establish the Office of Elections Cybersecurity (OEC) in the Secretary of State’s (SOS) office. The OEC has two primary missions. First, it is responsible for coordinating efforts between the SOS and local elections officials to reduce the likelihood and severity of cyber incidents that could interfere with the security or integrity of elections in California. The OEC is also tasked with monitoring and counteracting false or misleading information regarding the electoral process that is published online or on other platforms that may suppress voter participation, cause confusion, or disrupt the ability to ensure a secure election. According to the OEC’s website, the office serves California with the sole purpose of keeping every Californian’s vote safe from online interference, especially the spread of mis- and disinformation.
In 2019, the Legislature approved and Governor Newsom signed AB 730 (Berman), Chapter 493, Statutes of 2019. AB 730 sought to address concerns that deepfake technology could be used to spread misinformation in political campaigns. Legislative analyses of AB 730 described “deepfake technology” as software capable of producing a realistic looking video of someone saying or doing something they did not actually say or do.
AB 730 prohibits anyone from distributing deceptive audio or visual media with actual malice and the intent to injure a candidate’s reputation or to deceive a voter, unless the media includes a disclaimer that it has been manipulated. AB 730 does not apply exclusively to deepfakes, but also to any intentional manipulation of audio or visual images where a reasonable person would be misled into believing it was authentic. Notably, AB 730 focused on materially deceptive representations of candidates, and not on deceptive media of other aspects of the electoral process.
AB 730 included a January 1, 2023 sunset date, but in 2022, the Legislature approved AB 972 (Berman), Chapter 745, Statutes of 2022, to extend the sunset date to January 1, 2027.
Tech Accord to Combat Deceptive Use of AI in 2024. In February 2024, 20 technology companies signed the “Tech Accord to Combat Deceptive Use of AI in 2024 Elections.” This set of commitments seeks to combat harmful AI-generated content meant to deceive voters. The signatories included Adobe, Amazon, Anthropic, Arm, ElevenLabs, Google, IBM, Inflection AI, LinkedIn, McAfee, Meta, Microsoft, Nota, OpenAI, Snap Inc., Stability AI, TikTok, Trend Micro, Truepic, and X.
The signatories committed to taking the following steps through this year:
- Develop and implement technology to mitigate risks related to deceptive AI content.
- Assess and better understand the risks presented by deceptive AI election content.
- Seek ways to detect the distribution of deceptive AI election content.
- Seek to address deceptive AI election content.
- Share best practices and explore pathways to share tools throughout the industry.
- Provide transparency to the public.
- Continue to engage with stakeholders to better understand the global risk landscape.
- Support efforts to raise public awareness regarding deceptive AI election content.
Other States. According to the National Conference of State Legislatures, 16 states (Alabama, Arizona, California, Colorado, Florida, Idaho, Indiana, Michigan, Minnesota, Mississippi, New Mexico, Oregon, Texas, Utah, Washington, and Wisconsin) enacted legislation designed to address deceptive media, including but not limited to, AI.
COMMENTS
- According to the Author: “AB 2655 will ensure that online platforms restrict the spread of election-related deceptive deepfakes meant to prevent voters from voting or to deceive them based on fraudulent content. Deepfakes are a powerful and dangerous tool in the arsenal of those that want to wage disinformation campaigns, and they have the potential to wreak havoc on our democracy by attributing speech and conduct to a person that is false or that never happened. Advances in technology make it easy for practically anyone to generate this deceptive content, making it that much more important that we identify and restrict its spread before it has the chance to deceive voters and undermine our democracy.”
- First Amendment Considerations. The First Amendment to the United States (US) Constitution provides in relevant part “Congress shall make no law…abridging the freedom of speech…” Similarly, Section 2 of Article I of the California Constitution provides in relevant part “Every person may freely speak, write, and publish his or her sentiments on all subjects, being responsible for the abuse of this right. A law may not restrain or abridge liberty of speech or press.”
This bill seeks to regulate the distribution by online platforms of media containing intentionally manipulated images, audio, or video related to candidates, elections officials, elected officials, and election materials and equipment under certain circumstances. A question could be raised about whether this bill is consistent with the right to freedom of speech that is guaranteed by the US and California Constitutions. The US Supreme Court has ruled that even false statements are protected by the First Amendment (United States v. Alvarez (2012), 567 U.S. 709). When a law burdens core political speech, the restrictions on speech generally must be “narrowly tailored to serve an overriding state interest,” McIntyre v. Ohio Elections Commission (1995), 514 US 334.
This bill targets deceptive content that could undermine the public’s trust in elections, prevent voters from voting, and distort the electoral process. The US Supreme Court generally has found the protection of the integrity of elections is an overriding (or compelling) government interest (Id. at 349; Burson v. Freeman (1992) 504 U.S. 191, 199). A challenge of this bill on First Amendment grounds would likely hinge on whether the court found this bill’s provisions to be narrowly tailored.
- Banned Sometimes, Labeled Other Times. This bill requires online media platforms with more than one million users to detect materially deceptive content, block, and prevent the posting or distribution of materially deceptive content related to elections from 120 days before an election and, in some cases, 60 days after an election.
However, for the other six to eight months of the year, the platforms would only have to label the materially deceptive content as such.
The Committee may wish to consider whether this distinction is appropriate and why, if content is materially deceptive, the content should not be banned entirely from the platform, regardless of how close to an election such content is posted or displayed.
- Election Timing. This bill requires platforms to block content beginning 120 days before an election and ending as many as 60 days after the election. As drafted, this would include any election in California. As a result, in counties where there are large numbers of elections (statewide, legislative, and congressional elections are held in every even-numbered year and many cities hold local elections in odd-numbered years), platforms may be blocking content almost constantly in order to ensure compliance with the provisions of this bill.
- Double Referral. If approved by this committee, AB 2655 will be referred to the Committee on Judiciary for further consideration.
RELATED/PRIOR LEGISLATION
AB 2355 (W. Carrillo) of 2024 requires a campaign committee that creates, originally publishes, or originally distributes a political advertisement to include a disclosure stating that the audio, image, or video was generated or substantially altered using AI. AB 2355 is being considered by this committee.
AB 2839 (Pellerin) of 2024 prohibits the distribution of campaign advertisements and other election communications containing materially deceptive and digitally altered or created images or audio or video files with the intent to influence an election or solicit funds for a candidate or campaign. AB 2839 is being considered by this committee.
PRIOR ACTION
Assembly Floor: | 56 - 1 |
Assembly Appropriations Committee: | 11 - 1 |
Assembly Committee: | 6 - 1 |
POSITIONS
Sponsor: California Initiative for Technology & Democracy
Support: Disability Rights California
League of Women Voters in California
Oppose: ACLU California Action
California Chamber of Commerce
Computer and Communications Industry
NetChoice
Software and Information Industry Association
TECHNET
-- END --
ASSEMBLY THIRD READING
ATR Bill Id:AB 2655
Author:(Berman and Pellerin)
As Amended Ver:May 20, 2024
Majority vote
SUMMARY
Requires large online platforms, as defined, to block the posting or sending of materially deceptive and digitally modified or created content related to elections, or to label that content, during specified periods before and after an election.
Major Provisions
- Requires a large online platform (platform), using state-of-the-art, best available tools to detect digitally modified or created content, to block and prevent the posting or sending of materially deceptive and digitally modified or created content related to elections, or to label that content, as follows:
- Requires the platform to block content if it is posted in the period beginning 120 days before the election through the day of the election (or through the 60th day after the election in the case of content that depicts or pertains to elections officials or elections equipment and materials) if the platform knows or should know that the content portrays any of the following:
- A candidate for elective office portrayed as doing or saying something that the candidate did not do or say.
- An elections official portrayed as doing or saying something in connection with the performance of their elections-related duties that the official did not do or say.
- An elected official portrayed as doing or saying something that influences the election that the elected official did not do or say.
- A voting machine, ballot, voting site, or other property or equipment related to an election that is portrayed in a materially false way.
- Requires the platform to label content as inauthentic, fake, or false if it is posted in the period beginning one year before the election through the day of the election (or through the 60th day after the election in the case of content that depicts or pertains to elections officials, the electoral college, elections equipment and materials, or the canvass of election results), or during a governmental process related to redistricting, if the platform knows or should know that the content meets either of the following requirements:
- The content would be required to be blocked by the platform, except that it was posted or sent outside of the time period during which the blocking requirement applies; or,
- The content is in an advertisement supporting or opposing a candidate for elective office or in an election communication, as defined, but is not content of the type that is required to be blocked as detailed above.
- Requires the platform to block content if it is posted in the period beginning 120 days before the election through the day of the election (or through the 60th day after the election in the case of content that depicts or pertains to elections officials or elections equipment and materials) if the platform knows or should know that the content portrays any of the following:
- Requires a platform to provide a way for Californians to report content that was not blocked or labeled as required. Permits a candidate, elected official, or elections official who reported content and who does not receive a response in 36 hours or who disagrees with the response, or the Attorney General (AG), a district attorney, or a city attorney, to seek injunctive or other equitable relief against a platform to compel compliance with this bill. Provides that the plaintiff bears the burden of establishing a violation through clear and convincing evidence. Requires the court to award a prevailing plaintiff reasonable attorney's fees and costs. Provides that such an action is entitled to precedence in court, as specified.
- Provides that this bill does not apply to satire or parody, or to a regularly-published news periodical that publishes materially deceptive content that contains a clear disclosure that the content is not an accurate representation of reality.
- Defines various terms for the purposes of this bill, including the following:
- "Materially deceptive and digitally modified or created content" to mean digital content that has been intentionally manipulated such that both of the following conditions are met:
- The content is the product of digital manipulation such that it so closely resembles authentic content that a reasonable person would not be able to readily distinguish the digitally manipulated or created content from the authentic content.
- The person or entity who attempted to post or send, or who did post or send, the content did so knowing the portrayal was false, or did so with reckless disregard for whether the portrayal was false, as specified.
- "Large online platform" to mean a public-facing internet website, web application, or digital application that had at least one million California users in the preceding 12 months.
- "Materially deceptive and digitally modified or created content" to mean digital content that has been intentionally manipulated such that both of the following conditions are met:
COMMENTS
The use of false and deceptive information in campaigns to influence election outcomes is not a new phenomenon. Laws aimed at curbing such practices and preserving the integrity of elections have a long history in California. In 1850, the First Session of the California State Legislature created penalties for election misconduct, including for "deceiving [an elector] and causing him to vote for a different person for any office than such elector desired or intended to vote for" (Chapter 38, Statutes of 1850).
Advancements in technology have made it increasingly simple to produce false and misleading media that closely resembles authentic content. Moreover, platforms like social media have facilitated the rapid dissemination of deceptive media to large audiences at minimal cost. Given these developments, the potential threat posed by manipulated media to future elections' integrity may be more significant than in the past.
Past legislative efforts have addressed concerns about manipulated media's use to deceive voters during elections. Those laws, however, are limited, and are designed primarily to target the harms to candidates that may result from the distribution of manipulated media of those candidates. In contrast, this bill aims to regulate materially deceptive and digitally altered media depicting not only candidates, but also elections officials and elected officials who are not candidates. Additionally, this bill targets media that portrays elections materials and equipment in materially deceptive ways. The author and supporters of this bill believe that these provisions will safeguard voters against deceitful media that could undermine trust in the electoral process.
The Legislature is considering a number of bills this year that seek to address deceptive and digitally altered elections-related content in an effort to protect the integrity of elections in California. While other legislation related to this topic applies broadly to the distribution of such content through various mediums, this bill specifically targets the distribution of deceptive content through online platforms, including social media. Recognizing that those online platforms can facilitate the rapid spread of deceptive content, this bill seeks to minimize that potential by obligating large online platforms to block or label offending content. In order to do that, the platforms necessarily will need to be able to identify the content that must be blocked or labeled.
In recognition that the regulation of the distribution of content can create free speech concerns, this bill contains various provisions that tailor the content to which it applies, such that it targets content that has the highest likelihood of deceiving voters and undermining electoral integrity. While that tailoring does limit the content that online platforms would be required to block or label, it also adds additional factors that platforms must consider in order to identify content that is required to be blocked or labeled under this bill.
Along with other limitations, this bill applies only to content that 1) is distributed during specified time periods around elections and election processes, 2) includes media relating to elections or the electoral process in specified ways, 3) that was intentionally manipulated digitally to be materially deceptive, and 4) that is not satire or parody. Each of these limitations adds additional factors that online platforms would need to consider when determining whether a specific communication must be blocked or labeled by this bill. Given the number of elections and candidates in California at any given time, making the determinations at scale about which content must be blocked or labeled likely will be considerably more challenging than making those determinations on a case-by-case basis in a court of law.
A question could be raised about whether this bill is consistent with the right to freedom of speech that is guaranteed by the United States (US) and California constitutions. The US Supreme Court has ruled that even false statements are protected by the First Amendment (United States v. Alvarez (2012), 567 U.S. 709). When a law burdens core political speech, the restrictions on speech generally must be "narrowly tailored to serve an overriding state interest," McIntyre v. Ohio Elections Commission (1995), 514 US 334.
This bill targets deceptive content that could undermine trust in elections, prevent voters from voting, and distort the electoral process. The US Supreme Court generally has found that the protection of the integrity of elections is an overriding (or compelling) government interest (Id. at 349; Burson v. Freeman (1992) 504 U.S. 191, 199). A challenge of this bill on First Amendment grounds, then, likely would hinge on whether the court found this bill's provisions to be narrowly tailored. This bill includes provisions to limit its scope to communications posing the greatest threat to election integrity in an effort to tailor its provisions. Whether these limitations adequately protect this bill from a potential constitutional challenge is unclear. However, while these limitations may help protect the bill against a constitutional challenge, they may also make it harder for the bill to achieve its aims of limiting the spread of materially deceptive communications that have the potential to undermine election integrity.
AB 2355 (Wendy Carrillo), which is pending on the Assembly Floor, requires any political advertisement, as specified, that is generated or substantially altered using artificial intelligence (AI), to include a disclaimer stating that fact.
AB 2839 (Pellerin), which is pending on the Assembly Floor, prohibits the distribution of campaign advertisements and other election communications that contain media that has been digitally altered in a deceptive way, except as specified.
Please see the policy committee analysis for a full discussion of this bill.
According to the Author
"AB 2655 will ensure that online platforms restrict the spread of election-related deceptive deepfakes meant to prevent voters from voting or to deceive them based on fraudulent content. Deepfakes are a powerful and dangerous tool in the arsenal of those that want to wage disinformation campaigns, and they have the potential to wreak havoc on our democracy by attributing speech and conduct to a person that is false or that never happened. Advances in AI make it easy for practically anyone to generate this deceptive content, making it that much more important that we identify and restrict its spread before it has the chance to deceive voters and undermine our democracy."
Arguments in Support
The sponsor of this bill, the California Initiative for Technology & Democracy, a Project of California Common Cause, writes in support, "Those trying to influence campaigns – conspiracy theorists, foreign states, online trolls, and candidates themselves – are already creating and distributing election-threatening deepfake images, audio, and video content in the US and around the world…[AB] 2655 strikes the right balance by seeking to ban, for a strictly limited time before and after elections, the online spread of the worst deepfakes and disinformation maliciously intended to prevent voters from voting or getting them to vote erroneously based on fraudulent content. The bill also requires that other fake online content related to elections and elections processes (such as redistricting), which is also designed to undermine election procedures and democratic institutions, must be labeled as fake, again just for a limited time. The bill only applies to the largest online platforms with the greatest reach of potential election disinformation, and we believe it is fully implementable today based on tools these companies already possess. The companies covered by the bill's requirements are all already subject to similar requirements under the European Union's Digital Services Act, which is designed to, among other things, crack down on election interference. AB 2655's approach is narrowly tailored and does not extend the law to hot button controversies or inflammatory claims – it does not ask social media platforms to adjudicate controversial opinions post by post. It simply stops the use of obviously, demonstrably untrue and provably false content meant to impermissibly influence our elections at peak election times."
Arguments in Opposition
A joint letter of opposition submitted by Chamber of Progress, Computer and Communications Industry Association, NetChoice, Software & Information Industry Association, and TechNet states, "Responsible digital services providers take aggressive steps to moderate dangerous and illegal content, consistent with their terms of service. The companies deliver on the commitments made to their user communities with a mix of automated tools and human review…AB 2655 appears to be based on the false assumption that online platforms definitively know whether any particular piece of content has been manipulated in such a way that is defined under the bill. While digital services may employ tools to identify and detect these materials with some degree of certainty, it is an evolving and imperfect science in its current form. AB 2655 also presumes that online platforms are an appropriate arbiter of deciding what constitutes accurate election information. However, most digital services are not equipped with the tools or expertise to make such judgments. AB 2655 would impose significant operationally and practically challenging requirements on large online platforms who may not be best suited to achieve the bill's laudable and important goal of ensuring California's elections remain free and fair."
FISCAL COMMENTS
According to the Assembly Appropriations Committee:
- The AG reports no impact from this bill, as the AG anticipates local prosecutors are more likely to be the primary agencies bringing forward such cases.
- By authorizing a claim by a candidate, elected official, or elections official, the AG, or a district or city attorney against a large online platform for failing to block or label specified content, this bill may result in an increased number of civil actions that also receive precedence when filed in court, resulting in potentially significant cost pressures of an unknown amount to the courts (Trial Court Trust Fund (TCTF)). It is unclear how many new actions may be filed statewide, but disinformation and hostility regarding elections has grown in recent years and one hour of court time is estimated to cost $1,000 in workload. Although courts are not funded on the basis of workload, increased pressure on staff and the TCTF may create a need for increased court funding from the General Fund (GF) to perform existing duties. The Governor's proposed 2024-25 state budget includes $83.1 million ongoing GF to backfill declining TCTF revenue.
According to the Legislative Analyst's Office, the GF faces a structural deficit in the tens of billions of dollars over the next several fiscal years.
VOTES
ASM ELECTIONS: 6-1-1
YES: Pellerin, Bennett, Berman, Cervantes, Low, Weber
NO: Essayli
ABS, ABST OR NV: Lackey
ASM JUDICIARY: 9-0-3
YES: Kalra, Bauer-Kahan, Bryan, Connolly, Haney, Maienschein, McKinnor, Pacheco, Reyes
ABS, ABST OR NV: Dixon, Sanchez, Waldron
ASM APPROPRIATIONS: 11-1-3
YES: Wicks, Arambula, Bryan, Calderon, Wendy Carrillo, Mike Fong, Grayson, Haney, Hart, Pellerin, Villapudua
NO: Dixon
ABS, ABST OR NV: Sanchez, Jim Patterson, Ta
UPDATED
VERSION: May 20, 2024
CONSULTANT: Ethan Jones / ELECTIONS / (916) 319-2094 FN: 0003116
Date of Hearing: May 8, 2024
ASSEMBLY COMMITTEE ON APPROPRIATIONS
Buffy Wicks, Chair
ABPCA Bill Id:AB 2655
Author:(Berman) – As Amended Ver:April 24, 2024
Policy Committee: | Elections | Vote: | 6 - 1 |
Judiciary | 9 - 0 | ||
|
|
Urgency: No State Mandated Local Program: No Reimbursable: No
SUMMARY:
This bill establishes the Defending Democracy from Deepfake Deception Act of 2024 (Act) for the purpose of preventing online dissemination of manipulated media and disinformation meant to deceive voters and prevent voting activity.
Specifically, this bill:
- Requires, during specified periods before and after an election, a large online platform to block the posting or sending of materially deceptive and digitally modified or created content related to elections, as well as label certain additional content as inauthentic, fake, or false.
- Requires a large online platform to develop procedures for California residents to report content that has not been blocked or labeled in compliance with the Act. Exempt content includes satire or parody, as well as content required to be blocked or labeled but published by an online newspaper or periodical of general circulation with a clear disclosure that the content is not accurately representative.
- Authorizes a California resident who filed a report with the platform and either did not receive a response within 36 hours or disagrees with the response, the Attorney General (AG), and a district or city attorney to seek injunctive relief and reasonable attorney’s fees and costs against a platform for non-compliance with the Act, and assigns such actions precedence when filed in court.
- Requires a platform that blocks or labels a piece of content to maintain a copy of the content for at least five years and make the content available, upon request, to the Secretary of State, the Fair Political Practices Commission, and researchers.
FISCAL EFFECT:
- The AG reports no impact from this bill, as the AG anticipates local prosecutors are more likely to be the primary agencies bringing forward such cases.
- By authorizing a claim by an aggrieved California resident, the AG, or a district or city attorney against a large online platform for failing to block or label specified content, this bill may result in an increased number of civil actions that also receive precedence when filed in court, resulting in potentially significant cost pressures of an unknown amount to the courts (Trial Court Trust Fund (TCTF)). It is unclear how many new actions may be filed statewide, but disinformation and hostility regarding elections has grown in recent years and one hour of court time is estimated to cost $1,000 in workload. Although courts are not funded on the basis of workload, increased pressure on staff and the TCTF may create a need for increased court funding from the General Fund (GF) to perform existing duties. The Governor’s proposed 2024-25 state budget includes $83.1 million ongoing GF to backfill declining TCTF revenue.
According to the Legislative Analyst’s Office, the GF faces a structural deficit in the tens of billions of dollars over the next several fiscal years.
COMMENTS:
- Purpose. According to the author:
AB 2655 will ensure that online platforms restrict the spread of election-related deceptive deepfakes meant to prevent voters from voting or to deceive them based on fraudulent content. Deepfakes are a powerful and dangerous tool in the arsenal of those that want to wage disinformation campaigns, and they have the potential to wreak havoc on our democracy by attributing speech and conduct to a person that is false or that never happened.
- Support and Opposition. This bill is sponsored by the California Initiative for Technology and Democracy, a project of California Common Cause, which argues:
AB 2655’s approach is narrowly tailored and does not extend the law to hot button controversies or inflammatory claims – it does not ask social media platforms to adjudicate controversial opinions post by post. It simply stops the use of obviously, demonstrably untrue and provably false content meant to impermissibly influence our elections at peak election times.
This bill is also supported by voter empowerment groups, civil rights groups, and labor organizations.
This bill is opposed, for different reasons, by associations representing online platforms and civil liberties groups. A coalition of industry groups argues:
AB 2655 appears to be based on the false assumption that online platforms definitively know whether any particular piece of content has been manipulated in such a way that is defined under the bill. While digital services may employ tools to identify and detect these materials with some degree of certainty, it is an evolving and imperfect science in its current form.
ACLU California Action argues:
Digitally modified content or content created using artificial intelligence (AI) tools is also entitled to these protections, unless the content falls within recognized First Amendment exceptions such as libel or fraud. The ‘novelty of deepfake technology and the speed with which it is improving’ do not justify relaxing the stringent protections afforded to political speech by the First Amendment.
- Manipulated Campaign Communications. The use of false and deceptive information in campaigns to influence elections is not a new phenomenon, with laws aimed at curbing such practices and preserving the integrity of elections dating back to 1850, the first session of the state legislature. However, advancements in technology have made it increasingly simple to produce false and misleading media that closely resembles authentic content. Moreover, platforms like social media have facilitated the rapid dissemination of deceptive media to large audiences at minimal cost.
In response, AB 730 (Berman), Chapter 493, Statutes of 2019, prohibited the distribution of materially deceptive audio or visual media with actual malice and intent to injure a candidate’s reputation or to deceive a voter to vote for or against a candidate, unless the media includes a disclaimer that the content is manipulated. As such, AB 730, as well as most previous laws, was designed primarily to target the harm caused to a candidate by the distribution of manipulated media. In contrast, this bill requires a large online platform to block or label materially deceptive and digitally modified media depicting elected officials who are not candidates; elections officials; and elections materials, equipment, and processes. For example, AI was used by a democratic operative and a magician to generate deepfake audio of President Biden’s voice used in a robocall to dissuade voters from participating in the primary election – the author intends this bill to prohibit the spread of such materially deceptive and digitally created content.
This bill also requires a platform to develop procedures for residents to report content that is not blocked or labeled in compliance with the Act. The platform must respond to the resident within 36 hours, describing the action taken or not taken regarding the content. If the resident does not receive such a response within 36 hours or disagrees with the outcome, the resident, in addition to the AG or a district or city attorney, may file suit against the platform to compel compliance with the Act.
- Related Legislation. AB 2355 (Wendy Carrillo) requires a political advertisement generated in whole or in part using AI to include such a disclaimer. AB 2355 is pending hearing in this committee.
AB 2839 (Pellerin) prohibits the distribution of certain campaign advertisements and other election communications that are materially deceptive and digitally altered or created. AB 2839 is pending hearing in this committee.
Analysis Prepared by: Irene Ho / APPR. / (916) 319-2081
Date of Hearing: April 23, 2024
ASSEMBLY COMMITTEE ON JUDICIARY
Ash Kalra, Chair
ABPCA Bill Id:AB 2655
Author:(Berman) – As Amended Ver:April 1, 2024
As Proposed to be Amended
SUBJECT: Defending Democracy from Deepfake Deception Act of 2024
KEY ISSUE: should large online platforms be required to block (or in some cases label) materially deceptive and digitally modified or created content related to an election or election process, during a prescribed period of time before or after an election?
SYNOPSIS
According to the author, disinformation powered by Artificial Intelligence (AI), which can be distributed to millions of social media users in an instant, poses a serious threat to our political discourse, our elections, and indeed our democracy. For example, “deepfakes” can generate false sounds and images that could lead to the most discerning viewer to falsely conclude that a candidate, elected official, or election worker said or did something they did not do. Such disinformation not only distorts the truth, it has the potential to undermine people’s confidence in our political institutions. While disinformation can threaten political discourse at any time, the author believes that it is especially harmful during an election season, when uncorrected disinformation may influence an election result or create false concerns about the legitimacy of an election in its immediate aftermath.
This bill, therefore, would require a large online platform to block “materially deceptive and digitally modified or created” content that portrays any candidate, elected official, or elections official doing or saying something they did not do or say; it would also require them to block deceptive material that concerns voting machines, ballots, voting sites, or other procedures or equipment related to an election. In addition, deceptive material about broader “elections processes,” that are not subject to blocking requirement, would need to contain a label that they are materially deceptive and digitally modified. The bill would allow any resident of California to inform the platform that covered material had not been properly blocked or labeled, and if the platform does not respond within 36 hours, or if the reporting resident does not agree with the response, the resident may bring an action for injunctive relief. The bill would also allow the Attorney General, or any district attorney or city attorney, to also seek injunctive relief.
This bill passed out of the Assembly Elections Committee on a 6-1 vote. It is sponsored by the California Initiative for Technology and Democracy, a project of California Common Cause, and supported by several political reform groups and labor organizations, among others. The bill is opposed by groups representing the information and technology industry and by ACLU Action California. The opposition argues that the bill would be ineffective, unconstitutional, and preempted by federal law. The author will take several definitional amendments in this Committee, which are reflected in the Summary, below, and discussed in the analysis.
SUMMARY: Requires large online platforms, as defined, to block the posting or sending of materially deceptive and digitally modified or created content related to elections, or to label that content, during specified periods before and after an election. Specifically, this bill:
- Makes findings and declarations about the growing use of generative artificial intelligence (AI), deepfakes, and related technologies to disseminate disinformation that distorts our electoral process and undermines trust in elections.
- Requires a large online platform (platform), using state-of-the-art, best available tools to detect digitally modified or created content, to develop procedures for blocking and preventing the posting or sending of materially deceptive and digitally modified or created content, and to block and prevent that content if the platform knows or should know that the content meets the following requirements, during a specified time period, of any of the following:
- A candidate portrayed as doing or saying something that the candidate did not do or say.
- An elections official portrayed as doing or saying something in connection with the performance of their elections-related duties that the official did not do or say.
- An elected official portrayed as doing or saying something that influences the election that the elected official did not do or say.
- A voting machine, ballot, voting site, or other property or equipment related to an election that is portrayed in a materially false way.
- Prohibits a platform, notwithstanding the above, from preventing candidates from posting deceptive and manipulated material about themselves so long as the content includes a prescribed disclaimer.
- Provides that the above prohibitions apply only during the period 120 days before and through election day; however, they apply to content relating to elections officials, voting sites, voting machines, ballots, or related equipment from 120 days before the election to 60 days after the election.
- Requires a platform to develop procedures for labeling materially deceptive and digitally modified or created content that pertains to election processes, but that is not subject to the blocking provisions above. Requires the label to indicate that the content is inauthentic, fake, or false if the platform knows or should know as much. Provides that the labeling requirement applies during the period beginning one year before the election or election process, as specified.
- Requires the platform to provide a way for Californians to report content that was not blocked or labeled as required, and requires the platform to respond to the report within 36 hours and to describe any actions taken. If the platform does not respond within 36 hours, or if the reporting resident disagrees with the response, the reporting resident may bring an action for injunctive or other equitable relief, and a prevailing plaintiff may recover reasonable attorney’s fees and costs. Allows the Attorney General, a district attorney, or a city attorney to similarly seek injunctive relief and obtain fees and costs.
- Specifies that the provisions of this bill do not apply to a regularly published online newspaper, magazine, or periodical, as specified.
- Defines the following terms for purposes of the above as follows:
- “Elections official” means any of the following: an elections official as defined in Elections Code Section 320; the Secretary of State and their staff; a temporary worker, poll worker, or member of a precinct board; any other person charged with holding or conducting an election, a canvas, or performing another election-related duty.
- “Election processes” means any government process related to an election, including, but not limited to, elections, candidates, vote counting, redistricting, and proceedings or processes of the Electoral College.
- “Materially deceptive and digitally modified or created content” means an image or an audio or video recording or other digital content, including a chatbot, that has been intentionally manipulated such that all of the following conditions are met:
- The digital content is the product of digital manipulation, artificial intelligence, or machine learning, including deep learning techniques, that merges, combines, replaces, or superimposes content onto an image or an audio or video recording, creating an image or an audio or video recording that appears authentic, or that otherwise generates an inauthentic image or an audio or video recording that appears authentic, and that contains a false portrayal of any of the following: a candidate for elective office, elected official, elections official, voting machine, ballot, voting site, other property or equipment related to an election, or elections process; and provides that “false portrayal” means the content would cause a reasonable person to have a fundamentally different understanding or impression of the content than the person would have if they were hearing or seeing the unaltered, original version of the content.
- The person or entity who attempted to post or send, or who did post or send, the content did so knowing the portrayal was false, or did so with reckless disregard for whether the portrayal was false. If the content is intentionally manipulated and contains a false portrayal as specified in subparagraph (A), there shall be a rebuttable presumption that the person or entity knew the portrayal was false or that they acted with reckless disregard for whether the portrayal was false.
- Clarifies that “Materially deceptive and digitally modified or created content” does not include any image or audio or video recording that contains only minor modifications that do not lead to significant changes to the perceived contents or meaning of the content. Minor changes include changes to the brightness or contrast of images, removal of background noise in audio, and other minor changes that do not impact the content of the image or audio or video recording.
- “Large online platform” means a public-facing internet website, web application, or digital application, including a social network, video sharing platform, advertising network, or search engine that had at least 1 million California users during the preceding 12 months.
- “Artificial intelligence” means an engineered or machine-based system that varies in its level of autonomy and that can, for explicit or implicit objectives, infer from the input it receives how to generate outputs that can influence physical or virtual environments.
EXISTING LAW:
- Prohibits a person, committee, or other entity, until January 1, 2027, from distributing with actual malice, within 60 days of an election at which a candidate for elective office will appear on the ballot, materially deceptive audio or visual media of a candidate with the intent to injure the candidate’s reputation or to deceive a voter into voting for or against the candidate.
- Defines “materially deceptive audio or visual media,” for these purposes, as an image or an audio or visual recording of a candidate’s appearance, speech or conduct that has been intentionally manipulated in a manner that both of the following are true about the image or audio or video recording: (1) It would falsely appear to a reasonable person to be authentic; and (2) it would cause a reasonable person to have a fundamentally different understanding or impression of the expressive content of the image or audio or video recording than the person would have if the person were hearing or seeing the unaltered, original version of the image or audio or video recording.
- Provides that this prohibition does not apply if the audio or visual media includes a disclaimer stating “This (image/video/audio) has been manipulated,” and the disclaimer complies with specified requirements.
- Permits a candidate whose voice or likeness appears in deceptive audio or visual media distributed in violation of this provision to seek injunctive relief, as specified, and general or special damages and reasonable attorney’s fees and costs, as specified. Specifies that in any civil action brought pursuant to these provisions, the plaintiff bears the burden of establishing the violation through clear and convincing evidence.
- Provides that this prohibition shall not be construed to alter or negate any rights, obligations, or immunities of an interactive service provider under Section 230 of the federal Communications Decency Act.
- Provides that this prohibition does not apply to a radio or television broadcasting station; an internet website; a regularly published newspaper, magazine, or other periodical of general circulation, including an internet or electronic publication; or media that constitute satire or parody. (Elections Code Section 20010.)
- Prohibits a person, firm, association, corporation, campaign committee, or organization, beginning January 1, 2027, with actual malice, from producing, distributing, publishing, or broadcasting campaign material, as defined, that contains either of the following types of pictures or photographs, as specified, unless the campaign material includes a disclaimer that the picture is not an accurate representation of fact:
- A picture or photograph of a person or persons into which the image of a candidate for public office is superimposed.
- A picture or photograph of a candidate for public office into which the image of another person or persons is superimposed. (Elections Code Section 20010.)
FISCAL EFFECT: As currently in print this bill is keyed fiscal.
COMMENTS: According to the author:
AB 2655 will ensure that online platforms restrict the spread of election-related deceptive deepfakes meant to prevent voters from voting or to deceive them based on fraudulent content. Deepfakes are a powerful and dangerous tool in the arsenal of those that want to wage disinformation campaigns, and they have the potential to wreak havoc on our democracy by attributing speech and conduct to a person that is false or that never happened. Advances in AI make it easy for practically anyone to generate this deceptive content, making it that much more important that we identify and restrict its spread before it has the chance to deceive voters and undermine our democracy.
Therefore, in order to ensure California elections are free and fair, online platforms must prevent the online spread of election-related deceptive deepfakes and disinformation meant to prevent voters from voting or to deceive them based on fraudulent content.
Existing laws maintaining “election integrity.” As aptly noted in the analysis of this bill by the Assembly Elections Committee, the “use of false and deceptive information in campaigns to influence election outcomes is not a new phenomenon,” nor are the laws “aimed at curbing such practices” new. Indeed, in 1850, the First Session of the California State Legislature created penalties for “deceiving [an elector] and causing him to vote for a different person for any office than such elector desired or intended to vote for.” (Chapter 38, Statutes of 1850.) Existing California law has greatly elaborated on these initial legislative efforts. For example, provisions in the Elections Code prohibit the distribution of false and misleading information about qualifications to vote or about the days, dates, times, and places where voting may occur; prohibit the misleading use of government seals in campaign literature (Elections Code Section 18304); and prohibit coercing or deceiving people into voting in a way that is inconsistent with the person’s intent (Elections Code Sections 18302, 18304, 18573 and 18573.5).
In the last five years, the Legislature has turned its attention to “materially deceptive” audio and visual materials that portray a candidate. For example, AB 730 (Chap. 493, Stats. 2019) responded to reports that so-called “deepfake” technology, a software that allows someone to produce audios and videos that look and appear remarkably real to even the most discerning person. For example, in 2018, BuzzFeed and the film director Jordan Peele published a very realistic-looking deepfake showing former President Obama calling then-President Donald Trump a “total and complete dipshit.” Obama did not say that. Peele and BuzzFeed did not use the video to try to influence a political election – indeed halfway through the video the ruse was revealed – but to show the potential for abuse of deepfake technology. Responding to this and similar reports, AB 730 prohibited the distribution of materially deceptive audio or visual media with the intent to injure the candidate’s reputation or to deceive a voter into voting for or against a candidate.
AB 730 was itself an amendment to California’s “Truth in Political Advertising Act” of 1998, which had prohibited campaign material that deceptively altered a picture of a candidate (for example by superimposing one person’s image upon another) unless the picture contained a disclaimer. This 1998 bill was introduced in response to the use of technologies like “photo shopping,” which now seem quaint compared to deepfakes, including deepfakes generated by artificial intelligence. The bill now before the Committee, like AB 730 before it, is apparently an effort to stay one step ahead of evolving technologies, which not only create more realistic-looking deceptions, but make it possible to quickly create, alter, and distribute fake images to millions of people in the blink of an eye (or the click of a mouse).
This bill and existing law: who? AB 2655 elaborates upon and modifies existing law in a variety of ways. Most significant, existing law prohibits a person, committee, or other entity from distributing, with actual malice, “materially deceptive” audio or visual media of a candidate with the intent to injure the candidate’s reputation or to deceive a voter into voting for or against the candidate. Existing law prohibits the distribution of such material within 60 days of the election in which the targeted candidate is running for office. The prohibitions in AB 2655, on the other hand, do not apply to the person or entity that created the materially deceptive material, but to the “large online platform” on which it is posted. AB 2655 requires the online platform to develop and implement procedures to block or prevent the posting of the covered content.
This bill and existing law: what? This bill also differs from existing law in terms of covered content. Existing law applies only to manipulated material that misrepresents the “candidate” saying or doing something the candidate did not do or say. This bill similarly applies to content deceptively depicting the candidate, but also applies to images portraying an “elections official” doing or saying something they did not do or say, as well as images depicting a voting machine, ballot, voting site, or voting equipment in a materially false way. In addition, this bill would require the platform to label (but not necessarily block) materially deceptive content about “election processes” (as opposed to a specific candidate, election official, or voting site). Although the distinction between what material must be blocked versus what material must be labeled is not entirely clear, the intent of the author and sponsor is that the more the material singles out a particular candidate or election official, the more it must be blocked; whereas material that deals with “election processes” more generally need only be labeled. The author has agreed to work with the Committee as the bill moves forward to clarify the distinction between the blocking and labeling requirements.
This bill and existing law: when? AB 2655 also expands and modifies the relevant time period that prohibitions are in force. Existing law prohibits someone from distributing deceptive material during the 60-day period before an election. This bill, however, requires the platform to block covered material a period beginning 120 days before the election and through the day of the election. However, if the deceptive material pertains to an election official, or deceptively depicts a voting machine, ballot, voting site, or property or equipment related to an election, the platform must also block the content for 60 days after the election. Presumably, this post-election period is intended to prevent false claims – similar to those made in 2020 – that the election process was irregular or otherwise “rigged.” The labeling requirement covers an even larger time frame; it applies during the period beginning one year before the “election.” The bill also requires labeling for the period beginning one year before an “election process.” “Election processes” – as opposed to an “election” – is defined to include any government process “related” to an election, “including, but not limited to,” elections, candidates, vote counting, redistricting, and proceedings or processes of the electoral college.” Because an “election” has a known date, it should be fairly easy for the platform to figure out when the year-long labeling period starts. However, if the platform must also label one year prior to an “election process,” when does a process like “redistricting” start? Does it start with each new census? Does it start when the legislative body (or in some states a commission) meet to draw up new district lines? Moreover, the definition of “election process” is not limited to the items listed, expressly stating “including, but is not limited to,’ those items. As discussed above, the author has agreed to work with the Committee as the bill moves forward to clarify the distinction between materials that the platform is required to “block” and those it is required to “label.”
This bill and existing law: how enforced? In addition to differences as to who, what, and when, this bill also differs in how violations would be enforced. Existing law permits only the “candidate” whose voice or likeness appears in the deceptive material to bring an action for injunctive relief, general or special damages, and reasonable attorney’s fee and cost. However, under existing law the candidate bears the burden of establishing a violation by “clear and convincing evidence.” Enforcement provisions in this bill are much different. The bill allows any “California resident” to report to the platform that content was not blocked or labeled as required. If the platform does not respond within 36 hours, or if the resident disagrees with the response, the resident may seek injunctive relief to compel compliance and, if the resident prevails, shall be awarded reasonable attorney’s fees and costs. Thus, not only can any California resident – not just a person depicted or otherwise affected – seek injunctive relief, they apparently do not have the burden of proving a violation by clear and convincing evidence. If they “disagree” with the platform’s response, that’s enough; they can seek injunctive relief and a court would decide to issue on the likelihood of success on merits, the general rule for injunctive relief. The author may wish to consider, as the bill moves forward, increasing the standard of proof to the higher clear and convincing evidence; and limiting who may seek relief.
First Amendment concerns. Because this bill imposes a government mandate that online platforms must block expressive material based upon its content, it implicates the First Amendment. The First Amendment provides that “Congress shall make no law . . . prohibiting the freedom of speech.” As interpreted by the courts and incorporated against the states by the due process clause of the 14th Amendment, the First Amendment prevents any government entity (not just Congress) from enacting any law or adopting any policy that burdens freedom of speech. In addition, Article I, Section 2 of the California Constitution guarantees to every person the freedom to “speak, write, and publish his or her sentiments on all subjects, being responsible for the abuse of this right.” Moreover, the First Amendment not only protects the right to speak, as a logical corollary it protects the “right to receive information and ideas.” (Stanley v Georgia (1969) 394 U.S. 557, 564.) This bill would interfere with both the expression and reception of information based upon its content. Moreover, not only does this bill single out particular content, the content relates to political candidates and elections. This is potentially problematic because the First Amendment affords the “broadest protection” to the “discussion of public issues” and “political expression in order to assure the unfettered interchange of ideas for the bringing about of political and social changes desired by the people.” (McIntyre v Ohio Election Commission (1997) 514 U.S. 334.) It is difficult to imagine any content more related to “political expression” and “discussion of public issues” than content about candidates and elections. The fact that the bill restricts speech that is “materially deceptive” or “false” does not matter, for the U.S. Supreme Court has been unequivocal that the First Amendment protects even “false” speech. The remedy for false speech is more true speech, and false speech tends to call forth true speech. (United States v Alvarez (2012) 567 U.S. 709.)
The bill will likely meet the “compelling interest” threshold, if not the “narrowly tailored” threshold. The right to free speech is not absolute. As Justice Holmes noted in a dissenting opinion over a century ago, the First Amendment does not protect a right to falsely cry fire in a crowded theater. (Schenck v. United States (1919) 249 U.S. 47.) The proponents of this bill may, not unreasonably, think that pervasive disinformation about candidates and elections, especially during an election season, is just as dangerous. In reviewing the law, the Court would apply strict scrutiny. This means that government can impose even content-based restrictions on protected speech if they have a “compelling government interest” and they use “narrowly tailored means” to achieve that interest.
Even the opponents of this bill appear to concede that maintaining “election integrity” is a “weighty” and, presumably, “compelling” government interest. Therefore, it seems likely that if this bill is enacted and subsequently challenged, a court will accept that there is a “compelling interest” but still need to consider whether the “means” are “narrowly tailored.” The opponents of this measure claim that it is not narrowly tailored. They contend that while covered platforms may have state-of-the-art tools that allow them to identify content that has been “digitally modified,” there is no technology to determine if content is “materially deceptive.” The bill defines “materially deceptive,” in relevant part, to mean content that is “intentionally manipulated” so that it appears “authentic” but contains a “false portrayal” of a candidate, elected official, elections official, voting machine, ballot, voting site, other property or equipment related to an election, or elections process. The purpose of narrow tailoring is to ensure that no more speech is infringed or burdened than is necessary. However, the opponents of this bill – both the industry groups and the ACLU – believe that with no sure means to determine what is “materially deceptive,” the platforms will err on the side of blocking content, thus burdening more speech than is necessary.
The findings and declarations in the bill state that the “labeling information required by this bill is narrowly tailored to provide consumers with factual information about the inauthenticity of particular images, audio, video, or text content in order to prevent consumer deception.” Tellingly, there is no similar claim about the blocking requirement being narrowly tailored. In any event, it will be a court – not the findings and declarations of the bill – that will determine whether the bill is narrowly tailored. The court may consider, for example, if there are other less restrictive and more effective means of protecting election integrity.
Section 230 concerns and “editorial discretion.” In addition to implicating the First Amendment, this bill may also be preempted by Section 230 of the federal Communications Decency Act. In relevant part, Section 230 provides two express protections for online platforms and their ability to moderate online content. First, Section 230 declares that an online platform cannot be held liable for content posted by third parties. The rationale for this immunity is premised on the idea that online platforms, unlike newspapers, do not exercise editorial discretion; rather, like telephone companies or “common carriers” they are merely a conduit for the expression of ideas by others. Those others, not the platform, are liable for any harm caused by the content. Second, somewhat in tension with this immunity, Section 230 expressly provides that online platforms are not liable if they block or remove material because they disapprove of its content. While it is important to remember that the First Amendment is distinct from Section 230, this protection flows from First Amendment principles. First, because the online platform is not a government actor, it cannot violate the First Amendment. Second, as a private actor, the platform has its own First Amendment right not to be associated with speech it finds objectionable. Some scholars have noted that the two protections in Section 230 are based on contrary premises. Immunity from liability from postings by third parties assumes that platforms do not exercise editorial discretion. Their right to remove content without liability, and their own free speech claims, on the other hand, assume that they do exercise editorial discretion. [For a concise overview of Section 230 and its intersection with the First Amendment, see Bollinger and Stone, Social Media, Freedom of Speech, and the Future of our Democracy (Oxford University Press, 2022), especially pp. xxiii-xl; on efforts to reform Section 230, see pp.103-120.] Whatever the merits or demerits of Section 230 may be, it is federal law and appears to grant social media platforms the right to moderate content on their platforms and immunizes them from liability for content posted by the third party.
Cases pending before the U.S. Supreme Court. In February of this year, the U.S. Supreme Court heard arguments about two state laws that may have far-reaching consequences for both First Amendment case law and the status of Section 230. Largely in response to social media platforms barring former President Donald Trump from their platforms in the wake of the January 6 riots, both Texas and Florida enacted laws that limited the ability of social media platforms to control content on their platforms. The Florida law fines platforms if they ban a candidate for office in their state, and requires platforms to disclose information about their moderation policies. The Texas law prohibits platforms from removing content based on its “viewpoint.” Both of these laws directly challenge the provision in Section 230 that expressly allows platforms to remove content. Both laws also raise First Amendment concerns about the platforms’ right not to be associated with views with which they disagree. Both laws provide an interesting point of comparison with the bill under review: Texas and Florida prohibit a platform from denying access to certain people or blocking content on certain topics, while this bill would require the platforms to remove content. The Court is expected to issue a ruling in June of this year. How that ruling would affect this bill is unclear, especially given that this bill moves in the opposite direction of the Florida and Texas laws. However, if the Court decides in favor of the platforms – which many commentators think they will, at least in part – it might suggest that the Court believes that platforms should be given more freedom to self-moderate content without state interference. (See David McCabe, “Social media companies are bracing for Supreme Court arguments on Monday that could fundamentally alter how the platforms police their sites,” New York Times, February 25, 2024; and Adam Liptak, “The Supreme Court seemed skeptical on laws in Florida and Texas,” Id. February 26, 2024.)
Like constitutional and preemption questions, there is no obvious or certain answer as to whether this bill violates the First Amendment or Section 230. The Court may provide some insight soon enough.
Proposed Author Amendments. The author will take the following amendments to the definitions section of the bill:
(a) “Artificial intelligence” means an engineered or machine-based system that varies in its level of autonomy and that can, for explicit or implicit objectives, infer from the input it receives how to generate outputs that can influence physical or virtual environments.
(b) (1) “Elections official” means any of the following persons, but only in their capacity as a person charged with holding or conducting an election, conducting a canvass, assisting with the holding or conducting of an election or a canvas, or performing another duty related to administering the provisions of this code:
(A) An elections official as defined in Section 320.
(B) The Secretary of State and their staff.
(C) A temporary worker, poll worker, or member of a precinct board.
(D) Any other person charged with holding or conducting an election, conducting a canvass, assisting with the holding or conducting of an election or a canvas, or performing another duty related to administering the provisions of this code.
(2) The requirements of this chapter relating to content portraying an elections official apply only if the large online platform knows or should know that the person is an elections official.
(c) (b) “Election processes” means any government process related to an election, including, but not limited to, elections, candidates, vote counting, redistricting, and proceedings or processes of the electoral college.
(d) (c) (1) “Materially deceptive and digitally modified or created content” means an image or an audio or video recording or other digital content, including a chatbot, that has been intentionally manipulated such that all of the following conditions are met:
(A) (i) The digital content is the product of digital manipulation, including, but not limited to, artificial intelligence, artificial intelligence, or machine learning, including deep learning techniques, that merges, combines, replaces, or superimposes content onto an image or an audio or video recording, creating an image or an audio or video recording that appears authentic, or that otherwise generates an inauthentic image or an audio or video recording that appears authentic, and that but contains a false portrayal of any of the following: a candidate for elective office, elected official, elections official, voting machine, ballot, voting site, other property or equipment related to an election, or elections process.
(ii) For purposes of this subdivision, “false portrayal” means the content would cause a reasonable person to have a fundamentally different understanding or impression of the content than the person would have if they were hearing or seeing the unaltered, original an authentic version of the content.
(B) The person or entity who attempted to post or send, or who did post or send, the content did so knowing the portrayal was false, or did so with reckless disregard for whether the portrayal was false. If the content is intentionally manipulated and contains a false portrayal as specified in subparagraph (A), there shall be a rebuttable presumption that the person or entity knew the portrayal was false or that they acted with reckless disregard for whether the portrayal was false.
(2) “Materially deceptive and digitally modified or created content” does not include any image or audio or video recording that contains only minor modifications that do not lead to significant changes to the perceived contents or meaning of the content. Minor changes include changes to the brightness or contrast of images, removal of background noise in audio, and other minor changes that do not impact the content of the image or audio or video recording.
(e) (d) “Large online platform” means a public-facing internet website, web application, or digital application, including a social network, video sharing platform, advertising network, or search engine that had at least 1,000,000 California users during the preceding 12 months.
ARGUMENTS IN SUPPORT: The sponsor of this bill – The California Initiative for Technology and Democracy (CITED) – writes in support of AB 2655:
California and the nation are entering the first-ever generative artificial intelligence (AI) election, in which disinformation powered by generative AI will pollute our information ecosystems like never before. . . Those trying to influence campaigns – conspiracy theorists, foreign states, online trolls, and candidates themselves – are already creating and distributing election-threatening deepfake images, audio, and video content in the US and around the world. . . Examples of this occurring in U.S. elections include Ron Desantis using AI-generated images to attack his opponent in his presidential run, foreign states caught attempting to influence American politics through social media, and just this month, a supporter of former President Trump creating a deepfake image of Trump with Black Americans designed to persuade Black voters to support Trump. These examples demonstrate the power of generative AI-fueled disinformation to skew election results and weaken our faith in our democracy.
AB 2655 strikes the right balance by seeking to ban, for a strictly limited time before and after elections, the online spread of the worst deepfakes and disinformation maliciously intended to prevent voters from voting or getting them to vote erroneously based on fraudulent content. . . AB 2655's approach is narrowly tailored and does not extend the law to hot button controversies or inflammatory claims – it does not ask social media platforms to adjudicate controversial opinions post by post. It simply stops the use of obviously, demonstrably untrue and provably false content meant to impermissibly influence our elections at peak election times.
ARGUMENTS IN OPPOSITION: Opponents of AB 2655 contend that the bill is unnecessary, unwise, unconstitutional, or some combination thereof. A coalition of groups representing the technology and information industry (industry opponents) contend that responsible digital service providers already “take aggressive steps to moderate dangerous and illegal content, consistent with their terms of service. The companies deliver on the commitments made to their user communities with a mix of automated tools and human review.” For example, industry opponents point to the several online businesses that voluntarily participate in “the Digital Trust & Safety Partnership (DTSP) to develop and implement best practices to ensure a safer and more trustworthy internet, and have recently reported on the efforts to implement these commitments.”
Industry opponents believe that AB 2655 falsely assumes that online platforms “definitively know whether any particular piece of content has been manipulated in such a way that is defined under the bill. While digital services may employ tools to identify and detect these materials with some degree of certainty, it is an evolving and imperfect science in its current form. AB 2655 also presumes that online platforms are an appropriate arbiter of deciding what constitutes accurate election information. However, most digital services are not equipped with the tools or expertise to make such judgments.”
In addition to these practical and operational concerns, industry opponents also question the effectiveness of the bill’s approach. For example, they point out that the bill only applies to the largest online platforms, specifically those with at least one million California users. Therefore the bill would not include platforms like Truth Social or Parler (which may relaunch this year) even though they are the ones that produce most of the concern. Opponents also point to the “sweeping” enforcement provisions, most notably the provision that allows “any California resident” to notify the platform of content that, in the resident’s opinion, should have been blocked or labeled. The bill, opponents note, would allow this resident “to bring a civil action against a large online platform if the platform has not responded within 36 hours or if the reporting resident disagrees with the platform’s response.” Confronted with such a restricted timeline and the threat of a civil action, the opponents contend, platforms will “remove significantly more content, including content that has accurate election information and content that is not materially deceptive.”
While industry opponents concentrate on problems of implementation and effectiveness, ACLU California Action focuses on the bill’s constitutional problems. ACLU agrees that protecting “election integrity is a weighty governmental interest,” but under the First Amendment, “that interest may be accomplished . . . only by means that are narrowly tailored.” ACLU points to ample First Amendment case law holding that discussion “of public issues and debate on the qualifications of candidates are integral to the operation of the system of government established by our Constitution.” Quoting Buckley v. Valeo (1976), ACLU writes that the First Amendment affords “the broadest protection to such political expression in order to assure the unfettered interchange of ideas for the bringing about of political and social changes desired by the people,” and, quoting New York Times v Sullivan (1964), ACLU notes that debate on public issues must remain “uninhibited, robust, and wide-open.” Moreover, ACLU notes, the First Amendment affords protection “to even allegedly false statements about public officials and public figures.” ACLU fears that faced with “the prospect of vetting millions of different posts to determine if they are ‘materially deceptive and digitally modified or created,’ many platforms may instead choose to aggressively censor or prohibit speech out of caution, including speech by candidates or relating to entire political topics.” Citing Brown v. Entertainment Merchants Association (2011) and Burstyn v Wilson (1952), ACLU concludes that however much the technology may change, “the basic principles of freedom of speech and the press” do not vary with each new medium of communication. ACLU believes that the provisions of AB 2655, as currently drafted, “threaten to intrude on those rights and deter that vital speech.”
REGISTERED SUPPORT / OPPOSITION:
Support
California Initiative for Technology & Democracy (sponsor)
AFSCME California
Asian Americans Advancing Justice - Asian Law Caucus
Asian Americans and Pacific Islanders for Civic Empowerment
Asian Law Alliance
Bay Rising
California Clean Money Campaign
California Initiative for Technology & Democracy, a Project of California Common CAUSE
California State Sheriffs' Association
California Voter Foundation
Chinese Progressive Association
Courage California
Disability Rights California
Hmong Innovating Politics
Indivisible CA Statestrong
Inland Empire United
League of Women Voters of California
Partnership for the Advancement of New Americans
SEIU California
The Partnership for the Advancement of New Americans
Verified Voting
Opposition
ACLU California Action
Chamber of Progress
Computer and Communications Industry Association
Electronic Frontier Foundation
Internet.Works
NetChoice
Software & Information Industry Association
TechNet
Opposition unless amended
Oakland Privacy
Analysis Prepared by: Tom Clark / JUD. / (916) 319-2334
Date of Hearing: April 23, 2024
ASSEMBLY COMMITTEE ON JUDICIARY
Ash Kalra, Chair
ABPCA Bill Id:AB 2655
Author:(Berman) – As Amended Ver:April 1, 2024
As Proposed to be Amended
SUBJECT: Defending Democracy from Deepfake Deception Act of 2024
KEY ISSUE: should large online platforms be required to block (or in some cases label) materially deceptive and digitally modified or created content related to an election or election process, during a prescribed period of time before or after an election?
SYNOPSIS
According to the author, disinformation powered by Artificial Intelligence (AI), which can be distributed to millions of social media users in an instant, poses a serious threat to our political discourse, our elections, and indeed our democracy. For example, “deepfakes” can generate false sounds and images that could lead to the most discerning viewer to falsely conclude that a candidate, elected official, or election worker said or did something they did not do. Such disinformation not only distorts the truth, it has the potential to undermine people’s confidence in our political institutions. While disinformation can threaten political discourse at any time, the author believes that it is especially harmful during an election season, when uncorrected disinformation may influence an election result or create false concerns about the legitimacy of an election in its immediate aftermath.
This bill, therefore, would require a large online platform to block “materially deceptive and digitally modified or created” content that portrays any candidate, elected official, or elections official doing or saying something they did not do or say; it would also require them to block deceptive material that concerns voting machines, ballots, voting sites, or other procedures or equipment related to an election. In addition, deceptive material about broader “elections processes,” that are not subject to blocking requirement, would need to contain a label that they are materially deceptive and digitally modified. The bill would allow any resident of California to inform the platform that covered material had not been properly blocked or labeled, and if the platform does not respond within 36 hours, or if the reporting resident does not agree with the response, the resident may bring an action for injunctive relief. The bill would also allow the Attorney General, or any district attorney or city attorney, to also seek injunctive relief.
This bill passed out of the Assembly Elections Committee on a 6-1vote. It is sponsored by the California Initiative for Technology and Democracy, a project of California Common Cause, and supported by several political reform groups, among others. The bill is opposed by groups representing the information and technology industry and by ACLU Action California. The opposition argues that the bill would be ineffective, unconstitutional, and preempted by federal law. As the analysis suggests, some of these arguments have considerable merit. The author will take several definitional amendments in this Committee, which are reflected in the Summary, below, and discussed in the analysis.
SUMMARY: Requires large online platforms, as defined, to block the posting or sending of materially deceptive and digitally modified or created content related to elections, or to label that content, during specified periods before and after an election. Specifically, this bill:
- Makes findings and declarations about the growing use of generative artificial intelligence (AI), deepfakes, and related technologies to disseminate disinformation that distorts our electoral process and undermines trust in elections.
- Requires a large online platform (platform), using state-of-the-art, best available tools to detect digitally modified or created content, to develop procedures for blocking and preventing the posting or sending of materially deceptive and digitally modified or created content, and to block and prevent that content if the platform knows or should know that the content meets the following requirements, during a specified time period, of any of the following:
- A candidate portrayed as doing or saying something that the candidate did not do or say.
- An elections official portrayed as doing or saying something in connection with the performance of their elections-related duties that the official did not do or say.
- An elected official portrayed as doing or saying something that influences the election that the elected official did not do or say.
- A voting machine, ballot, voting site, or other property or equipment related to an election that is portrayed in a materially false way.
- Prohibits a platform, notwithstanding the above, from preventing candidates from posting deceptive and manipulated material about themselves so long as the content includes a prescribed disclaimer.
- Provides that the above prohibitions apply only during the period 120 days before and through election day; however, they apply to content relating to elections officials, voting sites, voting machines, ballots, or related equipment from 120 days before the election to 60 days after the election.
- Requires a platform to develop procedures for labeling materially deceptive and digitally modified or created content that pertains to election processes, but that is not subject to the blocking provisions above. Requires the label to indicate that the content is inauthentic, fake, or false if the platform knows or should know as much. Provides that the labeling requirement applies during the period beginning one year before the election or election process, as specified.
- Requires the platform to provide a way for Californians to report content that was not blocked or labeled as required, and requires the platform to respond to the report within 36 hours and to describe any actions taken. If the platform does not respond within 36 hours, or if the reporting resident disagrees with the response, the reporting resident may bring an action for injunctive or other equitable relief, and a prevailing plaintiff may recover reasonable attorney’s fees and costs. Allows the Attorney General, a district attorney, or a city attorney to similarly seek injunctive relief and obtain fees and costs.
- Specifies that the provisions of this bill do not apply to a regularly published online newspaper, magazine, or periodical, as specified.
- Defines the following terms for purposes of the above as follows:
- “Elections official” means any of the following: an elections official as defined in Elections Code Section 320; the Secretary of State and their staff; a temporary worker, poll worker, or member of a precinct board; any other person charged with holding or conducting an election, a canvas, or performing another election-related duty.
- “Election processes” means any government process related to an election, including, but not limited to, elections, candidates, vote counting, redistricting, and proceedings or processes of the Electoral College.
- “Materially deceptive and digitally modified or created content” means content that is the product of digital manipulation, including but not limited to, artificial intelligence that appears authentic but contains a false portrayal of any of the following: a candidate for elective office, an elected official, an elections official, or a voting machine, ballot, voting site, other property or equipment related to an election, or elections process.
- “Large online platform” means a public-facing internet website, web application, or digital application, including a social network, video sharing platform, advertising network, or search engine that had at least 1 million California users during the preceding 12 months.
- “Artificial intelligence” means an engineered or machine-based system that varies in its level of autonomy and that can, for explicit or implicit objectives, infer from the input it receives how to generate outputs that can influence physical or virtual environments.
EXISTING LAW:
- Prohibits a person, committee, or other entity, until January 1, 2027, from distributing with actual malice, within 60 days of an election at which a candidate for elective office will appear on the ballot, materially deceptive audio or visual media of a candidate with the intent to injure the candidate’s reputation or to deceive a voter into voting for or against the candidate.
- Defines “materially deceptive audio or visual media,” for these purposes, as an image or an audio or visual recording of a candidate’s appearance, speech or conduct that has been intentionally manipulated in a manner that both of the following are true about the image or audio or video recording: (1) It would falsely appear to a reasonable person to be authentic; and (2) it would cause a reasonable person to have a fundamentally different understanding or impression of the expressive content of the image or audio or video recording than the person would have if the person were hearing or seeing the unaltered, original version of the image or audio or video recording.
- Provides that this prohibition does not apply if the audio or visual media includes a disclaimer stating “This (image/video/audio) has been manipulated,” and the disclaimer complies with specified requirements.
- Permits a candidate whose voice or likeness appears in deceptive audio or visual media distributed in violation of this provision to seek injunctive relief, as specified, and general or special damages and reasonable attorney’s fees and costs, as specified. Specifies that in any civil action brought pursuant to these provisions, the plaintiff bears the burden of establishing the violation through clear and convincing evidence.
- Provides that this prohibition shall not be construed to alter or negate any rights, obligations, or immunities of an interactive service provider under Section 230 of the federal Communications Decency Act.
- Provides that this prohibition does not apply to a radio or television broadcasting station; an internet website; a regularly published newspaper, magazine, or other periodical of general circulation, including an internet or electronic publication; or media that constitute satire or parody. (Elections Code Section 20010.)
- Prohibits a person, firm, association, corporation, campaign committee, or organization, beginning January 1, 2027, with actual malice, from producing, distributing, publishing, or broadcasting campaign material, as defined, that contains either of the following types of pictures or photographs, as specified, unless the campaign material includes a disclaimer that the picture is not an accurate representation of fact:
- A picture or photograph of a person or persons into which the image of a candidate for public office is superimposed.
- A picture or photograph of a candidate for public office into which the image of another person or persons is superimposed. (Elections Code Section 20010.)
FISCAL EFFECT: As currently in print this bill is keyed fiscal.
COMMENTS: According to the author:
AB 2655 will ensure that online platforms restrict the spread of election-related deceptive deepfakes meant to prevent voters from voting or to deceive them based on fraudulent content. Deepfakes are a powerful and dangerous tool in the arsenal of those that want to wage disinformation campaigns, and they have the potential to wreak havoc on our democracy by attributing speech and conduct to a person that is false or that never happened. Advances in AI make it easy for practically anyone to generate this deceptive content, making it that much more important that we identify and restrict its spread before it has the chance to deceive voters and undermine our democracy.
Therefore, in order to ensure California elections are free and fair, online platforms must prevent the online spread of election-related deceptive deepfakes and disinformation meant to prevent voters from voting or to deceive them based on fraudulent content.
Existing laws maintaining “election integrity.” As aptly noted in the analysis of this bill by the Assembly Elections Committee, the “use of false and deceptive information in campaigns to influence election outcomes is not a new phenomenon,” nor are the laws “aimed at curbing such practices” new. Indeed, in 1850, the First Session of the California State Legislature created penalties for “deceiving [an elector] and causing him to vote for a different person for any office than such elector desired or intended to vote for.” (Chapter 38, Statutes of 1850.) Existing California law has greatly elaborated on these initial legislative efforts. For example, provisions in the Elections Code prohibit the distribution of false and misleading information about qualifications to vote or about the days, dates, times, and places where voting may occur; prohibit the misleading use of government seals in campaign literature (Elections Code Section 18304); and prohibit coercing or deceiving people into voting in a way that is inconsistent with the person’s intent (Elections Code Sections 18302, 18304, 18573 and 18573.5).
In the last five years, the Legislature has turned its attention to “materially deceptive” audio and visual materials that portray a candidate. For example, AB 730 (Chap. 493, Stats. 2019) responded to reports that so-called “deepfake” technology, a software that allows someone to produce audios and videos that look and appear remarkably real to even the most discerning person. For example, in 2018, BuzzFeed and the film director Jordan Peele published a very realistic-looking deepfake showing former President Obama calling then-President Donald Trump a “total and complete dipshit.” Obama did not say that. Peele and BuzzFeed did not use the video to try to influence a political election – indeed halfway through the video the ruse was revealed – but to show the potential for abuse of deepfake technology. Responding to this and similar reports, AB 730 prohibited the distribution of materially deceptive audio or visual media with the intent to injure the candidate’s reputation or to deceive a voter into voting for or against a candidate.
AB 730 was itself an amendment to California’s “Truth in Political Advertising Act” of 1998, which had prohibited campaign material that deceptively altered a picture of a candidate (for example by superimposing one person’s image upon another) unless the picture contained a disclaimer. This 1998 bill was introduced in response to the use of technologies like “photo shopping,” which now seem quaint compared to deepfakes, including deepfakes generated by artificial intelligence. The bill now before the Committee, like AB 730 before it, is apparently an effort to stay one step ahead of evolving technologies, which not only create more realistic-looking deceptions, but make it possible to quickly create, alter, and distribute fake images to millions of people in the blink of an eye (or the click of a mouse).
This bill and existing law: who? AB 2655 elaborates upon and modifies existing law in a variety of ways. Most significant, existing law prohibits a person, committee, or other entity from distributing, with actual malice, “materially deceptive” audio or visual media of a candidate with the intent to injure the candidate’s reputation or to deceive a voter into voting for or against the candidate. Existing law prohibits the distribution of such material within 60 days of the election in which the targeted candidate is running for office. The prohibitions in AB 2655, on the other hand, do not apply to the person or entity that created the materially deceptive material, but to the “large online platform” on which it is posted. AB 2655 requires the online platform to develop and implement procedures to block or prevent the posting of the covered content.
This bill and existing law: what? This bill also differs from existing law in terms of covered content. Existing law applies only to manipulated material that misrepresents the “candidate” saying or doing something the candidate did not do or say. This bill similarly applies to content deceptively depicting the candidate, but also applies to images portraying an “elections official” doing or saying something they did not do or say, as well as images depicting a voting machine, ballot, voting site, or voting equipment in a materially false way. In addition, this bill would require the platform to label (but not necessarily block) materially deceptive content about “election processes” (as opposed to a specific candidate, election official, or voting site). Although the distinction between what material must be blocked versus what material must be labeled is not entirely clear, apparently the intent of the author and sponsor is that the more the material singles out a particular candidate or election official, the more it must be blocked; whereas material that deals with “election processes” more generally need only be labeled. If the bill moves out of this Committee, the author may wish to consider language that will clarify the intended distinction between the blocking and labeling requirements.
This bill and existing law: when? AB 2655 also expands and modifies the relevant time period that prohibitions are in force. Existing law prohibits someone from distributing deceptive material during the 60-day period before an election. This bill, however, requires the platform to block covered material a period beginning 120 days before the election and through the day of the election. However, if the deceptive material pertains to an election official, or deceptively depicts a voting machine, ballot, voting site, or property or equipment related to an election, the platform must also block the content for 60 days after the election. Presumably, this post-election period is intended to prevent false claims – similar to those made in 2020 – that the election process was irregular or otherwise “rigged.” The labeling requirement covers an even larger time frame; it applies during the period beginning one year before the “election.” Somewhat confusingly, the bill also requires labeling for the period beginning one year before an “election process.” “Election processes” – as opposed to an “election” – is defined to include any government process “related” to an election, “including, but not limited to,” elections, candidates, vote counting, redistricting, and proceedings or processes of the electoral college.” Because an “election” has a known date, it should be fairly easy for the platform to figure out when the year-long labeling period starts. However, if the platform must also label one year prior to an “election process,” when does a process like “redistricting” start? Does it start with each new census? Does it start when the legislative body (or in some states a commission) meet to draw up new district lines? Moreover, the definition of “election process” is not limited to the items listed, expressly stating “including, but is not limited to,’ those items. As with the distinction between materials that the platform is required to “block” and those it is required to “label,” the lack of clarity stems from confusion over the distinction between “election processes” and “elections.” Both for example, appear to include “candidates,” “elections,” and “vote counting.”
This bill and existing law: how enforced? In addition to differences as to who, what, and when, this bill also differs in how violations would be enforced. Existing law permits only the “candidate” whose voice or likeness appears in the deceptive material to bring an action for injunctive relief, general or special damages, and reasonable attorney’s fee and cost. However, under existing law the candidate bears the burden of establishing a violation by “clear and convincing evidence.” Enforcement provisions in this bill are much different. The bill allows any “California resident” to report to the platform that content was not blocked or labeled as required. If the platform does not respond within 36 hours, or if the resident disagrees with the response, the resident may seek injunctive relief to compel compliance and, if the resident prevails, shall be awarded reasonable attorney’s fees and costs. Thus, not only can any California resident – not just a person depicted or otherwise affected – seek injunctive relief, they apparently do not have the burden of proving a violation by clear and convincing evidence. If they “disagree” with the platform’s response, that’s enough; they can seek injunctive relief and a court would decide to issue on the likelihood of success on merits, the general rule for injunctive relief.
First Amendment concerns. Because this bill imposes a government mandate that online platforms must block expressive material based upon its content, it creates significant First Amendment concerns. The First Amendment provides that “Congress shall make no law . . . prohibiting the freedom of speech.” As interpreted by the courts and incorporated against the states by the due process clause of the 14th Amendment, the First Amendment prevents any government entity (not just Congress) from enacting any law or adopting any policy that burdens freedom of speech. In addition, Article I, Section 2 of the California Constitution guarantees to every person the freedom to “speak, write, and publish his or her sentiments on all subjects, being responsible for the abuse of this right.” Moreover, the First Amendment not only protects the right to speak, as a logical corollary it protects the “right to receive information and ideas.” (Stanley v Georgia (1969) 394 U.S. 557, 564.) This bill would interfere with both the expression and reception of information based upon its content. Moreover, not only does this bill single out particular content, the content relates to political candidates and elections. This is potentially problematic because the First Amendment affords the “broadest protection” to the “discussion of public issues” and “political expression in order to assure the unfettered interchange of ideas for the bringing about of political and social changes desired by the people.” (McIntyre v Ohio Election Commission (1997) 514 U.S. 334.) It is difficult to imagine any content more related to “political expression” and “discussion of public issues” than content about candidates and elections. The fact that the bill restricts speech that is “materially deceptive” or “false” does not matter, for the U.S. Supreme Court has been unequivocal that the First Amendment protects even “false” speech. The remedy for false speech is more true speech, and false speech tends to call forth true speech. (United States v Alvarez (2012) 567 U.S. 709.)
The bill will likely meet the “compelling interest” threshold, if not the “narrowly tailored” threshold. The right to free speech is not absolute. As Justice Holmes noted in a dissenting opinion over a century ago, the First Amendment does not protect a right to falsely cry fire in a crowded theater. (Schenck v. United States (1919) 249 U.S. 47.) The proponents of this bill may, not unreasonably, think that pervasive disinformation about candidates and elections, especially during an election season, is just as dangerous; but that analogy will not be enough for the Court. Rather, a court will apply strict scrutiny. This means that government can impose even content-based restrictions on protected speech if they have a “compelling government interest” and they use “narrowly tailored means” to achieve that interest.
Even the opponents of this bill appear to concede that maintaining “election integrity” is a “weighty” and, presumably, “compelling” government interest. Therefore, it seems likely that if this bill is enacted and subsequently challenged, a court will accept that there is a “compelling interest” but still need to consider whether the “means” are “narrowly tailored.” The opponents of this measure claim that it is not narrowly tailored. They contend that while covered platforms may have state-of-the-art tools that allow them to identify content that has been “digitally modified,” there is no technology to determine if content is “materially deceptive.” The bill defines “materially deceptive,” in relevant part, to mean content that is “intentionally manipulated” so that it appears “authentic” but contains a “false portrayal” of a candidate, elected official, elections official, voting machine, ballot, voting site, other property or equipment related to an election, or elections process. The purpose of narrow tailoring is to ensure that no more speech is infringed or burdened than is necessary. However, the opponents of this bill – both the industry groups and the ACLU – believe that with no sure means to determine what is “materially deceptive,” the platforms will err on the side of blocking content, thus burdening more speech than is necessary.
Even the findings and declarations of the bill in print seem to concede, by its silence, that not all parts of the bill are narrowly tailored. The findings and declarations confidently assert that the “labeling information required by this bill is narrowly tailored to provide consumers with factual information about the inauthenticity of particular images, audio, video, or text content in order to prevent consumer deception.” Tellingly, there is no similar claim about the blocking requirement being narrowly tailored. In any event, it will be a court – not the findings and declarations of the bill – that will determine whether the bill is narrowly tailored. The court may consider, for example, if there are other less restrictive and more effective means of protecting election integrity.
Section 230 concerns and “editorial discretion.” In addition to implicating the First Amendment, this bill may also be preempted by Section 230 of the federal Communications Decency Act. In relevant part, Section 230 provides two express protections for online platforms and their ability to moderate online content. First, Section 230 declares that an online platform cannot be held liable for content posted by third parties. The rationale for this immunity is premised on the idea that online platforms, unlike newspapers, do not exercise editorial discretion; rather, like telephone companies or “common carriers” they are merely a conduit for the expression of ideas by others. Those others, not the platform, are liable for any harm caused by the content. Second, somewhat in tension with this immunity, Section 230 expressly provides that online platforms are not liable if they block or remove material because they disapprove of its content. While it is important to remember that the First Amendment is distinct from Section 230, this protection flows from First Amendment principles. First, because the online platform is not a government actor, it cannot violate the First Amendment. Second, as a private actor, the platform has its own First Amendment right not to be associated with speech it finds objectionable. Some scholars have noted that the two protections in Section 230 are based on contrary premises. Immunity from liability from postings by third parties assumes that platforms do not exercise editorial discretion. Their right to remove content without liability, and their own free speech claims, on the other hand, assume that they do exercise editorial discretion. [For a concise overview of Section 230 and its intersection with the First Amendment, see Bollinger and Stone, Social Media, Freedom of Speech, and the Future of our Democracy (Oxford University Press, 2022), especially pp. xxiii-xl; on efforts to reform Section 230, see pp.103-120.] Whatever the merits or demerits of Section 230 may be, it is federal law and appears to grant social media platforms the right to moderate content on their platforms and immunizes them from liability for content posted by the third party.
Cases pending before the U.S. Supreme Court. In February of this year, the U.S. Supreme Court heard arguments about two state laws that may have far-reaching consequences for both First Amendment case law and the status of Section 230. Largely in response to social media platforms barring former President Donald Trump from their platforms in the wake of the January 6 riots, both Texas and Florida enacted laws that limited the ability of social media platforms to control content on their platforms. The Florida law fines platforms if they ban a candidate for office in their state, and requires platforms to disclose information about their moderation policies. The Texas law prohibits platforms from removing content based on its “viewpoint.” Both of these laws directly challenge the provision in Section 230 that expressly allows platforms to remove content. Both laws also raise First Amendment concerns about the platforms’ right not to be associated with views with which they disagree. Both laws provide an interesting point of comparison with the bill under review: Texas and Florida prohibit a platform from denying access to certain people or blocking content on certain topics, while this bill would require the platforms to remove content. The Court is expected to issue a ruling in June of this year. How that ruling would affect this bill is unclear, especially given that this bill moves in the opposite direction of the Florida and Texas laws. However, if the Court decides in favor of the platforms – which many commentators think they will, at least in part – it might suggest that the Court believes that platforms should be given more freedom to self-moderate content without state interference. (See David McCabe, “Social media companies are bracing for Supreme Court arguments on Monday that could fundamentally alter how the platforms police their sites,” New York Times, February 25, 2024; and Adam Liptak, “The Supreme Court seemed skeptical on laws in Florida and Texas,” Id. February 26, 2024.)
Like constitutional and preemption questions, there is no obvious or certain answer as to whether this bill violates the First Amendment or Section 230. The Court may provide some insight soon enough.
Proposed Author Amendments. The author will take the following amendments to the definitions section of the bill:
(a) “Artificial intelligence” means an engineered or machine-based system that varies in its level of autonomy and that can, for explicit or implicit objectives, infer from the input it receives how to generate outputs that can influence physical or virtual environments.
(b) (1) “Elections official” means any of the following persons, but only in their capacity as a person charged with holding or conducting an election, conducting a canvass, assisting with the holding or conducting of an election or a canvas, or performing another duty related to administering the provisions of this code:
(A) An elections official as defined in Section 320.
(B) The Secretary of State and their staff.
(C) A temporary worker, poll worker, or member of a precinct board.
(D) Any other person charged with holding or conducting an election, conducting a canvass, assisting with the holding or conducting of an election or a canvas, or performing another duty related to administering the provisions of this code.
(2) The requirements of this chapter relating to content portraying an elections official apply only if the large online platform knows or should know that the person is an elections official.
(c) (b) “Election processes” means any government process related to an election, including, but not limited to, elections, candidates, vote counting, redistricting, and proceedings or processes of the electoral college.
(d) (c) (1) “Materially deceptive and digitally modified or created content” means an image or an audio or video recording or other digital content, including a chatbot, that has been intentionally manipulated such that all of the following conditions are met:
(A) (i) The digital content is the product of digital manipulation, including, but not limited to, artificial intelligence, artificial intelligence, or machine learning, including deep learning techniques, that merges, combines, replaces, or superimposes content onto an image or an audio or video recording, creating an image or an audio or video recording that appears authentic, or that otherwise generates an inauthentic image or an audio or video recording that appears authentic, and that but contains a false portrayal of any of the following: a candidate for elective office, elected official, elections official, voting machine, ballot, voting site, other property or equipment related to an election, or elections process.
(ii) For purposes of this subdivision, “false portrayal” means the content would cause a reasonable person to have a fundamentally different understanding or impression of the content than the person would have if they were hearing or seeing the unaltered, original an authentic version of the content.
(B) The person or entity who attempted to post or send, or who did post or send, the content did so knowing the portrayal was false, or did so with reckless disregard for whether the portrayal was false. If the content is intentionally manipulated and contains a false portrayal as specified in subparagraph (A), there shall be a rebuttable presumption that the person or entity knew the portrayal was false or that they acted with reckless disregard for whether the portrayal was false.
(2) “Materially deceptive and digitally modified or created content” does not include any image or audio or video recording that contains only minor modifications that do not lead to significant changes to the perceived contents or meaning of the content. Minor changes include changes to the brightness or contrast of images, removal of background noise in audio, and other minor changes that do not impact the content of the image or audio or video recording.
(e) (d) “Large online platform” means a public-facing internet website, web application, or digital application, including a social network, video sharing platform, advertising network, or search engine that had at least 1,000,000 California users during the preceding 12 months.
ARGUMENTS IN SUPPORT: The sponsor of this bill – The California Initiative for Technology and Democracy (CITED) – writes in support of AB 2655:
California and the nation are entering the first-ever generative artificial intelligence (AI) election, in which disinformation powered by generative AI will pollute our information ecosystems like never before. . . Those trying to influence campaigns – conspiracy theorists, foreign states, online trolls, and candidates themselves – are already creating and distributing election-threatening deepfake images, audio, and video content in the US and around the world. . . Examples of this occurring in U.S. elections include Ron Desantis using AI-generated images to attack his opponent in his presidential run, foreign states caught attempting to influence American politics through social media, and just this month, a supporter of former President Trump creating a deepfake image of Trump with Black Americans designed to persuade Black voters to support Trump. These examples demonstrate the power of generative AI-fueled disinformation to skew election results and weaken our faith in our democracy.
AB 2655 strikes the right balance by seeking to ban, for a strictly limited time before and after elections, the online spread of the worst deepfakes and disinformation maliciously intended to prevent voters from voting or getting them to vote erroneously based on fraudulent content. . . AB 2655's approach is narrowly tailored and does not extend the law to hot button controversies or inflammatory claims – it does not ask social media platforms to adjudicate controversial opinions post by post. It simply stops the use of obviously, demonstrably untrue and provably false content meant to impermissibly influence our elections at peak election times.
ARGUMENTS IN OPPOSITION: Opponents of AB 2655 contend that the bill is unnecessary, unwise, unconstitutional, or some combination thereof. A coalition of groups representing the technology and information industry (industry opponents) contend that responsible digital service providers already “take aggressive steps to moderate dangerous and illegal content, consistent with their terms of service. The companies deliver on the commitments made to their user communities with a mix of automated tools and human review.” For example, industry opponents point to the several online businesses that voluntarily participate in “the Digital Trust & Safety Partnership (DTSP) to develop and implement best practices to ensure a safer and more trustworthy internet, and have recently reported on the efforts to implement these commitments.”
Industry opponents believe that AB 2655 falsely assumes that online platforms “definitively know whether any particular piece of content has been manipulated in such a way that is defined under the bill. While digital services may employ tools to identify and detect these materials with some degree of certainty, it is an evolving and imperfect science in its current form. AB 2655 also presumes that online platforms are an appropriate arbiter of deciding what constitutes accurate election information. However, most digital services are not equipped with the tools or expertise to make such judgments.”
In addition to these practical and operational concerns, industry opponents also question the effectiveness of the bill’s approach. For example, they point out that the bill only applies to the largest online platforms, specifically those with at least one million California users. Therefore the bill would not include platforms like Truth Social or Parler (which may relaunch this year) even though they are the ones that produce most of the concern. Opponents also point to the “sweeping” enforcement provisions, most notably the provision that allows “any California resident” to notify the platform of content that, in the resident’s opinion, should have been blocked or labeled. The bill, opponents note, would allow this resident “to bring a civil action against a large online platform if the platform has not responded within 36 hours or if the reporting resident disagrees with the platform’s response.” Confronted with such a restricted timeline and the threat of a civil action, the opponents contend, platforms will “remove significantly more content, including content that has accurate election information and content that is not materially deceptive.”
While industry opponents concentrate on problems of implementation and effectiveness, ACLU California Action focuses on the bill’s constitutional problems. ACLU agrees that protecting “election integrity is a weighty governmental interest,” but under the First Amendment, “that interest may be accomplished . . . only by means that are narrowly tailored.” ACLU points to ample First Amendment case law holding that discussion “of public issues and debate on the qualifications of candidates are integral to the operation of the system of government established by our Constitution.” Quoting Buckley v. Valeo (1976), ACLU writes that the First Amendment affords “the broadest protection to such political expression in order to assure the unfettered interchange of ideas for the bringing about of political and social changes desired by the people,” and, quoting New York Times v Sullivan (1964), ACLU notes that debate on public issues must remain “uninhibited, robust, and wide-open.” Moreover, ACLU notes, the First Amendment affords protection “to even allegedly false statements about public officials and public figures.” ACLU fears that faced with “the prospect of vetting millions of different posts to determine if they are ‘materially deceptive and digitally modified or created,’ many platforms may instead choose to aggressively censor or prohibit speech out of caution, including speech by candidates or relating to entire political topics.” Citing Brown v. Entertainment Merchants Association (2011) and Burstyn v Wilson (1952), ACLU concludes that however much the technology may change, “the basic principles of freedom of speech and the press” do not vary with each new medium of communication. ACLU believes that the provisions of AB 2655, as currently drafted, “threaten to intrude on those rights and deter that vital speech.”
REGISTERED SUPPORT / OPPOSITION:
Support
California Initiative for Technology & Democracy (sponsor)
AFSCME California
Asian Americans Advancing Justice - Asian Law Caucus
Asian Americans and Pacific Islanders for Civic Empowerment
Asian Law Alliance
Bay Rising
California Clean Money Campaign
California Initiative for Technology & Democracy, a Project of California Common CAUSE
California State Sheriffs' Association
California Voter Foundation
Chinese Progressive Association
Courage California
Disability Rights California
Hmong Innovating Politics
Indivisible CA Statestrong
Inland Empire United
League of Women Voters of California
Partnership for the Advancement of New Americans
SEIU California
The Partnership for the Advancement of New Americans
Verified Voting
Opposition
ACLU California Action
Chamber of Progress
Computer and Communications Industry Association
Electronic Frontier Foundation
Internet.Works
NetChoice
Software & Information Industry Association
TechNet
Opposition unless amended
Oakland Privacy
Analysis Prepared by: Tom Clark / JUD. / (916) 319-2334
Date of Hearing: April 10, 2024
ASSEMBLY COMMITTEE ON ELECTIONS
Gail Pellerin, Chair
ABPCA Bill Id:AB 2655 (
Author:Berman) – As Amended Ver:April 1, 2024
SUBJECT: Elections: deceptive audio or visual media.
SUMMARY: Requires large online platforms, as defined, to block the posting or sending of materially deceptive and digitally modified or created content related to elections, or to label that content, during specified periods before and after an election. Specifically, this bill:
- Requires a large online platform (platform), using state-of-the-art, best available tools to detect digitally modified or created content, to develop procedures for blocking and preventing the posting or sending of materially deceptive and digitally modified or created content, and to block and prevent that content if the platform knows or should know that the content meets the following requirements, during a specified time period, of any of the following:
- A candidate portrayed as doing or saying something that the candidate did not do or say.
- An elections official portrayed as doing or saying something in connection with the performance of their elections-related duties that the official did not do or say.
- An elected official portrayed as doing or saying something that influences the election that the elected official did not do or say.
- A voting machine, ballot, voting site, or other property or equipment related to an election that is portrayed in a materially false way.
- A candidate portrayed as doing or saying something that the candidate did not do or say.
- Prohibits a platform from preventing a candidate, notwithstanding the prohibition detailed above, from portraying themself as doing or saying something that the candidate did not do or say if the digital content includes a disclaimer stating: “This (image/video/audio) has been manipulated.” Requires this disclaimer to comply with the following:
- In the case of visual media, requires the text of the disclaimer to appear in a size that is easily readable, as specified.
- In the case of a video, requires the disclaimer to appear for the duration of the video.
- In the case of media that consists of audio only, requires the disclaimer to be read in a manner that can be easily heard by the average listener at both the beginning and the end of the audio. For audio that is longer than two minutes, the disclaimer must also be included during the audio at intervals of not more than two minutes each.
- In the case of visual media, requires the text of the disclaimer to appear in a size that is easily readable, as specified.
- Provides that the provisions detailed above apply only during the following time periods:
- Beginning 120 days before any election through the day of the election.
- With respect to content pertaining to elections officials, or that depicts or pertains to elections equipment and materials, beginning 120 days before the election and ending on the 60th day after the election.
- Beginning 120 days before any election through the day of the election.
- Requires a platform, using state-of-the-art, best available tools to detect digitally modified or created content, to develop procedures for labeling materially deceptive and digitally modified or created content that pertains to election processes and that is not subject to the blocking provisions outlined above, and to label such content as inauthentic, fake, or false if the platform knows or should know that the digitally modified or created content meets the requirements of this bill.
- Requires the label to permit users to click or tap on it and to inspect all available provenance data about the digitally modified or created content in an easy-to-understand format.
- Provides that the labeling requirement detailed above applies only during the following time periods:
- The period beginning one year before the election and through the day of the election that is specified in or implicated by the content.
- The period beginning one year before the election process and through the final day of the election process that is specified in or implicated by the content.
- If the content depicts or pertains to elections officials, the period beginning one year before the election or election process that is specified in or implicated by the content and ending on the 60th day after that election or the 60th day after the final day of that election process, as applicable.
- The period beginning one year before the election and through the day of the election that is specified in or implicated by the content.
- Requires the label to permit users to click or tap on it and to inspect all available provenance data about the digitally modified or created content in an easy-to-understand format.
- Requires a platform to provide a way for Californians to report content that was not blocked or labeled as required. Requires the platform to respond within 36 hours and to describe any action taken.
- Permits a Californian who reported content and who does not receive a response in 36 hours or who disagrees with the response, the Attorney General (AG), a district attorney, or a city attorney to seek injunctive or other equitable relief against a platform to compel compliance with this bill. Requires the court to award a prevailing plaintiff reasonable attorney’s fees and costs. Provides that such an action is entitled to precedence in court, as specified.
- Requires any label or disclaimer that must appear on content under this bill to appear in English, and in the same language as the content if the content isn’t in English.
- Requires a platform to maintain a copy of any content that it blocks or labels under this bill for at least five years from the election or election process specified or implicated in the content. Requires the platform to make that content available to the Secretary of State, the Fair Political Practices Commission, and researchers, if requested.
- Provides that this bill does not apply to any of the following:
- A regularly published online newspaper, magazine, or other periodical of general circulation that routinely carries news and commentary of general interest, and that publishes any materially deceptive and digitally altered or digitally created image, audio, or video recording that an online platform is required to block or label by this bill, if the publication contains a clear disclosure that the materially deceptive and digitally altered or digitally created image or audio or video recording does not accurately represent any actual event, occurrence, appearance, speech, or expressive conduct.
- Materially deceptive audio or visual media that constitutes satire or parody.
- A regularly published online newspaper, magazine, or other periodical of general circulation that routinely carries news and commentary of general interest, and that publishes any materially deceptive and digitally altered or digitally created image, audio, or video recording that an online platform is required to block or label by this bill, if the publication contains a clear disclosure that the materially deceptive and digitally altered or digitally created image or audio or video recording does not accurately represent any actual event, occurrence, appearance, speech, or expressive conduct.
- Defines the following terms, for the purposes of this bill:
- “Election processes” to mean any government process related to an election, including, but not limited to, elections, candidates, vote counting, redistricting, and proceedings or processes of the electoral college.
- “Materially deceptive and digitally modified or created content” to mean an image or an audio or video recording or other digital content, including a chatbot, that has been intentionally manipulated such that all of the following conditions are met:
- The digital content is the product of digital manipulation, artificial intelligence (AI), or machine learning, including deep learning techniques, that merges, combines, replaces, or superimposes content onto an image or an audio or video recording, creating an image or an audio or video recording that appears authentic, or that otherwise generates an inauthentic image or an audio or video recording that appears authentic.
- The content contains a false portrayal of a candidate for elective office, an elected official, an elections official, or a voting machine, ballot, voting site, other property or equipment related to an election, or elections process. Provides that a “false portrayal” means the content would cause a reasonable person to have a fundamentally different understanding or impression of the content than the person would have if they were hearing or seeing the unaltered, original version of the content.
- The person or entity who attempted to post or send, or who did post or send, the content did so knowing the portrayal was false, or did so with reckless disregard for whether the portrayal was false. Provides that if the content is intentionally manipulated and contains a false portrayal, there is a rebuttable presumption that the person or entity knew the portrayal was false or that they acted with reckless disregard for whether the portrayal was false.
- The digital content is the product of digital manipulation, artificial intelligence (AI), or machine learning, including deep learning techniques, that merges, combines, replaces, or superimposes content onto an image or an audio or video recording, creating an image or an audio or video recording that appears authentic, or that otherwise generates an inauthentic image or an audio or video recording that appears authentic.
- “Large online platform” to mean a public-facing internet website, web application, or digital application, including a social network, video sharing platform, advertising network, or search engine that had at least 1 million California users during the preceding 12 months.
- “Election processes” to mean any government process related to an election, including, but not limited to, elections, candidates, vote counting, redistricting, and proceedings or processes of the electoral college.
- Provides that content that contains only minor modifications that do not lead to significant changes to the perceived contents or meaning of the content are not “materially deceptive and digitally modified or created content” for the purposes of this bill, as specified.
- Contains various findings and declarations and contains a severability clause.
EXISTING STATE LAW:
- Prohibits a person, committee, or other entity, until January 1, 2027, from distributing with actual malice, within 60 days of an election at which a candidate for elective office will appear on the ballot, materially deceptive audio or visual media of a candidate with the intent to injure the candidate’s reputation or to deceive a voter into voting for or against the candidate.
- Defines “materially deceptive audio or visual media,” for these purposes, as an image or an audio or visual recording of a candidate’s appearance, speech or conduct that has been intentionally manipulated in a manner that both of the following are true about the image or audio or video recording:
- It would falsely appear to a reasonable person to be authentic; and,
- It would cause a reasonable person to have a fundamentally different understanding or impression of the expressive content of the image or audio or video recording than the person would have if the person were hearing or seeing the unaltered, original version of the image or audio or video recording.
- Provides that this prohibition does not apply if the audio or visual media includes a disclaimer stating “This (image/video/audio) has been manipulated,” and the disclaimer complies with specified requirements.
- Permits a candidate whose voice or likeness appears in deceptive audio or visual media distributed in violation of this provision to seek the following relief:
- Injunctive or other equitable relief prohibiting the distribution of the materially deceptive audio or visual media in violation of this bill. Provides that such an action is entitled to precedence in court, as specified.
- General or special damages against the person, committee, or other entity that distributed that audio or visual media. Permits the court to award reasonable attorney’s fees and costs to a prevailing party in such an action.
- Provides that in any civil action brought pursuant to these provisions, the plaintiff bears the burden of establishing the violation through clear and convincing evidence.
- Provides that this prohibition shall not be construed to alter or negate any rights, obligations, or immunities of an interactive service provider under Section 230 of the federal Communications Decency Act.
- Provides that this prohibition does not apply to any of the following:
- A radio or television broadcasting station, as specified, in either of the following circumstances:
- When it broadcasts materially deceptive audio or visual media as part of a bona fide newscast, news interview, news documentary, or on-the-spot coverage of bona fide news events, if the broadcast clearly acknowledges through content or disclosure that there are questions about the authenticity of the audio or visual media, as specified.
- When it is paid to broadcast materially deceptive audio or visual media.
- When it broadcasts materially deceptive audio or visual media as part of a bona fide newscast, news interview, news documentary, or on-the-spot coverage of bona fide news events, if the broadcast clearly acknowledges through content or disclosure that there are questions about the authenticity of the audio or visual media, as specified.
- An internet website, or a regularly published newspaper, magazine, or other periodical of general circulation, including an internet or electronic publication, that routinely carries news and commentary of general interest, and that publishes materially deceptive audio or visual media covered by this prohibition, if the publication clearly states that the media does not accurately represent the speech or conduct of the candidate.
- Materially deceptive audio or visual media that constitute satire or parody. (Elections Code §20010, as amended by Section 3 of Chapter 745 of the Statutes of 2022)
- A radio or television broadcasting station, as specified, in either of the following circumstances:
- Defines “materially deceptive audio or visual media,” for these purposes, as an image or an audio or visual recording of a candidate’s appearance, speech or conduct that has been intentionally manipulated in a manner that both of the following are true about the image or audio or video recording:
- Prohibits a person, firm, association, corporation, campaign committee, or organization, beginning January 1, 2027, with actual malice, from producing, distributing, publishing, or broadcasting campaign material, as defined, that contains either of the following types of pictures or photographs, as specified, unless the campaign material includes a disclaimer that the picture is not an accurate representation of fact:
- A picture or photograph of a person or persons into which the image of a candidate for public office is superimposed.
- A picture or photograph of a candidate for public office into which the image of another person or persons is superimposed. (Elections Code §20010, as amended by Section 4 of Chapter 745 of the Statutes of 2022)
EXISTING FEDERAL LAW provides, pursuant to Section 230 of the federal Communications Decency Act, that no provider or user of an interactive computer service shall be treated as the publisher or speaker of any information provided by another information content provider. (47 U.S.C. §230)
FISCAL EFFECT: Unknown
COMMENTS:
- Purpose of the Bill: According to the author:
AB 2655 will ensure that online platforms restrict the spread of election-related deceptive deepfakes meant to prevent voters from voting or to deceive them based on fraudulent content. Deepfakes are a powerful and dangerous tool in the arsenal of those that want to wage disinformation campaigns, and they have the potential to wreak havoc on our democracy by attributing speech and conduct to a person that is false or that never happened. Advances in AI make it easy for practically anyone to generate this deceptive content, making it that much more important that we identify and restrict its spread before it has the chance to deceive voters and undermine our democracy.
- Threats of Manipulated Media in Campaign Communications: The use of false and deceptive information in campaigns to influence election outcomes is not a new phenomenon. Laws aimed at curbing such practices and preserving the integrity of elections have a long history in California. In 1850, the First Session of the California State Legislature created penalties for election misconduct, including for “deceiving [an elector] and causing him to vote for a different person for any office than such elector desired or intended to vote for” (Chapter 38, Statutes of 1850). California law today includes various provisions criminalizing deceptive tactics that undermine election integrity or interfere with voters’ ability to participate in elections. This includes laws that prohibit distribution of false and misleading information about qualifications to vote or about the days, dates, times, and places where voting may occur (Elections Code §18302); prohibit the misleading use of government seals in campaign literature (Elections Code §18304); and prohibit coercing or deceiving people into voting in a way that was inconsistent with the person’s intent (Elections Code §§18573, 18573.5).
Advancements in technology have made it increasingly simple to produce false and misleading media that closely resembles authentic content. Moreover, platforms like social media have facilitated the rapid dissemination of deceptive media to large audiences at minimal cost. Given these developments, the potential threat posed by manipulated media to future elections' integrity may be more significant than in the past.
As described in greater detail below, past legislative efforts have addressed concerns about manipulated media’s use to deceive voters during elections. Those laws, however, are limited, and are designed primarily to target the harms to candidates that may result from the distribution of manipulated media of those candidates. In contrast, this bill aims to regulate materially deceptive and digitally altered media depicting not only candidates, but also elections officials and elected officials who are not candidates. Additionally, this bill targets media that portrays elections materials and equipment in materially deceptive ways. The author and supporters of this bill believe that these provisions will safeguard voters against deceitful media that could undermine trust in the electoral process. - Recent Examples of Materially Deceptive Campaign Communications: As evidence of the need for this bill, the author points to the following incidents, as reported in the media:
- AI tools were used to create deepfake video ads of British Prime Minister Sunak on Facebook.
- A Chinese disinformation campaign in Taiwanese elections made use of deepfakes, and AI-generated videos, images and audio clips.
- AI tools additionally were used to disrupt elections in Argentina, Bangladesh, Pakistan, Slovakia.
- Governor Ron DeSantis created a deepfake of former President Trump hugging Anthony Fauci.
- AI was used by a democratic operative and a magician to generate deepfake audio of President Biden’s voice that was used in a robocalls to dissuade voters from voting in the primary election.
- Previous Legislation Related to Materially Deceptive Media in Campaigns: In 2019, in response to concerns that deepfake technology could be used to spread misinformation in political campaigns, the Legislature approved and Governor Newsom signed AB 730 (Berman), Chapter 493, Statutes of 2019. Deepfake technology refers to software capable of producing a realistic looking video of someone saying or doing something that they did not, in fact, say or do. This technology has advanced rapidly in recent years thanks to the use of AI to help train the software.
AB 730 prohibits the distribution of materially deceptive audio or visual media with actual malice with the intent to injure a candidate’s reputation or to deceive a voter into voting for or against a candidate, unless the materially deceptive audio or visual media includes a disclaimer that it has been manipulated. AB 730 does not apply exclusively to deepfakes, but rather applies to any intentional manipulation of audio or visual images that results in a version that a reasonable observer would believe to be authentic. Nonetheless, the increasing availability and advancing capability of deepfake technology was the immediate impetus for that bill.
AB 730 was designed as an update to California’s “Truth in Political Advertising Act,” a law enacted in 1998 (through the passage of AB 1233 (Leach), Chapter 718, Statutes of 1998) that prohibited campaign material that contains a picture of a person into which a candidate's image is superimposed, or contains a picture of a candidate into which another person's image is superimposed, except if a specified disclaimer was included. The Truth in Political Advertising Act was introduced in response to the use of photoshopped pictures in campaign materials, and accordingly was designed to target the manipulation of photographs in campaign materials. In the 20 years following its passage, however, it was never amended to update the law to address more modern techniques of manipulating campaign materials in a manner that can mislead voters. AB 730 replaced the Truth in Political Advertising Act with a law that regulates not only altered photographs in campaign materials, but also audio and video media that have been altered in a materially deceptive manner.
AB 730 included a January 1, 2023 sunset date. In 2022, however, the Legislature approved AB 972 (Berman), Chapter 745, Statutes of 2022, which extended the sunset date to January 1, 2027. AB 972 did not otherwise change the provisions of AB 730. If the current January 1, 2027 sunset date is not repealed or extended, the original Truth in Political Advertising Act as enacted by AB 1233 of 1998 would go back into effect.
Because the impetus for AB 730 was concern about the potential that people might create deepfake media appearing to be accurate representations of the conduct of candidates for office, its provisions apply exclusively to images or audio or video recordings of a candidate’s appearance, speech, or conduct. Relatedly, candidates for elective office who are the target of materially deceptive media are the only entities that can seek injunctive relief or damages under AB 730. Materially deceptive images, audio, or video that appear in campaign communications are not covered by AB 730 if that media is not of a candidate. For instance, if a candidate digitally manipulated video or a photo of a campaign rally to make the crowd look significantly larger than it actually was, such manipulation would not be covered by AB 730 as long as the manipulated image or video was not materially deceptive about a candidate’s appearance, speech, or conduct. Similarly, manipulated and materially deceptive content in advertisements related to ballot measures, or in communications that seek to undermine confidence in the electoral process but that do not mention candidates directly, generally would not be covered by AB 730. - Online Platforms and Materially Deceptive Content: This is one of two bills being considered by the committee today that seek to address materially deceptive and digitally altered elections-related content in an effort to protect the integrity of elections in California. While the related bill that is being considered today (AB 2839 (Pellerin)) applies broadly to the distribution of such content through various mediums, this bill specifically targets the distribution of deceptive content through online platforms, including social media. Recognizing that those online platforms can facilitate the rapid spread of deceptive content, this bill seeks to minimize that potential by obligating large online platforms to block or label offending content. In order to do that, the platforms necessarily will need to be able to identify the content that must be blocked or labeled.
In recognition that the regulation of the distribution of content can create free speech concerns, this bill contains various provisions that tailor the content to which it applies, such that it targets content that has the highest likelihood of deceiving voters and undermining electoral integrity. While that tailoring does limit the content that online platforms would be required to block or label, it also adds additional factors that platforms must consider in order to identify content that is required to be blocked or labeled under this bill.
Along with other limitations, this bill applies only to content that (1) is distributed during specified time periods around elections and election processes, (2) includes media relating to elections or the electoral process in specified ways, (3) that was intentionally manipulated digitally to be materially deceptive, and (4) that is not satire or parody. Each of these limitations adds additional factors that online platforms would need to consider when determining whether a specific communication must be blocked or labeled by this bill.
For example, in order to determine whether it must block content that portrays a candidate for election as doing or saying something that the candidate did not do or say, the platform would need to know not only that the person portrayed in the content was a candidate for office, but also the date (or dates) of the election when the candidate will appear on the ballot. Similarly, it would need to determine whether the candidate had actually said or done the thing that the candidate is portrayed as doing. While some of that information will be widely available and well known in some cases (e.g., the identity of major party candidates for President of the United States in presidential general elections and the dates of federal elections), it will be more arcane in other situations. Given the number of elections (including standalone local and special elections) and candidates (including write-in candidates and candidates for local elections in smaller jurisdictions) in California at any given time, making the determinations at scale about which content must be blocked or labeled likely will be considerably more challenging than making those determinations on a case-by-case basis in a court of law.
This bill includes numerous provisions that recognize that platforms will face challenges in making some of these determinations, and in limiting those platforms’ obligations and liabilities accordingly. For instance, recent amendments to this bill provide for the large online platforms to use “state-of-the-art, best available tools” for detecting digitally modified or created content, recognizing that the identification of such content with perfect accuracy is impossible. Other recent amendments provide that an online platform is obligated to block or label content only if the platform “knows or should know” that the content meets the requirements of the bill, and provide that the bill’s requirements relating to content portraying an elections official applies only if the platform knows or should know that the person is an elections official. Furthermore, unlike the related legislation that targets materially deceptive election content more broadly, this bill does not provide for damages to be awarded against platforms that fail to comply with their obligations. Instead, the only legal relief available under this bill is injunctive relief, and a person (other than the AG, a district attorney, or a city attorney) would be required to report the content to the platform and give the platform an opportunity to block or label the content before the person could seek that relief. Notwithstanding these provisions, the extent to which large online platforms will be able to accurately block and label materially deceptive elections-related content at scale, as contemplated by this bill, is unclear. - Free Speech Considerations: The First Amendment to the United States (US) Constitution, which also applies to states under the Fourteenth Amendment, provides in relevant part "Congress shall make no law…abridging the freedom of speech…” Similarly, Section 2 of Article I of the California Constitution provides in relevant part "Every person may freely speak, write, and publish his or her sentiments on all subjects, being responsible for the abuse of this right. A law may not restrain or abridge liberty of speech or press."
This bill seeks to regulate the distribution by online platforms of media containing intentionally manipulated images, audio, or video related to candidates, elections officials, elected officials, and election materials and equipment under certain circumstances. A question could be raised about whether this bill is consistent with the right to freedom of speech that is guaranteed by the US and California constitutions. The US Supreme Court has ruled that even false statements are protected by the First Amendment (United States v. Alvarez (2012), 567 U.S. 709). When a law burdens core political speech, the restrictions on speech generally must be "narrowly tailored to serve an overriding state interest," McIntyre v. Ohio Elections Commission (1995), 514 US 334.
This bill targets deceptive content that could undermine trust in elections, prevent voters from voting, and distort the electoral process. The US Supreme Court generally has found that the protection of the integrity of elections is an overriding (or compelling) government interest (Id. at 349; Burson v. Freeman (1992) 504 U.S. 191, 199). A challenge of this bill on First Amendment grounds, then, likely would hinge on whether the court found this bill’s provisions to be narrowly tailored.
As discussed in more detail above, this bill includes provisions to limit its scope to communications posing the greatest threat to election integrity. Whether these limitations adequately protect this bill from a potential constitutional challenge is a question that falls more squarely within the jurisdiction of the Assembly Judiciary Committee, where this bill will be heard next if it is approved by this committee. However, while these limitations may help protect the bill against a constitutional challenge, they may also make it harder for the bill to achieve its aims of limiting the spread of materially deceptive communications that have the potential to undermine election integrity. - Arguments in Support: The sponsor of this bill, the California Initiative for Technology & Democracy, a Project of California Common Cause, writes in support:
Those trying to influence campaigns – conspiracy theorists, foreign states, online trolls, and candidates themselves – are already creating and distributing election-threatening deepfake images, audio, and video content in the US and around the world. This threat is not imaginary: generative AI has been used in various ways – most of them deeply deceptive – to influence the national elections in Slovakia, Bangladesh, Argentina, Pakistan, and elsewhere, including in our own country. Examples of this occurring in U.S. elections include Ron Desantis using AI-generated images to attack his opponent in his presidential run, foreign states caught attempting to influence American politics through social media, and just this month, a supporter of former President Trump creating a deepfake image of Trump with Black Americans designed to persuade Black voters to support Trump…
[AB] 2655 strikes the right balance by seeking to ban, for a strictly limited time before and after elections, the online spread of the worst deepfakes and disinformation maliciously intended to prevent voters from voting or getting them to vote erroneously based on fraudulent content. The bill also requires that other fake online content related to elections and elections processes (such as redistricting), which is also designed to undermine election procedures and democratic institutions, must be labeled as fake, again just for a limited time. The bill only applies to the largest online platforms with the greatest reach of potential election disinformation, and we believe it is fully implementable today based on tools these companies already possess. The companies covered by the bill’s requirements are all already subject to similar requirements under the European Union's Digital Services Act, which is designed to, among other things, crack down on election interference.
AB 2655's approach is narrowly tailored and does not extend the law to hot button controversies or inflammatory claims – it does not ask social media platforms to adjudicate controversial opinions post by post. It simply stops the use of obviously, demonstrably untrue and provably false content meant to impermissibly influence our elections at peak election times.
- Arguments in Opposition: A joint letter of opposition submitted by Chamber of Progress, Computer and Communications Industry Association, NetChoice, Software & Information Industry Association, and TechNet states:
Responsible digital services providers take aggressive steps to moderate dangerous and illegal content, consistent with their terms of service. The companies deliver on the commitments made to their user communities with a mix of automated tools and human review. In 2021, a number of online businesses announced that they had been voluntarily participating in the Digital Trust & Safety Partnership (DTSP) to develop and implement best practices to ensure a safer and more trustworthy internet, and have recently reported on the efforts to implement these commitments.
AB 2655 appears to be based on the false assumption that online platforms definitively know whether any particular piece of content has been manipulated in such a way that is defined under the bill. While digital services may employ tools to identify and detect these materials with some degree of certainty, it is an evolving and imperfect science in its current form. AB 2655 also presumes that online platforms are an appropriate arbiter of deciding what constitutes accurate election information. However, most digital services are not equipped with the tools or expertise to make such judgments.
AB 2655 would impose significant operationally and practically challenging requirements on large online platforms who may not be best suited to achieve the bill’s laudable and important goal of ensuring California’s elections remain free and fair.
- Related Legislation: AB 2355 (Wendy Carrillo), which is also being heard in this committee today, requires a political advertisement that is generated in whole or in part using AI to include a disclaimer stating that fact.
AB 2839 (Pellerin), which is also being heard in this committee today, prohibits the distribution of campaign advertisements and other election communications that are materially deceptive and digitally altered or created, except as specified. - Double-Referral: This bill has been double-referred to the Assembly Judiciary Committee.
REGISTERED SUPPORT / OPPOSITION:
Support
California Initiative for Technology & Democracy, a Project of California Common Cause (Sponsor)
Asian Americans Advancing Justice - Asian Law Caucus
Asian Americans and Pacific Islanders for Civic Empowerment
Asian Law Alliance
California Clean Money Campaign
California State Sheriffs' Association
California Voter Foundation
Courage California
Disability Rights California
Indivisible CA Statestrong
Inland Empire United
The Partnership for the Advancement of New Americans
Verified Voting
Opposition
Chamber of Progress
Computer and Communications Industry Association
Electronic Frontier Foundation
NetChoice
Oakland Privacy (unless amended)
Software & Information Industry Association
TechNet
Analysis Prepared by: Ethan Jones / ELECTIONS / (916) 319-2094
Discussed in Hearing
57SEC
Assembly Floor
3MIN
Senate Floor
46SEC
Senate Standing Committee on Appropriations
13MIN
Senate Standing Committee on Judiciary
15MIN
Senate Standing Committee on Elections and Constitutional Amendments
1MIN
Assembly Floor
28MIN
Assembly Standing Committee on Judiciary
51SEC
