Existing law, in a civil action brought by, on behalf of, or for the benefit of a minor, or nonminor dependent, against a person who engaged in an act of commercial sexual exploitation, as defined, of a minor or nonminor dependent, authorizes the trier of fact to either impose a fine, civil penalty, or other penalty, or other remedy in an amount up to 3 times greater than authorized by the statute or to award a civil penalty not exceeding $50,000 and not less than $10,000 for each act of commercial sexual exploitation committed by the defendant if the tier of fact makes an affirmative finding regarding certain factors, including whether the defendants conduct was directed to more than one minor or nonminor dependent, as prescribed.

This bill would, beginning January 1, 2025, prohibit a social media platform, as defined, from knowingly facilitating, aiding, or abetting commercial sexual exploitation, as specified. The bill would require a court to award statutory damages not exceeding $4,000,000 and not less than $1,000,000 for each act of commercial sexual exploitation facilitated, aided, or abetted by the social media platform. The bill would define facilitate, aid, or abet to mean to deploy a system, design, feature, or affordance that is a substantial factor in causing minor users to be victims of commercial sexual exploitation. The bill would prohibit a social media platform from being deemed to be in violation of this provision if it demonstrates certain mitigating facts, including that the social media platform instituted and maintained a program of at least biannual audits of its designs, algorithms, practices, affordances, and features to detect designs, algorithms, practices, affordances, or features that have the potential to cause or contribute to violations of that provision, as prescribed.

Existing law, the California Consumer Privacy Act of 2018, grants to a consumer various rights with respect to personal information, as defined, that is collected by a business, as defined, including the right to request that a business delete personal information about the consumer that the business has collected from the consumer.

This bill would, beginning January 1, 2025, require a social media platform to, among other things, provide, in a mechanism that is reasonably accessible to users, a means for a user who is a California resident to report material to the social media platform that the user reasonably believes meets certain criteria, including that the reported material is child sexual abuse material, as defined, in which the reporting user is an identifiable minor depicted in the reported material.

This bill would, beginning January 1, 2025, require the social media platform to permanently block the instance of reported material and make reasonable efforts to remove and block other instances of the same reported material from being viewable on the social media platform, as prescribed, if there is a reasonable basis to believe that the reported material is child sexual abuse material that is displayed, stored, or hosted on the social media platform, and the report contains basic identifying information, as specified, sufficient to permit the social media platform to locate the reported material. The bill would make a violator of those provisions liable to the reporting user, including for statutory damages of no more than $250,000 per violation, $125,000 per violation, or $75,000 per violation, as specified.

Existing law, in a civil action brought by, on behalf of, or for the benefit of a minor, or nonminor dependent, against a person who engaged in an act of commercial sexual exploitation, as defined, of a minor or nonminor dependent, authorizes the trier of fact to either impose a fine, civil penalty, or other penalty, or other remedy in an amount up to 3 times greater than authorized by the statute or to award a civil penalty not exceeding $50,000 and not less than $10,000 for each act of commercial sexual exploitation committed by the defendant if the tier of fact makes an affirmative finding regarding certain factors, including whether the defendants conduct was directed to more than one minor or nonminor dependent, as prescribed.

This bill would, beginning January 1, 2025, prohibit a social media platform, as defined, from knowingly facilitating, aiding, or abetting commercial sexual exploitation, as specified. The bill would require a court to award statutory damages not exceeding $4,000,000 and not less than $1,000,000 for each act of commercial sexual exploitation facilitated, aided, or abetted by the social media platform. The bill would define facilitate, aid, or abet to mean to deploy a system, design, feature, or affordance that is a substantial factor in causing minor users to be victims of commercial sexual exploitation. The bill would prohibit a social media platform from being deemed to be in violation of this provision if it demonstrates certain mitigating facts, including that the social media platform instituted and maintained a program of at least biannual audits of its designs, algorithms, practices, affordances, and features to detect designs, algorithms, practices, affordances, or features that have the potential to cause or contribute to violations of that provision, as prescribed.

Existing law, the California Consumer Privacy Act of 2018, grants to a consumer various rights with respect to personal information, as defined, that is collected by a business, as defined, including the right to request that a business delete personal information about the consumer that the business has collected from the consumer.

This bill would, beginning January 1, 2025, require a social media platform to, among other things, provide, in a mechanism that is reasonably accessible to users, a means for a user who is a California resident to report material to the social media platform that the user reasonably believes meets certain criteria, including that the reported material is child sexual abuse material, as defined, in which the reporting user is an identifiable minor depicted in the reported material.

This bill would, beginning January 1, 2025, require the social media platform to permanently block the instance of reported material and make reasonable efforts to remove and block other instances of the same reported material from being viewable on the social media platform, as prescribed, if there is a reasonable basis to believe that the reported material is child sexual abuse material that is displayed, stored, or hosted on the social media platform, and the report contains basic identifying information, as specified, sufficient to permit the social media platform to locate the reported material. The bill would make a violator of those provisions liable to the reporting user, including for statutory damages of no more than $250,000 per violation, $125,000 per violation, or $75,000 per violation, as specified.

Existing law, in a civil action brought by, on behalf of, or for the benefit of a minor, or nonminor dependent, against a person who engaged in an act of commercial sexual exploitation, as defined, of a minor or nonminor dependent, authorizes the trier of fact to either impose a fine, civil penalty, or other penalty, or other remedy in an amount up to 3 times greater than authorized by the statute or to award a civil penalty not exceeding $50,000 and not less than $10,000 for each act of commercial sexual exploitation committed by the defendant if the tier of fact makes an affirmative finding regarding certain factors, including whether the defendants conduct was directed to more than one minor or nonminor dependent, as prescribed.

This bill would would, beginning January 1, 2025, prohibit a social media platform, as defined, from knowingly facilitating, aiding, or abetting commercial sexual exploitation. exploitation, as specified. The bill would require a court to award statutory damages not exceeding $4,000,000 and not less than $1,000,000 for each act of commercial sexual exploitation facilitated, aided, or abetted by the social media platform. The bill would define facilitate, aid, or abet to mean to deploy a system, design, feature, or affordance that is a substantial factor in causing minor users to be victims of commercial sexual exploitation. The bill would prohibit a social media platform from being deemed to be in violation of this provision if it demonstrates certain mitigating facts, including that the social media platform instituted and maintained a program of at least quarterly biannual audits of its designs, algorithms, practices, affordances, and features to detect and correct designs, algorithms, practices, affordances, or features that have the potential to cause or contribute to violations of that provision, as prescribed.

Existing law, the California Consumer Privacy Act of 2018, grants to a consumer various rights with respect to personal information, as defined, that is collected by a business, as defined, including the right to request that a business delete personal information about the consumer that the business has collected from the consumer.

This bill would would, beginning January 1, 2025, require a social media platform to, among other things, provide, in a mechanism that is reasonably accessible to users, a means for a user who is a California resident to report material to the social media platform that the user reasonably believes meets certain criteria, including that the reported material is child sexual abuse material, as defined, in which the reporting user is an identifiable minor depicted in the reported material.

This bill would would, beginning January 1, 2025, require the social media platform to permanently block the instance of reported material and make reasonable efforts to remove and block other instances of the same reported material from being viewable on the social media platform, as prescribed, if there is a reasonable basis to believe that the reported material is child sexual abuse material that is displayed, stored, or hosted on the social media platform, and the report contains basic identifying information, as specified, sufficient to permit the social media platform to locate the reported material. The bill would make a violator of those provisions liable to the reporting user, including for statutory damages of no more than $250,000 per violation or violation, $125,000 per violation, or $75,000 per violation, as specified.

Existing law, in a civil action brought by, on behalf of, or for the benefit of a minor, or nonminor dependent, against a person who engaged in an act of commercial sexual exploitation, as defined, of a minor or nonminor dependent, authorizes the trier of fact to either impose a fine, civil penalty, or other penalty, or other remedy in an amount up to 3 times greater than authorized by the statute or to award a civil penalty not exceeding $50,000 and not less than $10,000 for each act of commercial sexual exploitation committed by the defendant if the tier of fact makes an affirmative finding regarding certain factors, including whether the defendants conduct was directed to more than one minor or nonminor dependent, as prescribed.

This bill would prohibit a social media platform, as defined, from knowingly, recklessly, or negligently knowingly facilitating, aiding, or abetting commercial sexual exploitation. The bill would require a court to award statutory damages of $5,000,000 for a violation that is knowing or reckless not exceeding $4,000,000 and not less than $1,000,000 for each act of commercial sexual exploitation facilitated, aided, or abetted by the social media platform, as specified. platform. The bill would define facilitate, aid, or abet to mean to deploy a system, design, feature, or affordance that is a substantial factor in causing minor users to be victims of commercial sexual exploitation. The bill would prohibit a social media platform from being deemed to be in violation of this provision if it demonstrates certain mitigating facts, including that the social media platform instituted and maintained a program of at least quarterly audits of its designs, algorithms, practices, affordances, and features to detect and correct designs, algorithms, practices, affordances, or features that have the potential to cause or contribute to violations of that provision, as prescribed.

Existing law, the California Consumer Privacy Act of 2018, grants to a consumer various rights with respect to personal information, as defined, that is collected by a business, as defined, including the right to request that a business delete personal information about the consumer that the business has collected from the consumer.

This bill would require a social media platform to, among other things, provide, in a form mechanism that is reasonably accessible to users, a means for a user who is a California resident to report material to the social media platform that the user reasonably believes meets certain criteria, including that the reported material is child sexual abuse material, as defined, in which the reporting user is an identifiable minor depicted in the reported material.

This bill would require the social media platform to permanently block the instance of reported material and make reasonable efforts to remove and block other instances of the same reported material from being viewable on the social media platform, as prescribed, if there is a reasonable basis to believe that the reported material is child sexual abuse material that is displayed, stored, or hosted on the social media platform. platform, and the report contains basic identifying information, as specified, sufficient to permit the social media platform to locate the reported material. The bill would make a violator of those provisions liable to the reporting user, including for statutory damages of no more than $250,000 per violation or $125,000 per violation, as specified.

Existing law, in a civil action brought by, on behalf of, or for the benefit of a minor, or nonminor dependent, against a person who engaged in an act of commercial sexual exploitation, as defined, of a minor or nonminor dependent, authorizes the trier of fact to either impose a fine, civil penalty, or other penalty, or other remedy in an amount up to 3 times greater than authorized by the statute or to award a civil penalty not exceeding $50,000 and not less than $10,000 for each act of commercial sexual exploitation committed by the defendant if the tier of fact makes an affirmative finding regarding certain factors, including whether the defendants conduct was directed to more than one minor or nonminor dependent, as prescribed.

This bill would prohibit a social media platform, as defined, from knowingly, recklessly, or negligently facilitating, aiding, or abetting commercial sexual exploitation. The bill would require a court to award statutory damages not exceeding of $5,000,000 and not less than $1,000,000 for a violation that is knowing or reckless for each act of commercial sexual exploitation facilitated, aided, or abetted by the social media platform. platform, as specified. The bill would define facilitate, aid, or abet to mean to deploy a system, design, feature, or affordance that is a substantial factor in causing minor users to be victims of commercial sexual exploitation. The bill would prohibit a social media platform from being deemed to be in violation of this provision if it demonstrates certain mitigating facts, including that the social media platform instituted and maintained a program of at least quarterly audits of its designs, algorithms, practices, affordances, and features to detect and correct designs, algorithms, practices, affordances, or features that have the potential to cause or contribute to violations of that provision, as prescribed.

Existing law, the California Consumer Privacy Act of 2018, grants to a consumer various rights with respect to personal information, as defined, that is collected by a business, as defined, including the right to request that a business delete personal information about the consumer that the business has collected from the consumer.

This bill would require a social media platform to, among other things, provide, in a form that is reasonably accessible to users, a means for a user who is a California resident to report material to the social media platform that the user reasonably believes meets certain criteria, including that the reported material is child sexual abuse material, as defined, in which the reporting user is an identifiable minor depicted in the reported material.

This bill would require the social media platform to permanently block the reported material from being viewable on the social media platform, as prescribed, if the reported material is child sexual abuse material that is displayed, stored, or hosted on the social media platform. The bill would make a violator of those provisions liable to the reporting user, including for statutory damages of no more than $250,000 per violation, violation or $125,000 per violation, as specified.

Existing law, in a civil action brought by, on behalf of, or for the benefit of a minor, or nonminor dependent, against a person who engaged in an act of commercial sexual exploitation, as defined, of a minor or nonminor dependent, authorizes the trier of fact to either impose a fine, civil penalty, or other penalty, or other remedy in an amount up to 3 times greater than authorized by the statute or to award a civil penalty not exceeding $50,000 and not less than $10,000 for each act of commercial sexual exploitation committed by the defendant if the tier of fact makes an affirmative finding regarding certain factors, including whether the defendants conduct was directed to more than one minor or nonminor dependent, as prescribed.

This bill would prohibit a social media platform, as defined, from knowingly, recklessly, or negligently facilitating, aiding, or abetting commercial sexual exploitation. The bill would require a court to award a civil penalty statutory damages not exceeding $5,000,000 and not less than $1,000,000 for each act of commercial sexual exploitation facilitated, aided, or abetted by the social media platform. The bill would define facilitate, aid, or abet to include, but not be limited to, being directly involved in, or actively participating in, the development, deployment, or enforcement of mean to deploy a system, design, feature, or affordance that the social media platform knows, or should know, foreseeably causes child is a substantial factor in causing minor users to be victims of commercial sexual exploitation.

Existing law, the California Consumer Privacy Act of 2018, grants to a consumer various rights with respect to personal information, as defined, that is collected by a business, as defined, including the right to request that a business delete personal information about the consumer that the business has collected from the consumer. Existing law authorizes a minor to disaffirm a contract before majority or within a reasonable time afterwards or, in case of the minors death within that period, by the minors heirs or personal representative.

Existing law, in a civil action brought by, on behalf of, or for the benefit of a minor, or nonminor dependent, against a person who engaged in an act of commercial sexual exploitation, as defined, of a minor or nonminor dependent, authorizes the trier of fact to either impose a fine, civil penalty, or other penalty, or other remedy in an amount up to 3 times greater than authorized by the statute or to award a civil penalty not exceeding $50,000 and not less than $10,000 for each act of commercial sexual exploitation committed by the defendant if the tier of fact makes an affirmative finding regarding certain factors, including whether the defendants conduct was directed to more than one minor or nonminor dependent, as prescribed.

This bill would prohibit a social media platform, as defined, from knowingly, recklessly, or negligently facilitating, aiding, or abetting commercial sexual exploitation. The bill would require a court to award a civil penalty not exceeding $5,000,000 and not less than $1,000,000 for each act of commercial sexual exploitation facilitated, aided, or abetted by the social media platform. The bill would define facilitate, aid, or abet to include, but not be limited to, being directly involved in, or actively participating in, the development, deployment, or enforcement of a system, design, feature, or affordance that the social media platform knows, or should know, foreseeably causes child users to be victims of commercial sexual exploitation.

Existing law, the California Consumer Privacy Act of 2018, grants to a consumer various rights with respect to personal information, as defined, that is collected by a business, as defined, including the right to request that a business delete personal information about the consumer that the business has collected from the consumer. Existing law authorizes a minor to disaffirm a contract before majority or within a reasonable time afterwards or, in case of the minors death within that period, by the minors heirs or personal representative.

This bill would authorize a minor victim of commercial sexual exploitation who disaffirms a contract with a social media platform to submit a request to the social media platform, pursuant to a certain procedure, that the social media platform permanently block from being viewable on the social media platform specific photographs or videos, or versions or reproductions in which the minor victim is readily identifiable, as specified, depicting the minor that are related to the minor victims commercial sexual exploitation. The bill would require a social media platform to comply with that request within 30 days of receiving the request.

This bill would authorize a person who is at least 18 years of age who was a minor victim of commercial sexual exploitation to submit a similar request in the same fashion as described above and would require the social media platform to comply with that request within 30 days of receiving the request. The bill would make a social media platform that violates those provisions liable for reasonable attorneys fees and a civil penalty of up to $250,000 for each violation, as specified.

Existing law makes liable for actual and statutory damages a person who knowingly uses anothers name, voice, signature, photograph, or likeness, in any manner, on or in products, merchandise, or goods, or for purposes of advertising or selling, or soliciting purchases of, products, merchandise, goods or services, without that persons prior consent, or, in the case of a minor, the prior consent of the persons parent or legal guardian.

This bill would make it a violation of that provision to violate the provision of this bill described above with respect to a request made by a person who is at least 18 years of age who was a minor victim of commercial sexual exploitation.

CONCURRENCE IN SENATE AMENDMENTS

CSA1 Bill Id:AB 1394

Author:(Wicks and Flora)

As Amended Ver:September 8, 2023

Majority vote

SUMMARY

Requires social media platforms to provide a mechanism for users to report child sexual abuse material in which they are depicted; provides platforms 30 to 60 days after receiving a report to verify the content of the material and block it from reappearing. Also provides victims of commercial sexual exploitation the right to sue social media platforms for having deployed features that were a substantial factor in causing their exploitation.

1. Defines "child sexual abuse material" to mean either of the following: a) child pornography, as defined under federal law; or b) obscene matter, as defined under state law, that depicts a minor personally engaging in, or personally simulating, sexual conduct.
2. Requires a social media platform to provide, in a form that is reasonably accessible, a means for a user who is a California resident to report material to the platform that the user reasonably believes meets all of the following criteria: a) it is child sexual abuse material; b) the user themselves is an identifiable minor depicted in the reported material; and c) the reported material is displayed, stored, or hosted on the social media platform.
3. Requires the social media platform to block material reported under 2), above, if the reported material meets both of the following criteria: a) there is a reasonable basis to believe that the reported material is child sexual abuse material; b) the reported material is displayed, stored, or hosted on the social media platform; and c) the report contains basic identifying information, such as an account identifier, to permit the platform to locate the material.
4. Requires a social media platform to collect from the reporting user information reasonably sufficient to enable the platform to contact the user in writing. The social media platform must meet the following schedule of written communications with the reporting user:
   1. Within 36 hours, the reporting user must be notified that the platform received their report.
   2. Every seven days thereafter, the reporting user must be provided an update as to the status of the social media platform's handling of the reported material, until a final written determination is issued under c), above.
      1. No later than 30 days after the material was first reported, the platform must issue a final written determination stating one of the following: (1)the reported material has been determined to be child sexual abuse material that was displayed, stored, or hosted on the social media platform and has been blocked on the social media platform; (2) the reported material has been determined not to be child sexual abuse material; or (3) the reported material has been determined not to be displayed, stored, or hosted on the social media platform.
5. Provides that if the social media platform cannot comply with the requirements in the timeframe set forth above, due to circumstances beyond its reasonable control, then the platform must comply with these requirements no later than 60 days after the date on which the material was first reported.
6. Holds a social media company liable to a reporting user if it fails to comply with the requirements in paragraphs 2) to 5), above. Available remedies include any actual damages sustained by the reporting user as a result of the violation, and statutory damages of no more than $250,000 per violation (reduced to no more than $125,000 per violation if a company fails to comply with these requirements, but eventually permanently blocks the reported material before a lawsuit is filed. Reduces statutory damages to no more than $75,000 per violation, if the social media platform permanently removes child sexual abuse material identified by the National Center for Missing and Exploited Children within 36 hours.
7. Holds social media platforms liable if they knowingly facilitate, aid, or abet commercial sexual exploitation of minors.
8. Defines "facilitate, aid, or abet" as the deployment of a system, design, feature, or affordance that is a substantial factor in causing minor users to be victims of commercial sexual exploitation.
9. Provides that a court is to award statutory damages of between $1 million and $4 million for each act of commercial sexual exploitation facilitated, aided, or abetted by the social media platform. Further provides that if a platform knowingly or recklessly facilitates, aids, or abets commercial sexual exploitation, or if a violation is part of a pattern or practice of doing so, then the court shall award statutory damages of $5 million per violation.
10. Establishes a process for a social media platform to not be deemed to have facilitated, aided, or abetted commercial sexual exploitation via a)audits of its designs, algorithms, practices, affordances, and features that occur at least biannually and b) mitigation or elimination within 30 days of any such designs, algorithms, practices, affordances, and features that present a reasonably foreseeable risk of violation or contributing to a violation.

Senate Amendments

Delay effective date of bill to January 1, 2025; require that reports of child sexual abuse material include identifying information sufficient for a platform to locate the reported material; reduce statutory damages if the platform participates in the National Center for Missing and Exploited Children program; remove "recklessly" and "negligently" (leaving "knowingly") as standards for facilitating, aiding, or abetting commercial sexual exploitation; change the standard for action to correct a risk identified in an audit from "de minimis" to "reasonably foreseeable"; remove requirement that independent third party perform audit; change frequency of audits from quarterly to biannually, and time for board to receive audit from 30 to 90 days; and establish that the "knowing" standard can be met if child sexual abuse material is reported to a platform for four consecutive months, if the material meets 2); clarifies that portions of multiservice social media platforms that use end-to-end encryption are not subject to this bill; and exempts from this bill’s requirements internet-based services and applications that are owned or operated by 501(c)(3) nonprofits.

COMMENTS

Background. Child sexual abuse material, commonly referred to under the acronym "CSAM," is tragically pervasive on the internet—and not only in its illicit corners, on the so-called "dark web," but also on popular social media websites and applications that billions of people use every day. In 2020, household names such as Discord, Reddit, TikTok, Twitch, and Twitter each self-reported thousands of instances of CSAM on their systems to the National Center for Missing & Exploited Children; Facebook reported millions. (NCMEC, 2020 CyberTipline Reports by Electronic Service Providers (2021), available at https://www.missingkids.org/content/dam/missingkids/pdfs/2020-reports-by-esp.pdf.)

This tragedy is compounded by the fact that certain websites and applications are not only a convenient means for sharing CSAM, but arguably induce its production:

A Forbes review of hundreds of recent TikTok livestreams reveals how viewers regularly use the comments to urge young girls to perform acts that appear to toe the line of child pornography—rewarding those who oblige with TikTok gifts, which can be redeemed for money, or off-platform payments to Venmo, PayPal or Cash App accounts that users list in their TikTok profiles. (Levine, How TikTok Live Became ‘A Strip Club Filled With 15-Year-Olds,’ Forbes (Apr. 27, 2022), available at https://www.forbes.com/sites/alexandralevine/2022/04/27/how-tiktok-live-became-a-strip-club-filled-with-15-year-olds/.)

This measure proposes to address these challenges in two ways, each described below.

First, this bill is meant to ensure that social media platforms promptly block child sexual abuse material reported by victims. This bill establishes a comprehensive process for users to report material that they reasonably believe to be 1)CSAM, 2) in which they themselves are depicted as identifiable minors, and 3) displayed, stored, or hosted on a social media platform. Following a report, a platform would have 30 days (with a potential extension to 60 days) to verify whether the material is CSAM, and if it is, block it from appearing on the platform. The steps in this process are set forth in paragraphs 2) to 5), above, of the SUMMARY above. Some important points to note about this process are as follows.

1. Social media platforms can use a third-party service to collect CSAM reports, and thereby avoid incurring liability that might arise from coming into possession of CSAM, particularly CSAM that is not already present on their systems.
2. Social media platforms would only be required to block specific CSAM reported by users.
3. Social media platforms must acknowledge user reports and keep users updated on the progress of their requests.
4. Social media platforms would retain the ability to pursue users who make fraudulent reports. The bill provides that nothing in its text limits or impairs in any way a cause of action for deceit involving an intentional misrepresentation of fact.

Second, this bill would hold platforms liable if they cause commercial sexual exploitation. Existing California law holds adults civilly liable for the tort of commercial sexual exploitation: sexually exploiting minors in exchange for money, property, or something else of value. This bill proposes to extend such liability to social media platforms that knowingly facilitate, aid, or abet commercial sexual exploitation. "Facilitate, aid, or abet" is defined as "deploy[ing] a system, design, feature, or affordance that is a substantial factor in causing minor users to be victims of commercial sexual exploitation." The "substantial factor" test is generally understood to be the appropriate causation standard under California tort law. (Rutherford v. Owens-Illinois, Inc. (1997) 16 Cal. 4th 953, 968-9.)

It is critical to note that the bill focuses on the system, design, feature, or affordance at issue, not particular content accessible through the site. This wording is meant to insulate the provision in question from being struck down under Section 230 of the federal Communications Decency Act, 47 United States Code Section 230(c). Lemmon v. Snap (9th Cir. 2021) 995 F.3d 1085, 1094 provides that even under Section 230, interactive computer services "face the prospect of liability, even for their 'neutral tools,' so long as plaintiffs' claims do not blame them for the content that third parties generate with those tools."

The liability standard under this bill would not be easy to meet. Because of Section 230, the ability to share digital images or videos on a social media platform, in and of itself, is likely to be insufficient to trigger liability—even if the images or videos shared were produced as a result of commercial sexual exploitation. But, as now worded, it appears that the bill would allow social media platforms to be held liable for deploying features that cause commercial sexual exploitation, without running afoul of Section 230. To the extent this dissuades platforms from deploying features that induce commercial sexual exploitation, this appears a beneficial measure.

According to the Author

We are not serious about ending the horror of sexual exploitation and trafficking of children if we do not aggressively address the prominent role gigantic social media platforms knowingly play in facilitating such exploitation and trafficking and re-traumatizing survivors for a lifetime afterwards.

Arguments in Support

LookUp.Live lauds the agency this bill would provide survivors:

We…appreciate that this bill also empowers survivors haunted by the prospect of images and videos about them lingering on platforms to require this content be rendered invisible, with the ability to recover penalties against platforms that deny their requests.

Arguments in Opposition

California Chamber of Commerce, Civil Justice Association of California, NetChoice, and Technet warn the bill could inadvertently harm minors:

[P]latforms could choose to disable features that could be misused by perpetrators to traffic children, such as direct messaging, chat forums, or other communicative features. They could also prohibit anyone under the age of 18 from using their platforms. In either instance, more lawful speech is impacted and burdened than intended, thus creating a chilling effect on providing and hosting lawful content and speech on social media platforms. Though well-intentioned, this bill will result in more lawful speech being removed and fewer online spaces for teens to communicate and share ideas with one another.

FISCAL COMMENTS

Anticipated increases in court costs, although, according to the Senate Appropriations Committee, "[i]t is unknown how many additional actions would be brought as a result of the implementation of this bill."

VOTES:

ASM PRIVACY AND CONSUMER PROTECTION: 8-0-3

YES: Gabriel, Bauer-Kahan, Bennett, Irwin, Kalra, Lowenthal, Papan, Wicks

ABS, ABST OR NV: Joe Patterson, Essayli, Vince Fong

ASM JUDICIARY: 10-0-1

YES: Maienschein, Connolly, Dixon, Haney, Kalra, Pacheco, Papan, Reyes, Robert Rivas, Sanchez

ABS, ABST OR NV: Essayli

ASSEMBLY FLOOR: 77-0-3

YES: Addis, Aguiar-Curry, Alanis, Alvarez, Arambula, Bains, Bauer-Kahan, Bennett, Berman, Boerner, Bonta, Bryan, Calderon, Juan Carrillo, Wendy Carrillo, Cervantes, Connolly, Megan Dahle, Davies, Dixon, Flora, Mike Fong, Friedman, Gabriel, Gallagher, Garcia, Gipson, Grayson, Haney, Hart, Holden, Hoover, Irwin, Jackson, Jones-Sawyer, Kalra, Lackey, Lee, Low, Lowenthal, Maienschein, Mathis, McCarty, McKinnor, Muratsuchi, Stephanie Nguyen, Ortega, Pacheco, Papan, Jim Patterson, Joe Patterson, Pellerin, Petrie-Norris, Quirk-Silva, Ramos, Reyes, Luz Rivas, Robert Rivas, Rodriguez, Blanca Rubio, Sanchez, Santiago, Schiavo, Soria, Ta, Ting, Valencia, Villapudua, Waldron, Wallis, Ward, Weber, Wicks, Wilson, Wood, Zbur, Rendon

ABS, ABST OR NV: Chen, Essayli, Vince Fong

UPDATED

VERSION: September 8, 2023

CONSULTANT: Jith Meganathan / P. & C.P. / (916) 319-2200 FN: 0002426

 THIRD READING  

Bill No: AB 1394

Author: Wicks (D) and Flora (R), et al.

Amended: 9/8/23 in Senate

Vote: 21

SENATE JUDICIARY COMMITTEE: 11-0, 7/11/23

AYES: Umberg, Wilk, Allen, Ashby, Caballero, Durazo, Laird, Min, Niello, Stern, Wiener

SENATE APPROPRIATIONS COMMITTEE: 7-0, 9/1/23

AYES: Portantino, Jones, Ashby, Bradford, Seyarto, Wahab, Wiener

ASSEMBLY FLOOR: 77-0, 5/31/23 - See last page for vote

SUBJECT: Commercial sexual exploitation: child sexual abuse material: civil actions

SOURCE: American Association of University Women California

Children’s Advocacy Institute

Common Sense Media

DIGEST:

This bill requires social media platforms to provide a reporting mechanism for suspected child sexual abuse material and requires them to permanently block the material, as provided. It also prohibits platforms from knowingly facilitating, aiding, or abetting minor’s commercial sexual exploitation.

Senate Floor Amendments of 9/8/23 provide a partial safe harbor in connection with the reporting mechanism created by the bill and simplify the safe harbor, and associated audit, with the other portion of the bill. The amendments extend the compliance timeline, clarify the application of the knowledge standard, and delay implementation for one year.

ANALYSIS:

Existing federal law:

1. Provides that no provider or user of a website shall be treated as the publisher or speaker of any information provided by another information content provider, and that no provider of a website shall be held liable on account of any action voluntarily taken in good faith to restrict the availability of materials that the provider determines to be obscene, lewd, lascivious, filthy, excessively violent, harassing, or otherwise objectionable, whether or not such material is constitutionally protected. (47 U.S.C. § 230(c) (Section 230).)
2. Provides that no cause of action may be brought and no liability may be imposed under any state or local law that is inconsistent with Section 230. (47 U.S.C. § 230(e).)

Existing state law:

1. Defines “commercial sexual exploitation” as an act committed for the purpose of obtaining property, money, or anything else of value, in exchange for, or as a result of, a sexual act of a minor or a nonminor dependent, including sex trafficking or sexual exploitation of a minor. (Civ. Code § 3345.1.)
2. Defines “obscene matter” as matter, taken as a whole, that to the average person, applying contemporary statewide standards, appeals to the prurient interest, that, taken as a whole, depicts or describes sexual conduct in a patently offensive way, and that, taken as a whole, lacks serious literary, artistic, political, or scientific value. (Pen. Code § 311.)
3. Prohibits a person who intentionally distributes the image of the intimate body part or parts of another identifiable person, or an image of the person depicted engaged in an act of sexual intercourse, sodomy, oral copulation, sexual penetration, or an image of masturbation by the person depicted or in which the person depicted participates, under circumstances in which the persons agree or understand that the image shall remain private, the person distributing the image knows or should know that distribution of the image will cause serious emotional distress, and the person depicted suffers that distress. (Pen. Code § 647(j)(4)(A).)

This bill:

1. Requires a social media platform to provide, in a mechanism that is reasonably accessible to users, a means for a user who is a California resident to report material to the platform that the user reasonably believes meets all of the following criteria:
   1. The reported material is CSAM;
   2. The reporting user is an identifiable minor depicted in the reported material;
   3. The reported material is displayed, stored, or hosted on the social media platform.
2. Requires a social media platform to do all of the following:
   1. Permanently block the instance of reported material from being viewable if there is a reasonable basis to believe that the material is CSAM and it is displayed, stored, or hosted on the platform. The report must contain basic identifying information, as provided.
   2. Make reasonable efforts to remove and block other instances of the same blocked reported material from being viewable on the social media platform.
   3. Collect information reasonably sufficient to enable the social media platform to contact a reporting user.
   4. Contact a reporting user in writing by a method chosen by the user.
   5. Provide written confirmation to a reporting user that the social media platform received the report within 24 hours of when the material was first reported that informs the reporting user of the schedule of regular written updates that the social media platform is required to make pursuant hereto.
   6. Provide a written update to the reporting user as to the status of the social media platform’s handling of the reported material seven days after the written confirmation was provided and every seven days thereafter until the final written determination is provided.
   7. Issue a final written determination to the reporting user stating one of the following:
      1. The reported material has been determined to be CSAM and has been blocked on the social media platform.
      2. The reported material has been determined not to be CSAM.
      3. The reported material has been determined not to be displayed, stored, or hosted on the social media platform.
   8. Comply with these requirements no later than 30 days after the date on which the material was first reported, or within 60 days with notice, as provided.
3. Makes a social media company that fails to comply with the above liable to a reporting user for actual and statutory damages, as well as other specified remedies. The bill creates safe harbors for platforms to limit their liability, as provided. The bill creates a rebuttable presumption that the reporting user is entitled to statutory damages if a social media platform fails to comply with the requirements in 2).
4. Prohibits a social media platform from knowingly facilitating, aiding, or abetting commercial sexual exploitation. Requires a court to award statutory damages of at least $1 million but no more than $4 million for each act in violation.
5. Provides that a social media platform shall be deemed to have knowledge under paragraph 4) if all of the following are true:
   1. Material was reported to a social media platform using the required mechanism under 1) for four consecutive months.
   2. The criteria set forth in 1) (a)-(c) are established with respect to that reported material.
   3. The reported material was first displayed, stored, or hosted on the platform after January 1, 2025.
6. Provides that “facilitate, aid, or abet” means to deploy a system, design, feature, or affordance that is a substantial factor in causing minor users to be victims of commercial sexual exploitation.
7. Provides a safe harbor from liability for facilitating, aiding, or abetting commercial sexual exploitation where the platform demonstrates all of the following:
   1. The social media platform instituted and maintained a program of at least biannual audits of its designs, algorithms, practices, affordances, and features to detect designs, algorithms, practices, affordances, or features that have the potential to cause or contribute to violations.
   2. The social media platform, within 30 days of the completion of an audit, took action designed to mitigate or eliminate the reasonably foreseeable risk that a design, algorithm, practice, affordance, or feature violates, or contributes to a violation of 4).
   3. The platform provided to each member of its board of directors a true and correct copy of each audit within 90 days of completion accompanied by a description of any action taken.
8. Excludes a standalone direct messaging service that provides end-to-end encrypted communication, or the portion of a multiservice platform that uses end-to-end encrypted communication, from the definition of social media platform, which otherwise cross references that already in statute. Specified nonprofit organizations are also excluded.
9. Defines “child sexual abuse material” as either child pornography or obscene matter that depicts a minor personally engaging in, or personally simulating, sexual conduct. “Child pornography” and “obscene matter” have the same meaning as existing federal and state law, respectively.
10. Includes non-waiver and severability clauses. Makes various findings and declarations regarding the sexual abuse, exploitation, and trafficking of children on social media platforms. Delays the operative date until 1/1/25.

Background

Child sexual abuse material (CSAM) generally refers to any visual depiction of sexually explicit conduct involving a minor. CSAM is widely distributed online. In 2021 alone, more than 29 million reports of suspected child sexual exploitation were reported by online platforms.

This bill seeks to address the incidence of CSAM on social media platforms in two distinct ways. First, the bill requires platforms to establish a mechanism for minor users to report suspected CSAM they are depicted in and requires the platforms to collect information from and report information to those users. The platform is required to permanently block the CSAM. Platforms in violation are subject to civil liability, but are provided safe harbors to limit that liability.

Second, the bill prohibits social media platforms from knowingly facilitating, aiding, or abetting commercial sexual exploitation of minors. “Facilitate, aid, or abet” means to deploy a system, design, feature, or affordance that is a substantial factor in causing minor users to be victims of commercial sexual exploitation. Violations are subject to statutory damages of up to $4,000,000 but no less than $1,000,000 for each act of exploitation. The bill provides a safe harbor where the platform has undertaken biannual audits and taken action designed to mitigate or eliminate the reasonably foreseeable risk that a design, algorithm, practice, affordance, or feature violates, or contributes to a violation, of the bill.

This bill is co-sponsored by the Children’s Advocacy Institute, Common Sense Media, and the American Association of University Women California. It is supported by various advocacy groups, including Jewish Family and Children’s Services. It is opposed by a coalition of organizations, including TechNet and the California Chamber of Commerce. For a more thorough discussion, please see the Senate Judiciary Committee analysis of the bill, which is incorporated herein by reference.

Comments

The scourge of child sexual abuse material

Globally, the volume of CSAM increased dramatically during the pandemic as both children and predators spent more time online than ever before.^[1] Child protection experts, including the anti-child-trafficking organization Thorn and INHOPE, a global network of CSAM hotlines, predict the problem will only continue to grow.^[2] In 2020 alone, the Meta family of social media platforms reported over 20 million instances of child exploitative content.^[3] The National Center for Missing and Exploited Children (NCMEC) reported a 97.5 percent increase in reports compared to 2019, and speculated the increase was possibly spurred by risks to children who are isolated at home with abusers and more online than ever during the COVID-19 pandemic. While the problem is global, the United States is a substantial locus of it. Research indicates that the United States hosts more CSAM online than any other country in the world.^[4]

This bill seeks to address this problem in two distinct ways. The first is by requiring a reporting mechanism be created for users with attendant obligations on the social media platform to investigate and remove suspected CSAM. The second is by imposing civil liability on platforms for knowingly facilitating, aiding, or abetting commercial sexual exploitation of minors through their systems, designs, features, or affordances. Essentially, the bill holds platforms accountable where they knowingly deploy systems, designs, features, or affordances that are a substantial factor in causing minor users to be victims of commercial sexual exploitation. Safe harbors are provided to limit or avoid liability.

FISCAL EFFECT: Appropriation: No Fiscal Com.: No Local: No

According to the Senate Appropriations Committee, unknown court workload cost pressures to adjudicate alleged violations of the prohibition on knowingly facilitating or aiding and abetting exploitative material and for failing to take reasonable steps to block such material (Trial Court Trust Fund, General Fund).

SUPPORT: (Verified 9/11/23)

American Association of University Women – California (co-source)

Children’s Advocacy Institute (co-source)

Common Sense Media (co-source)

#halfthestory

3Strands Global Foundation

Fairplay

Jewish Family and Children's Services of San Francisco, the Peninsula, Marin and Sonoma Counties

OPPOSITION: (Verified 9/11/23)

California Chamber of Commerce

Civil Justice Association of California

Computer and Communications Industry Association

Electronic Frontier Foundation

NetChoice

TechNet

ARGUMENTS IN SUPPORT: The Children’s Advocacy Institute writes:

It is self-evident that unless social media platforms devote far more of their resources to the morally compelled but unprofitable task of preventing their spaces from being used by those seeking to exploit children sexually, we will never successfully reduce the sexual exploitation of children. This need to establish legal minimum requirements is especially true and urgent during a time when the largest platform is laying off tens of thousands of workers to increase its profits.

ARGUMENTS IN OPPOSITION: TechNet argues:

Section 230 of the Communications Decency Act (47 U.S.C. §230) generally protects platforms from liability for content that users generate with limited exceptions. This protection enables platforms to host third party content and to moderate third-party content on their platforms without fear of liability. Without the protections of Section 230, the internet ecosystem would be dramatically different with a limited ability for users to post, share, read, view, and discover the content of others.

Fortunately, Section 230 explicitly preempts state laws such as AB 1394 that would conflict with this protection. This bill creates liability for platforms based on third party content by applying to any feature that allows users to encounter content. It effectively assumes all features are harmful and imposes liability on a site for offering any of those potentially harmful features. Platforms’ algorithms and features that allow users to encounter or share content from other users are inextricably linked to the underlying content. It would also impose liability for failure to remove content, which the Ninth Circuit has held falls squarely within the preemption of Section 230.5 Therefore, by imposing liability on platforms for their moderation decisions and content serving features, AB 1394 conflicts with Section 230 and is likely preempted.

ASSEMBLY FLOOR: 77-0, 5/31/23

AYES: Addis, Aguiar-Curry, Alanis, Alvarez, Arambula, Bains, Bauer-Kahan, Bennett, Berman, Boerner, Bonta, Bryan, Calderon, Juan Carrillo, Wendy Carrillo, Cervantes, Connolly, Megan Dahle, Davies, Dixon, Flora, Mike Fong, Friedman, Gabriel, Gallagher, Garcia, Gipson, Grayson, Haney, Hart, Holden, Hoover, Irwin, Jackson, Jones-Sawyer, Kalra, Lackey, Lee, Low, Lowenthal, Maienschein, Mathis, McCarty, McKinnor, Muratsuchi, Stephanie Nguyen, Ortega, Pacheco, Papan, Jim Patterson, Joe Patterson, Pellerin, Petrie-Norris, Quirk-Silva, Ramos, Reyes, Luz Rivas, Robert Rivas, Rodriguez, Blanca Rubio, Sanchez, Santiago, Schiavo, Soria, Ta, Ting, Valencia, Villapudua, Waldron, Wallis, Ward, Weber, Wicks, Wilson, Wood, Zbur, Rendon

NO VOTE RECORDED: Chen, Essayli, Vince Fong

Prepared by: Christian Kurpiewski / JUD. / (916) 651-4113

9/11/23 12:47:44

**** END ****

1. Rhiannon Williams, The US now hosts more child sexual abuse material online than any other country (April 26, 2023) MIT Technology Review, https://www.technologyreview.com/2022/04/26/1051282/the-us-now-hosts-more-child-sexual-abuse-material-online-than-any-other-country/. All internet citations are current as of August 30, 2023. ↑

2. Ibid. ↑

3. Samantha Cole, Facebook Reported 20 Million Instances of Child Sexual Abuse in 2020 (February 24, 2021) Motherboard, Tech by Vice, https://www.vice.com/en/article/7k9an4/facebook-pornhub-child-abuse-content-ncmec-report-2020. ↑

4. See fn. 1. ↑

SENATE COMMITTEE ON APPROPRIATIONS

Senator Anthony Portantino, Chair

2023 - 2024 Regular Session

AB 1394 (Wicks) - Commercial sexual exploitation: child sexual abuse material: civil actions

Bill Summary:

AB 1394 would prohibit social media platforms from knowingly facilitating, aiding, or abetting commercial sexual exploitation, as specified. The bill would also require social media platforms to provide a reporting mechanism for suspected child sexual abuse material and require them to permanently block that material.

Fiscal Impact:

Unknown court workload cost pressures to adjudicate alleged violations of the prohibition on knowingly facilitating or aiding and abetting exploitative material and for failing to take reasonable steps to block such material (Trial Court Trust Fund, General Fund). See Staff Comments for additional detail.

Background:

Child sexual abuse material (CSAM) generally refers to any visual depiction of sexually explicit conduct involving a minor. CSAM is widely distributed online. In 2021 alone, more than 29 million reports of suspected child sexual exploitation were reported by online platforms. Globally, the volume of CSAM increased dramatically during the pandemic as both children and predators spent more time online than ever before. Some research indicates that the United States hosts more CSAM online than any other country in the world.

Proposed Law:

• Requires a social media platform to provide, in a form that is reasonably accessible to users, a means for a user who is a California resident to report material to the platform that the user reasonably believes meets all of the following criteria:
  • The reported material is Child Sexual Abuse Material (CSAM);
  • The reporting user is an identifiable minor depicted in the reported material;
  • The reported material is displayed, stored, or hosted on the social media platform.
    
• Requires a social media platform to permanently block an instance of reported material from being viewable if:
  • There is a reasonable basis to believe the material is CSAM;
  • It is displayed, stored, or hosted on the platform; and,
  • The report contains basic identifying information, such as an account identifier, sufficient to permit the social media platform to locate the reported material. Specifies that a social media platform shall not require a report to contain a specific piece of information for these purposes.
    
• Provides that a social media platform shall make reasonable efforts to remove and block other instances of the same reported material blocked pursuant to this subdivision from being viewable on the social media platform.
• Requires a social media platform to provide written confirmation to a reporting user that the social media platform received that person’s report that meets specified criteria.
• Requires a social media platform to provide a written update to the reporting user as to the status of the social media platform’s handling of the reported material, as specified.
• Requires a social media platform to issue a final written determination to the reporting user stating one of the following:
  • The reported material has been determined to be CSAM and has been blocked on the social media platform.
  • The reported material has been determined not to be CSAM.
  • The reported material has been determined not to be displayed, stored, or hosted on the social media platform.
    
• Requires a social media platform to comply with these requirements no later than 30 days after the date on which the material was first reported, or within 60 days with notice, as provided.
• Makes a social media company that fails to comply with the above liable to a reporting user for all of the following:
  • Any actual damages sustained by the reporting user as a result of the violation;
  • Statutory damages of no more than $250,000, or if the platform has permanently blocked the material, as provided, no more than $125,000, per violation, considering the willfulness and severity of the violation and whether the platform has previously violated this law;
  • Costs of the action, together with reasonable attorney’s fees;
  • Any other relief that the court deems proper.
    
• Creates a rebuttable presumption that the reporting user is entitled to statutory damages if a social media platform fails to comply with specified requirements.
• Prohibits a social media platform from knowingly, facilitating, aiding, or abetting commercial sexual exploitation. Requires a court to award statutory damages of at least $1 million but no more than $4 million for each act of commercial sexual exploitation facilitated, aided, or abetted by the social media platform.
• Provides a safe harbor from liability for facilitating, aiding, or abetting commercial sexual exploitation where the platform demonstrates all of the following:
  • The social media platform instituted and maintained a program of at least quarterly audits of its designs, algorithms, practices, affordances, and features to detect designs, algorithms, practices, affordances, or features that have the potential to cause or contribute to violations.
  • The social media platform corrected, within 30 days of the completion of an audit, any design, algorithm, practice, affordance, or feature discovered by the audit to present more than a de minimis risk of violating these provisions, and any correction has endured without interruption and is in full force and effect.
  • The audit included the participation of an independent nonprofit or law enforcement agency expert in the prevention of the commercial sexual exploitation of minors and an officer thereof verifies that the platform provided that person access to the platform’s personnel, data, records, and technology as required for the nonprofit or law enforcement agency expert to confirm the results and the completion of the corrections.
  • The platform provided to each member of its board of directors a true and correct copy of each audit within 30 days of completion accompanied by a description of any corrections made.

Related Legislation:

• SB 287 (Skinner, 2023) subjects social media platforms to civil liability for damages caused by their practices, affordances, designs, algorithms, or features, as provided. It provides a safe harbor where certain auditing practices are carried out. SB 287 was ordered to the inactive file at the request of the author.
• SB 680 (Skinner, 2023) is substantially similar to SB 287. SB 680 is currently in the Assembly Appropriations Committee.
• SB 646 (Cortese, 2023) creates liability for the distribution of certain “actionable material,” which includes illicit pictures of minors and images or depictions of minors that serve as the basis for criminal and civil liability at the federal level. SB 646 is currently in the Assembly Appropriations Committee.

Staff Comments:

This bill seeks to address the incidence of CSAM on social media platforms in two distinct ways. First, the bill requires platforms to establish a mechanism for minor users to report suspected CSAM they are depicted in and requires the platforms to collect information from and report information to those users. The platform is required to permanently delete the CSAM. Platforms in violation are subject to civil liability including statutory damages of up to $250,000 per violation. Second, the bill prohibits social media platforms from knowingly facilitating, aiding, or abetting commercial sexual exploitation of minors. Violations are subject to statutory damages of up to $4,000,000 but no less than $1,000,000 for each act of exploitation.

Regulation of social media platforms, particularly as it relates to young users, is an area of the law that is rife with complex litigation. One consolidated federal case in the Northern District of California, for example, has been brought on behalf of approximately 235 plaintiffs, including school districts, individuals and local and state governments, who allege numerous claims and injuries against defendants Meta, Snap, ByteDance (Tik Tok), and Google (YouTube).

By creating additional liability for social media platforms, this bill is likely to result in court workload cost pressures. It is unknown how many additional actions would be brought as a result of the implementation of this bill. However, it generally costs about $8,000 to operate a courtroom for one eight-hour day. Although courts are not funded on the basis of workload, increased pressure on the trial courts and staff may create a need for increased funding for courts from the General Fund (GF) to perform existing duties. Numerous trial court operations are funded through the imposition and collection of criminal fines and fees. However, the Legislature has reduced and eliminated criminal fines and fees over the past decade. As a result, the 2023-24 budget includes an ongoing annual allocation of $105.1 million backfill from the General Fund in order to address declining revenue to the Trial Court Trust Fund.

-- END --

SENATE COMMITTEE ON APPROPRIATIONS

Senator Anthony Portantino, Chair

2023 - 2024 Regular Session

AB 1394 (Wicks) - Commercial sexual exploitation: child sexual abuse material: civil actions

Bill Summary:

AB 1394 would prohibit social media platforms from knowingly facilitating, aiding, or abetting commercial sexual exploitation, as specified. The bill would also require social media platforms to provide a reporting mechanism for suspected child sexual abuse material and require them to permanently block that material.

Fiscal Impact:

Unknown court workload cost pressures to adjudicate alleged violations of the prohibition on knowingly facilitating or aiding and abetting exploitative material and for failing to take reasonable steps to block such material (Trial Court Trust Fund, General Fund). See Staff Comments for additional detail.

Background:

Child sexual abuse material (CSAM) generally refers to any visual depiction of sexually explicit conduct involving a minor. CSAM is widely distributed online. In 2021 alone, more than 29 million reports of suspected child sexual exploitation were reported by online platforms. Globally, the volume of CSAM increased dramatically during the pandemic as both children and predators spent more time online than ever before. Some research indicates that the United States hosts more CSAM online than any other country in the world.

Proposed Law:

• Requires a social media platform to provide, in a form that is reasonably accessible to users, a means for a user who is a California resident to report material to the platform that the user reasonably believes meets all of the following criteria:
  • The reported material is Child Sexual Abuse Material (CSAM);
  • The reporting user is an identifiable minor depicted in the reported material;
  • The reported material is displayed, stored, or hosted on the social media platform.
    
• Requires a social media platform to permanently block an instance of reported material from being viewable if:
  • There is a reasonable basis to believe the material is CSAM;
  • It is displayed, stored, or hosted on the platform; and,
  • The report contains basic identifying information, such as an account identifier, sufficient to permit the social media platform to locate the reported material. Specifies that a social media platform shall not require a report to contain a specific piece of information for these purposes.
    
• Provides that a social media platform shall make reasonable efforts to remove and block other instances of the same reported material blocked pursuant to this subdivision from being viewable on the social media platform.
• Requires a social media platform to provide written confirmation to a reporting user that the social media platform received that person’s report that meets specified criteria.
• Requires a social media platform to provide a written update to the reporting user as to the status of the social media platform’s handling of the reported material, as specified.
• Requires a social media platform to issue a final written determination to the reporting user stating one of the following:
  • The reported material has been determined to be CSAM and has been blocked on the social media platform.
  • The reported material has been determined not to be CSAM.
  • The reported material has been determined not to be displayed, stored, or hosted on the social media platform.
    
• Requires a social media platform to comply with these requirements no later than 30 days after the date on which the material was first reported, or within 60 days with notice, as provided.
• Makes a social media company that fails to comply with the above liable to a reporting user for all of the following:
  • Any actual damages sustained by the reporting user as a result of the violation;
  • Statutory damages of no more than $250,000, or if the platform has permanently blocked the material, as provided, no more than $125,000, per violation, considering the willfulness and severity of the violation and whether the platform has previously violated this law;
  • Costs of the action, together with reasonable attorney’s fees;
  • Any other relief that the court deems proper.
    
• Creates a rebuttable presumption that the reporting user is entitled to statutory damages if a social media platform fails to comply with specified requirements.
• Prohibits a social media platform from knowingly, facilitating, aiding, or abetting commercial sexual exploitation. Requires a court to award statutory damages of at least $1 million but no more than $4 million for each act of commercial sexual exploitation facilitated, aided, or abetted by the social media platform.
• Provides a safe harbor from liability for facilitating, aiding, or abetting commercial sexual exploitation where the platform demonstrates all of the following:
  • The social media platform instituted and maintained a program of at least quarterly audits of its designs, algorithms, practices, affordances, and features to detect designs, algorithms, practices, affordances, or features that have the potential to cause or contribute to violations.
  • The social media platform corrected, within 30 days of the completion of an audit, any design, algorithm, practice, affordance, or feature discovered by the audit to present more than a de minimis risk of violating these provisions, and any correction has endured without interruption and is in full force and effect.
  • The audit included the participation of an independent nonprofit or law enforcement agency expert in the prevention of the commercial sexual exploitation of minors and an officer thereof verifies that the platform provided that person access to the platform’s personnel, data, records, and technology as required for the nonprofit or law enforcement agency expert to confirm the results and the completion of the corrections.
  • The platform provided to each member of its board of directors a true and correct copy of each audit within 30 days of completion accompanied by a description of any corrections made.

Related Legislation:

• SB 287 (Skinner, 2023) subjects social media platforms to civil liability for damages caused by their practices, affordances, designs, algorithms, or features, as provided. It provides a safe harbor where certain auditing practices are carried out. SB 287 was ordered to the inactive file at the request of the author.
• SB 680 (Skinner, 2023) is substantially similar to SB 287. SB 680 is currently in the Assembly Appropriations Committee.
• SB 646 (Cortese, 2023) creates liability for the distribution of certain “actionable material,” which includes illicit pictures of minors and images or depictions of minors that serve as the basis for criminal and civil liability at the federal level. SB 646 is currently in the Assembly Appropriations Committee.

Staff Comments:

This bill seeks to address the incidence of CSAM on social media platforms in two distinct ways. First, the bill requires platforms to establish a mechanism for minor users to report suspected CSAM they are depicted in and requires the platforms to collect information from and report information to those users. The platform is required to permanently delete the CSAM. Platforms in violation are subject to civil liability including statutory damages of up to $250,000 per violation. Second, the bill prohibits social media platforms from knowingly facilitating, aiding, or abetting commercial sexual exploitation of minors. Violations are subject to statutory damages of up to $4,000,000 but no less than $1,000,000 for each act of exploitation.

Regulation of social media platforms, particularly as it relates to young users, is an area of the law that is rife with complex litigation. One consolidated federal case in the Northern District of California, for example, has been brought on behalf of approximately 235 plaintiffs, including school districts, individuals and local and state governments, who allege numerous claims and injuries against defendants Meta, Snap, ByteDance (Tik Tok), and Google (YouTube).

By creating additional liability for social media platforms, this bill is likely to result in court workload cost pressures. It is unknown how many additional actions would be brought as a result of the implementation of this bill. However, it generally costs about $8,000 to operate a courtroom for one eight-hour day. Although courts are not funded on the basis of workload, increased pressure on the trial courts and staff may create a need for increased funding for courts from the General Fund (GF) to perform existing duties. Numerous trial court operations are funded through the imposition and collection of criminal fines and fees. However, the Legislature has reduced and eliminated criminal fines and fees over the past decade. As a result, the 2023-24 budget includes an ongoing annual allocation of $105.1 million backfill from the General Fund in order to address declining revenue to the Trial Court Trust Fund.

-- END --

SENATE JUDICIARY COMMITTEE

Senator Thomas Umberg, Chair

2023-2024 Regular Session

Bill No: AB 1394 (Author:Wicks)

Version: May 25, 2023

Hearing Date: July 11, 2023

Fiscal: No

Urgency: No

Consultant: CK

SUBJECT

Commercial sexual exploitation: child sexual abuse material: civil actions

DIGEST

This bill requires social media platforms to provide a reporting mechanism for suspected child sexual abuse material and requires them to permanently block the material, as provided. The bill prohibits platforms from knowingly, recklessly, or negligently facilitating, aiding, or abetting commercial sexual exploitation of minors.

EXECUTIVE SUMMARY

Child sexual abuse material (CSAM) generally refers to any visual depiction of sexually explicit conduct involving a minor. CSAM is widely distributed online. In 2021 alone, more than 29 million reports of suspected child sexual exploitation were reported by online platforms.

This bill seeks to address the incidence of CSAM on social media platforms in two distinct ways. First, the bill requires platforms to establish a mechanism for minor users to report suspected CSAM they are depicted in and requires the platforms to collect information from and report information to those users. The platform is required to permanently delete the CSAM. Platforms in violation are subject to civil liability including statutory damages of up to $250,000 per violation.

Second, the bill prohibits social media platforms from knowingly, recklessly, or negligently facilitating, aiding, or abetting commercial sexual exploitation of minors. “Facilitate, aid, or abet” means to deploy a system, design, feature, or affordance that is a substantial factor in causing minor users to be victims of commercial sexual exploitation. Violations are subject to statutory damages of up to $4,000,000 but no less than $1,000,000 for each act of exploitation. The bill provides a safe harbor where the platform has undertaken quarterly audits and corrected designs, algorithms, practices, affordances, and features that risk violation, as provided.

This bill is co-sponsored by the Children’s Advocacy Institute, Common Sense Media, and the American Association of University Women California. It is supported by various advocacy groups, including Jewish Family and Children’s Services. It is opposed by a coalition of organizations, including TechNet and the California Chamber of Commerce.

PROPOSED CHANGES TO THE LAW

Existing federal law:

1. Provides that no provider or user of a website shall be treated as the publisher or speaker of any information provided by another information content provider, and that no provider of a website shall be held liable on account of any action voluntarily taken in good faith to restrict the availability of materials that the provider determines to be obscene, lewd, lascivious, filthy, excessively violent, harassing, or otherwise objectionable, whether or not such material is constitutionally protected. (47 U.S.C. § 230(c) (Section 230).)
2. Provides that no cause of action may be brought and no liability may be imposed under any state or local law that is inconsistent with Section 230. (47 U.S.C. § 230(e).)
3. Authorizes any person who, while a minor, was a victim of a violation of 18 U.SC. § 1589, 1590, 1591, 2241(c), 2242, 2243, 2251, 2251A, 2252, 2252A, 2260, 2421, 2422, or 2423, and who suffers personal injury as a result of such violation, regardless of whether the injury occurred while such person was a minor, to sue in any appropriate United States District Court and provides for recovery of the actual damages such person sustains or liquidated damages in the amount of $150,000, and the cost of the action, including reasonable attorney’s fees and other litigation costs reasonably incurred. The court may also award punitive damages and such other preliminary and equitable relief as the court determines to be appropriate. There is no statute of limitations for such actions. (18 U.S.C. § 2255.)
4. Provides a right to free speech and expression. (U.S. Const., 1st amend; Cal. Const., art 1, § 2.)
5. Recognizes certain judicially created exceptions to the rights of freedom of speech and expression. (E.g., Virginia v. Black (2003) 538 U.S. 343, 359.)
6. Defines “child pornography” as any visual depiction of sexually explicit conduct, where any of the following is true:
   1. The production of such visual depiction involves the use of a minor engaging in sexually explicit conduct.
   2. The visual depiction is a digital image, computer image, or computer-generated image that is, or is indistinguishable from, that of a minor engaging in sexually explicit conduct.
   3. The visual depiction has been created, adapted, or modified to appear that an identifiable minor is engaging in sexually explicit conduct. (18 U.S.C. § 2256.)

Existing law:

1. Defines “commercial sexual exploitation” as an act committed for the purpose of obtaining property, money, or anything else of value, in exchange for, or as a result of, a sexual act of a minor or a nonminor dependent. The definition includes the following crimes:
   1. sex trafficking of a minor,
   2. pimping of a minor,
   3. pandering of a minor,
   4. procurement of a child under 16 years of age for lewd and lascivious acts,
   5. solicitation of a child for an act of prostitution, and
   6. sexual exploitation of a minor. (Civ. Code § 3345.1.)
2. Defines “social media platform” as a public or semipublic internet-based service or application that has users in California and that meets both of the following criteria:
   1. A substantial function of the service or application is to connect users in order to allow them to interact socially with each other within the service or application. (A service or application that provides email or direct messaging services does not meet this criterion based solely on that function.)
   2. The service or application allows users to do all of the following:
      1. Construct a public or semipublic profile for purposes of signing into and using the service or application.
      2. Populate a list of other users with whom an individual shares a social connection within the system.
      3. Create or post content viewable by other users, including on message boards, in chat rooms, or through a landing page or main feed that presents the user with content generated by other users. (Bus. & Prof. Code § 22675(e).)
3. Defines “social media company” as a person or entity that owns or operates one or more social media platforms. (Bus. & Prof. Code § 22675(d).)
4. Defines “obscene matter” as matter, taken as a whole, that to the average person, applying contemporary statewide standards, appeals to the prurient interest, that, taken as a whole, depicts or describes sexual conduct in a patently offensive way, and that, taken as a whole, lacks serious literary, artistic, political, or scientific value. (Pen. Code § 311.)
5. Prohibits a person who intentionally distributes the image of the intimate body part or parts of another identifiable person, or an image of the person depicted engaged in an act of sexual intercourse, sodomy, oral copulation, sexual penetration, or an image of masturbation by the person depicted or in which the person depicted participates, under circumstances in which the persons agree or understand that the image shall remain private, the person distributing the image knows or should know that distribution of the image will cause serious emotional distress, and the person depicted suffers that distress. (Pen. Code § 647(j)(4)(A).)

This bill:

1. Requires a social media platform to provide, in a form that is reasonably accessible to users, a means for a user who is a California resident to report material to the platform that the user reasonably believes meets all of the following criteria:
   1. The reported material is CSAM;
   2. The reporting user is an identifiable minor depicted in the reported material;
   3. The reported material is displayed, stored, or hosted on the social media platform.
2. Requires a social media platform to do all of the following:
   1. Permanently block the reported material from being viewable if the material is both CSAM and is displayed, stored, or hosted on the platform.
   2. Collect information reasonably sufficient to enable the social media platform to contact a reporting user.
   3. Contact a reporting user in writing by a method chosen by the user.
   4. Provide written confirmation to a reporting user that the social media platform received the report within 24 hours of when the material was first reported that informs the reporting user of the schedule of regular written updates that the social media platform is required to make pursuant hereto.
   5. Provide a written update to the reporting user as to the status of the social media platform’s handling of the reported material seven days after the written confirmation was provided and every seven days thereafter until the final written determination is provided.
   6. Issue a final written determination to the reporting user stating one of the following:
      1. The reported material has been determined to be CSAM and has been blocked on the social media platform.
      2. The reported material has been determined not to be CSAM.
      3. The reported material has been determined not to be displayed, stored, or hosted on the social media platform.
   7. Comply with these requirements no later than 30 days after the date on which the material was first reported, or within 60 days with notice, as provided.
3. Makes a social media company that fails to comply with the above liable to a reporting user for all of the following:
   1. Any actual damages sustained by the reporting user as a result of the violation;
   2. Statutory damages of no more than $250,000, or if the platform has permanently blocked the material, as provided, no more than $125,000, per violation, considering the willfulness and severity of the violation and whether the platform has previously violated this law;
   3. Costs of the action, together with reasonable attorney’s fees;
   4. Any other relief that the court deems proper.
4. Creates a rebuttable presumption that the reporting user is entitled to statutory damages if a social media platform fails to comply with the requirements in 2).
5. Prohibits a social media platform from knowingly, recklessly, or negligently facilitating, aiding, or abetting commercial sexual exploitation. Requires a court to award statutory damages of at least $1 million but no more than $4 million for each act in violation, except it shall award $5 million for knowing or reckless violations.
6. Provides that “facilitate, aid, or abet” means to deploy a system, design, feature, or affordance that is a substantial factor in causing minor users to be victims of commercial sexual exploitation.
7. Provides a safe harbor from liability for facilitating, aiding, or abetting commercial sexual exploitation where the platform demonstrates all of the following:
   1. The social media platform instituted and maintained a program of at least quarterly audits of its designs, algorithms, practices, affordances, and features to detect designs, algorithms, practices, affordances, or features that have the potential to cause or contribute to violations.
   2. The social media platform corrected, within 30 days of the completion of an audit, any design, algorithm, practice, affordance, or feature discovered by the audit to present more than a de minimis risk of violating these provisions, and any correction has endured without interruption and is in full force and effect.
   3. The audit included the participation of an independent nonprofit or law enforcement agency expert in the prevention of the commercial sexual exploitation of minors and an officer thereof verifies that the platform provided that person access to the platform’s personnel, data, records, and technology as required for the nonprofit or law enforcement agency expert to confirm the results and the completion of the corrections.
   4. The platform provided to each member of its board of directors a true and correct copy of each audit within 30 days of completion accompanied by a description of any corrections made.
8. Excludes a standalone direct messaging service that provides end-to-end encrypted communication from the definition of social media platform, which otherwise cross references that already in statute.
9. Defines “child sexual abuse material” as either child pornography or obscene matter that depicts a minor personally engaging in, or personally simulating, sexual conduct. “Child pornography” and “obscene matter” have the same meaning as existing federal and state law, respectively.
10. Includes non-waiver and severability clauses.
11. Makes various findings and declarations regarding the sexual abuse, exploitation, and trafficking of children on social media platforms.

COMMENTS

• • • 1. The scourge of child sexual abuse material

Globally, the volume of CSAM increased dramatically during the pandemic as both children and predators spent more time online than ever before.^[1] Child protection experts, including the anti-child-trafficking organization Thorn and INHOPE, a global network of CSAM hotlines, predict the problem will only continue to grow.^[2] In 2020 alone, the Meta family of social media platforms reported over 20 million instances of child exploitative content.^[3] The National Center for Missing and Exploited Children (NCMEC) reported a 97.5 percent increase in reports compared to 2019, and speculated the increase was possibly spurred by risks to children who are isolated at home with abusers and more online than ever during the COVID-19 pandemic.

While the problem is global, the United States is a substantial locus of it. Research indicates that the United States hosts more CSAM online than any other country in the world.^[4] The country accounted for 30 percent of the global total of CSAM URLs at the end of March 2022, according to the Internet Watch Foundation, a UK-based organization that works to spot and take down abusive content.^[5]

A number of factors have been identified to explain this reality:

[T]he rapidly growing CSAM problem in the US is attributable to a number of more long-term factors. The first is the country’s sheer size and the fact that it’s home to the highest number of data centers and secure internet servers in the world, creating fast networks with swift, stable connections that are attractive to CSAM hosting sites.

The second is that the vast scale of CSAM dwarfs the resources dedicated to weeding it out. This imbalance means that bad actors feel they’re able to operate with impunity within the US because the chance of them getting in trouble, even if caught, is “vanishingly small,” says Hany Farid, a professor of computer science at the University of California, Berkeley, and the co-developer of PhotoDNA, a technology that turns images into unique digital signatures, known as hashes, to identify CSAM.

Similarly, while companies in the US are legally required to report CSAM to the National Center for Missing & Exploited Children (NCMEC) once they’ve been made aware of it or face a fine of up to $150,000, they’re not required to proactively search for it.

Besides “bad press” there isn’t much punishment for platforms that fail to remove CSAM quickly, says Lloyd Richardson, director of technology at the Canadian Centre for Child Protection. “I think you’d be hard pressed to find a country that’s levied a fine against an electronic service provider for slow or non-removal of CSAM,” he says.

• • • 1. Platform liability for failing to address CSAM

This bill seeks to address this problem in two distinct ways. The first is by requiring a reporting mechanism be created for users with attendant obligations on the social media platform to investigate and remove suspected CSAM. The second is by imposing civil liability on platforms for knowingly, recklessly, or negligently facilitating, aiding, or abetting commercial sexual exploitation of minors through their systems, designs, features, or affordances.

• • • • 1. Reporting mechanism

The bill requires a social media platform to establish a mechanism for minor users to report CSAM they are identifiably depicted in that is on the platform. CSAM includes “child pornography,” as defined in federal law, and obscene matter, as defined in the Penal Code, that depicts a minor personally engaging in, or personally simulating, sexual conduct.

The social media platform is required to get the preferred contact information of the minor reporting the CSAM and provide written confirmation, within 24 hours, that the report was received, as well as notice of the platform’s reporting requirements imposed by this bill. The platform is then required to provide the minor weekly updates until a final written determination is issued. That determination must either state the reported material was determined to be CSAM and was blocked; it was determined not to be CSAM; or it was determined not to be displayed, stored, or hosted on the platform. All of this must occur within 30 days, unless certain extenuating circumstances exist warranting an additional 30 days with notice to the minor, as provided.

If the reported material is in fact CSAM displayed, stored, or hosted on the social media platform, the social media platform is required to “permanently block” the material from being viewable. The coalition in opposition raises concerns with this provision:

[W]e believe there needs to be some clarification that platforms are only required to permanently block the material reported and not altered versions or reproductions. We believe this was the intent of the recent amendments but want to ensure that companies are able to comply with a realistic standard. Currently, platforms and their automated systems are highly effective at identifying CSAM that has been assigned a hash value by NCMEC’s CyberTipline or Take it Down programs. In the vast majority of cases, that material is prevented from upload or removed before any user ever sees it. However, alterations or reproductions can get around those hash values, and in those instances, platforms cannot guarantee the ability to permanently block that new content. It seems that the intent of the amendments was to require a new report for alterations or reproductions rather than impose liability on a platform for them. If that is the case, we believe this should be fairly easy to clarify.

The coalition requests that the bill differentiate from the original reported “instance” of CSAM from other instances of it that may be altered or reproduced. Opposition also argues that some of the provisions included in the reporting mechanism section are difficult to abide by and has requested amendments to streamline the process.

In response to these concerns, the author has agreed to the following amendments:

Amendment

Amend Section 3273.61(a)-(d) to read:

A social media platform shall do all of the following:

(a) Provide, in a form mechanism that is reasonably accessible to users, a means for a user who is a California resident to report material to the social media platform that the user reasonably believes meets all of the following criteria:

(1) The reported material is child sexual abuse material.

(2) The reporting user is an identifiable minor depicted in the reported material.

(3) The reported material is displayed, stored, or hosted on the social media platform.

(b) Collect information reasonably sufficient to enable the social media platform to contact, pursuant to subdivision (c), a reporting user.

(c) A social media platform shall contact a reporting user in writing by a method, including, but not limited to, a mailing address, a telephone number for purposes of sending text messages, or an email address, that meets both of the following criteria:

(1) The method is chosen by the reporting user.

(2) The method is not a method that is within the control of the social media company that owns or operates the social media platform.

(d)(1) Permanently block the instance of reported material from being viewable on the social media platform if the reported material meets both of the following criteria:

(i) There is a reasonable basis to believe the reported material is child sexual abuse material.

(ii) The reported material is displayed, stored, or hosted on the social media platform.

(iii) The report contains basic identifying information, such as an account identifier, sufficient to permit the social media platform to locate the reported material. No specific piece of information shall be required by social media platforms for purposes of this subdivision.

(2) Make reasonable efforts to remove and block other instances of the same reported material from being viewable on the social media platform.^[6]

A social media platform in violation is liable to a reporting minor for actual damages, costs and fees, and other appropriate relief. In addition, a platform is liable for statutory damages of no more than $250,000 per violation. However, the cap is $125,000 if the platform permanently blocked the reported material before the complaint is filed. The court is instructed to set the amount based on a consideration of the willfulness and severity of the violation and any history of violations. The bill establishes a rebuttable presumption that statutory damages are warranted if the platform fails to carry out its obligation in communicating with the reporting minor and permanently blocking the reported material.

Opposition argues that public enforcement would be a better method in place of the right of action the bill grants to the minors depicted in CSAM. They argue the bill could lead to “uneven policy outcomes” and that the private enforcement makes it “difficult for platforms to implement lessons learned from litigation.”

• • • • 1. Liability for commercial sexual exploitation of a minor

The second method for holding platforms accountable is creating liability for commercial sexual exploitation. Existing law defines “commercial sexual exploitation” as an act committed for the purpose of obtaining property, money, or anything else of value in exchange for, or as a result of, a sexual act of a minor or nonminor dependent, including an act that constitutes a violation of specified laws, including:

• Sex trafficking of a minor.
• Pimping of a minor.
• Pandering of a minor.
• Procurement of a child under 16 years of age for lewd and lascivious acts.
• Solicitation of a child for specified purposes in violation of the law.
• An act of sexual exploitation.

The bill prohibits a social media platform from knowingly, recklessly, or negligently facilitating, aiding, or abetting commercial sexual exploitation of minors. “Facilitate, aid, or abet” means to deploy a system, design, feature, or affordance that is a substantial factor in causing minor users to be victims of commercial sexual exploitation. According to the Judicial Council of California Civil Jury Instructions:

A substantial factor in causing harm is a factor that a reasonable person would consider to have contributed to the harm. It must be more than a remote or trivial factor. It does not have to be the only cause of the harm.

[Conduct is not a substantial factor in causing harm if the same harm would have occurred without that conduct.]^[7]

Opposition argues that this standard “is an incredibly low bar, and combined with the massive penalties it will encourage a flood of litigation leading to unintended consequences.”

A civil action can be brought by, or on behalf of, or for the benefit of, a person who is a minor and is a victim of commercial sexual exploitation facilitated, aided, or abetted by a social media platform. The court is directed to award statutory damages against a platform in violation of no more than $4 million but not less than $1 million for each act of commercial sexual exploitation facilitated, aided, or abetted by the social media platform. Where the violation is found to be knowing or reckless, a court must award statutory damages of $5 million.

The bill provides a safe harbor by which a platform can avoid liability by carrying out at least quarterly audits of its designs, algorithms, practices, affordances, and features to detect designs, algorithms, practices, affordances, or features that have the potential to cause or contribute to violations. Those found to post a more than de minimis risk of violating the law must be corrected within 30 days. In addition, any correction must have endured without interruption and is in full force and effect.

To successfully take advantage of the safe harbor, the platform must ensure that an independent nonprofit or law enforcement agency expert participated in the audit and an officer of that nonprofit or the law enforcement agency expert verifies that the platform provided that person access to its personnel, data, records, and technology as required to confirm the results and the completion of the corrections. As a final step, the platform must have provided each member of its board of directors a copy of each audit within 30 days of the audit being completed accompanied by a description of any corrections made.

Writing in opposition, a coalition of technology and business groups argues this effectively prohibits direct messaging:

Despite recent amendments excluding standalone direct messaging from the bill, AB 1394 still applies to direct messaging that is integrated into a social media platform. AB 1394 effectively prohibits the use of any feature or design, including direct messaging, that could be misused in violation of our rules to victimize children and imposes a penalty of between $1 million and $5 million for each violation. Simply offering direct messaging makes our platforms liable. In order to avoid this liability and continue offering direct messaging, companies would have no choice but to access, monitor, and moderate content in direct and private messages. That is impossible for sites that encrypt messages.

In response, the author has agreed to amend the definition of “social media platform” to exclude not only a standalone direct messaging service that provides end-to-end encrypted communication but also the portion of a multi-service platform that uses end-to-end encrypted communication.

The American Association of University Women California, a co-sponsor of the bill, writes in support:

AB 1394 responds to the soaring rates of child sexual exploitation and child sex trafficking online by amending current law to hold social media platforms accountable when the platforms knowingly or negligently facilitate child exploitation or trafficking. Social media platforms would be subject to significant statutory damages for survivors who would be authorized to file claims against the platform. Survivors would also be able to require any images and videos that remain on the platform to be rendered invisible. A platform’s failure to follow this requirement would be subject to additional penalties.

A coalition of groups in support, including Jewish Family and Children’s Services, writes: “AB 1394 would amend existing law permitting survivors to sue those that trafficked or exploited them to include large social media platforms. It would also permit suits against platforms that refuse a child and/or guardian's demand to render their images and videos invisible.”

• • • 1. Legal considerations

As with most of the legislation seeking to govern the moderation or prohibition of internet content, legal questions arise around whether the specific approach of any proposed law runs afoul of the First Amendment or is preempted by Section 230.

1. First Amendment

The First Amendment, as applied to the states through the Fourteenth Amendment, prohibits Congress or the states from passing any law “abridging the freedom of speech.”^[8] “[A]s a general matter, the First Amendment means that government has no power to restrict expression because of its message, its ideas, its subject matter, or its content.”^[9] However, while the amendment is written in absolute terms, the courts have created a handful of narrow exceptions to the First Amendment’s protections, including “true threats,”^[10] “fighting words,”^[11] incitement to imminent lawless action,^[12] defamation,^[13] and obscenity.^[14]

Expression on the internet is given the same measure of protection granted to in-person speech or statements published in a physical medium.^[15] Accordingly, a social media user may generally post content and comments free from government regulation, but may incur civil or criminal liability if their comment falls within one of the First Amendment exceptions. At the same time, social media platforms themselves—as private businesses—are not subject to the constraints of the First Amendment and may limit or prohibit users’ speech on their sites as they see fit.^[16]

The United States Supreme Court has held that posting on social networking and/or social media sites constitutes communicative activity protected by the First Amendment.^[17] As a general rule, the government “may not suppress lawful speech as the means to suppress unlawful speech.”^[18]

A constitutional challenge to a restriction on speech is generally analyzed under one of two frameworks, depending on whether the courts deem it to be “content neutral” or “content based,” i.e., targeting a particular type of speech. A law is content neutral when it “serves purposes unrelated to the content of the expression.”^[19] On the other hand, a law is content based when the proscribed speech is “defined solely on the basis of the content of the suppressed speech.”^[20]  

If a law is determined to be content neutral it will be subject to intermediate scrutiny, which requires that the law “be ‘narrowly tailored to serve a significant government interest.’ ”^[21] In other words, the law “‘need not be the least restrictive or least intrusive means of’ serving the government’s interests,” but “’may not regulate expression in such a manner that a substantial portion of the burden on speech does not serve to advance its goals.’”^[22] 

If a restriction on speech is determined to be content based, it will be subject to strict scrutiny.^[23] A restriction is content based “if it require[s] ‘enforcement authorities’ to ‘examine the content of the message that is conveyed to determine whether’ a violation has occurred.”^[24] Content-based restrictions subject to strict scrutiny are “presumptively unconstitutional.”^[25] A restriction can survive strict scrutiny only if it uses the least-restrictive means available to achieve a compelling government purpose.^[26]

Although this bill is a content-based regulation, as it requires examination of whether the content is CSAM, the targeted speech arguably falls within an exception to the First Amendment.

The United States Supreme Court in Miller v. California, (1973) 413 U.S. 15, 24, established the prevailing three-prong test for determining whether certain material should be deemed obscenity and therefore unprotected speech.

The basic guidelines for the trier of fact must be: (a) whether “the average person, applying contemporary community standards” would find that the work, taken as a whole, appeals to the prurient interest . . . ; (b) whether the work depicts or describes, in a patently offensive way, sexual conduct specifically defined by the applicable state law; and (c) whether the work, taken as a whole, lacks serious literary, artistic, political, or scientific value.

However, the court revisited the issue when the alleged obscene material was CSAM:

The Miller standard, like its predecessors, was an accommodation between the State’s interests in protecting the “sensibilities of unwilling recipients” from exposure to pornographic material and the dangers of censorship inherent in unabashedly content-based laws. Like obscenity statutes, laws directed at the dissemination of child pornography run the risk of suppressing protected expression by allowing the hand of the censor to become unduly heavy. For the following reasons, however, we are persuaded that the States are entitled to greater leeway in the regulation of pornographic depictions of children.^[27]

Among the reasons laid out by the court are the states’ clear interest in safeguarding the physical and psychological well-being of minors; the fact that “distribution of photographs and films depicting sexual activity by juveniles is intrinsically related to the sexual abuse of children”; and the “value of permitting live performances and photographic reproductions of children engaged in lewd sexual conduct is exceedingly modest, if not de minimis.”

The court then laid out the scope of this new exception and the modified Miller-standard to be applied:

There are, of course, limits on the category of child pornography which, like obscenity, is unprotected by the First Amendment. As with all legislation in this sensitive area, the conduct to be prohibited must be adequately defined by the applicable state law, as written or authoritatively construed. Here the nature of the harm to be combated requires that the state offense be limited to works that visually depict sexual conduct by children below a specified age. The category of “sexual conduct” proscribed must also be suitably limited and described.

The test for child pornography is separate from the obscenity standard enunciated in Miller, but may be compared to it for the purpose of clarity. The Miller formulation is adjusted in the following respects: A trier of fact need not find that the material appeals to the prurient interest of the average person; it is not required that sexual conduct portrayed be done so in a patently offensive manner; and the material at issue need not be considered as a whole. We note that the distribution of descriptions or other depictions of sexual conduct, not otherwise obscene, which do not involve live performance or photographic or other visual reproduction of live performances, retains First Amendment protection.^[28]

Here the material at issue is one of two things. The first is “child pornography” as defined under federal law. “Child pornography” includes any visual depiction of sexually explicit conduct, where any of the following is true:

• The production of such visual depiction involves the use of a minor engaging in sexually explicit conduct.
• The visual depiction is a digital image, computer image, or computer-generated image that is, or is indistinguishable from, that of a minor engaging in sexually explicit conduct.
• The visual depiction has been created, adapted, or modified to appear that an identifiable minor is engaging in sexually explicit conduct.^[29]

The second category of CSAM under the bill involves “obscene material.” This is defined as matter, taken as a whole, that to the average person, applying contemporary statewide standards, appeals to the prurient interest, that, taken as a whole, depicts or describes sexual conduct in a patently offensive way, and that, taken as a whole, lacks serious literary, artistic, political, or scientific value.^[30] The definition itself is based off of the Miller standard. However, it is only considered CSAM under the bill when the obscene matter depicts a minor personally engaging in, or personally simulating, sexual conduct. This material arguably meets the Ferber standard and falls outside of First Amendment protection.

However, a coalition of business and technology groups argues that the bill still fails to pass constitutional muster because it creates a chilling effect on lawful speech and will result in takedowns of protected material for fear of the massive civil liability that could result:

AB 1394 raises several constitutional concerns and its overbreadth creates a

significant chilling effect on lawful speech. For example, Section 2 of the bill creates a strong incentive to over-remove content any time a request is submitted. Platforms deal with millions of pieces of content every single day. If confronted with a notice to take down content, they will err on the side of caution and remove it due to the significant liability exposure. . . .

To the extent that this bill creates a de facto ban on users under the age of 18, AB 1394 also directly interferes with expressive rights of the minors who will be banned from social media services. As the Supreme Court emphasized in Packingham v. North Carolina, 582 U.S. ___, 137 S.Ct. 1730 (2017): “For many,” social media platforms “are the principal sources for knowing current events, checking ads for employment, speaking and listening in the modern public square, and otherwise exploring the vast realms of human thought and knowledge,” such that “to foreclose access to social media altogether is to prevent the user from engaging in the legitimate exercise of First Amendment rights.” Id. at 1737. To the extent that AB 1394 has the practical effect of foreclosing minors’ access to social media “altogether” (e.g., because AB 1394 makes it practically impossible for social media platforms to offer their services to children in California), the law would raise grave concerns under the First Amendment.

Children have First Amendment rights both to receive information and to express themselves. While protecting children from self-harm is an important interest, AB 1394 makes no attempt to even reasonably scope the restrictions on social media platforms to that goal, let alone to “narrowly tailor” the law as the Constitution requires. Accordingly, broad regulations restricting youth expression have been struck down on First Amendment grounds. Holding unconstitutional a California law prohibiting the sale of violent video games to minors the Supreme Court declared that, “whatever the challenges of applying the Constitution to ever advancing technology, the basic principles of freedom of speech and the press, like the First Amendment’s command, do not vary when a new and different medium for communication appears.”^[31]

1. Conflict with Section 230 of the Communications Decency Act, 47 U.S.C. § 230

In addition to the First Amendment, the other primary source governing content on social media is Section 230. Section 230 does not apply to the users of social media (or the internet generally), but rather applies to the platforms themselves. In the early 1990s, prior to the enactment of Section 230, two trial court orders—one in the United States District Court for the Southern District of New York, and New York state court—suggested that internet platforms could be held liable for allegedly defamatory statements made by the platforms’ users if the platforms engaged in any sort of content moderation (e.g., filtering out offensive material).^[32] In response, two federal legislators and members of the burgeoning internet industry crafted a law that would give internet platforms immunity from liability for users’ statements, even if they might have reason to know that the statements might be false, defamatory, or otherwise actionable.^[33] The result—Section 230—was relatively uncontroversial at the time, in part because of the relative novelty of the internet and in part because Section 230 was incorporated into a much more controversial internet regulation scheme that was the subject of greater debate.^[34]

Section 230 begins with findings and a statement of policy that extol the value of the internet and the intention to let the internet develop without significant government regulation.^[35] The crux of Section 230 is then laid out in two parts. The first provides that “[n]o provider or user of an interactive computer service shall be treated as the publisher or speaker of any information provided by another information content provider.”^[36] The second provides a safe harbor for content moderation, by stating that no provider or user shall be held liable because of good-faith efforts to restrict access to material that is “obscene, lewd, lascivious, filthy, excessively violent, harassing, or otherwise objectionable, whether or not such material is constitutionally protected.”^[37]

Together, these two provisions give platforms immunity from any civil or criminal liability that could be incurred by user statements, while explicitly authorizing platforms to engage in their own content moderation without risking that immunity.

Section 230 specifies that it does not preempt federal criminal laws, but that “[n]o cause of action may be brought and no liability may be imposed under any State law that is inconsistent with this section.”^[38]

Section 230 uses terminology generally applicable in defamation cases (e.g., “publisher,” “speaker”), but courts interpreting Section 230 did not limit its application to the defamation context. Instead, courts have applied Section 230 in a vast range of cases to immunize internet platforms from “virtually all suits arising from third-party content.”^[39] Courts have even extended Section 230 immunity to situations where the platform’s moderator affirmatively solicited the information, selected the user’s statement for publication, and/or edited the content.^[40]

A coalition of industry groups, including NetChoice, believes the bill as written is preempted by Section 230:

Section 230 of the Communications Decency Act (47 U.S.C. §230) generally protects platforms from liability for content that users generate with limited exceptions. This protection enables platforms to host third party content and to moderate third-party content on their platforms without fear of liability.

Without the protections of Section 230, the internet ecosystem would be dramatically different with a limited ability for users to post, share, read, view, and discover the content of others.

Fortunately, Section 230 explicitly preempts state laws such as AB 1394 that would conflict with this protection. This bill creates liability for platforms based on third party content by applying to any feature that allows users to encounter content. It effectively assumes all features are harmful and imposes liability on a site for offering any of those potentially harmful features. Platforms’ algorithms and features that allow users to encounter or share content from other users are inextricably linked to the underlying content. It would also impose liability for failure to remove content, which the Ninth Circuit has held falls squarely within the preemption of Section 230. Therefore, by imposing liability on platforms for their moderation decisions and content serving features, AB 1394 conflicts with Section 230 and is likely preempted.

A brief look at recent, relevant case law is necessary to assess these Section 230 concerns.

First, the Ninth Circuit Court of Appeals in Barnes v. Yahoo!, Inc. (9th Cir. 2009) 570 F.3d 1096, 1100-01 established the prevailing three-part test for certain claims pursuant to Section 230: “[I]t appears that subsection (c)(1) only protects from liability (1) a provider or user of an interactive computer service (2) whom a plaintiff seeks to treat, under a state law cause of action, as a publisher or speaker (3) of information provided by another information content provider.”

A pair of highly anticipated companion cases were recently decided by the Supreme Court in May 2023, Gonzalez v. Google LLC (2023) 143 S. Ct. 1191, and Twitter, Inc. v. Taamneh (2023) 143 S. Ct. 1206, which presented the highest court with the task of determining the scope of Section 230’s protective shield and the valid bases for holding platforms liable for content and conduct carried out on their platforms. The cases below were brought by the families of several victims of ISIS attacks in various parts of the world. The defendants were several social media platforms. The court in the Gonzalez decision declined to address the relevant Section 230 concerns. However, the Court in Taamneh, while not discussing Section 230 application at all, provided some arguably useful guidance:

In this case, the failure to allege that the platforms here do more than transmit information by billions of people—most of whom use the platforms for interactions that once took place via mail, on the phone, or in public areas—is insufficient to state a claim that defendants knowingly gave substantial assistance and thereby aided and abetted ISIS’ acts. A contrary conclusion would effectively hold any sort of communications provider liable for any sort of wrongdoing merely for knowing that the wrongdoers were using its services and failing to stop them. That would run roughshod over the typical limits on tort liability and unmoor aiding and abetting from culpability.^[41]

Particularly relevant here, since its passage, Congress has created one exemption to Section 230 to allow online platforms (including social media platforms) to be held liable for online content promoting or facilitating sexual exploitation or sex trafficking of children.^[42] This exemption was enacted in 2018, as part of the Stop Enabling Sex Traffickers Act and the Allow States to Fight Online Sex Trafficking Act (SESTA-FOSTA) legislation package.^[43] The Ninth Circuit has recently analyzed the scope of the SESTA-FOSTA exception:

In 2018, Congress amended section 230 by passing FOSTA. Pub. L. No. 115-164, 132 Stat. 1253. Among other things, FOSTA provides that section 230 immunity does not apply to certain sex trafficking claims. Pursuant to 47 U.S.C. § 230(e)(5)(A), “[n]othing in [section 230] . . . shall be construed to impair or limit . . . any claim in a civil action brought under section 1595 of title 18, if the conduct underlying the claim constitutes a violation of section 1591 of that title.” In turn, this provision of FOSTA incorporates two sections of the Trafficking Victims Protection Reauthorization Act (TVPRA), 18 U.S.C. § 1589 et seq. First, section 1595 of the TVPRA provides a civil cause of action for violations of the federal trafficking laws. 18 U.S.C. § 1595(a). It permits trafficking victims to sue the perpetrators of their trafficking as well as anyone who “knowingly benefits . . . from participation in a venture which that person knew or should have known” was engaged in sex trafficking. Id.

Section 1591, on the other hand, is the federal criminal child sex trafficking statute. Like section 1595, section 1591 covers both perpetrators and beneficiaries of trafficking. Id. § 1591(a). However, the standard for beneficiary liability pursuant to section 1591 is higher: to be held criminally liable as a beneficiary, a defendant must have actual knowledge of the trafficking and must “assist[], support[], or facilitat[e]” the trafficking venture. Id. § 1591(e)(4).

In sum: websites are generally immune from liability for user-posted content, but that immunity does not cover civil child sex trafficking claims if the “conduct underlying the claim” violates 18 U.S.C. §1591.^[44]

The court concluded that “it is clear that FOSTA requires that a defendant-website violate the criminal statute by directly sex trafficking or, with actual knowledge, ‘assisting, supporting, or facilitating’ trafficking, for the immunity exception to apply.”^[45]

This bill establishes liability for facilitating, aiding, or abetting commercial sexual exploitation. However, the standard that it applies is knowingly, recklessly, or negligently. Given the case law interpreting Section 230 and its exemptions, this standard is arguably extremely susceptible to challenge. In order to more closely tailor the basis for liability to the Supreme Court’s guidance, the author has agreed to amend the bill to limit liability in the latter section of the bill to where a platform knowingly facilitates, aids, or abets commercial sexual exploitation, and removes the reckless and negligence bases:

Amendment

Amend Section 3345.1(g)(1)-(2) as follows:

(g) (1) A social media platform shall not knowingly, recklessly, or negligently facilitate, aid, or abet commercial sexual exploitation.

(2) (A) For a violation of this subdivision that is knowing or reckless, a court shall award statutory damages of five million dollars ($5,000,000).

(B) For a violation of this subdivision that is not described in subparagraph (A), a court shall award statutory damages not exceeding four million dollars ($4,000,000) and not less than one million dollars ($1,000,000) for each act of commercial sexual exploitation facilitated, aided, or abetted by the social media platform.^[46]

SUPPORT

#halfthestory

3strands Global Foundation

American Association of University Women - California

Children’s Advocacy Institute

Common Sense Media

Fairplay

Jewish Family and Children's Services of San Francisco, the Peninsula, Marin and Sonoma Counties

OPPOSITION

California Chamber of Commerce

Civil Justice Association of California

Computer and Communications Industry Association

Electronic Frontier Foundation

NetChoice

TechNet

RELATED LEGISLATION

Pending Legislation:

SB 287 (Skinner, 2023) subjects social media platforms to civil liability for damages caused by their practices, affordances, designs, algorithms, or features, as provided. It provides a safe harbor where certain auditing practices are carried out. SB 287 is currently on the Senate Floor.

SB 646 (Cortese, 2023) creates liability for the distribution of certain “actionable material,” which includes illicit pictures of minors and images or depictions of minors that serve as the basis for criminal and civil liability at the federal level. SB 646 is currently in the Assembly Appropriations Committee.

SB 680 (Skinner, 2023) is substantially similar to SB 287. SB 680 is currently in the Assembly Privacy and Consumer Protection Committee.

SB 764 (Padilla, 2023) prohibits a social media platform from adopting or implementing a policy or practice related to the targeting of content to minors that prioritizes user engagement of minor users over the safety, health, and well-being of the minor users if the social media platform knows or, should know that it has caused harm to minor users or it is reasonably foreseeable that it will cause harm to minor users. SB 764 is currently pending before the Senate Judiciary Committee.

SB 845 (Stern, 2023) requires large social media platforms, as defined, to create, maintain, and make available to third-party safety software providers a set of real-time application programming interfaces, through which a child or a parent or legal guardian of a child may delegate permission to a third-party safety software provider to manage the child’s online interactions, content, and account settings on the large social media platform on the same terms as the child, and for other purposes. SB 845 is pending before the Senate Judiciary Committee.

AB 955 (Petrie-Norris, 2023) makes the sale of fentanyl on a social media platform a crime punishable by imprisonment in a county jail for three, six, or nine years (higher than the existing penalty for selling fentanyl, which is imprisonment in a county jail for two, three, or four years). AB 955 is pending before the Assembly Public Safety Committee.

Prior Legislation:

SB 1056 (Umberg, Ch. 881, Stats. 2022) required a social media platform, as defined, to clearly and conspicuously state whether it has a mechanism for reporting violent posts, as defined; and allows a person who is the target, or who believes they are the target, of a violent post to seek an injunction to have the violent post removed.

AB 587 (Gabriel, Ch. 269, Stats. 2022) required social media companies, as defined, to post their terms of service and report certain information to the Attorney General on a quarterly basis.

AB 1628 (Ramos, Ch. 432, Stats. 2022) required a social media platform, as defined, that operates in this state to create and publicly post a policy statement including specified information pertaining to the use of the platform to illegally distribute controlled substances, until January 1, 2028.

AB 2273 (Wicks, Ch. 320, Stats. 2022) established the California Age-Appropriate Design Code Act, placing a series of obligations and restriction on businesses that provide online services, products, or features likely to be accessed by a child.

AB 2408 (Cunningham, 2022) would have prohibited a social media platform from using a design, feature, or affordance that the platform knew, or which by the exercise of reasonable care it should have known, causes child users to become addicted to the platform. AB 2408 died in the Senate Appropriations Committee.

AB 2571 (Bauer-Kahan, Ch. 77, Stats. 2022) prohibits firearm industry members from advertising or marketing, as defined, firearm-related products to minors. This bill restricts the use of minors’ personal information in connection with marketing or advertising firearm-related products to those minors.

AB 2879 (Low, Ch. 700, Stats. 2022) requires a social media platform to disclose its cyberbullying reporting procedures in its terms of service and to have a mechanism for reporting cyberbullying that is available to individuals whether or not they have an account on the platform.

AB 1114 (Gallagher, 2021) would have required a social media company located in California to develop a policy or mechanism to address content or communications that constitute unprotected speech, including obscenity, incitement of imminent lawless action, and true threats, or that purport to state factual information that is demonstrably false. AB 1114 died in the Assembly Arts, Entertainment, Sports, Tourism, and Internet Media Committee.

SB 388 (Stern, 2021) would have required a social media platform company, as defined, that, in combination with each subsidiary and affiliate of the service, has 25,000,000 or more unique monthly visitors or users for a majority of the preceding 12 months, to report to the Department of Justice by April 1, 2022, and annually thereafter, certain information relating to its efforts to prevent, mitigate the effects of, and remove potentially harmful content. SB 388 died in the Senate Judiciary Committee.

PRIOR VOTES:

Assembly Floor (Ayes 77, Noes 0)

Assembly Judiciary Committee (Ayes 10, Noes 0)

Assembly Privacy and Consumer Protection Committee (Ayes 8, Noes 0)

**************

1. Rhiannon Williams, The US now hosts more child sexual abuse material online than any other country (April 26, 2023) MIT Technology Review, https://www.technologyreview.com/2022/04/26/1051282/the-us-now-hosts-more-child-sexual-abuse-material-online-than-any-other-country/. All internet citations are current as of July 2, 2023. ↑

2. Ibid. ↑

3. Samantha Cole, Facebook Reported 20 Million Instances of Child Sexual Abuse in 2020 (February 24, 2021) Motherboard, Tech by Vice, https://www.vice.com/en/article/7k9an4/facebook-pornhub-child-abuse-content-ncmec-report-2020. ↑

4. See fn. 1. ↑

5. Ibid. ↑

6. This includes conforming amendments that include other references to “instances” but do not make any changes to subdivisions (e) through (h) of this section. ↑

7. 1 CACI 430 (2023). ↑

8. U.S. Const., 1st & 14th amends. ↑

9. Ashcroft v. American Civil Liberties Union (2002) 535 U.S. 564, 573. ↑

10. Snyder v. Phelps (2011) 562 U.S. 443, 452. ↑

11. Cohen v. California (1971) 403 U.S. 15, 20. ↑

12. Virginia v. Black (2003) 538 U.S. 343, 359. ↑

13. R.A.V. v. St. Paul (1992) 505 U.S. 377, 383. ↑

14. Ibid. ↑

15. Reno v. ACLU (1997) 521 U.S. 844, 870. ↑

16. E.g., Hudgens v. NLRB (1976) 424 U.S. 507, 513. Some have argued that certain social media platforms are so essential to the freedom of expression that they should be treated as common carriers subject to the First Amendment. ↑

17. E.g., Packingham v. North Carolina (2017) 137 S.Ct. 1730, 1735-1736. ↑

18. Ashcroft v. Free Speech Coalition (2002) 535 U.S. 234, 255; see also United States v. Alvarez (2012) 567 U.S. 709, 717 (Supreme Court “has rejected as ‘startling and dangerous’ a ‘free-floating test for First Amendment coverage…[based on] an ad hoc balancing of relative social costs and benefits’ ” [alterations in original]). ↑

19. Ward v. Rock Against Racism (1989) 491 U.S. 781, 791.   ↑

20. FCC v. League of Women Voters (1984) 468 U.S. 364, 383.  ↑

21. Packingham, supra, 137 S.Ct. at p. 1736. ↑

22. McCullen v. Coakley (2014) 573 U.S. 464, 486 (McCullen). ↑

23. Id. at p. 478.  ↑

24. Id. at p. 479. ↑

25. Reed v. Town of Gilbert (2015) 135 S.Ct. 2218, 2226 (Reed). ↑

26. United States v. Playboy Entertainment Group (2000) 529 U.S. 803, 813. ↑

27. New York v. Ferber (1982) 458 U.S. 747, 756. ↑

28. Id. at 764-65. It should also be noted that the United States Supreme Court in Ashcroft v. Free Speech Coalition (2002) 535 U.S. 234, 239, ruled that the federal Child Pornography Prevention Act’s ban on virtual child pornography was unconstitutionally overbroad as it proscribed speech which was neither child pornography nor obscene. ↑

29. 18 U.S.C. § 2256. ↑

30. Pen. Code § 311. ↑

31. Citing Brown v Entertainment Merchants Ass’n, (2011) 564 U.S. 786, 790. ↑

32. See Cubby, Inc. v. Compuserve, Inc. (S.D.N.Y. 1991) 776 F.Supp. 135, 141; Stratton Oakmont v. Prodigy Servs. Co. (N.Y. Sup. Ct., May 26, 1995) 1995 N.Y. Misc. LEXIS 229, *10-14. These opinions relied on case law developed in the context of other media, such as whether book stores and libraries could be held liable for distributing defamatory material when they had no reason to know the material was defamatory. (See Cubby, Inc., 776 F. Supp. at p. 139; Smith v. California (1959) 361 U.S. 147, 152-153.) ↑

33. Kosseff, The Twenty-Six Words That Created The Internet (2019) pp. 57-65. ↑

34. Id. at pp. 68-73. Section 230 was added to the Communications Decency Act of 1996 (title 5 of the Telecommunications Act of 1996, Pub. L. 104-104, 110 Stat. 56), which would have imposed criminal liability on internet platforms if they did not take steps to prevent minors from obtaining “obscene or indecent” material online. The Supreme Court invalidated the CDA, except for Section 230, on the basis that it violated the First Amendment. (See Reno, supra, 521 U.S. at p. 874.) ↑

35. 47 U.S.C. § 230(a) & (b). ↑

36. Id., § 230(c)(1). ↑

37. Id., § 230(c)(1) & (2). ↑

38. Id., § 230(e)(1) & (3). ↑

39. Kosseff, supra, fn. 13, at pp. 94-95; see, e.g., Doe v. MySpace Inc. (5th Cir. 2008) 528 F.3d 413, 421-422; Carfano v. Metrospalsh.com, Inc. (9th Cir. 2003) 339 F.3d 1119, 1125; Zeran v. America Online, Inc. (4th Cir. 1997) 129 F.3d 327, 333-334. ↑

40. See, e.g., Jones v. Dirty World Entertainment Recordings LLC (6th Cir. 2014) 755 F.3d 398, 415; Batzel v. Smith (9th Cir. 2003) 333 F.3d 1018, 1030-1031; cf. Blumenthal v. Drudge (D.D.C. 1998) 992 F.Supp. 44, 51-52. ↑

41. Taamneh, 143 S. Ct. at 1213. ↑

42. Id., § 230(e)(5). ↑

43. See P.L. 115-164, 113 Stat. 1253. It should be noted that the United States Government Accountability Office (GAO) found that SESTA-FOSTA made it more difficult for law enforcement to gather information about actual sex trafficking, and the main effect of it was to cause online platforms to shut down pages featuring legitimate activities for fear of liability. See GAO Report to Congressional Committees, Sex Trafficking: Online Platforms and Federal Prosecutions, No. 21-385 (June 2021), pp. 20-25, https://www.gao.gov/assets/gao-21-385.pdf. Congress has also altered the liability of Section 230 with respect to hosting copyrighted material by allowing platforms to be held liable for users’ copyright violations unless the platform blocks access to alleged infringing material upon receiving a notice of infringement. See 17 U.S.C. § 512, the Digital Millennium Copyright Act (DCMA). ↑

44. Does v. Reddit, Inc. (9th Cir. 2022) 51 F.4th 1137, 1140-41. ↑

45. Id. at 1145. ↑

46. These amendments do not make any changes to the other paragraphs of this subdivision or the other subdivisions of this section. ↑

ASSEMBLY THIRD READING

ATR Bill Id:AB 1394

Author:(Wicks and Flora)

As Amended Ver:May 25, 2023

Majority vote

SUMMARY

Requires social media platforms to provide a mechanism for users to report child sexual abuse material in which they are depicted; provides platforms 30-60 days after receiving a report to verify the content of the material and block it from reappearing. Also provides victims of commercial sexual exploitation the right to sue social media platforms for having deployed features that were a substantial factor in causing their exploitation.

Major Provisions

1. Defines "child sexual abuse material" to mean either of the following:
   1. Child pornography, as defined under federal law.
   2. Obscene matter, as defined under state law, that depicts a minor personally engaging in, or personally simulating, sexual conduct.
2. Requires a social media platform to provide, in a form that is reasonably accessible, a means for a user who is a California resident to report material to the platform that the user reasonably believes meets all of the following criteria:
   1. It is child sexual abuse material.
   2. The user themselves is an identifiable minor depicted in the reported material.
   3. The reported material is displayed, stored, or hosted on the social media platform.
3. Requires the social media platform to block material reported under 2), above, if the reported material meets both of the following criteria:
   1. The reported material is child sexual abuse material.
   2. The reported material is displayed, stored, or hosted on the social media platform.
4. Requires a social media platform to collect from the reporting user information reasonably sufficient to enable the platform to contact the user in writing. The social media platform must meet the following schedule of written communications with the reporting user:
   1. Within 24 hours, the reporting user must be notified that the platform received their report.
   2. Every seven days thereafter, the reporting user must be provided an update as to the status of the social media platform's handling of the reported material, until a final written determination is issued under c), below.
   3. No later than 30 days after the material was first reported, the platform must issue a final written determination stating one of the following:
      1. The reported material has been determined to be child sexual abuse material that was displayed, stored, or hosted on the social media platform and has been blocked on the social media platform.
      2. The reported material has been determined not to be child sexual abuse material.
      3. The reported material has been determined not to be displayed, stored, or hosted on the social media platform.
5. Provides that if the social media platform cannot comply with the requirements in the timeframe set forth above, due to circumstances beyond its reasonable control, then:
   1. The platform must comply with these requirements no later than 60 days after the date on which the material was first reported.
   2. The platform must provide written notice of the delay to the reporting user no later than 48 hours from the time the social media platform knew the delay was likely to occur, using the contact method collected under 4), above.
6. Holds a social media company liable to a reporting user if it fails to comply with the requirements in paragraphs 2) to 5), above. Available remedies include any actual damages sustained by the reporting user as a result of the violation, and statutory damages of no more than $250,000 per violation. This amount is reduced to no more than $125,000 per violation if a company fails to comply with the requirements in paragraphs 2) to 5), above, but eventually permanently blocks the reported material before a lawsuit is filed. In determining the amount of statutory damages, the court is to consider the willfulness and severity of the violation, as well as previous violations by the social media company.
7. Holds social media platforms liable if they knowingly, recklessly, or negligently facilitate, aid, or abet commercial sexual exploitation of minors.
8. Defines "facilitate, aid, or abet" as the deployment of a system, design, feature, or affordance that is a substantial factor in causing minor users to be victims of commercial sexual exploitation.
9. Provides that a court is to award statutory damages of between $1 million and $4 million for each act of commercial sexual exploitation facilitated, aided, or abetted by the social media platform. Further provides that if a platform knowingly or recklessly facilitates, aids, or abets commercial sexual exploitation, or if a violation is part of a pattern or practice of doing so, then the court shall award statutory damages of $5 million per violation.
10. Establishes a process for a social media platform to not be deemed to have facilitated, aided, or abetted commercial sexual exploitation via (a) audits of its designs, algorithms, practices, affordances, and features that occur at least quarterly and (b) correction within 30 days of any such designs, algorithms, practices, affordances, and features that present more than a minimal risk of violation.

COMMENTS

Background. Child sexual abuse material, commonly referred to under the acronym “CSAM,” is tragically pervasive on the internet – and not only in its illicit corners, on the so-called “dark web,” but also on popular social media websites and applications that billions of people use every day. In 2020, household names such as Discord, Reddit, TikTok, Twitch, and Twitter each self-reported thousands of instances of CSAM on their systems to the National Center for Missing & Exploited Children; Facebook reported millions. (NCMEC, 2020 CyberTipline Reports by Electronic Service Providers (2021), available at [https://www.missingkids.org/content/dam/missingkids/pdfs/2020-reports-by-esp.pdf.])

This tragedy is compounded by the fact that certain websites and applications are not only a convenient means for sharing CSAM, but arguably induce its production:

A Forbes review of hundreds of recent TikTok livestreams reveals how viewers regularly use the comments to urge young girls to perform acts that appear to toe the line of child pornography—rewarding those who oblige with TikTok gifts, which can be redeemed for money, or off-platform payments to Venmo, PayPal or Cash App accounts that users list in their TikTok profiles. (Levine, How TikTok Live Became 'A Strip Club Filled With 15-Year-Olds,' Forbes (Apr. 27, 2022), available at [https://www.forbes.com/sites/alexandralevine/2022/04/27/how-tiktok-live-became-a-strip-club-filled-with-15-year-olds/.])

This measure proposes to address these challenges in two ways, each described below.

First, this bill is meant to ensure that social media platforms promptly block child sexual abuse material reported by victims. This bill establishes a comprehensive process for users to report material that they reasonably believe to be (i) CSAM, (ii) in which they themselves are depicted as identifiable minors, and (iii) displayed, stored, or hosted on a social media platform. Following a report, a platform would have 30 days (with a potential extension to 60 days) to verify whether the material is CSAM, and if it is, block it from appearing on the platform. The steps in this process are set forth in paragraphs 2) - 5) of the SUMMARY above. Some important points to note about this process are as follows.

1. Social media platforms can use a third-party service to collect CSAM reports, and thereby avoid incurring liability that might arise from coming into possession of CSAM, particularly CSAM that is not already present on their systems.
2. Social media platforms would only be required to block specific CSAM reported by users.
3. Social media platforms must acknowledge user reports and keep users updated on the progress of their requests.
4. Social media platforms would retain the ability to pursue users who make fraudulent reports. The bill provides that nothing in its text limits or impairs in any way a cause of action for deceit involving an intentional misrepresentation of fact.

Second, this bill would hold platforms liable if they cause commercial sexual exploitation. Existing California law holds adults civilly liable for the tort of commercial sexual exploitation: sexually exploiting minors in exchange for money, property, or something else of value. This bill proposes to extend such liability to social media platforms that knowingly, recklessly, or negligently facilitate, aid, or abet commercial sexual exploitation. "Facilitate, aid, or abet" is defined as "deploy[ing] a system, design, feature, or affordance that is a substantial factor in causing minor users to be victims of commercial sexual exploitation." The "substantial factor" test is generally understood to be the appropriate causation standard under California tort law. (Rutherford v. Owens-Illinois, Inc. (1997) 16 Cal. 4th 953, 968-9.)

It is critical to note that the bill focuses on the system, design, feature, or affordance at issue, not particular content accessible through the site. This wording is meant to insulate the provision in question from being struck down under Section 230 of the federal Communications Decency Act, United States Code Title 47, Section 230(c). There are limits to Section 230's preemptive effects, however; the law is "not meant to create a lawless no-man's-land on the Internet." (Council of San Fernando Valley v. Roommates.com, LLC (9th Cir. 2007) 489 F.3d 921.) Lemmon v. Snap (9th Cir. 2021) 995 F.3d 1085, 1094 provides that even under Section 230, interactive computer services "face the prospect of liability, even for their ‘neutral tools,’ so long as plaintiffs' claims do not blame them for the content that third parties generate with those tools."

The liability standard under this bill would not be easy to meet. Because of Section 230, the ability to share digital images or videos on a social media platform, in and of itself, is likely to be insufficient to trigger liability—even if the images or videos shared were produced as a result of commercial sexual exploitation. But, as now worded, it appears that the bill would allow social media platforms to be held liable for deploying features that cause commercial sexual exploitation, without running afoul of Section 230. To the extent this dissuades platforms from deploying features that induce commercial sexual exploitation, this appears a beneficial measure.

According to the Author

We are not serious about ending the horror of sexual exploitation and trafficking of children if we do not aggressively address the prominent role gigantic social media platforms knowingly play in facilitating such exploitation and trafficking and re-traumatizing survivors for a lifetime afterwards.

Arguments in Support

LookUp.Live lauds the agency this bill would provide survivors:

We…appreciate that this bill also empowers survivors haunted by the prospect of images and videos about them lingering on platforms to require this content be rendered invisible, with the ability to recover penalties against platforms that deny their requests.

Arguments in Opposition

California Chamber of Commerce, Civil Justice Association of California, NetChoice, and Technet warn the bill could inadvertently harm minors:

[P]latforms could choose to disable features that could be misused by perpetrators to traffic children, such as direct messaging, chat forums, or other communicative features. They could also prohibit anyone under the age of 18 from using their platforms. In either instance, more lawful speech is impacted and burdened than intended, thus creating a chilling effect on providing and hosting lawful content and speech on social media platforms. Though well-intentioned, this bill will result in more lawful speech being removed and fewer online spaces for teens to communicate and share ideas with one another.

FISCAL COMMENTS

None.

VOTES

ASM PRIVACY AND CONSUMER PROTECTION: 8-0-3

YES: Gabriel, Bauer-Kahan, Bennett, Irwin, Kalra, Lowenthal, Papan, Wicks

ABS, ABST OR NV: Joe Patterson, Essayli, Vince Fong

ASM JUDICIARY: 10-0-1

YES: Maienschein, Connolly, Dixon, Haney, Kalra, Pacheco, Papan, Reyes, Robert Rivas, Sanchez

ABS, ABST OR NV: Essayli

UPDATED

VERSION: May 25, 2023

CONSULTANT: Jith Meganathan / P. & C.P. / (916) 319-2200 FN: 0000989

ASSEMBLY THIRD READING

ATR Bill Id:AB 1394

Author:(Wicks and Flora)

As Amended Ver:April 10, 2023

Majority vote

SUMMARY

Requires social media platforms to provide a mechanism for users to report child sexual abuse material in which they are depicted; provides platforms 30-60 days after receiving a report to verify the content of the material and block it from reappearing. Also provides victims of commercial sexual exploitation the right to sue social media platforms for having deployed features that were a substantial factor in causing their exploitation.

Major Provisions

1. Defines "child sexual abuse material" to mean either of the following:
   1. Child pornography, as defined under federal law.
   2. Obscene matter, as defined under state law, that depicts a minor personally engaging in, or personally simulating, sexual conduct.
2. Requires a social media platform to provide, in a form that is reasonably accessible, a means for a user who is a California resident to report material to the platform that the user reasonably believes meets all of the following criteria:
   1. It is child sexual abuse material.
   2. The user themselves is an identifiable minor depicted in the reported material.
   3. The reported material is displayed, stored, or hosted on the social media platform.
3. Requires the social media platform to block material reported under 2), above, if the reported material meets both of the following criteria:
   1. The reported material is child sexual abuse material.
   2. The reported material is displayed, stored, or hosted on the social media platform.
4. Requires a social media platform to collect from the reporting user information reasonably sufficient to enable the platform to contact the user in writing. The social media platform must meet the following schedule of written communications with the reporting user:
   1. Within 24 hours, the reporting user must be notified that the platform received their report.
   2. Every seven days thereafter, the reporting user must be provided an update as to the status of the social media platform's handling of the reported material, until a final written determination is issued under c), below.
   3. No later than 30 days after the material was first reported, the platform must issue a final written determination stating one of the following:
      1. The reported material has been determined to be child sexual abuse material that was displayed, stored, or hosted on the social media platform and has been blocked on the social media platform.
      2. The reported material has been determined not to be child sexual abuse material.
      3. The reported material has been determined not to be displayed, stored, or hosted on the social media platform.
5. Provides that if the social media platform cannot comply with the requirements in the timeframe set forth above, due to circumstances beyond its reasonable control, then:
   1. The platform must comply with these requirements no later than 60 days after the date on which the material was first reported.
   2. The platform must provide written notice of the delay to the reporting user no later than 48 hours from the time the social media platform knew the delay was likely to occur, using the contact method collected under 4).
6. Holds a social media company liable to a reporting user if it fails to comply with the requirements in paragraphs 2) to 5), above. Available remedies include any actual damages sustained by the reporting user as a result of the violation, and statutory damages of no more than $250,000 per violation. In determining the amount of statutory damages, the court is to consider the willfulness and severity of the violation, as well as previous violations by the social media company.
7. Holds social media platforms liable if they knowingly, recklessly, or negligently facilitate, aid, or abet commercial sexual exploitation of minors.
8. Defines "facilitate, aid, or abet" as the deployment of a system, design, feature, or affordance that is a substantial factor in causing minor users to be victims of commercial sexual exploitation.
9. Provides that a court is to award statutory damages of between $1 million and $5 million for each act of commercial sexual exploitation facilitated, aided, or abetted by the social media platform.

COMMENTS

Background. Child sexual abuse material, commonly referred to under the acronym "CSAM," is tragically pervasive on the internet—and not only in its illicit corners, on the so-called "dark web," but also on popular social media websites and applications that billions of people use every day. In 2020, household names such as Discord, Reddit, TikTok, Twitch, and Twitter each self-reported thousands of instances of CSAM on their systems to the National Center for Missing & Exploited Children; Facebook reported millions. (NCMEC, 2020 CyberTipline Reports by Electronic Service Providers (2021), available at https://www.missingkids.org/content/dam/missingkids/pdfs/2020-reports-by-esp.pdf.)

This tragedy is compounded by the fact that certain websites and applications are not only a convenient means for sharing CSAM, but arguably induce its production:

A Forbes review of hundreds of recent TikTok livestreams reveals how viewers regularly use the comments to urge young girls to perform acts that appear to toe the line of child pornography—rewarding those who oblige with TikTok gifts, which can be redeemed for money, or off-platform payments to Venmo, PayPal or Cash App accounts that users list in their TikTok profiles. (Levine, How TikTok Live Became 'A Strip Club Filled With 15-Year-Olds,' Forbes (Apr. 27, 2022), available at https://www.forbes.com/sites/alexandralevine/2022/04/27/how-tiktok-live-became-a-strip-club-filled-with-15-year-olds/.)

This measure proposes to address these challenges in two ways, each described below.

First, this bill is meant to ensure that social media platforms promptly block child sexual abuse material reported by victims. This bill establishes a comprehensive process for users to report material that they reasonably believe to be (i) CSAM, (ii) in which they themselves are depicted as identifiable minors, and (iii) displayed, stored, or hosted on a social media platform. Following a report, a platform would have 30 days (with a potential extension to 60 days, if required by factors beyond the platform's control) to verify whether the material is CSAM, and if it is, block it from appearing on the platform. The steps in this process are set forth in paragraphs 2) - 5) of the SUMMARY above. Some important points to note about this process are as follows.

1. Social media platforms can use a third-party service to collect CSAM reports, and thereby avoid incurring liability that might arise from coming into possession of CSAM, particularly CSAM that is not already present on their systems.
2. Social media platforms would only be required to block the specific CSAM reported by users.
3. Social media platforms must acknowledge user reports and keep users updated on the progress of their requests.
4. Social media platforms would retain the ability to pursue users who make fraudulent reports. The bill provides that nothing in its text limits or impairs in any way a cause of action for deceit involving an intentional misrepresentation of fact.

This bill would hold platforms liable if they cause commercial sexual exploitation. Existing California law holds adults civilly liable for the tort of commercial sexual exploitation: sexually exploiting minors in exchange for money, property, or something else of value. This bill proposes to extend such liability to social media platforms that knowingly, recklessly, or negligently facilitate, aid, or abet commercial sexual exploitation.

"Facilitate, aid, or abet" is defined as "deploy[ing] a system, design, feature, or affordance that is a substantial factor in causing minor users to be victims of commercial sexual exploitation." The "substantial factor" test is generally understood to be the appropriate causation standard under California tort law. (Rutherford v. Owens-Illinois, Inc. (1997) 16 Cal. 4th 953, 968-9.)

It is critical to note that the bill focuses on the system, design, feature, or affordance at issue, not particular content accessible through the site. This wording is meant to insulate the provision in question from being struck down under Section 230 of the federal Communications Decency Act, 47 United States Code Section230(c). There are limits to Section 230's preemptive effects, however; the law is "not meant to create a lawless no-man's-land on the Internet." (Council of San Fernando Valley v. Roommates.com, LLC (9th Cir. 2007) 489 F.3d 921, aff'd en banc (9th Cir. 2008) 521 F.3d 1157, 1164.) Lemmon v. Snap (9th Cir. 2021) 995 F.3d 1085, 1094 provides that even under Section 230, interactive computer services "face the prospect of liability, even for their 'neutral tools,' so long as plaintiffs' claims do not blame them for the content that third parties generate with those tools."

The liability standard under this bill would not be easy to meet. Because of Section 230, the ability to share digital images or videos on a social media platform, in and of itself, is likely to be insufficient to trigger liability—even if the images or videos shared were produced as a result of commercial sexual exploitation. But, as now worded, it appears that the bill would allow social media platforms to be held liable for deploying features that cause commercial sexual exploitation, without running afoul of Section 230. To the extent this dissuades platforms from deploying features that induce commercial sexual exploitation, this appears a beneficial measure.

According to the Author

We are not serious about ending the horror of sexual exploitation and trafficking of children if we do not aggressively address the prominent role gigantic social media platforms knowingly play in facilitating such exploitation and trafficking and re-traumatizing survivors for a lifetime afterwards.

Social media platforms that simply avoid being directly involved or active participants in child sexual exploitation in the first place and who respond with a minimum of compassion to the pleas of survivors to remove images and videos documenting their abuse have nothing to fear from this measure.

Arguments in Support

LookUp.Live lauds the agency this bill would provide survivors:

We…appreciate that this bill also empowers survivors haunted by the prospect of images and videos about them lingering on platforms to require this content be rendered invisible, with the ability to recover penalties against platforms that deny their requests.

Arguments in Opposition

An opposition coalition, consisting of the California Chamber of Commerce, the Civil Justice Association of California, NetChoice, and Technet, warns that this bill could inadvertently harm minors:

[P]latforms could choose to disable features that could be misused by perpetrators to traffic children, such as direct messaging, chat forums, or other communicative features. They could also prohibit anyone under the age of 18 from using their platforms. In either instance, more lawful speech is impacted and burdened than intended, thus creating a chilling effect on providing and hosting lawful content and speech on social media platforms. Though well-intentioned, this bill will result in more lawful speech being removed and fewer online spaces for teens to communicate and share ideas with one another.

FISCAL COMMENTS

None.

VOTES

ASM PRIVACY AND CONSUMER PROTECTION: 8-0-3

YES: Gabriel, Bauer-Kahan, Bennett, Irwin, Kalra, Lowenthal, Papan, Wicks

ABS, ABST OR NV: Joe Patterson, Essayli, Vince Fong

ASM JUDICIARY: 10-0-1

YES: Maienschein, Connolly, Dixon, Haney, Kalra, Pacheco, Papan, Reyes, Robert Rivas, Sanchez

ABS, ABST OR NV: Essayli

UPDATED

VERSION: April 10, 2023

CONSULTANT: Jith Meganathan / P. & C.P. / (916) 319-2200 FN: 0000298

Date of Hearing: May 2, 2023

ASSEMBLY COMMITTEE ON JUDICIARY

Brian Maienschein, Chair

ABPCA Bill Id:AB 1394

Author:(Wicks) – As Amended Ver:April 10, 2023

SUBJECT: Commercial sexual exploitation: child sexual abuse material: civil actions

KEY ISSUES:

1. SHOULD SOCIAL MEDIA PLATFORMS, AS DEFINED, BE REQUIRED TO IMPLEMENT AND MAINTAIN A MECHANISM BY WHICH A USER MAY REPORT CHILD SEXUAL ABUSE MATERIAL TO THE SOCIAL MEDIA PLATFORM; AND SHOULD A REPORTING USER HAVE A CAUSE OF ACTION AGAINST THE PLATFORM IF THEY FAIL TO FOLLOW THESE PROCEDURES?
2. SHOULD EXISTING LAW BE EXPANDED TO HOLD A SOCIAL MEDIA PLATFORM LIABLE FOR KNOWINGLY, RECKLESSLY, OR NEGLIGNETLY FACILITATING, AIDING, OR ABETTING THE COMMERCIAL SEXUAL EXPLOITATION OF A MINOR?

SYNOPSIS

Child sexual abuse material (CSAM) refers to content, including pictures and videos, which capture the sexual abuse of children. CSAM is deplorable and yet tragically prevalent throughout social media. In 2021, the National Center for Missing and Exploited Children (NCMEC) estimated that over 29 million reports of suspected child sexual exploitation were made by online platforms. Those 29 million reports included nearly 85 million individual images and videos of CSAM.

This bill proposes to address the scourge of CSAM on social media platforms through two separate mechanisms. First, the bill directs social media platforms to implement a mechanism by which a user may submit a report of suspected CSAM in which they are depicted as a minor, that is displayed, stored, or hosted on the social media platform. The bill establishes a right of action for a reporting user to bring against the platform for the platform’s failure to adhere to the procedure established by this bill. Second, the bill proposes an extension of currently existing law. Civil Code Section 3345.1 currently establishes a cause of action against a person who engaged in any act of commercial sexual exploitation in a claim brought by, or on behalf of, a minor victim of commercial sexual exploitation. The bill proposes to make social media platforms liable under this section if they knowingly, recklessly, or negligently facilitate, aid, or abet commercial sexual exploitation, and proposes striking potential damages for a violation of the previous prohibition: between $1,000,000 and $5,000,000 for each act of commercial sexual exploitation that is facilitated, aided, or abetted by the platform.

This bill is co-sponsored by the Children’s Advocacy Institute, Common Sense Media, and the American Association of University Women. It is also supported by several media watchdog groups. It is opposed by a coalition of tech industry and business advocates. This bill was previously heard by the Assembly Committee on Privacy and Consumer Protection, which passed the bill by a vote of 8-0.

SUMMARY: Requires social media platforms to provide a mechanism for platform users to report child sexual abuse material in which they are depicted; provides platforms with 30-60 days to verify the content of the material and block it from reappearing after receiving a report. Also provides victims of commercial sexual exploitation with the right to sue social media platforms that knowingly, recklessly or negligently facilitate, aid, or abet their exploitation. Specifically, this bill:

1. Makes various findings and declarations regarding the sexual abuse, exploitation, and trafficking of children on social media platforms.
2. Defines the following for purposes of this bill:
   1. “Child pornography” has the same meaning as existing federal law.
   2. “Child sexual abuse material” means either child pornography or obscene matter that depicts a minor personally engaging in, or personally simulating, sexual conduct.
   3. “Identifiable minor” has the same meaning as existing federal law.
   4. “Minor” has the same meaning as existing federal law.
   5. “Obscene matter” has the same meaning as existing state law.
   6. “Reporting user” means a natural person who reports material to a social media platform using the means provided by the social media platform pursuant to 3).
   7. “Social media company” has the same meaning as existing state law.
   8. “Social media platform” has the same meaning as existing law, except that it does not include a standalone direct messaging service that provides end-to-end encrypted communication.
3. Requires a social media platform to do all of the following:
   1. Provide, in a form that is reasonably accessible to users, a means for a user who is a California resident to report material to the social media platform that the user reasonably believes meets all of the following criteria:
      1. The reported material is child sexual abuse material;
      2. The reporting user is an identifiable minor depicted in the reported material;
      3. The reported material is displayed, stored, or hosted on the social media platform.
   2. Collect information reasonably sufficient to enable the social media platform to contact, pursuant to c), a reporting user.
   3. Contact a reporting user in writing by a method including, but not limited to, a mailing address, a telephone number for purposes of sending text message, or an email address, that is chosen by the user and is not a method that is within the control of the social media company that owns or operates the social media platform.
   4. Permanently block the reported material from being viewable on the social media platform if the reported material is both child sexual abuse material and is displayed, stored, or hosted on the social media platform.
   5. Provide written confirmation to a reporting user that the social media platform received that person’s report within 24 hours of when the material was first reported using the information collected from the user under b) that informs the reporting user of the schedule of regular written updates that the social media platform is required to make pursuant to f).
   6. Provide a written update to the reporting user as to the status of the social media platform’s handling of the reported material using the information collected from the reporting user pursuant to b).
      1. Requires the written update to be provided seven days after the date on which the written confirmation required under e) was provided and every seven days thereafter until the final written determination required pursuant to g) is provided.
   7. Issue a final written determination to the reporting user stating one of the following:
      1. That the reported material has been determined to be child sexual abuse material that was displayed, stored, or hosted on the social media platform and has been blocked on the social media platform.
      2. That the reported material has been determined not to be child sexual abuse material.
      3. That the reported material has been determined not to be displayed, stored, or hosted on the social media platform.
   8. Comply with a) – g) no later than 30 days after the date on which the material was first reported.
4. Requires a social media platform who cannot comply with h), due to circumstances beyond the reasonable control of the social media platform, to comply with a) – g) no later than 60 days on which the material was first reported. Requires the social media platform to promptly provide written notice of the delay, no later than 48 hours from the time the social media platform knew the delay was likely to occur, to the reporting user.
5. Makes a social media company that fails to comply with 3) liable to a reporting user for all of the following:
   1. Any actual damages sustained by the reporting user as a result of the violation;
   2. Statutory damages of no more than $250,000 per violation, considering the willfulness and severity of the violation and whether the social media platform has previously violated the provisions of this bill;
   3. Costs of the action, together with reasonable attorney’s fees, as determined by the court;
   4. Any other relief that the court deems proper.
6. Creates a rebuttable presumption that the reporting user is entitled to statutory damages if a social media platform fails to comply with 3).
7. Makes a waiver of the provisions of this bill as contrary to public policy and void and unenforceable.
8. Makes provisions of 1) – 7) severable.
9. Prohibits a social media platform from knowingly, recklessly, or negligently facilitating, aiding, or abetting commercial sexual exploitation.
10. Directs a court to award statutory damages not exceeding $5,000,000 for a violation of 9).
11. Defines the following for purposes of 9) and 10):
    1. “Facilitate, aid, or abet” means deploy a system, design, feature, or affordance that is a substantial factor in causing minor users to be victims of commercial sexual exploitation.
    2. “Social media platform” has the same meaning as existing law, but does not include a standalone direct messaging service that provides end-to-end encrypted communication.

EXISTING LAW:

1. Establishes, under Section 230 of the Communications Decency Act, that no provider or user of an interactive computer service shall be treated as the publisher or speaker of any information provided by another information content provider. (47 U.S.C. Section 230(c)(1).)
2. Exempts from Section 230 protection violations of federal criminal law; intellectual property law; state law that is consistent with Section 230; communications privacy law; and sex trafficking law. (47 U.S.C. Section 230(e).)
3. Defines “child pornography” as any visual depiction of sexually explicit conduct, where any of the following is true:
   1. The production of such visual depiction involves the use of a minor engaging in sexually explicit conduct.
   2. The visual depiction is a digital image, computer image, or computer-generated image that is, or is indistinguishable from, that of a minor engaging in sexually explicit conduct.
   3. The visual depiction has been created, adapted, or modified to appear that an identifiable minor is engaging in sexually explicit conduct. (18 U.S.C. Section 2256.)
4. Defines “social media platform” as a public or semipublic internet-based service or application that has users in California and that meets both of the following criteria:
   1. A substantial function of the service or application is to connect users in order to allow them to interact socially with each other within the service or application. (A service or application that provides email or direct messaging services does not meet this criterion based solely on that function.)
   2. The service or application allows users to do all of the following:
      1. Construct a public or semipublic profile for purposes of signing into and using the service or application.
      2. Populate a list of other users with whom an individual shares a social connection within the system.
      3. Create or post content viewable by other users, including, but not limited to, on message boards, in chat rooms, or through a landing page or main feed that presents the user with content generated by other users. (Bus. & Prof. Code Section 22675(e).)
5. Defines “social media company” as a person or entity that owns or operates one or more social media platforms. (Bus. & Prof. Code Section 22675 (d).)
6. Defines “obscene matter” as matter, taken as a whole, that to the average person, applying contemporary statewide standards, appeals to the prurient interest, that, taken as a whole, depicts or describes sexual conduct in a patently offensive way, and that, taken as a whole, lacks serious literary, artistic, political, or scientific value. (Penal Code Section 311.) 
7. Defines the tort of deceit as including an intentional misrepresentation of fact, i.e., the suggestion, as a fact, of that which is not true, by one who does not believe it to be true. (Civil Code Section 1710 (1).)
8. Defines “commercial sexual exploitation” as an act committed for the purpose of obtaining property, money, or anything else of value, in exchange for, or as a result of, a sexual act of a minor or a nonminor dependent. The definition includes, but is not limited to, the crimes of (i) sex trafficking of a minor, (ii) pimping of a minor, (iii) pandering of a minor, (iv) procurement of a child under 16 years of age for lewd and lascivious acts, (v) solicitation of a child for an act of prostitution, and (vi) sexual exploitation of a minor. (Civil Code Section 3345.1.)

FISCAL EFFECT: As currently in print this bill is keyed non-fiscal.

COMMENTS: Child sexual abuse material (CSAM) refers to content, including pictures and videos, which capture the sexual abuse of children. CSAM is deplorable and yet tragically prevalent throughout social media. In 2021, the National Center for Missing and Exploited Children (NCMEC) estimated that over 29 million reports of suspected child sexual exploitation were reported by online platforms. Those 29 million reports included nearly 85 million individual images and videos of CSAM. (What is Child Sexual Abuse Material (CSAM), (Rape, Abuse & Incest National Network (RAINN)) (August 25, 2022) available at: https://www.rainn.org/news/what-child-sexual-abuse-material-csam.)

A spring 2022 report by Forbes detailed how the exploitation manifests on the platform TikTok in particular. The review explored “how viewers regularly use the comments to urge young girls to perform acts that appear to toe the line of child pornography – rewarding those who oblige with TikTok gifts, which can be redeemed for money, or off-platform payments to Venmo, PayPal or CashApp accounts that users list in their TikTok profiles.” (How TikTok Live Became ‘A Strip Cub Filled With 15-Year-Olds’, Alexandra S. Levine (Forbes) April 27, 2022 available at: https://www.forbes.com/sites/alexandralevine/2022/04/27/how-tiktok-live-became-a-strip-club-filled-with-15-year-olds/?sh=734b448162d7.) Further, the article describes:

The transactions are happening in a public online forum open to viewers almost anywhere on the planet. Some of the demands are explicit – like asking girls to kiss each other, spread their legs or flash the camera – and some harder to detect, masked with euphemisms. Commenters say ‘outfit check’ to get a complete look at a girl’s body; ‘pedicure check’ to see their feet; ‘there’s a spider on your wall’ to get girls to turn around and show their rears; and ‘play rock-paper-scissors to encourage girls to flirt-fight or wrestle with each other. Phrases like ‘put your arms up’ or ‘touch the ceiling’ are often directed at girls in crop tops so viewers can see their breasts and stomachs. And many simply coax girls to show their tongues and belly buttons or do handstands and splits. In return, the girls are showered with virtual gifts, like flowers, hearts, ice cream cones and lollipops, that can be converted to cash. (Ibid.)

Congress has also engaged in efforts to curb the prevalence of CSAM on social media platforms during the last three years, including this past month with the reintroduction of the Eliminating Abusive and Rampant Neglect of Interactive Technologies (EARN IT) Act. The EARN IT Act would remove immunity for companies that “knowingly facilitate or profit” from sexually explicit images of children on their platforms. (BREAKING: Congress Introduces Bill Compelling Big Tech to Address Growing Threat of Online Child Sexual Exploitation, (RAINN) April 20, 2023 available at: https://www.rainn.org/news/breaking-congress-introduces-bill-compelling-big-tech-address-growing-threat-online-child.) Previous efforts to move this legislation have ultimately failed due to concerns that the bill would negatively impact the protections intended to be provided by Section 230 of the federal Communications Decency Act. (Congress Re-Introduces Bill to Protect Kids From Online Sexual Abuse. Here’s What to Know, Solcyre Burga (TIME) April 23. 2023 available at: https://time.com/6273895/congress-re-introduces-bill-to-protect-kids-from-online-sexual-abuse/.)

This bill proposes to address the scourge of CSAM on social media platforms through two separate mechanisms. First, the bill directs social media platforms to implement a mechanism by which a user may submit a report of suspected CSAM in which they are depicted as a minor, that is displayed, stored, or hosted on the social media platform. A social media platform that receives such a report would then be subject to various deadlines. The deadlines require the platform to keep the user updated as to the progress of its investigation, and to complete any investigation, including an ultimate determination regarding whether the material is CSAM, within 30 days of the date of the report. If the material is found to be CSAM and is hosted, stored, or displayed on the platform, the social media platform is required to permanently block the material from being viewable on the platform. Importantly, the bill grants the platform an extension of 30 days (for a total of 60 days) to complete the investigation if they are unable to complete the required investigation due to circumstances beyond their control. The bill additionally grants the reporting user a right of action against the social media platform if it fails to comply with any of the above requirements, and authorizes recovery for actual damages; statutory damages of up to $250,000 per violation; costs of the action, including reasonable attorney’s fees; and any other relief the court deems proper.

The second mechanism through which the bill proposes to help stem the tide of CSAM on social media platforms is through an extension of existing law. Civil Code Section 3345.1 currently makes a person who engages in any act of commercial sexual exploitation civilly liable in a claim brought by, or on behalf of, a minor victim of commercial sexual exploitation. This bill proposes an expansion of both the existing right of action, and the potential recovery. First, the bill proposes to extend liability under this section to a social media platform that knowingly, recklessly, or negligently facilitates, aids, or abets commercial sexual exploitation. Second, the bill proposes to make a social media platform civilly liable for a violation of this provision and authorizes recovery of statutory damages, ranging from $1,000,000 to $5,000,000 for each act of commercial sexual exploitation that is facilitated, aided, or abetted by the platform. The bill defines “facilitate, aid, or abet” to mean that a social media platform deploys a system, design, feature, or affordance that is a substantial factor in causing minor users to be victims of commercial sexual exploitation.

According to the author:

We are not serious about ending the horror of sexual exploitation and trafficking of children if we do not aggressively address the prominent role gigantic social media platforms knowingly play in facilitating such exploitation and trafficking and re-traumatizing survivors for a lifetime afterwards.

Social media platforms that simply avoid being directly involved or active participants in child sexual exploitation in the first place and who respond with a minimum of compassion to the pleas of survivors to remove images and videos documenting their abuse have nothing to fear from this measure.

This bill requires social media companies to create a mechanism that users can use to report child sexual abuse material and mandates basic communication about their efforts to remove the material.

Section 230 of the Federal Communications Decency Act. The federal Communications Decency Act (CDA) provides that “[n]o provider or user of an interactive computer service shall be treated as the publisher or speaker of any information provided by another information content provider,” and affords broad protection from civil liability for the good faith content moderation decisions of interactive computer services. (47 U.S.C. Sec. 230(c)(1) and (2).) Though Section 230 was originally passed in response to judicial inconsistency with respect to the liability of internet service providers under statutes pertaining to “publishers” of content created by others, it has since been interpreted to confer operators of social media platforms and other online services with broad immunity from liability for content posted on their platforms by others.

Section 230 also indicates that “[n]othing in this section shall be construed to prevent any State from enforcing any State law that is consistent with this section,” but further provides that “[n]o cause of action may be brought and no liability may be imposed under any State or local law that is inconsistent with this section.” (47 U.S.C. Sec. 230(e)(3).) The latter provision has generally been interpreted to expressly preempt any state law that has the effect of treating a social media or other online platform as the publisher of information posted by other users, including prescriptive requirements relating to content moderation. This is consistent with the law’s original intent, which was to ensure that internet platforms facilitating the sharing of content can do so without considerable risk of liability in the event that content is not meticulously policed.

Since the development of social media platforms, the extent of Section 230’s immunity shield has been heavily litigated. Questions have arisen regarding what elements of a user’s experience and therefore of a social media platform’s business model, benefit from section 230, and which fall outside its scope. In Fair Housing Council v. Roommates.com, LLC, Roommates.com was sued by a coalition of fair housing councils in California for allegedly violating the federal Fair Housing Act and state housing discrimination laws. (Fair Housing Council of San Fernando valley v. Roommates.com, LLC, 521 F.3d 1157 (2007).) The claim was based largely on the website’s search function which appeared to prefer certain profiles over others, seemingly on the basis of elements of the user’s identity, including their gender and sexual orientation, which were collected by Roommate.com through a mandatory questionnaire developed by the website. The District Court ruled in favor of Roommates.com, holding that the website was protected by Section 230. The councils subsequently appealed to the Ninth Circuit, which held that the website’s use of the information they required users to submit in order to utilize the website rendered Roommate.com outside the protection of Section 230. In its analysis, the court reasoned that a “website operator can be both a service provider and a content provider: If it passively displays content that is created entirely by third parties, then it is only a service provider with respect to that content. But as to content that it creates itself, or is ‘responsible, in whole or in part’ for creating or developing, the website is also a content provider.” (Roommates.com, LLC, 521 F.3d at p. 1163 (2007).) With regard to Roommate.com’s own role in developing the users’ profiles, the court argued, “Roommate is ‘responsible’ at least ‘in part’ for each subscriber’s profile page, because ever such page is a collaborative effort between Roommate and the subscriber.” (Id. at p. 1167)

Just two years later in Barnes v. Yahoo! Inc., the Ninth Circuit established a three-part test for determining whether a website benefits from the liability shield of Section 230. According to Barnes, Section 230 “only protects from liability (1) a provider or user of an interactive computer service (2) whom a plaintiff seeks to treat, under a state law cause of action, as a publisher or speaker (3) of information provided by another information content provider.” (Barnes v. Yahoo! Inc., 570 F.3d 1096, 1100 - 1101 (2009).)

The opposition argues that this bill is preempted by Section 230. Specifically, they posit that:

[T]his bill creates liability for platforms based on third party content by applying to any feature that allows users to encounter content. It effectively assumes all features are harmful and imposes liability n a site for offering any of those potentially harmful features. Platforms’ algorithms and features that allow users to encounter or share content from other users are inextricably linked to the underlying content. It would also impose liability for failure to remove content, which the Ninth Circuit has held falls squarely within the preemption of Section 230.

First, it is important to establish that Section 230 was not designed to create a “lawless no-man’s-land on the internet.” (Roommates.com, LLC, 521 F.3d at p. 1164). Under current case law, it is arguably more appropriate to say media platforms that pass Barnes’s three-part test may enjoy the provision’s protections, while others that do not risk potential liability.

The two separate provisions of this bill present a distinct analysis regarding their viability under Section 230. First, the reporting requirement imposed by Section Two of the bill arguably falls outside the scope of the immunity shield provided by the CDA. Rather than imposing liability based upon the content posted by a third party, the bill would make a social media platform liable only for their failure to appropriately comply with the 30 or 60-day process laid out by the bill. Additionally, as to the opposition’s contention that imposing liability for a platform’s failure to remove content also runs afoul of Section 230, once again it is arguable that the provisions do not make a platform liable for failure to remove content, but rather for failure to adhere to the requirements outlined by the bill. A social media platform that abides by the timelines and notice requirements established by Section Two of the bill and ultimately determines reported material is not CSAM and therefore does not block the reported content, for example, would not necessarily be liable under this bill. Stated differently, a social media platform that fails to complete an investigation would risk liability, regardless of the content that is reported.

The second mechanism proposed by this bill that would expose a platform to potential liability applies when the platform knowingly, recklessly, or negligently facilitated, aided, or abetted commercial sexual exploitation of a minor. The bill further defines “facilitate, aid, or abet,” to mean “deploy a system, design, feature, or affordance that is a substantial factor in causing minor users to be victims of commercial sexual exploitation.” As with Section Two, this mechanism appears to attempt to avoid section 230 concerns by eschewing liability premised on the content of a third party. Instead, the language hinges liability on the creation of a platform’s own mechanisms.

In Lemmon v. Snap, Snap (commonly known as Snapchat) attempted to argue that the CDA immunized them from liability in a negligent design lawsuit, brought by parents whose children were killed in a car accident that the parents claim was a result of Snap’s negligent design of a new feature. (Lemmon v. Snap, Inc. (2021) 995 F.3d 1085.) In support of their position, Snap argued that Section 230 of the Communications Decency Act shielded them from liability for the publication of third party materials. (Id. at pp. 1090 – 1091.) The Ninth Circuit applied the three-pronged test set forth by Barnes v. Yahoo! Inc. (2009) 570 F.3d 1096 described above. (Id. at p. 1091) The court ultimately decided against Snap “because the Parents’ claim turns on Snap’s design of Snapchat.” (Id. at p. 1092).

While the merits of the parents’ claim in Lemmon were not ultimately adjudicated, nonetheless this case seems to indicate that basing liability on a platform’s own feature that they knew or reasonably should have known would be a substantial factor in causing harm to its users may preserve a law from being struck down under a Section 230 preemption argument.

The opposition claims that the bill’s language establishing liability for “knowingly, recklessly, or negligently” facilitating, aiding or abetting commercial sexual exploitation exposes them to extensive liability. Assuming a plaintiff is able to successfully establish that a social media platform knowingly, recklessly, or negligently facilitated, aided, or abetted their commercial sexual exploitation through the deployment of a feature that was a substantial factor in the exploitation, the potential economic liability under this provision is astronomical, to be sure. A single claim could result in statutory damages against the social media platform of up to $5,000,000, and, as previously discussed, the amount of child sexual abuse material across platforms is immense. However, there are two important elements to consider when determining the validity of this policy. First, social media platforms are often multi-billion dollar corporations. In 2021, Meta reported total revenue of over 33 billion dollars and a net income of over ten billion dollars. (Meta Reports Fourth Quarter and Full Year 2021 Results, Meta available at: https://investor.fb.com/investor-news/press-release-details/2022/Meta-Reports-Fourth-Quarter-and-Full-Year-2021-Results/default.aspx.) Second, as detailed below, it is arguably unlikely that any currently existing feature of a social media platform will trigger liability under this language.

In order to trigger liability under the bill, the platform must deploy a design, feature, or affordance that is a substantial factor in causing minor users to be victims of commercial sexual exploitation. The court in Roommates made a useful distinction between the types of tools platforms may develop and deploy. According to the court in Roommates, “a website helps to develop unlawful content, and thus falls within the exception to section 230, if it contributes materially to the alleged illegality of the conduct.” (Roommates.com, 521 F.3d 1157 at p. 1168.) Further, the court reasoned “ordinary search engines do not use unlawful criteria to limit the scope of searches conducted on them, nor are they designed to achieve illegal ends[.]” (Id. at p. 1167.) While conducted for an analysis of Section 230 immunity, the distinction is helpful in determining whether such a tool was a substantial factor in the distribution or development of CSAM, as a similar argument can be made here. While there is no ultimate definition of what constitutes a “substantial factor,” and an analysis would necessarily be fact-specific, it is likely that merely using an otherwise neutral feature on a platform that is used by billions of other users for purposes other than the sexual exploitation of a minor would not likely be found to be a “substantial factor” in the commercial sexual exploitation of a child. Therefore, it appears that this language would allow a social media platform to defend against meritless claims.

However, to the extent that potential statutory damages provided for in this bill are strikingly high, the author may wish to consider implementing a safe harbor provision to allow a social media platform to remove a feature that they become aware has been used to facilitate the distribution of CSAM.

Despite opposition arguments that this bill regulates speech and content, thereby violating First Amendment principles, the bill does not appear to violate the First Amendment. The groups in opposition specifically point to Section Two of the bill and claim the following:

[T]the bill creates a strong incentive to over-remove content any time a request is submitted. […] [Social media platforms] will have no choice but to [remove content] even if the content does not violate their policies because the risk of incurring a $250,000 statutory penalty, damages, and attorneys’ fees is too high. […] Alternatively, platforms could chose to disable features that could be misused by perpetrators to traffic children, such as direct messaging, chat forums, or other communicative features. They could also prohibit anyone under the age of 18 from using their platforms. In either instance, more lawful speech is impacted and burdened than intended, thus creating a chilling effect on providing and hosting lawful content and speech on social media platforms. Though well-intended, this bill will result in more lawful speech being removed and fewer online spaces for teens to communicate and share ideas with one another. To the extent that this bill creates a de facto ban on users under the age of 18, AB 1394 also directly interferes with expressive rights of the minors who will be banned from social media platforms.

The First Amendment of the United States Constitution provides that, “Congress shall make no law abridging the freedom of speech, or of the press.” [Emphasis added.] As applied to the states through the Fourteenth Amendment, and as interpreted by the courts, the First Amendment prohibits any law or policy, at any level of government, from abridging freedom of speech. Supreme Court jurisprudence has identified various categories of speech and determined how much each is protected by the First Amendment. These categories can first be broken down into two primary categories – protected or unprotected. Protected speech benefits from the full shield of the First Amendment and cannot be regulated by the government. Unprotected speech is speech that has been found to be “of such slight social value as a step to truth that any benefit that may be derived from them is clearly outweighed by the social interest in order and morality,” and thus may be subject to government regulation. (R.A.V. v. St. Paul (1992) 505 U.S. 377, 383.) Unprotected speech can be broken down further into various categories including: true threats; obscenity; defamation; and incitement to imminent lawless action. Obscenity is defined through the following three-pronged test referred to as the Miller test. (Miller v. California, 413 U.S. 15 (1972).)

As an initial matter in the discussion of potential First Amendment concerns arising under this bill, it is difficult to imagine that social media platforms, who rely on the sheer number of minors on their platforms for revenue generation, will limit minors’ access to their products to such an extent as to create a de facto ban on all users under 18 years of age. At this point, this concern seems addressed in full. However, even assuming the risk of a de facto ban to be true, this bill likely would still survive a First Amendment challenge.

First, the material which social media platforms are directed to block is undoubtedly obscene and therefore not protected by the First Amendment. The opponents therefore argue that it is actually the subsequent necessary implications of this bill that would result in it running afoul of First Amendment protections. In sum, that the economic risk of failing to address every instance of CSAM is so great, that social media platforms will overcorrect and in so doing result in a silencing of minors and other content on the platform. However, as posed by the opposition, it is arguably the social media platform’s overreaction to the threat of liability, and not the prohibition against certain acts proposed by this bill, that would potentially limit minors’ access to social media. Therefore, it does not appear that this bill faces constitutionality concerns with respect to the First Amendment.

The U.S. Supreme Court is Considering Section 230 and the extent to which it shields social media platforms from liability for content posted by third parties. In November 2015, the Islamic State (ISIS) took responsibility for a series of coordinated terrorist attacks in Paris. Among the at least 130 people killed was 23-year-old American Nohemi Gonzalez. Shortly after Nohemi’s death, her father filed suit against Google, Twitter, and Facebook, arguing that the platforms were liable for aiding and abetting international terrorism by failing to appropriately respond to, or address, terrorist organizations’ use of their services. (Gonzalez v. Google LLC, (Oyez) available at: https://www.oyez.org/cases/2022/21-1333.) They additionally argued that Google was not immune under Section 230 because their algorithm recommended ISIS videos to their users who ultimately conducted the acts of terrorism. The petitioners lost in both district court and on appeal to the Ninth Circuit, and subsequently appealed to the United States Supreme Court. The Court held oral argument on the matter in February of this year. A decision determining whether Section 230(c)(1) immunizes certain platforms when they make targeted recommendations of information provided by a third party provider or only limits their liability when they engage in traditional editorial functions, is expected this summer. (Gonzalez v. Google LLC, (SCOTUSblog) available at: https://www.scotusblog.com/case-files/cases/gonzalez-v-google-llc/.)

ARGUMENTS IN SUPPORT: This bill is co-sponsored by the Children’s Advocacy Institute, Common Sense Media, and the American Association of University Women. It is supported by several media watchdog groups. In support of the measure, Fairplay writes the following:

Social media companies are aware that sexual abuse, exploitation, and trafficking of children are originating on their platforms. Their reliance on inadequate self-policing is facilitating the dramatic increase in online sexual exploitation. Their inaction to protect our kids is even more important when we are seeing that Californian kids are spending more time on these companies’ platforms. In 2021, CommonSense Media reported that 38% of tweens and 84% of teens log onto social media; 18% of tweens and 62% of teens report everyday use.

Unfortunately, our kids’ social media use doesn’t end at a simple joke sent to their friends. Since 2000, traffickers have recruited 55% of sex trafficking victims online, usually through social 2 media platforms. One quarter of 9-17 year olds report having had a sexually explicit interaction online with someone they believed to be an adult. Nearly 1 in 3 LGBTQ+ minors (32%) reported an online sexual encounter with someone they believed to be over 18, 10% higher their non-LGBTQ+ peers.

It is self-evident than unless social media platforms devote far more of their resources to the moral but unprofitable task of preventing their spaces from being used by those seeking to sexually exploit children, we will never successfully reduce the sexual exploitation of children.

Social media platforms are central to the sexual exploitation of children. Unless platforms are required to devote far more time and resources to preventing and fixing the child sexual exploitation they themselves are facilitating in the first place, children will continue to be sexually exploited in ever-increasing numbers.

The Children’s Advocacy Institute at the University of San Diego School of Law, a co-sponsor of this measure, writes the following:

A 2020 report by the Canadian Centre for Child Protection revealed that social media companies design their products in a way that makes reporting and removing images and videos portraying their exploitation difficult, if not impossible, for survivors. As one survivor told the Center, “I spend hours every day searching for my own content, reporting thousands of accounts and posts sharing CSAM. When platforms don’t actively look for or prevent this content from being uploaded, the burden falls on me to have these images removed.”

It is self-evident that unless social media platforms devote far more of their resources to the morally compelled but unprofitable task of preventing their spaces from being used by those seeking to exploit children sexually, we will never successfully reduce the sexual exploitation of children. This need to establish legal minimum requirements is especially true and urgent during a time when the largest platform is laying off tens of thousands of workers to increase its profits.

To that end, AB 1394 has two parts.

Part One: Prevention. Motivating Platforms To Do Far More By Clarifying Their Accountability When They “Facilitate, Aid, Or Abet” Child Sex Trafficking Or Exploitation.

[…]

This part of the bill is a harm prevention measure that seeks to impose a downside financial consequence for a platform that is directly involved in or actively participating in the sexual exploitation of children. It also aims to offer survivors harmed by some of the wealthiest corporations in the world a pathway to obtaining the money needed to pay for their ongoing treatment and their suffering when and if those platforms were a “substantial factor” in causing their trauma.

Part Two: Remediation And Preventing A Never Ending Trauma

Where part one of the bill is about ensuring certain vast corporations have a financial reason aggressively to prevent their platforms from being used to facilitate sexual trafficking and exploitation of children, the second part of the bill is about helping survivors deal with the consequences of their exploitation when platforms fail to help them.

Thus, the second part of the bill requires platforms, when notified by a survivor, to render images or videos of children who have been sexually exploited permanently invisible, helping survivors to reduce their lifelong fear of repeated exploitation.

[…]

The bill seeks to ensure that survivors seeking to remove images and videos documenting their abuse – these are images of a crime – are treated with a bare minimum of respect, especially given the heinousness of what is portrayed.

[…]

AB 1394’s Penalties:

[…]

How big do financial disincentives need to be when dealing with companies of such vast size? When the FTC fined Facebook a record $5 billion, its stock rose the next day and the fine was described as a “joke.”

Likewise with the penalties proposed for a platform’s failure to remove horrifying images and videos upon a survivor’s request: $250,000 for each failure to heed the survivor’s plea. Given that what is being asked to be removed shows crimes being committed and should not be visible anyway under any barely adequate system, these fines are modest.

Such fines are also precedented in far less compelling circumstances. For example, Business & Professions Code section 5116.2 permits the Board of Accountancy, which also in part regulates vast corporations (the “Big Four” Accounting firms), to assess a penalty of up $1 million for a first violation and not more than $5 million for subsequent violations. Surely, motivating compliance to prevent children from being sexually exploited and prevent them, if exploited, from being re-victimized is not less worthy.

VI. Conclusion.

Unless platforms devote far more time and resources to preventing and fixing the child sexual exploitation they are facilitating in the first place, children will continue to be sexually exploited in ever-increasing numbers. Please support these children and survivors by supporting AB 1394.

ARGUMENTS IN OPPOSITION: This bill is opposed by a coalition of tech-industry advocates, including TechNet, NetChoice, the Electronic Frontier Foundation, the Civil Justice Association of California, and the Chamber of Commerce. The coalition led by TechNet writes:

TechNet member companies take multi-faceted approaches to combat CSAM and commercial sexual exploitation on their services by creating and sharing software detection tools as well as partnering with local, state, and federal law enforcement agencies and the National Center for Missing and Exploited Children (NCMEC). NCMEC takes in federally mandated reports from our companies, assesses them, and refers them to federal, state, local and international law enforcement for investigation. Through our partnerships and collaborations, TechNet members have made considerable investments and pioneered new technologies to fight CSAM.

Platforms Aggressively Combat Commercial Sexual Exploitation and CSAM

[…] Federal law, 18 USC § 2258A, requires that online providers report instances of child sexual abuse material (CSAM) to the CyberTipline at the National Center for Missing and Exploited Children (NCMEC). NCMEC takes in reports, assesses them, and refers them to federal, state, local and international law enforcement for investigation. In addition, the law also requires providers to retain important data related to their reports in the event of a law enforcement investigation. Since the early 2000s, the tech industry has been involved in efforts to use hash values to detect items previously identified as violations, identify the legal changes necessary for sharing of hash values among companies and the NCMEC, and legal changes to allow the advancement of the next generation of detection technologies. In 2009, PhotoDNA, an image matching software that can detect known CSAM, was developed and donated allowing NCMEC to license it for no cost to entities who want to use its fingerprint-analysis process to identify repeat versions of previously reported images. Additional detection tools have been developed such as Google’s CSAI Match and TechNet member companies have donated technical and financial resources to modernize the CyberTipline and to develop tools for better tracking and coordination of investigations by law enforcement

AB 1394 requires the surveillance and moderation of direct messages

TechNet members employ numerous tools to moderate, remove, and eliminate harmful content and the accounts that post it on our platforms. We do not moderate or monitor direct messages between users because those messages are intended to be private between consumers. Many of our companies encrypt or provide the option to encrypt private, direct messages to provide greater privacy and consumer trust.

Despite recent amendments excluding standalone direct messaging from the bill, AB 1394 still applies to direct messaging that is integrated into a social media platform. AB 1394 effectively prohibits the use of any feature or design, including direct messaging, that could be misused in violation of our rules to victimize children and imposes a penalty of between $1 million and $5 million for each violation. Simply offering direct messaging makes our platforms liable. In order to avoid this liability and continue offering direct messaging, companies would have no choice but to monitor and moderate content in direct and private messages. That violates consumer privacy and is impossible for sites that encrypt messages.

Privacy is a value held not just by the recipient of a message but also by the sender. A de facto requirement for companies to access a message therefore violates not only the recipient’s privacy but also the sender’s and vice versa. The government should not create a de facto requirement to monitor private, direct messages, especially those belonging to child users, and doing so directly contradicts California’s history of strengthening individuals’ privacy rights.

Finally, direct messaging is not unique to social media platforms. Many websites and applications offer users the ability to message each other. However, this bill curiously only applies to social media platforms.

[…]

Punitive structure of the bill creates an unachievable standard

If a platform offers a design, feature, or affordance that is misused to victimize a child, they are liable for between $1 million and $5 million per violation. A jury can additionally award treble damages for certain factors, making the penalties between $3 and $15 million per violation. This section dramatically expands what it means to facilitate, aid or abet by adopting a negligence standard that in effect requires social media companies to dismantle their platforms piece by piece. The bill defines facilitate, aid or abet to mean “to deploy a system, design, feature, or affordance that is a substantial factor in causing minor users to be victims of commercial sexual exploitation.” “Substantial factor” has been interpreted as a stand in for the “but for” test for causation, that is, “but for” the defendant’s conduct, the plaintiff’s harm would not have occurred. We think this is an incredibly low bar and combined with the massive penalties will encourage a flood of litigation leading to unintended consequences.

Additionally, once a platform becomes aware that a predator has misused a feature on their platform, they arguably know or should know that it can be misused again. In order to avoid liability, this bill requires that they disable it. For example, it is well documented that predators use direct messaging to contact potential victims. Additionally, any platform that suggests potential connections, friends, or even allows users to connect with others could inadvertently connect a teen user with a predator (predators are already known to their child abusers in 93% of cases). Any forum or message board where users can interact with each other could be used by a predator to connect with potential victims. All those features are central to the primary functions of a social media platform and would have to be disabled to avoid liability.

The only incentive this bill provides is to completely dismantle any designs, features, or affordances that could be misused by a trafficker. Even then it may not be enough to completely avoid liability and the only real way to prevent the misuse of these features is to cease operations or ban anyone under the age of 18 from accessing the platform.

Pending related legislation. AB 1282 (Lowenthal, 2023) would require the state Mental Health Services Oversight and Accountability Commission to report to the Legislature a statewide strategy to understand, communicate, and mitigate mental health risks associated with the use of social media by children and youth. The bill is currently referred to the Assembly Appropriations Committee.

SB 287 (Skinner, 2023) would prohibit a social media platform from using a design, algorithm, practice, affordance, or feature that the platform knows or should have known causes child users to experience specified harms, including receiving content that facilitates purchase of a controlled substance and developing an eating disorder. The bill is currently referred to the Senate Appropriations Committee.

AB 2273 (Wicks, 2022) established the California Age-Appropriate Design Code. Chap. 320, Stats. 2022

AB 2408 (Cunningham, 2022) would have prohibited a social media platform from using a design, feature, or affordance that the platform knows, or should know by the exercise of reasonable care, causes a child user to become addicted to the platform. The bill was held in the Senate Appropriations Committee.

AB 1138 (Gallagher, 2019) would have prohibited a for-profit social media website or application from allowing a person under 16 years of age to create an account without first obtaining the consent of the person’s parent or guardian. The bill was vetoed by Governor Newsom.

AB 2105 (Maienschein, 2018) authorized treble damages in a civil action brought by, or on behalf of, or for the benefit of, a person who is a minor or nonminor dependent and is a victim of commercial sexual exploitation. Chap. 166, Stats. 2018

REGISTERED SUPPORT / OPPOSITION:

Support

Children's Advocacy Institute at the University of San Diego School of Law (co-sponsor)

Common Sense Media (co-sponsor)
American Association of University Women – California (co-sponsor)

#halfthestory
3strands Global Foundation
AAUW - California
Fairplay
Junior Leagues of California State Public Affairs Committee (CALSPAC)
Lookup.live

One individual

Opposition

California Chamber of Commerce
Civil Justice Association of California
Electronic Frontier Foundation
Netchoice
TechNet

Analysis Prepared by: Manuela Boucher-de la Cadena / JUD. / (916) 319-2334

Date of Hearing: April 11, 2023

ASSEMBLY COMMITTEE ON PRIVACY AND CONSUMER PROTECTION

Jesse Gabriel, Chair

ABPCA Bill Id:AB 1394 (

Author:Wicks) – As Amended Ver:April 10, 2023

SUBJECT: Commercial sexual exploitation: civil actions

SYNOPSIS

Child sexual abuse material, commonly referred to under the acronym “CSAM,” is tragically pervasive on the internet—and not only in its illicit corners, on the so-called “dark web,” but also on popular social media websites and applications that billions of people use each day. This tragedy is compounded by the fact that certain websites and applications are not only a convenient means for sharing CSAM, but arguably induce its production.

This measure proposes to address these challenges in two ways. First, the bill would require social media platforms to provide a mechanism for users to report CSAM in which they are depicted; platforms would then generally have 30 days to verify that the material is CSAM and block it from reappearing. Second, the bill would provide victims of commercial sexual exploitation the right to sue social media platforms for deploying features that were a substantial factor in causing their exploitation.

Recent author’s amendments have significantly clarified the bill’s functioning and are responsive to many of the concerns that have been raised by the bill’s opponents.

The bill is co-sponsored by the American Association of University Women, Children’s Advocacy Institute at the University of San Diego School of Law, and Common Sense Media; it is supported by six other nonprofit organizations. The bill is opposed by Electronic Frontier Foundation and an opposition coalition consisting of the California Chamber of Commerce, the Civil Justice Association of California, NetChoice, and Technet.

If passed by this Committee, this bill will next be heard by the Assembly Judiciary Committee.

SUMMARY: Requires social media platforms to provide a mechanism for users to report child sexual abuse material in which they are depicted; provides platforms 30-60 days after receiving a report to verify the content of the material and block it from reappearing. Also provides victims of commercial sexual exploitation the right to sue social media platforms for having deployed features that were a substantial factor in causing their exploitation. Specifically, this bill:

1. Defines the following terms:
   1. “Child sexual abuse material” means either of the following:
      1. Child pornography, as defined under federal law.
      2. Obscene matter, as defined under state law, that depicts a minor personally engaging in, or personally simulating, sexual conduct.
   2. “Social media platform” and “social media company” have the same meanings as under the Business and Professions Code, except that, for purposes of this bill, the term “social media platform” is deemed to exclude standalone direct messaging services that provide end-to-end encrypted communication.
   3. “Reporting user” means a natural person who reports material to a social media platform using the means specified in 2) below.
2. Requires a social media platform to provide, in a form that is reasonably accessible, a means for a user who is a California resident to report material to the platform that the user reasonably believes meets all of the following criteria:
   1. It is child sexual abuse material.
   2. The user themselves is an identifiable minor depicted in the reported material.
   3. The reported material is displayed, stored, or hosted on the social media platform.
3. Requires the social media platform to block material reported under 2) if the reported material meets both of the following criteria:
   1. The reported material is child sexual abuse material.
   2. The reported material is displayed, stored, or hosted on the social media platform.
4. Requires a social media platform to collect from the reporting user information reasonably sufficient to enable the platform to contact the user in writing. The contact method, such as a mailing address, a telephone number (to send text messages), or an email address, must be chosen by the reporting user, and not be within the control of the social media company that owns or operates the social media platform.
5. Requires the social media platform to meet the following schedule of written communications with the reporting user, using the contact method collected under 4):
   1. Within 24 hours, the reporting user must be notified that the platform received their report.
   2. Every seven days thereafter, the reporting user must be provided an update as to the status of the social media platform’s handling of the reported material, until a final written determination is issued under c).
   3. No later than 30 days after the material was first reported, the platform must issue a final written determination stating one of the following:
      1. The reported material has been determined to be child sexual abuse material that was displayed, stored, or hosted on the social media platform and has been blocked on the social media platform.
      2. The reported material has been determined not to be child sexual abuse material.
      3. The reported material has been determined not to be displayed, stored, or hosted on the social media platform.
6. Provides that if the social media platform cannot comply with the requirements in the timeframe set forth above, due to circumstances beyond its reasonable control, then:
   1. The platform must comply with these requirements no later than 60 days after the date on which the material was first reported.
   2. The platform must provide written notice of the delay to the reporting user no later than 48 hours from the time the social media platform knew the delay was likely to occur, using the contact method collected under 4).
7. Holds a social media company liable to a reporting user if it fails to comply with the requirements in paragraphs 2) – 6). Available remedies include:
   1. Any actual damages sustained by the reporting user as a result of the violation.
   2. Statutory damages of no more than two hundred fifty thousand dollars ($250,000) per violation. In determining the amount of statutory damages, the court is to consider the willfulness and severity of the violation, as well as previous violations by the social media company.
   3. Costs of the action, together with reasonable attorney’s fees, as determined by the court.
   4. Any other relief that the court deems proper.
8. Establishes a rebuttable presumption that a social media platform that fails to comply with the requirements in paragraphs 2) – 6) within 60 days of the date on which child sexual abuse material was first reported is liable to the reporting user for statutory damages.
9. Declares that nothing in this bill limits or impairs in any way a cause of action for deceit involving an intentional misrepresentation of fact.
10. Makes uncodified findings and declarations in support of the foregoing provisions.
11. Holds social media platforms liable if they knowingly, recklessly, or negligently facilitate, aid, or abet commercial sexual exploitation of minors.
12. Defines “facilitate, aid, or abet” as the deployment of a system, design, feature, or affordance that is a substantial factor in causing minor users to be victims of commercial sexual exploitation.
13. Provides that a court is to award statutory damages of between $1,000,000 and $5,000,000 for each act of commercial sexual exploitation facilitated, aided, or abetted by the social media platform.
14. Provides that the requirements of this bill cannot be waived.
15. Includes a severability clause that governs the provisions of this bill.
16. Makes technical and clarifying amendments to existing statutes.

EXISTING FEDERAL LAW:

1. Establishes, under Section 230 of the Communications Decency Act, that no provider or user of an interactive computer service shall be treated as the publisher or speaker of any information provided by another information content provider. (47 U.S.C. § 230(c)(1).)
2. Exempts from Section 230 protection violations of federal criminal law; intellectual property law; state law that is consistent with Section 230; communications privacy law; and sex trafficking law. (47 U.S.C. § 230(e).)
3. Defines “child pornography” as any visual depiction of sexually explicit conduct, where any of the following is true:
   1. The production of such visual depiction involves the use of a minor engaging in sexually explicit conduct.
   2. The visual depiction is a digital image, computer image, or computer-generated image that is, or is indistinguishable from, that of a minor engaging in sexually explicit conduct.
   3. The visual depiction has been created, adapted, or modified to appear that an identifiable minor is engaging in sexually explicit conduct. (18 U.S.C. § 2256.)

EXISTING STATE LAW:

1. Defines “social media platform” as a public or semipublic internet-based service or application that has users in California and that meets both of the following criteria:
   1. A substantial function of the service or application is to connect users in order to allow them to interact socially with each other within the service or application. (A service or application that provides email or direct messaging services does not meet this criterion based solely on that function.)
   2. The service or application allows users to do all of the following:
      1. Construct a public or semipublic profile for purposes of signing into and using the service or application.
      2. Populate a list of other users with whom an individual shares a social connection within the system.
      3. Create or post content viewable by other users, including, but not limited to, on message boards, in chat rooms, or through a landing page or main feed that presents the user with content generated by other users. (Bus. & Prof. Code § 22675(e).)
2. Defines “social media company” as a person or entity that owns or operates one or more social media platforms. (Bus. & Prof. Code § 22675(d).)
3. Defines “obscene matter” as matter, taken as a whole, that to the average person, applying contemporary statewide standards, appeals to the prurient interest, that, taken as a whole, depicts or describes sexual conduct in a patently offensive way, and that, taken as a whole, lacks serious literary, artistic, political, or scientific value. (Pen. Code § 311.) 
4. Defines the tort of deceit as including an intentional misrepresentation of fact, i.e., the suggestion, as a fact, of that which is not true, by one who does not believe it to be true. (Civ. Code § 1710(1).)
5. Defines “commercial sexual exploitation” as an act committed for the purpose of obtaining property, money, or anything else of value, in exchange for, or as a result of, a sexual act of a minor or a nonminor dependent. The definition includes, but is not limited to, the crimes of (i) sex trafficking of a minor, (ii) pimping of a minor, (iii) pandering of a minor, (iv) procurement of a child under 16 years of age for lewd and lascivious acts, (v) solicitation of a child for an act of prostitution, and (vi) sexual exploitation of a minor. (Civ. Code § 3345.1.)

FISCAL EFFECT: As currently in print this bill is keyed non-fiscal.

COMMENTS:

1) Background. Child sexual abuse material, commonly referred to under the acronym “CSAM,” is tragically pervasive on the internet—and not only in its illicit corners, on the so-called “dark web,” but also on popular social media websites and applications that billions of people use every day. In 2020, household names such as Discord, Reddit, TikTok, Twitch, and Twitter each self-reported thousands of instances of CSAM on their systems to the National Center for Missing & Exploited Children; Facebook reported millions. (NCMEC, 2020 CyberTipline Reports by Electronic Service Providers (2021), available at https://www.missingkids.org/content/dam/missingkids/pdfs/2020-reports-by-esp.pdf.)

This tragedy is compounded by the fact that certain websites and applications are not only a convenient means for sharing CSAM, but arguably induce its production:

A Forbes review of hundreds of recent TikTok livestreams reveals how viewers regularly use the comments to urge young girls to perform acts that appear to toe the line of child pornography—rewarding those who oblige with TikTok gifts, which can be redeemed for money, or off-platform payments to Venmo, PayPal or Cash App accounts that users list in their TikTok profiles. (Levine, How TikTok Live Became ‘A Strip Club Filled With 15-Year-Olds,’ Forbes (Apr. 27, 2022)

This measure proposes to address these challenges in two ways. First, the bill would require social media platforms to provide a mechanism for users to report CSAM in which they are depicted; platforms would then generally have 30 days to verify that the material is CSAM and block it from reappearing. Second, the bill would provide victims of commercial sexual exploitation the right to sue social media platforms for deploying features that were a substantial factor in causing their exploitation.

Recent author’s amendments have significantly clarified the bill’s functioning and are responsive to many of the concerns that have been raised by the bill’s opponents.

2) Author’s statement. According to the author:

We are not serious about ending the horror of sexual exploitation and trafficking of children if we do not aggressively address the prominent role gigantic social media platforms knowingly play in facilitating such exploitation and trafficking and re-traumatizing survivors for a lifetime afterwards.

Social media platforms that simply avoid being directly involved or active participants in child sexual exploitation in the first place and who respond with a minimum of compassion to the pleas of survivors to remove images and videos documenting their abuse have nothing to fear from this measure.

3) This bill is meant to ensure that social media platforms promptly block child sexual abuse material reported by victims. This bill establishes a comprehensive process for users to report material that they reasonably believe to be (i) CSAM, (ii) in which the users themselves are depicted as identifiable minors, and (iii) displayed, stored, or hosted on a social media platform. Following a report, a platform would have 30 days (with a potential extension to 60 days, if required by factors beyond the platform’s control) to verify whether the material is CSAM, and if it is, block it from appearing on the platform. The steps in this process are set forth in paragraphs 2) - 6) of the SUMMARY above. Some important points to note about this process are as follows.

• Social media platforms can use a third-party service to collect CSAM reports, and thereby avoid incurring liability that might arise from coming into possession of CSAM, particularly CSAM that is not already present on their systems.

Federal law already requires online electronic service providers in the United States to report to the CyberTipline operated by the National Center for Missing & Exploited Children (NCMEC), a federally-chartered nonprofit, if they become aware of apparent CSAM that is within their control. (18 U.S.C. § 2258A.)

This bill does not specify that social media platforms must rely on NCMEC; they are free, to the extent permissible under federal and state law, to use another method of reporting. And if social media companies protest that NCMEC lacks the resources to handle an anticipated influx of reports under this bill, they can fund NCMEC directly rather than forcing the organization to rely on federal funding, currently at $40 million per year.

The key difference between this bill and the federal law cited above is that, under this bill, social media platforms would be required to block reported material from re-appearing once it is verified to be CSAM. There is currently no such requirement under federal law.

• Social media platforms would only be required to block the specific CSAM reported by users.

Opponents of this bill protested that, under the original version of this bill, “To avoid liability, companies must be perfect and catch every attempted upload of the material, even altered versions or reproductions, which is impossible to achieve.” As amended, the bill would only require that social media platforms block the CSAM that a user reports, not alterations or variations. NCMEC’s Take It Down tool, funded by Meta (Facebook’s parent), “works by assigning a unique digital fingerprint, called a hash value, to nude, partially nude, or sexually explicit images or videos of people under the age of 18. Online platforms can use hash values to detect these images or videos on their services and remove this content.” (NCMEC, Take It Down, available at https://takeitdown.ncmec.org/.) Once a hash is generated, social media platforms can use it to not only remove existing copies of the CSAM, but also rapidly compare image and video files that users attempt to upload for a match, analogous to the process that they use to scan incoming files for computer viruses.

• Social media platforms must acknowledge user reports and keep reporting users updated on the progress of their requests.

A complaint among some users who report CSAM to social media platforms is that they have no idea whether the platform has acted, or will act, to block the offending material. In such cases, survivors may “feel they have been left with no option but to take it upon themselves to monitor the presence of their own child sexual abuse images while attempting to get companies to remove it from their platforms.” (Canadian Centre for Child Protection, Reviewing Child Sexual Abuse Material Reporting Functions on Popular Platforms (2020) p. 7, available at https://protectchildren.ca/pdfs/C3P_ReviewingCSAMMaterialReporting_en.pdf.)

In response, this bill requires platforms to provide written updates to reporting users (i) when they submit reports, (ii) every seven days thereafter, and (iii) upon resolution of their report. Any delays beyond the statutorily-mandated 30 days must also be promptly reported to users. Finally, users must be able to choose the method by which they will receive these written updates.

• Social media platforms would retain the ability to pursue users who make fraudulent reports.

One criticism that bill opponents made of the original bill is that “if a bad actor wants to silence another user, they will report their account or content as CSAM or exploitation material.” This is a well-known phenomenon. A recent ProPublica story described scammers who attempt to get influencers banned on social media platforms, on behalf of rival influencers, for a fee. (Silverman and Fortis, A Scammer Who Tricks Instagram Into Banning Influencers Has Never Been Identified. We May Have Found Him, ProPublica (Mar. 26, 2023), available at https://www.propublica.org/article/instagram-fraudster-ban-influencer-accounts.)

In response, this bill has been amended to make clear that its provisions “shall not be construed to limit or impair in any way a cause of action under [Civil Code section 1710 (1)],” i.e., for an intentional misrepresentation of fact. If a fraudster uses the mechanism specified by this bill to report an influencer or creator’s non-CSAM image or video as CSAM in an attempt to get a social media platform to block it, nothing in this bill would protect that fraudster from civil liability for fraud damages. Both the person who created the content and the platform itself would have standing to sue for fraud in such a case.

Further, the requirement that social media platforms collect users’ contact information in order to provide them with written updates on the status of their reports could assist platforms, influencers, and creators in pursuing users who falsely report that images or videos are CSAM in an attempt to have this content blocked.

• Standalone services that provide end-to-end encryption for direct messages are exempted from the bill’s requirements.

Bill opponents protest that they “do not moderate or monitor direct messages between users because those messages are intended to be private between consumers.” As amended, this bill would not apply to standalone applications that provide 100% end-to-end encrypted direct messaging, as such services play an important role in ensuring the privacy of sensitive communications, such as political organizing and obtaining information about reproductive healthcare. But the bill would expect platforms to scan content shared through unencrypted messaging services, or through encrypted services integrated with other social media features, to block reported CSAM—just as these platforms scan uploaded material for computer viruses.

In conclusion, given the factors discussed above, it appears that the bill, in its present form, provides a targeted method for victims to report material that they reasonably believe to be CSAM in which they are depicted, and for social media companies to block this material from reappearing on their platforms, if it is verified to be CSAM. Both social media companies and injured third parties would retain the ability to sue users who make fraudulent reports of CSAM.

4) What this bill would do & analysis: holding platforms liable if they cause commercial sexual exploitation. Existing California law holds adults civilly liable for the tort of commercial sexual exploitation: sexually exploiting minors in exchange for money, property, or something else of value. This bill proposes to extend such liability to social media platforms that knowingly, recklessly, or negligently facilitate, aid, or abet commercial sexual exploitation.

As originally introduced, this provision drew significant opposition due to its definition of “facilitate, aid, or abet” as “includ[ing], but…not limited to, being directly involved in, or actively participating in, the development, deployment, or enforcement of a system, design, feature, or affordance that the social media platform knows, or should know, foreseeably causes child users to be victims of commercial sexual exploitation.”

Opponents protested:

• “This provision is vague, doesn’t give platforms clear direction, runs afoul of established [F]irst [A]mendment principles and is preempted by federal law.”
• “Any forum or message board where users can interact with each other could be used by a predator to connect with potential victims. All those features are central to the primary functions of a social media platform and would have to be disabled to avoid liability.”
• “This bill creates liability for platforms based on third party content by applying to any feature that allows users to encounter content. It effectively assumes all features are harmful and imposes liability on a site for offering any of those potentially harmful features. Platforms’ algorithms and features that allow users to encounter or share content from other users are inextricably linked to the underlying content.”

In response, this bill has been amended to meaningfully clarify the standard by which social media platforms could be held liable for commercial sexual exploitation. “Facilitate, aid, or abet” is now defined as “deploy[ing] a system, design, feature, or affordance that is a substantial factor in causing minor users to be victims of commercial sexual exploitation.” The “substantial factor” test is generally understood to be the appropriate causation standard under California tort law. (Rutherford v. Owens-Illinois, Inc. (1997) 16 Cal. 4th 953, 968-9.)

It is critical to note that the bill focuses on the system, design, feature, or affordance at issue, not particular content accessible through the site. This wording is meant to insulate the provision in question from being struck down under Section 230 of the federal Communications Decency Act, 47 U.S.C. § 230(c). “The touchstone of section 230(c) is that providers of interactive computer services are immune from liability for content created by third parties.” (Fair Housing Council of San Fernando Valley v. Roommates.com, LLC (9th Cir. 2007) 489 F.3d 921, 925, aff’d en banc (9th Cir. 2008) 521 F.3d 1157.) There are limits to Section 230’s preemptive effects, however; the law is “not meant to create a lawless no-man’s-land on the Internet.” (Ibid., 521 F.3d at 1164.)

Lemmon v. Snap (9th Cir. 2021) 995 F.3d 1085, 1094 provides that even under Section 230, interactive computer services “face the prospect of liability, even for their ‘neutral tools,’ so long as plaintiffs' claims do not blame them for the content that third parties generate with those tools.” In Lemmon, the Ninth Circuit held that Section 230 did not immunize social media provider Snap, Inc. for a personal injury suit brought by the parents of teen-agers who died in a high-speed traffic accident while driving 113 miles per hour. As described by the court:

Shortly before the crash, Landen [one of the teens] opened Snapchat, a smartphone application, to document how fast the boys were going. To keep its users engaged, Snapchat rewards them with “trophies, streaks, and social recognitions” based on the snaps they send. Snapchat, however, does not tell its users how to earn these various achievements. The app also permits its users to superimpose a "filter" over the photos or videos that they capture through Snapchat at the moment they take that photo or video. Landen used one of these filters—the “Speed Filter”—minutes before the fatal accident on May 28, 2017. […]

Many of Snapchat's users suspect, if not actually “believe,” that Snapchat will reward them for “recording a 100-MPH or faster [s]nap" using the Speed Filter. […]

Snapchat allegedly knew or should have known, before May 28, 2017, that its users believed that such a reward system existed and that the Speed Filter was therefore incentivizing young drivers to drive at dangerous speeds. Indeed, the Parents allege that there had been: a series of news articles about this phenomenon; an online petition that “called on Snapchat to address its role in encouraging dangerous speeding”; at least three accidents linked to Snapchat users’ pursuit of high-speed snaps; and at least one other lawsuit against Snap based on these practices. While Snapchat warned its users against using the Speed Filter while driving, these warnings allegedly proved ineffective. And, despite all this, “Snap did not remove or restrict access to Snapchat while traveling at dangerous speeds or otherwise properly address the danger it created.” (Ibid. at 1088-90.)

The Ninth Circuit concluded that “because the Parents’ claim neither treats Snap as a ‘publisher or speaker’ nor relies on ‘information provided by another information content provider,’ Snap does not enjoy immunity from this suit under § 230(c)(1).” (Ibid. at 1087.)

The liability standard under this bill would not be easy to meet. Because of Section 230, the ability to share digital images or videos on a social media platform, in and of itself, is likely to be insufficient to trigger liability—even if the images or videos shared were produced as a result of commercial sexual exploitation. Moreover, the Ninth Circuit did not rule on whether the Lemmon plaintiffs had adequately pled (much less proven) that Snapchat had legally caused the accident in question by releasing the Speed Filter. But, as now worded, it appears that the bill would allow social media platforms to be held liable for deploying features that cause commercial sexual exploitation without running afoul of Section 230. To the extent this dissuades platforms from deploying features that induce commercial sexual exploitation, in much the way that the Snapchat Speed Filter allegedly induced dangerous speeding, this appears a beneficial measure.

5) Related legislation. AB 1282 (Lowenthal, 2023) would require the state Mental Health Services Oversight and Accountability Commission to report to the Legislature a statewide strategy to understand, communicate, and mitigate mental health risks associated with the use of social media by children and youth. The bill is currently referred to the Assembly Health Committee.

SB 287 (Skinner, 2023) would prohibit a social media platform from using a design, algorithm, practice, affordance, or feature that the platform knows or should have known causes child users to experience specified harms, including receiving content that facilitates purchase of a controlled substance and developing an eating disorder. The bill is currently referred to the Senate Judiciary Committee.

AB 2273 (Wicks, Chap. 320, Stats. 2022) established the California Age-Appropriate Design Code.

AB 2408 (Cunningham, 2022) would have prohibited a social media platform from using a design, feature, or affordance that the platform knows, or should know by the exercise of reasonable care, causes a child user to become addicted to the platform. The bill was held in the Senate Appropriations Committee.

AB 1138 (Gallagher, 2019) would have prohibited a for-profit social media website or application from allowing a person under 16 years of age to create an account without first obtaining the consent of the person’s parent or guardian. The bill was vetoed by Governor Newsom.

AB 2105 (Maienschein, Chap. 166, Stats. 2018) authorized treble damages in a civil action brought by, or on behalf of, or for the benefit of, a person who is a minor or nonminor dependent and is a victim of commercial sexual exploitation.

ARGUMENTS IN SUPPORT: LookUp.Live lauds the agency this bill would provide survivors:

We…appreciate that this bill also empowers survivors haunted by the prospect of images and videos about them lingering on platforms to require this content be rendered invisible, with the ability to recover penalties against platforms that deny their requests.

Bill sponsor Children’s Advocacy Institute at the University of San Diego School of Law justifies the substantial statutory damages under this bill, as follows:

[The bill] aims to offer survivors a pathway to obtaining the money needed to pay for their ongoing treatment and their suffering. It is also a harm prevention measure that seeks to impose a downside financial consequence for a platform that is directly involved in or actively participating in the sexual exploitation of children

ARGUMENTS IN OPPOSITION: An opposition coalition, consisting of the California Chamber of Commerce, the Civil Justice Association of California, NetChoice, and Technet, warns that this bill could inadvertently harm minors:

[P]latforms could choose to disable features that could be misused by perpetrators to traffic children, such as direct messaging, chat forums, or other communicative features. They could also prohibit anyone under the age of 18 from using their platforms. In either instance, more lawful speech is impacted and burdened than intended, thus creating a chilling effect on providing and hosting lawful content and speech on social media platforms. Though well-intentioned, this bill will result in more lawful speech being removed and fewer online spaces for teens to communicate and share ideas with one another.

Electronic Frontier Foundation raises First Amendment concerns:

The design features, algorithms, and other tools social media companies use to disseminate, promote, or otherwise order content on their platforms are protected speech. Just as a newspaper must have the right to use their editorial judgment over the layout of articles in its publication, so must social media companies be able to control how information spreads across their platforms.

REGISTERED SUPPORT / OPPOSITION:

Support

American Association of University Women – California (co-sponsor)

Children’s Advocacy Institute (co-sponsor)

Common Sense Media (co-sponsor)

#HalfTheStory

3Strands Global Foundation

Fairplay

Junior Leagues of California State Public Affairs Committee

Lookup.live

Safe Social Media

Opposition

California Chamber of Commerce

Civil Justice Association of California

Electronic Frontier Foundation

NetChoice

Technet

Analysis Prepared by: Jith Meganathan / P. & C.P. / (916) 319-2200