Existing law regulates various businesses to, among other things, preserve and regulate competition, prohibit unfair trade practices, and regulate advertising.

This bill would, with certain exceptions, make it unlawful for any person to use a bot to communicate or interact with another person in California online with the intent to mislead the other person about its artificial identity for the purpose of knowingly deceiving the person about the content of the communication in order to incentivize a purchase or sale of goods or services in a commercial transaction or to influence a vote in an election. The bill would define various terms for these purposes. The bill would make these provisions operative on July 1, 2019.

Existing law regulates various businesses to, among other things, preserve and regulate competition, prohibit unfair trade practices, and regulate advertising.

This bill would, with certain exceptions, make it unlawful for any person to use a bot to communicate or interact with another person in California online with the intent to mislead the other person about its artificial identity for the purpose of knowingly deceiving the person about the content of the communication in order to incentivize a purchase or sale of goods or services in a commercial transaction or to influence a vote in an election. The bill would define various terms for these purposes. The bill would make these provisions operative on July 1, 2019.

Existing law regulates various businesses to, among other things, preserve and regulate competition, prohibit unfair trade practices, and regulate advertising.

This bill would, with certain exceptions, make it unlawful for any person to use a bot, as defined, bot to communicate or interact with another person in California online with intent to mislead the other person about its artificial identity for the purpose of knowingly deceiving the person about the content of the communication in order to incentivize a purchase or sale of goods or services in a commercial transaction or to influence a vote in an election. The bill would define various terms for these purposes. The bill would make these provisions operative on July 1, 2019.

Existing law regulates various businesses to, among other things, preserve and regulate competition, prohibit unfair trade practices, and regulate advertising.

This bill would would, with certain exceptions, make it unlawful for any person to use a bot, as defined, to communicate or interact with natural persons another person in California online with the intention of misleading and would provide that a person using a bot is presumed to not act with the intent to mislead if the person discloses that the bot is not a natural person. The bill would require an online platform to enable users to report violations of this prohibition, and would require the online platform to respond to the reports and, upon request, provide the Attorney General with specified related information. intent to mislead the other person about its artificial identity for the purpose of knowingly deceiving the person about the content of the communication in order to incentivize a purchase or sale of goods or services in a commercial transaction or to influence a vote in an election.

Existing law regulates various businesses to, among other things, preserve and regulate competition, prohibit unfair trade practices, and regulate advertising.

This bill would make it unlawful for any person to use a bot, as defined, to communicate or interact with natural persons in California online with the intention of misleading and would provide that a person using a bot is presumed to not act with the intent to mislead unless if the person discloses that the bot is not a natural person. The bill would require an online platform to enable users to report violations of this prohibition, and would require the online platform to respond to the reports, and to reports and, upon request, provide the Attorney General with specified related information.

Existing law regulates various businesses to, among other things, preserve and regulate competition, prohibit unfair trade practices, and regulate advertising.

This bill would make it unlawful for any person to use a bot, as defined, to communicate or interact with natural persons in California online, online with the intention of misleading and without clearly and conspicuously disclosing would provide that a person using a bot is presumed to act with the intent to mislead unless the person discloses that the bot is not a natural person. The bill would require an online platform to enable users to report violations of this prohibition, to respond to the reports, and to provide the Attorney General with specified related information.

The Alcoholic Beverage Control Act authorizes a retailer to return wine to a seller and the seller may accept the return of wine, except that the seller may not sell wine to the retailer for one year, unless the wine was returned under specified circumstances.

This bill would make nonsubstantive changes to this provision.

 UNFINISHED BUSINESS  

Bill No: SB 1001

Author:

:

Vote:

SENATE BUS., PROF. & ECON. DEV. COMMITTEE: 8-1, 4/16/18

AYES: Hill, Dodd, Galgiani, Glazer, Hernandez, Newman, Pan, Wilk

NOES: Fuller

SENATE JUDICIARY COMMITTEE: 5-1, 4/24/18

AYES: Jackson, Hertzberg, Monning, Stern, Wieckowski

NOES: Anderson

NO VOTE RECORDED: Moorlach

SENATE APPROPRIATIONS COMMITTEE: Senate Rule 28.8

SENATE FLOOR: 26-10, 5/30/18

AYES: Allen, Atkins, Beall, Bradford, Dodd, Galgiani, Glazer, Hernandez, Hertzberg, Hill, Hueso, Jackson, Lara, Leyva, McGuire, Mitchell, Monning, Newman, Pan, Portantino, Roth, Skinner, Stern, Wieckowski, Wiener, Wilk

NOES: Anderson, Bates, Berryhill, Fuller, Moorlach, Morrell, Nguyen, Nielsen, Stone, Vidak

NO VOTE RECORDED: Cannella, De León, Gaines

ASSEMBLY FLOOR: 64-14, 8/30/18 - See last page for vote

SUBJECT:

SOURCE: Author

DIGEST:

This bill, beginning July 1, 2019, prohibits a person from using of a bot to communicate or interact with another person in California online with intent to mislead the other person about its artificial identity for the purpose of knowingly deceiving the person about the content of the communication in order to incentivize a purchase or sale of goods or services in a commercial transaction or to influence a vote in an election.

Assembly Amendments add coauthors, update definitions, make clarifying changes, and remove requirements for online platforms to take actions including: (1) enabling users to report bots violating those provisions; (2) investigating and determining within 72 hours whether to act upon reports received by users; and (3) providing details of user reports and internal investigations to the Attorney General, upon request.

ANALYSIS:

Existing law:

1. Prohibits any person, firm, corporation or association, or any employee thereof, from making statements that are untrue or misleading in connection with communications regarding the sale of goods or services, including over the Internet. (Business and Professions Code (BPC) Section (§) 17500)
2. Prohibits a person, with a bad faith intent, to register, traffic in, or use a domain name that is identical or confusingly similar to the name of another living or deceased person. (BPC § 17525)
3. Prohibits any person or entity from sending a commercial e-mail advertisement from California or to a California electronic mail address if the e-mail advertisement contains falsified, misrepresented, or forged header information or if the e-mail advertisement has a misleading subject line. Permits the Attorney General, among other parties, to bring action against a person or entity that violates these provisions. (BPC §17529.5)
4. Prohibits a person or entity located within California from using a telephone facsimile machine, computer, or other device to send an unsolicited advertisement to a telephone facsimile machine. (BPC § 17538.43 (b)(1))
5. Requires a person or entity located in California to clearly mark the date and time of transmittal and identity and telephone number of the person or entity sending the message of any communication via a telephone facsimile machine.
   (BPC § 17538.43 (c)(1))
6. Requires individuals or partnerships doing business under fictitious names to file a fictitious business name certificate designed to make the identities of the persons doing business under the fictitious names available to the public. (BPC § 17900)

This bill:

1. Defines “bot” as an automated online account where all or substantially all of the actions or posts of that account are not the result of a person.
2. Defines “online” as appearing on any public-facing Internet Web site, Web application, or digital application, including a social network or publication.
3. Defines “online platform” as a public-facing Internet Web site, Web application, or digital application, including a social network or publication that has 10,000,000 or more unique monthly United States visitors or users for a majority of months during the preceding 12 months
4. Defines “person” as a natural person, corporation, limited liability company, partnership, joint venture, association, estate, trust, government, governmental subdivision or agency or other legal entity of any combination thereof.
5. Prohibits a person from using a bot to communicate or interact with another person in California online with the intention of misleading the other person about its artificial identity for the purpose of knowingly deceiving the person about the content of the communication in order to incentivize a purchase or sale of goods or services in a commercial transaction or to influence a vote in an election. Specifies that a person using a bot shall not be liable if the person discloses that it is a bot. Requires the disclosure to be clear, conspicuous, and reasonably designed to inform persons with whom the bot communicates or interacts that it is a bot.
6. Specifies that the duties and obligations imposed by this bill are cumulative with any other duties or obligations imposed by any other law. Specifies that the provisions of this bill are severable and do not impose a duty on service providers of online platforms, including Web hosting and Internet service providers.
7. Makes these provisions operative on July 1, 2019.

Background

Bots. Bots are software applications that run automated tasks on a network, such as the Internet, that can interact with computers systems or users. Bots can be purchased or created by individuals or organizations and many online platforms, including social media sites, create bots themselves and encourage users to build bots for use on their platforms. When interacting directly or indirectly with bots online, human users may not be able to discern if they are interacting with a bot or another human. For example, a human Twitter user may assume a Tweet was posted by another human when in actuality the content was generated by a bot. The latter type of interaction has become increasingly common as the number of automated users on social media sites continues to grow. A 2017 study by the University of Southern California and Indiana University found that an estimated 48 million users, or 15 percent of Twitter’s active users at the time, were automated accounts. In November 2017, Facebook disclosed to investors that up to 60 million bots may be on the site.

Utilization of bots online. Bots, particularly those that are automated accounts on social media sites, can be used to amplify the perceived popularity of people, products, and services online. In January 2018, the New York Times published an article, “The Follower Factory,” detailing an emerging market where those seeking to bolster their online presence can purchase followers, likes, and other engagements for their social media accounts from companies that control a large number of bots. One such company highlighted by The Times, Devumi, has reportedly sold Twitter followers to over 200,000 customers, including reality television starts, professional athletes, and politicians. To do so, Devumi draws upon its approximately 3.5 million bot accounts, at least 55,000 of which use the names, profile pictures, and other personal details of real Twitter users, representing “a kind of large-scale social identity theft.”

Just as companies like Devumi use bots to promote their customer’s accounts, individuals and groups can, and have, used bots to spread and amplify divisive views on social media platforms. For example, in February 2018, the U.S. Justice Department charged 13 Russians and three companies with creating fake profiles on social media sites to post contentious comments related to religion, race, and politics and then using bot accounts to like, share, and retweet the posts to help them gain traction during and after the 2016 Presidential election. Information provided to U.S. Congress on October 20, 2017 indicated that Russian agents published more than 131,000 messages on Twitter, uploaded over 1,000 videos to YouTube, and disseminated inflammatory posts that reached 126 million users on Facebook.

There have recently been efforts by other states and the federal government to regulate the use of bots and prevent the spread of misinformation and false advertising by automated accounts. However, there is currently no state that requires the disclosure of bots.

Comments

According to the author, “bots can be – and are often – weaponized to spread fake and misleading news, reshape political debates, and influence advertising audiences…SB 1001 sheds light on the fake accounts that simulate real people and spread waves of false information across social media platforms. On the Internet where the appearance of a mass audience can be monetized, it is critical to protect users by providing the tools to understand if their information is coming from a human or a bot account disguised as one. As long as bots are properly identified to let users know that they are a computer generated or automated account, users can at least be aware of who they are interacting with and judge the content accordingly.”

FISCAL EFFECT: Appropriation: Fiscal Com.: Local:

According to the Assembly Appropriations Committee, this bill will result in costs in the tens of thousands of dollars to the Department of Justice to investigate and prosecute violations.

SUPPORT: (Verified 8/28/18)

Common Sense Kids Action

OPPOSITION: (Verified 8/28/18)

California Chamber of Commerce

California Grocers Association

CompTIA

CTIA

Data and Marketing Association

Electronic Frontier Foundation

Internet Association

Internet Coalition

TechNet

ARGUMENTS IN SUPPORT: Common Sense Kids Action writes in support, noting that teens and children are “at risk of being targeted with unlabeled bots on [social media] platforms” which “are more likely to carry harmful messages: political propaganda, ethnic discrimination, or just false information.” They argue that “with so many children and teens online; families need confidence that they will be protected from unlabeled accounts that are spreading false information and harmful content. SB 1001 allows users to report these unlabeled bots and requires entities behind bots to clearly identify automated accounts.”

ARGUMENTS IN OPPOSITION: The California Chamber of Commerce, California Grocers Association, CompTIA, CTIA, Data and Marketing Association, Internet Association, Internet Coalition, and TechNet write in opposition, stating that while SB 1001 attempts to prevent specific problematic behaviors behind the use of certain types of bots, “aspects of the bill are vague, and the bill places onerous burdens on the operators of websites and applications.” Opponents argue that the requirements of this bill “are burdensome and could diminish the functionality and the speed of impacted websites, while failing to stop bad actors.”

ASSEMBLY FLOOR: 64-14, 8/30/18

AYES: Acosta, Aguiar-Curry, Arambula, Baker, Berman, Bloom, Bonta, Burke, Caballero, Calderon, Carrillo, Cervantes, Chau, Chiu, Chu, Cooley, Cooper, Cunningham, Daly, Eggman, Frazier, Friedman, Gabriel, Gallagher, Cristina Garcia, Eduardo Garcia, Gipson, Gloria, Gonzalez Fletcher, Gray, Grayson, Holden, Irwin, Jones-Sawyer, Kalra, Kamlager-Dove, Lackey, Levine, Limón, Low, Maienschein, Mayes, McCarty, Medina, Mullin, Muratsuchi, Nazarian, Obernolte, O'Donnell, Quirk, Quirk-Silva, Reyes, Rivas, Rodriguez, Rubio, Salas, Santiago, Mark Stone, Thurmond, Ting, Waldron, Weber, Wood, Rendon

NOES: Travis Allen, Bigelow, Brough, Chávez, Chen, Dahle, Flora, Harper, Kiley, Mathis, Melendez, Patterson, Steinorth, Voepel

NO VOTE RECORDED: Choi, Fong

Prepared by:

8/30/18 20:45:45

**** END ****

SENATE THIRD READING

STR Bill Id:SB 1001

Author:(Hertzberg)

As Amended Ver:August 24, 2018

Majority vote

SENATE VOTE: 26-10

SUMMARY: This bill makes it generally unlawful for any person to use a bot, as defined, to communicate or interact with another person in California online, with the intent to mislead the other person about its artificial identity with the purpose of knowingly deceiving the person about the content of the communication for commercial or political fraud purposes, as specified. Specifically, this bill:

1. Provides that it is unlawful for any person to use a bot to communicate or interact with another person in California online, with the intent to mislead the other person about its artificial identity for the purpose of knowingly deceiving the person about the content of the communication in order to incentivize a purchase or sale of goods or services in a commercial transaction or to influence a vote in an election.
2. Specifies that a person using a bot shall not be liable under this section if the person discloses that it is a bot. The disclosure must be clear, conspicuous, and reasonably designed to inform persons with whom the bot communicates or interacts that it is a bot.
3. Defines various terms for these purposes, including the following:
   1. "Bot" means an automated online account where all or substantially all of the actions or posts of that account are not the result of a person.
   2. "Online" means appearing on any public-facing Internet Web site, Web application, or digital application, including a social network or publication.
   3. "Online platform" means any public-facing Internet Web site, Web application, or digital application, including a social network or publication, that has 10,000,000 or more unique monthly United States visitors or users for a majority of months during the preceding 12 months.
   4. "Person" means a natural person, corporation, limited liability company, partnership, joint venture, association, estate, trust, government, governmental subdivision or agency, or other legal entity or any combination thereof.
   5. Specifies that this bill does not impose a duty on service providers of online platforms, including, but not limited to, web hosting and internet service providers.
   6. Provides a severability clause, as specified, and that the duties and obligations imposed by this chapter are cumulative with any other duties or obligation imposed by any other law.
4. Makes the above provisions operative on July 1, 2019.

EXISTING LAW:

1. Provides, under the United States (U.S.) Constitution, that "Congress shall make no law . . . abridging the freedom of speech, or of the press, or the right of the people peaceably to assemble, and to petition the government for a redress of grievances." (U.S. Constitution, 1st Amend., as applied to the states through the 14th Amendment's Due Process Clause; see Gitlow v. New York (1925) 268 U.S. 652.)
2. Provides under the California Constitution for the right of every person to freely speak, write and publish his or her sentiments on all subjects, being responsible for the abuse of this right. Existing law further provides that a law may not restrain or abridge liberty of speech or press. (California Constitution, Article I, Section 2(a).)
3. Provides as a matter of federal case law that the government may not, under the First Amendment to the United States Constitution, suppress political speech on the basis of the speaker's corporate identity. (Citizens United v. FEC (2010) 130 S.Ct. 876.)
4. Generally protects consumers and competitors against false or misleading advertising. (Business and Professions Code Section 17500 et seq.) Specifically, provides that it is unlawful for any person, firm, corporation or association, or any employee thereof, to make or disseminate before the public in this state, in any newspaper or other publication, or any advertising device, or in any other manner or means whatever, including over the internet, any statement concerning real or personal property or services, professional or otherwise, or concerning any circumstance or matter of fact connected with the proposed performance or disposition thereof, which is untrue or misleading, and which is known, or which by the exercise of reasonable care should be known, to be untrue or misleading. (Business & Professions Code Section 17500; emphasis added.) As used in the false advertising chapter, "person" includes any individual, partnership, firm, association, or corporation. (Business & Professions Code. Section 17506.)

FISCAL EFFECT: According to the Assembly Appropriations Committee, "Costs (Public Rights Division Law Enforcement Fund) in the tens of thousands of dollars to the Department of Justice (DOJ) to investigate and prosecute violations."

COMMENTS:

1. Purpose of this bill: This bill seeks to provide internet users the ability to distinguish between social media accounts owned and operated by "real" and "fake" persons – i.e., bot accounts impersonating a natural person or an entity – by requiring appropriate disclosures of "bot" accounts, as specified, to help prevent users from being intentionally misled about the content of the communication in a manner that could lead to commercial or election fraud, as specified. This bill is sponsored by Common Sense Kids Action.
2. Author's statement: According to the author, "[t]here is currently no law that relates to the disclosure of the use of bot accounts on social media. SB 1001 sheds light on the fake accounts that simulate real people and spread waves of false information across social media platforms. On the Internet where the appearance of a mass audience can be monetized, it is critical to protect users by providing the tools to understand if their information is coming from a human or a bot account disguised as one. As long as bots are properly identified to let users know that they are a computer generated or automated account, users can at least be aware of who they are interacting with and judge the content accordingly.
3. Bots (essentially) have speech rights, too: Fundamentally, this bill, like others brought before the Legislature this year, triggers a debate surrounding the First Amendment issues implicated by bot regulations. As a matter of constitutional law, even if bots do not directly have speech rights, they are connected to individuals (or corporations) who do. Indeed, bots do not appear out of thin air; they are created by a natural person or entity for a purpose. Namely, they enable the creator to associate themselves with the bot's "speech," or rather, to express themselves, or speak, through the bot. Bot accounts on social media can have legitimate and beneficial uses, let alone less favorable but still constitutionally-protected ones. As explained by the Electronic Frontier Foundation, bots can be "used for First Amendment protected activities. This includes poetry, political speech, parody, and satire—such as poking fun at people who cannot resist arguing, even with bots." Content produced by bots on behalf of the speaker arguably does not lose its First Amendment protection merely because it is "spoken" through a bot.

That being said, not all speech is always protected speech: the First Amendment reflects a fundamental right protecting the freedom of speech and expression; it is not an absolute right. The determination about whether a specific statute inappropriately restricts speech can vary depending on whether the content regulated is a lesser protected type of speech. For example, speech that falls under the category of obscenity or incitement is not afforded the same protections as conversational speech about current events or religious speech or artistic expression. Nor is commercial speech afforded the same protection as political speech. The latter categories receive greater scrutiny in analysis and require a demonstration that the laws are sufficiently narrowly tailored to achieve a compelling governmental interest. Ultimately, when conducting a constitutional analysis of government restrictions on freedom of speech, the analysis invariably requires a determination as to whether the restriction is content-neutral or content-based; unduly vague or overbroad; and whether the restriction acts as a prior restraint on speech. At times, questions around government regulations may also relate to whether the statute unconstitutionally prohibits anonymous speech, or actually compels speech, as the First Amendment also protects the right to speech anonymously, and the right to stay silent.

Here, the question becomes whether this bill meets such constitutional standards.

1. Amendments in policy committee narrowly tailored the bill: Recent amendments in policy committee have more clearly focused this bill on misleading speech in a narrowly tailored fashion. Specifically, as amended June 21, 2018, the bill makes it unlawful for any person to use a bot to communicate with or interact with another person in California, online, with the intent to mislead the person about its artificial identity for the purpose of knowingly deceiving the person about the content of the communication in order to incentivize a purchase or sale of goods or services in a commercial transaction or to influence a vote in an election.

Stated another way, it would be unlawful for a person to mislead another person about their account's bot identity in order to deceive that other person about the content of their communication for these narrow purposes which are related to commercial and political fraud. In doing so, this bill addresses more specific concerns about how political bots (whether created by natural persons or legal entities) are being used to somehow defraud the electorate and influence outcomes of elections, or how commercial bots could be used to defraud consumers (natural persons) or businesses (whether sole proprietorships or corporate legal entities) to elicit purchases or sales. Given their nexus to a more narrowly identifiable harm that the government has a compelling interest in preventing, these communications, if made the subject of the regulations, would presumably fall under a less protected category of First Amendment speech or activity (fraudulent commercial speech, for example) and more likely to survive a challenge in the courts.

As amended, the bill now makes clear that a person using a bot would not be liable under this section if it discloses that it is a bot. Notably, this statement is different from disclosing that it is "not a natural person," in recognition that a bot could be an online account mimicking a company as easily as it could mimic a natural person with the intent to deceive another person. Furthermore, this is not to say that misleading or fraudulent commercial or political activities that incentivize others to make purchases, sales, or votes based on those intentionally fraudulent representations would suddenly be made lawful if remedies can otherwise be pursued under other laws; it is only to say that the person would not be held liable for misleading the user about their "artificial identity" under this particular law. Lastly, the bill, as amended, clarifies that a "bot" is an automated online account (to narrow the bill to exclude "parody" accounts) that is designed to mimic or behave like the account of a person (again, as opposed to natural person).

While regulating political speech may nonetheless be precarious given the heavy presumptions of unconstitutionality around what is considered to be at the core of the First Amendment and receives the highest form of protection, by requiring a narrow disclosure for the compelling interest of preserving the integrity of the electoral system in a content neutral fashion, the amended version of this bill may have a better likelihood of surpassing strict scrutiny from the courts. Furthermore, the amended version of this bill appears to achieve the underlying goal of the sponsor of this bill, Common Sense Kids Action, which writes:

Sixty-six percent of American teens use Facebook, 47% use Twitter, and 39% say social media is their preferred news source. They are at risk of being targeted with unlabeled bots on those platforms, with an estimated 60 million (Facebook) and 48 million (Twitter) unlabeled bot accounts. These bots are more likely to carry harmful messages: political propaganda, ethnic discrimination, or just false information. Educators and parents are concerned, as only 44% of children say they can distinguish false news online from real.

With so many children and teens online[,] families need confidence that they will be protected from unlabeled accounts that are spreading false information and harmful content.

1. Related legislation: AB 1950 (Levine) of 2018, as introduced, would have prohibited an operator of a social media site with a physical presence in California from selling advertising to a computer software account or user that performs an automated task and is not verified as being controlled by a natural person. That bill was eventually gutted and amended into a bill related to the California Online Protection Privacy Act and was held in the Assembly Privacy and Consumer Protection Committee.

AB 3169 (Gallagher) of 2018 would have prohibited any person who operates a social media internet website or search engine located in California, as specified, from removing or manipulating content on the basis of the political affiliation or political viewpoint of that content. That bill died in the Assembly Privacy and Consumer Protection Committee.

Analysis Prepared by: Ronak Daylami / P. & C.P. / (916) 319-2200 FN: 0004877

SENATE THIRD READING

STR Bill Id:SB 1001

Author:(Hertzberg)

As Amended Ver:June 21, 2018

Majority vote

SENATE VOTE: 26-10

SUMMARY: This bill makes it generally unlawful for any person to use a bot, as defined, to communicate or interact with another person in California online, with the intent to mislead the other person about its artificial identity with the purpose of knowingly deceiving the person about the content of the communication for commercial or political fraud purposes, as specified. Specifically, this bill:

1. Provides that it is unlawful for any person to use a bot to communicate or interact with another person in California online, with the intent to mislead the other person about its artificial identity for the purpose of knowingly deceiving the person about the content of the communication in order to incentivize a purchase or sale of goods or services in a commercial transaction or to influence a vote in an election.
2. Specifies that a person using a bot shall not be liable under this section if the person discloses that it is a bot. The disclosure must be clear, conspicuous, and reasonably designed to inform persons with whom the bot communicates or interacts that it is a bot.
3. Defines various terms for these purposes, including the following:
   1. "Bot" means an automated online account on an online platform that is designed to mimic or behave like the account of a person.
   2. "Online" means appearing on any public-facing Internet Web site, Web application, or digital application, including a social network or publication.
   3. "Online platform" means any public-facing Internet Web site, Web application, or digital application, including a social network or publication.
   4. "Person" means a natural person, corporation, limited liability company, partnership, joint venture, association, estate, trust, government, governmental subdivision or agency, or other legal entity or any combination thereof.
   5. Specifies that this bill does not impose a duty on service providers of online platforms, including, but not limited to, web hosting and internet service providers.
   6. Provides a severability clause, as specified, and that the duties and obligations imposed by this chapter are cumulative with any other duties or obligation imposed by any other law.

EXISTING LAW:

1. Provides, under the United States (U.S.) Constitution, that "Congress shall make no law . . . abridging the freedom of speech, or of the press, or the right of the people peaceably to assemble, and to petition the government for a redress of grievances." (U.S. Constitution, 1st Amend., as applied to the states through the 14th Amendment's Due Process Clause; see Gitlow v. New York (1925) 268 U.S. 652.)
2. Provides under the California Constitution for the right of every person to freely speak, write and publish his or her sentiments on all subjects, being responsible for the abuse of this right. Existing law further provides that a law may not restrain or abridge liberty of speech or press. (California Constitution, Article I, Section 2(a).)
3. Provides as a matter of federal case law that the government may not, under the First Amendment to the United States Constitution, suppress political speech on the basis of the speaker's corporate identity. (Citizens United v. FEC (2010) 130 S.Ct. 876.)
4. Generally protects consumers and competitors against false or misleading advertising. (Business and Professions Code Section 17500 et seq.) Specifically, provides that it is unlawful for any person, firm, corporation or association, or any employee thereof, to make or disseminate before the public in this state, in any newspaper or other publication, or any advertising device, or in any other manner or means whatever, including over the internet, any statement concerning real or personal property or services, professional or otherwise, or concerning any circumstance or matter of fact connected with the proposed performance or disposition thereof, which is untrue or misleading, and which is known, or which by the exercise of reasonable care should be known, to be untrue or misleading. (Business & Professions Code Section 17500; emphasis added.) As used in the false advertising chapter, "person" includes any individual, partnership, firm, association, or corporation. (Business & Professions Code. Section 17506.)

FISCAL EFFECT: According to the Assembly Appropriations Committee, "Costs (Public Rights Division Law Enforcement Fund) in the tens of thousands of dollars to the Department of Justice (DOJ) to investigate and prosecute violations."

COMMENTS:

1. Purpose of this bill: This bill seeks to provide internet users the ability to distinguish between social media accounts owned and operated by "real" and "fake" persons – i.e., bot accounts impersonating a natural person or an entity – by requiring appropriate disclosures of "bot" accounts, as specified, to help prevent users from being intentionally misled about the content of the communication in a manner that could lead to commercial or election fraud, as specified. This bill is sponsored by Common Sense Kids Action.
2. Author's statement: According to the author, "[t]here is currently no law that relates to the disclosure of the use of bot accounts on social media. SB 1001 sheds light on the fake accounts that simulate real people and spread waves of false information across social media platforms. On the Internet where the appearance of a mass audience can be monetized, it is critical to protect users by providing the tools to understand if their information is coming from a human or a bot account disguised as one. As long as bots are properly identified to let users know that they are a computer generated or automated account, users can at least be aware of who they are interacting with and judge the content accordingly.
3. Bots (essentially) have speech rights, too: Fundamentally, this bill, like others brought before the Legislature this year, triggers a debate surrounding the First Amendment issues implicated by bot regulations. As a matter of constitutional law, even if bots do not directly have speech rights, they are connected to individuals (or corporations) who do. Indeed, bots do not appear out of thin air; they are created by a natural person or entity for a purpose. Namely, they enable the creator to associate themselves with the bot's "speech," or rather, to express themselves, or speak, through the bot. Bot accounts on social media can have legitimate and beneficial uses, let alone less favorable but still constitutionally-protected ones. As explained by the Electronic Frontier Foundation, bots can be "used for First Amendment protected activities. This includes poetry, political speech, parody, and satire—such as poking fun at people who cannot resist arguing, even with bots." Content produced by bots on behalf of the speaker arguably does not lose its First Amendment protection merely because it is "spoken" through a bot.

That being said, not all speech is always protected speech: the First Amendment reflects a fundamental right protecting the freedom of speech and expression; it is not an absolute right. The determination about whether a specific statute inappropriately restricts speech can vary depending on whether the content regulated is a lesser protected type of speech. For example, speech that falls under the category of obscenity or incitement is not afforded the same protections as conversational speech about current events or religious speech or artistic expression. Nor is commercial speech afforded the same protection as political speech. The latter categories receive greater scrutiny in analysis and require a demonstration that the laws are sufficiently narrowly tailored to achieve a compelling governmental interest. Ultimately, when conducting a constitutional analysis of government restrictions on freedom of speech, the analysis invariably requires a determination as to whether the restriction is content-neutral or content-based; unduly vague or overbroad; and whether the restriction acts as a prior restraint on speech. At times, questions around government regulations may also relate to whether the statute unconstitutionally prohibits anonymous speech, or actually compels speech, as the First Amendment also protects the right to speech anonymously, and the right to stay silent.

Here, the question becomes whether this bill meets such constitutional standards.

1. Amendments in policy committee narrowly tailored the bill: Recent amendments in policy committee have more clearly focused this bill on misleading speech in a narrowly tailored fashion. Specifically, as amended June 21, 2018, the bill makes it unlawful for any person to use a bot to communicate with or interact with another person in California, online, with the intent to mislead the person about its artificial identity for the purpose of knowingly deceiving the person about the content of the communication in order to incentivize a purchase or sale of goods or services in a commercial transaction or to influence a vote in an election.

Stated another way, it would be unlawful for a person to mislead another person about their account's bot identity in order to deceive that other person about the content of their communication for these narrow purposes which are related to commercial and political fraud. In doing so, this bill addresses more specific concerns about how political bots (whether created by natural persons or legal entities) are being used to somehow defraud the electorate and influence outcomes of elections, or how commercial bots could be used to defraud consumers (natural persons) or businesses (whether sole proprietorships or corporate legal entities) to elicit purchases or sales. Given their nexus to a more narrowly identifiable harm that the government has a compelling interest in preventing, these communications, if made the subject of the regulations, would presumably fall under a less protected category of First Amendment speech or activity (fraudulent commercial speech, for example) and more likely to survive a challenge in the courts.

As amended, the bill now makes clear that a person using a bot would not be liable under this section if it discloses that it is a bot. Notably, this statement is different from disclosing that it is "not a natural person," in recognition that a bot could be an online account mimicking a company as easily as it could mimic a natural person with the intent to deceive another person. Furthermore, this is not to say that misleading or fraudulent commercial or political activities that incentivize others to make purchases, sales, or votes based on those intentionally fraudulent representations would suddenly be made lawful if remedies can otherwise be pursued under other laws; it is only to say that the person would not be held liable for misleading the user about their "artificial identity" under this particular law. Lastly, the bill, as amended, clarifies that a "bot" is an automated online account (to narrow the bill to exclude "parody" accounts) that is designed to mimic or behave like the account of a person (again, as opposed to natural person).

While regulating political speech may nonetheless be precarious given the heavy presumptions of unconstitutionality around what is considered to be at the core of the First Amendment and receives the highest form of protection, by requiring a narrow disclosure for the compelling interest of preserving the integrity of the electoral system in a content neutral fashion, the amended version of this bill may have a better likelihood of surpassing strict scrutiny from the courts. Furthermore, the amended version of this bill appears to achieve the underlying goal of the sponsor of this bill, Common Sense Kids Action, which writes:

Sixty-six percent of American teens use Facebook, 47% use Twitter, and 39% say social media is their preferred news source. They are at risk of being targeted with unlabeled bots on those platforms, with an estimated 60 million (Facebook) and 48 million (Twitter) unlabeled bot accounts. These bots are more likely to carry harmful messages: political propaganda, ethnic discrimination, or just false information. Educators and parents are concerned, as only 44% of children say they can distinguish false news online from real.

With so many children and teens online[,] families need confidence that they will be protected from unlabeled accounts that are spreading false information and harmful content.

1. Related legislation: AB 1950 (Levine of 2018), as introduced, would have prohibited an operator of a social media site with a physical presence in California from selling advertising to a computer software account or user that performs an automated task and is not verified as being controlled by a natural person. That bill was eventually gutted and amended into a bill related to the California Online Protection Privacy Act and was held in the Assembly Privacy and Consumer Protection Committee.

AB 3169 (Gallagher of 2018) would have prohibited any person who operates a social media internet website or search engine located in California, as specified, from removing or manipulating content on the basis of the political affiliation or political viewpoint of that content. That bill died in the Assembly Privacy and Consumer Protection Committee.

Analysis Prepared by: Ronak Daylami / P. & C.P. / (916) 319-2200 FN: 0004267

Date of Hearing: August 15, 2018

ASSEMBLY COMMITTEE ON APPROPRIATIONS

Lorena Gonzalez Fletcher, Chair

SBPCA Bill Id:SB 1001

Author:(Hertzberg) – As Amended Ver:June 21, 2018

Urgency: No State Mandated Local Program: No Reimbursable: No

SUMMARY:

This bill makes it unlawful to use a “bot”, as defined, to communicate or interact online with a person with the intention of misleading the person regarding a purchase, transaction, or to influence a vote in an election, unless it is clearly and conspicuously disclosed that the bot is not a natural person.

FISCAL EFFECT:

Costs (Public Rights Division Law Enforcement Fund) in the tens of thousands of dollars to the Department of Justice (DOJ) to investigate and prosecute violations.

COMMENTS:

1. Background. Bots are software applications that run automated tasks on a network and can interact with computer systems or network users. There are a variety of different types of bots that are programmed to perform different tasks. While some bots are created for legitimate purposes, some types of bots are created with malicious intents, including scraping content, stealing passwords, spamming websites, or scalping tickets. When interacting with bots online, people may not be able to discern if they are interacting with a bot or another person. For example, users of a company’s customer support system may assume they are communicating directly with a customer service employee, but instead are texting with a bot. Similarly, Twitter users may assume a tweet was posted by another person when the content was actually generated by a bot. The latter type of interaction has become increasingly common as the number of bots on social media sites continues to grow.
2. Purpose. According to the author:

There is currently no law that relates to the disclosure of the use of bot accounts on social media. SB 1001 sheds light on the fake accounts that simulate real people and spread waves of false information across social media platforms. On the Internet where the appearance of a mass audience can be monetized, it is critical to protect users by providing the tools to understand if their information is coming from a human or a bot account disguised as one.

Analysis Prepared by: Jessica Peters / APPR. / (916) 319-2081

Date of Hearing: August 8, 2018

ASSEMBLY COMMITTEE ON APPROPRIATIONS

Lorena Gonzalez Fletcher, Chair

SBPCA Bill Id:SB 1001

Author:(Hertzberg) – As Amended Ver:June 21, 2018

Urgency: No State Mandated Local Program: No Reimbursable: No

SUMMARY:

This bill makes it unlawful to use a “bot”, as defined, to communicate or interact online with a person with the intention of misleading the person regarding a purchase, transaction, or to influence a vote in an election, unless it is clearly and conspicuously disclosed that the bot is not a natural person.

FISCAL EFFECT:

Costs (Public Rights Division Law Enforcement Fund) in the tens of thousands of dollars to the Department of Justice (DOJ) to investigate and prosecute violations.

COMMENTS:

1. Background. Bots are software applications that run automated tasks on a network and can interact with computer systems or network users. There are a variety of different types of bots that are programmed to perform different tasks. While some bots are created for legitimate purposes, some types of bots are created with malicious intents, including scraping content, stealing passwords, spamming websites, or scalping tickets. When interacting with bots online, people may not be able to discern if they are interacting with a bot or another person. For example, users of a company’s customer support system may assume they are communicating directly with a customer service employee, but instead are texting with a bot. Similarly, Twitter users may assume a tweet was posted by another person when the content was actually generated by a bot. The latter type of interaction has become increasingly common as the number of bots on social media sites continues to grow.
2. Purpose. According to the author:

There is currently no law that relates to the disclosure of the use of bot accounts on social media. SB 1001 sheds light on the fake accounts that simulate real people and spread waves of false information across social media platforms. On the Internet where the appearance of a mass audience can be monetized, it is critical to protect users by providing the tools to understand if their information is coming from a human or a bot account disguised as one.

Analysis Prepared by: Jessica Peters / APPR. / (916) 319-2081

Date of Hearing:

ASSEMBLY COMMITTEE ON ARTS, ENTERTAINMENT, SPORTS, TOURISM, AND INTERNET MEDIA

Kansen Chu, Chair

SBPCA Bill Id:SB 1001

Author:(Hertzberg) – As Amended Ver:June 21, 2018

SENATE VOTE: 26-10

SUBJECT: Bots: disclosure.

SUMMARY: This bill would make it unlawful for any person to use a bot, as defined, to communicate or interact with persons in California online, with the intention of misleading for specified purposes, and without clearly and conspicuously disclosing that the bot is not a natural person, as specified. Specifically, this bill:

1. Declares that it shall be unlawful for any person to use a bot to communicate or interact with another person in California online, with the intent to mislead the other person about its artificial identity for the purpose of knowingly deceiving the person about the content of the communication in order to either:
   1. Incentivize a purchase or sale of goods or services in a commercial transaction, or;
   2. To influence a vote in an election.
2. Provides that a person using a bot shall not be liable under this section if the person discloses that it is a bot.
3. Further provides that the disclosure required by this section shall be clear, conspicuous, and reasonably designed to inform persons with whom the bot communicates or interacts that it is a bot.
4. Contains the following definitions:
   1. “Bot” means an automated online account on an online platform that is designed to mimic or behave like the account of a person.
   2. “Online” means appearing on any public-facing Internet Web site, Web application, or digital application, including a social network or publication.
   3. “Online platform” means any public-facing Internet Web Site, Web application, or digital application, including a social network or publication.
   4. “Person” means a natural person, corporation, limited liability company, partnership, joint venture, association, estate, trust, government, governmental subdivision or agency, or other legal entity or any combination thereof.
5. Provides a severability clause, as specified, and states that the duties and obligations imposed by this chapter are cumulative with any other duties or obligation imposed by any other law.

EXISTING FEDERAL LAW:

1. Provides, under the U.S. Constitution, that “Congress shall make no law . . . abridging the freedom of speech, or of the press, or the right of the people peaceably to assemble, and to petition the government for a redress of grievances.” (U.S. Const., 1st Amend., as applied to the states through the 14th Amendment’s Due Process Clause; see Gitlow v. New York (1925) 268 U.S. 652.)
2. Provides as a matter of federal case law that the government may not, under the First Amendment to the United States Constitution, suppress political speech on the basis of the speaker’s corporate identity. (Citizens United v. FEC (2010) 130 S.Ct. 876.)

EXISTING STATE LAW:

1. Provides under the California Constitution for the right of every person to freely speak, write and publish his or her sentiments on all subjects, being responsible for the abuse of this right. Existing law further provides that a law may not restrain or abridge liberty of speech or press. (Cal. Const., art. I, Sec. 2(a).)
2. Prohibits any person, firm, corporation or association, or any employee thereof, from making statements that are untrue or misleading in connection with communications regarding the sale of goods or services, including over the Internet. (Business and Professions Code (BPC) Section (§) 17500)
3. Prohibits a person, with a bad faith intent, to register, traffic in, or use a domain name that is identical or confusingly similar to the name of another living or deceased person. (BPC § 17525)
4. Prohibits any person or entity from sending a commercial e-mail advertisement from California or to a California electronic mail address if the e-mail advertisement contains falsified, misrepresented, or forged header information or if the e-mail advertisement has a misleading subject line. Permits the AG, among other parties, to bring action against a person or entity that violates these provisions. (BPC §17529.5)
5. Prohibits a person or entity located within California from using a telephone facsimile machine, computer, or other device to send an unsolicited advertisement to a telephone facsimile machine. (BPC § 17538.43 (b)(1))
6. Requires a person or entity located in California to clearly mark the date and time of transmittal and identity and telephone number of the person or entity sending the message of any communication via a telephone facsimile machine. (BPC § 17538.43 (c)(1))
7. Requires individuals or partnerships doing business under fictitious names to file a fictitious business name certificate designed to make the identities of the persons doing business under the fictitious names available to the public. (BPC § 17900)

FISCAL EFFECT: According to the Senate Appropriations Committee, pursuant to Senate Rule 28.8, any additional state costs are not significant, and do not and will not require the appropriation of additional state funds, and the bill will cause no significant reduction in revenues.

COMMENTS:

1. Author's and supporter's statement of need for legislation: According to the author, “there is currently no law that relates to the disclosure of the use of bot accounts on social media. SB 1001 sheds light on the fake accounts that simulate real people and spread waves of false information across social media platforms. On the Internet where the appearance of a mass audience can be monetized, it is critical to protect users by providing the tools to understand if their information is coming from a human or a bot account disguised as one. As long as bots are properly identified to let users know that they are a computer generated or automated account, users can at least be aware of whom they are interacting with and judge the content accordingly.”

Common Sense Kids Action is the sponsor of SB 1001. They write the committee to share their belief that, "American democracy and civic discourse in 2018 remain under attack from bots and disinformation." Adding, "(A)s the advocacy arm of Common Sense Media, Common Sense Kids Action works to ensure that children can thrive in today's digital world. Sixty-six percent of American teens use Facebook, 47% use Twitter, and 39% say social media is their preferred news source. They are at risk with being targeted with unlabeled bots on those platforms, which have an estimated 60million (Facebook) and 48 million (Twitter) unlabeled bot accounts. …With so many kids online; families need confidence that they will be protected from unlabeled accounts that are spreading false information and harmful content." In order to demonstrate the havoc bots can wreck, the sponsors shared the following list of articles and accounts of the impact bots have on the online world:

Expert: Bots are poised to wreak havoc in the 2018 midterm elections. Samuel Woolley a technologist in Palo Alto reports that in “almost any” election happening in America in 2018, you see evidence of bots being used to manipulate the vote.

Mueller warns that election, politics meddling by foreigners is still happening. In June of 2018, Special Counsel Robert Mueller informed the United State District Court of ongoing Russian election interference, following the patterns of bots and disinformation from 2016.

Bots Hosted by American Social Media Platforms interfere with Mexico’s Election. Mexico has long faced some of the worst human rights and democracy abuse by bots, and it is continuing in the 2018 elections.

Unilever gets serious about influencer fraud. One of the world’s largest advertisers is rethinking its exposure to social media platforms populated by unlabeled bots that are taking advantage of public trust.

1. Background.
   1. The use and misuse of bots. According to research conducted by the Center for Complex Networks and Systems Research at Indiana University and the Information Sciences Institute at the University of Southern California, “[i]ncreasing evidence suggests that a growing amount of social media content is generated by autonomous entities known as social bots.” (Onur Varol, et al., Online Human-Bot Interactions: Detection, Estimation, and Characterization (Mar. 27, 2017). Their estimates suggest that as many as 15 percent of Twitter accounts are fake. This would mean roughly 48 million Twitter users are bots, automated to simulate real people. Facebook itself disclosed to its investors that the social media platform had “at least twice as many fake users as it previously estimated, indicating that up to 60 million automated accounts may roam the world’s largest social media platform.” (Confessore, et al., The Follower Factory (Jan. 27, 2018) The New York Times.

As explained by the Senate Committee on Business and Professions, Bots are software applications that run automated tasks on a network, such as the Internet, that can interact with computers systems or users. Bots can be purchased or created by individuals or organizations and many online platforms, including social media sites, create bots themselves and encourage users to build bots for use on their platforms. There are a variety of different types of bots that are programmed to perform different tasks. Examples include search engine crawler bots, such as GoogleBot or BingBot, that help search engines discover new content; copyright bots, such as YouTube Content ID, that help identify content posted online that violates copyright law; and chatbots that are designed to simulate conversations with human users, often in order to perform tasks such as booking a flight. Some types of bots are created with malicious intents, including scraping content, stealing passwords, spamming websites, or scalping tickets.

When interacting directly or indirectly with bots online, human users may not be able to discern if they are interacting with a bot or another human. For example, human users on a company’s customer support system may assume they are communicating directly with a human customer service employee, but instead are texting with a bot. Similarly, human Twitter users may assume a Tweet was posted by another human when in actuality the content was generated by a bot. The latter type of interaction has become increasingly common as the number of automated users on social media sites continues to grow.

Bots that are automated accounts on social media sites can be used to amplify the perceived popularity of people, products, and services online. In January 2018, the New York Times published an article, “The Follower Factory,” detailing an emerging market where those seeking to bolster their online presence can purchase followers, likes, and other engagements for their Facebook, Twitter, LinkedIn, YouTube, and other social media accounts from companies that control a large number of bots. One such company highlighted by The Times, Devumi, has reportedly sold Twitter followers to over 200,000 customers, including reality television starts, professional athletes, and politicians. To do so, Devumi draws upon its approximately 3.5 million bot accounts, at least 55,000 of which use the names, profile pictures, and other personal details of real Twitter users. Since a higher follower count implies a higher popularity or importance of the person or product, purchasing followers from companies like Devumi can lead to a real increase in human users engaging with the purchasers account and even lead to endorsement deals from companies seeking to capitalize on a user’s perceived online influence. Social media platforms use similar metrics to determine the popularity of an account, which can influence which accounts a platform recommends or which posts a platform determines are trending.

Just as companies like Devumi use bots to promote their customer’s accounts, individuals and groups can, and have, used bots to spread and amplify divisive views on social media platforms. In February 2018, the U.S. Justice Department charged 13 Russians and three companies with creating fake profiles on social media sites to post contentious comments related to religion, race, and politics and then using bot accounts to like, share, and retweet the posts to help them gain traction. Information provided to U.S. Congress on October 20, 2017 indicated that Russian agents published more than 131,000 messages on Twitter, uploaded over 1,000 videos to YouTube, and disseminated inflammatory posts that reached 126 million users on Facebook.

• 1. Regulation and legislation of bots. Many online platforms have their own policies regarding the use of bots, including Twitter which introduced new policies regarding the use of automated accounts in February 2018. While bots are still allowed on Twitter and developers are still encouraged by Twitter to create useful bots that add to the platform’s online community, individuals are now prohibited from using “any form of automation…to post identical or substantially similar content or perform actions such as Likes or Retweets across many accounts.” Twitter also has a variety of enforcement mechanisms in place, including requiring an account suspected of violating its policies to verify ownership with a phone number or email address. Twitter states that this enforcement mechanism helps it “identify violators who are operation multiple accounts for abusive purposes and take action on such accounts.” As its most severe enforcement action, Twitter permanently suspends accounts. Users can appeal permanent suspensions through the platform interface or by filing a report but it is unclear exactly how long Twitter’s investigative and enforcement process takes.

There have recently been efforts by other states and the federal government to regulate the use of bots and prevent the spread of misinformation and false advertising by automated accounts. In 2016, President Obama signed the Better Online Ticket Sales Act, or BOTS Act, (2016; 114^th Congress S. 3183) which prohibited the use of bots to scalp tickets by automatically purchase event tickets online and reselling them on secondary ticket-selling websites at a significantly higher price. This year, legislators in several states, including New York, Maryland, and California, as well as the U.S. Congress, have introduced legislation to regulate the purchasing of social media advertising, including by bots. However, there is currently no state that requires the disclosure of bots.

1. Recent amendments remove opposition – however discussions continue on potential clarification of "clear and conspicuous" disclosures. The author accepted narrowing amendment in the Assembly Committee on Privacy and Consumer Protection which removed opposition of the coalition of California business and technology organizations. There were however lingering concerns over the legal standards for what is "clear and conspicuous," which vary from mobile platforms to internet based sites. While these concerns do not rise to the level of opposition, the coalition remains in talks with the author about addressing challenges to implementation of this language.

There is guidance for the coalition, found in Federal Trade Commission (FTC) standards for online advertisements, and their views on what constitutes a clear and conspicuous disclosure on various platforms, and even sites such as Twitter and Instagram. As explained in the article, Clear & Conspicuous’: What It Now Means to the FTC in Digital Advertising, Rothbard, (2013), "The revised guidelines make clear that FTC truth-in-advertising principles apply to any medium, platform or device – no matter how new or small – and attempt to offer practical advice for complying with FTC disclosure requirements in online, social media and mobile advertising.

"What will make a disclosure “clear and conspicuous” in a digital ad? The FTC’s criteria are:

• Placement and prominence of the disclosure and how close it is to the related claim;
• Whether the disclosure is unavoidable;
• Whether other parts of the ad distract from the disclosure;
• Whether the disclosure needs to be repeated to ensure it’s seen; and
• Whether the language is understandable.

"The revamped guide also strongly discourages scrolling to find a disclosure, especially horizontal scrolling on smaller screens which consumers rarely do (any need to scroll horizontally can be solved by having a mobile-optimized site). When scrolling can’t be avoided, effective text or visual cues need to lead consumers to the disclosure. Pop-ups are another FTC disclosure non-favorite since they can be easily blocked and are often ignored.

"Where space constraints are acute, such as in tweets, the revised guidance encourages repetition of disclosures (such as republishing in 're-tweets') and allows abbreviated disclosures as long as consumers understand them. For example, 'Ad' should be adequate to indicate a tweet is sponsored, but not necessarily '#spon.'"

1. Constitutional concerns remain despite narrowing of legislation. The Assembly Committee on Privacy and Consumer Protection analysis of this bill contains an exhaustive discussion of constitutional principles and concerns raised by this bill. (Please see their committee analysis for more a in depth discussion). The recent amendments have taken care of many issues, but not all. In the final analysis it will be up to a court to determine if the measure passes constitutional muster. The following is a summary of issues raised by this measure as discussed in their analysis.

This bill would declare it to be a violation for any person (meaning both natural persons and corporations or other legal entities) to use a bot, as defined, to communicate or interact with persons in California online, with the intention of misleading a person about its artificial identity. Under this bill, a person using a bot would be presumed to not act with the intent to mislead if the person discloses that the bot is not a natural person.

Fundamentally, this bill, like others brought before the Legislature this year, triggers a debate surrounding the First Amendment issues implicated by bot regulations. As a matter of constitutional law, even if bots do not directly have speech rights, they are connected to individuals (or corporations) who do. Indeed, bots do not appear out of thin air; they are created by a natural person or entity for a purpose. Namely, they enable the creator to associate themselves with the bot’s “speech,” or rather, to express themselves, or speak, through the bot. Bot accounts on social media can have legitimate and beneficial uses, let alone less favorable but still constitutionally-protected ones. The content produced by bots on behalf of the speaker arguably does not lose its First Amendment protection merely because it is “spoken” through a bot.

That being said, not all speech is always protected speech: the First Amendment reflects a fundamental right protecting the freedom of speech and expression; it is not an absolute right. The determination about whether a specific statute inappropriately restricts speech can vary depending on whether the content regulated is a lesser protected type of speech. For example, speech that falls under the category of obscenity or incitement is not afforded the same protections as conversational speech about current events or religious speech or artistic expression. Nor is commercial speech afforded the same protection as political speech. The latter categories receive greater scrutiny in analysis and require a demonstration that the laws are sufficiently narrowly tailored to achieve a compelling governmental interest. Ultimately, when conducting a constitutional analysis of government restrictions on freedom of speech, the analysis invariably requires a determination as to whether the restriction is content-neutral or content-based; unduly vague or overbroad; and whether the restriction acts as a prior restraint on speech. At times, questions around government regulations may also relate to whether the statute unconstitutionally prohibits anonymous speech, or actually compels speech, as the First Amendment also protects the right to speech anonymously, and the right to stay silent.

Here, the question becomes whether this bill meets such constitutional standards, particularly as the underlying premise of this bill appears to be that any and all speech expressed by way of a bot is less trustworthy (or constitutes a fact that is relevant to its trustworthiness), or will inherently mislead people if the identity of the bot “speaker” is left undisclosed.

1. Prior and related legislation:
   1. AB 1950 (Levine) Legislation of 2018, as introduced, would have prohibited an operator of a social media site with a physical presence in California from selling advertising to a computer software account or user that performs an automated task and is not verified as being controlled by a natural person. Status: That bill was later gutted and amended into a bill related to the California Online Protection Privacy Act and was held in Assembly Committee on Privacy and Consumer Protection.
   2. AB 3169 (Gallagher), Legislation of 2018, would have prohibited any person who operates a social media internet website or search engine located in California, as specified, from removing or manipulating content on the basis of the political affiliation or political viewpoint of that content. Status: The bill failed passage in Assembly Committee on Privacy and Consumer Protection on April 3, 2018.
   3. AB 1832 (Calderon), Chapter 158, Statutes of 2014, expanded the provisions of AB 329 (Pan) which prohibited the intentional use or sale of software to circumvent a security measure, access control system, or other control or measure used to ensure an equitable ticket buying process, to any platform and not only on a ticket seller’s Internet Web site.
   4. AB 329 (Pan), Chapter 325, Statutes of 2013, made it a misdemeanor to intentionally use or sell software to circumvent a security measure, access control system, or other control or measure on a ticket seller’s Internet Web site that is used to ensure an equitable ticket buying process.

REGISTERED SUPPORT / OPPOSITION:

Support

Common Sense Kids Action (Sponsor)

Opposition

None on file

Analysis Prepared by: Dana Mitchell / A.,E.,S.,T., & I.M. / (916) 319-3450

Date of Hearing: June 19, 2018

ASSEMBLY COMMITTEE ON PRIVACY AND CONSUMER PROTECTION

Ed Chau, Chair

SBPCA Bill Id:SB 1001

Author:(Hertzberg) – As Amended Ver:May 25, 2018

SENATE VOTE: 26-10

SUBJECT: Bots: disclosure

SUMMARY: This bill would make it unlawful for any person to use a bot, as defined, to communicate or interact with natural persons in California online, with the intention of misleading and without clearly and conspicuously disclosing that the bot is not a natural person, as specified. Specifically, this bill would:

1. Provide that it is unlawful for any person to use a bot to communicate or interact with natural persons in California online, with the intention of misleading a natural person about its artificial identity. A person using a bot would be presumed to not act with the intent to mislead if the person discloses that the bot is not a natural person.
2. Specify that a statement will be considered a disclosure that a bot is not a person if it includes language stating the bot “is a bot” or is “auto-generated.”
3. Require that the disclosure be clear and conspicuous to all natural persons with whom the bot communicates or interacts, as specified. Would further provide that a statement would be deemed not made in a clear and conspicuous manner if it is difficult to read or hear or if the placement is easily overlooked.
4. Require an online platform to enable users to identify and report bots that the user suspects of violating the above provisions. Would require an online platform, after receiving notice of a bot as such, to expeditiously investigate and determine whether or not to disclose that the bot is not a natural person or remove the bot.
5. Specify that an online platform’s investigation and response is expeditious if it occurs within 72 hours of receipt of the notice.
6. Require, upon request of the Attorney General, that an online platform provide reports detailing notices received and actions taken in response.
7. Specify that the duties and obligations imposed by this bill do not apply to bots to the extent they are communicating on behalf of a corporation, limited partnership, limited liability company, or other form of business entity, with users or customers of that business, and the user or customer has specifically opted in to such communications after clear and conspicuous disclosure about their automated nature. Would provide further that this section does not impose a duty on service providers of online platforms, including, but not limited to, web hosting and internet service providers.
8. Provide a severability clause, as specified, and that the duties and obligations imposed by this chapter are cumulative with any other duties or obligation imposed by any other law.
9. Define various terms for these purposes, including the following:

• “Bot” to mean an online account that is designed to mimic or behave like the account of a natural person.
• “Online platform” to mean any public-facing internet website, web application, or digital application, including a social network or publication.
• “Person” to mean a natural person, corporation, limited liability company, partnership, joint venture, association, estate, trust, government, governmental subdivision or agency, or other legal entity or any combination thereof.

EXISTING LAW:

1. Provides, under the U.S. Constitution, that “Congress shall make no law . . . abridging the freedom of speech, or of the press, or the right of the people peaceably to assemble, and to petition the government for a redress of grievances.” (U.S. Const., 1st Amend., as applied to the states through the 14th Amendment’s Due Process Clause; see Gitlow v. New York (1925) 268 U.S. 652.)
2. Provides under the California Constitution for the right of every person to freely speak, write and publish his or her sentiments on all subjects, being responsible for the abuse of this right. Existing law further provides that a law may not restrain or abridge liberty of speech or press. (Cal. Const., art. I, Sec. 2(a).)
3. Provides as a matter of federal case law that the government may not, under the First Amendment to the United States Constitution, suppress political speech on the basis of the speaker’s corporate identity. (Citizens United v. FEC (2010) 130 S.Ct. 876.)
4. Generally protects consumers and competitors against false or misleading advertising. (Bus. & Prof. Code Sec. 17500 et seq.) Specifically, provides that it is unlawful for any person, firm, corporation or association, or any employee thereof, to make or disseminate before the public in this state, in any newspaper or other publication, or any advertising device, or in any other manner or means whatever, including over the internet, any statement concerning real or personal property or services, professional or otherwise, or concerning any circumstance or matter of fact connected with the proposed performance or disposition thereof, which is untrue or misleading, and which is known, or which by the exercise of reasonable care should be known, to be untrue or misleading. (Bus. & Prof. Code Sec. 17500; emphasis added.) As used in the false advertising chapter, “person” includes any individual, partnership, firm, association, or corporation. (Bus. & Prof. Code. Sec. 17506.)

FISCAL EFFECT: According to the Senate Appropriations Committee, pursuant to Senate Rule 28.8, any additional state costs are not significant, and do not and will not require the appropriation of additional state funds, and the bill will cause no significant reduction in revenues.

COMMENTS:

1. Purpose of this bill: This bill seeks to provide internet users the ability to distinguish between social media accounts owned and operated by “real” natural persons and “fake” persons – i.e., bot accounts impersonating a natural person – by requiring clear and conspicuous disclosures of “bot” or “auto-generated” accounts. This bill is sponsored by Common Sense Kids Action.
2. Author’s statement: According to the author, “There is currently no law that relates to the disclosure of the use of bot accounts on social media. SB 1001 sheds light on the fake accounts that simulate real people and spread waves of false information across social media platforms. On the Internet where the appearance of a mass audience can be monetized, it is critical to protect users by providing the tools to understand if their information is coming from a human or a bot account disguised as one. As long as bots are properly identified to let users know that they are a computer generated or automated account, users can at least be aware of who they are interacting with and judge the content accordingly.
3. Bots (essentially) have speech rights, too: This bill would declare it to be a violation for any person (meaning both natural persons and corporations or other legal entities) to use a bot, as defined, to communicate or interact with natural persons in California online, with the intention of misleading a natural person about its artificial identity. Under this bill, a person using a bot would be presumed to not act with the intent to mislead if the person discloses that the bot is not a natural person. A statement would be considered a disclosure for these purposes if it includes language stating that the bot “is a bot” or is “auto-generated.” The bill compels online platforms to engage in user-policing of violations by requiring online platforms to enable users to identify and report bots that the user suspects of violating such provisions. After receiving notice of a bot that is in violation, the bill requires an online platform to “expeditiously” investigate (i.e., within 72 hours) and determine whether or not to disclose that the bot is not a natural person or remove the bot.

Fundamentally, this bill, like others brought before the Legislature this year, triggers a debate surrounding the First Amendment issues implicated by bot regulations. As a matter of constitutional law, even if bots do not directly have speech rights, they are connected to individuals (or corporations) who do. Indeed, bots do not appear out of thin air; they are created by a natural person or entity for a purpose. Namely, they enable the creator to associate themselves with the bot’s “speech,” or rather, to express themselves, or speak, through the bot. Bot accounts on social media can have legitimate and beneficial uses, let alone less favorable but still constitutionally-protected ones. The content produced by bots on behalf of the speaker arguably does not lose its First Amendment protection merely because it is “spoken” through a bot.

Indeed, the Electronic Frontier Foundation (EFF) notes in opposition to this bill that “[b]ots often represent the speech of natural persons, processed through a computer program. The human speech underlying bots is protected by the First Amendment. It is also long settled that the computer code, including the code behind parody, art, or poetry bots on social media, is a form of protected speech under the First Amendment. Code consistently receives First Amendment protection because code, like a written musical score, ‘is an expressive means for the exchange of information and ideas.’”

That being said, not all speech is always protected speech: the First Amendment reflects a fundamental right protecting the freedom of speech and expression; it is not an absolute right. The determination about whether a specific statute inappropriately restricts speech can vary depending on whether the content regulated is a lesser protected type of speech. For example, speech that falls under the category of obscenity or incitement is not afforded the same protections as conversational speech about current events or religious speech or artistic expression. Nor is commercial speech afforded the same protection as political speech. The latter categories receive greater scrutiny in analysis and require a demonstration that the laws are sufficiently narrowly tailored to achieve a compelling governmental interest. Ultimately, when conducting a constitutional analysis of government restrictions on freedom of speech, the analysis invariably requires a determination as to whether the restriction is content-neutral or content-based; unduly vague or overbroad; and whether the restriction acts as a prior restraint on speech. At times, questions around government regulations may also relate to whether the statute unconstitutionally prohibits anonymous speech, or actually compels speech, as the First Amendment also protects the right to speech anonymously, and the right to stay silent.

Here, the question becomes whether this bill meets such constitutional standards, particularly as the underlying premise of this bill appears to be that any and all speech expressed by way of a bot is less trustworthy (or constitutes a fact that is relevant to its trustworthiness), or will inherently mislead people if the identity of the bot “speaker” is left undisclosed.

EFF argues in opposition to this bill, that SB 1001 does not meet such constitutional standards, noting the overbreadth of the bill and the chilling effect it would have on protected First Amendment activities: “SB 1001’s across the board bot-labeling mandate would sweep up all bots—including not only harmful bots, but also bots used for First Amendment protected activities. This includes poetry, political speech, parody, and satire—such as poking fun at people who cannot resist arguing, even with bots. By targeting all bots—instead of just the specific type of harmful bots driving the legislation—SB 1001 would chill the use of bots for such protected speech activities. The bill would, for example, restrict the speech of parodists or artists whose projects necessitate not disclosing that a bot is a bot.”

EFF argues that any bill attempting to address the harms of bots must be much more narrowly tailored than the current version of this bill:

As an organization dedicated to protecting the rights of all Internet users, we recognize that ill-intentioned bots, such as the Russian bots that interfered with the 2016 U.S. elections or spambots used for fraud or commercial gain, can cause real-life harm. But any law aimed at protecting against harmful bots must be crafted with precision—to avoid sweeping in lawful speech, to prevent the law from being abused as a tool for censoring or unmasking Internet users, and to ensure that the law will actually help prevent the harms it aims to protect against.

SB 1001 unfortunately fails in this regard. The bill applies to all “bots” on any public-facing websites, regardless of whether or not they cause harm, and will therefore sweep up perfectly lawful speech—such as speech from parody or artist accounts. As a result, the bill will not withstand First Amendment scrutiny. The bill also mandates a user-reporting system, ripe for abuse by those seeking to censor speech or unmask anonymous human speakers. Finally, due to the bill’s confusing definition of “bot”—“an online account that is designed to mimic or behave like the account of a natural person”—it would seem to apply to things like scheduled tweets, but not to auto-generated posts from fake organizational accounts, such as the Russian-controlled @Ten_GOP rogue account purporting to be the “unofficial Twitter of Tennessee Republicans.”

SB 1001 would silence or diminish the very voices it hopes to protect. Any law of this type, which restrains Internet speech, must be narrowly tailored to address proven harms; it is not appropriate to cast a wide net and sort it out later. If the California Legislature views this as an important issue meriting legislation, it should take the time to craft a carefully defined and thoughtfully tailored bill designed to address the precise harms the bill seeks to protect against. SB 1001 does not satisfy these criteria […]. (Internal footnote omitted.)

1. Content-neutral regulation versus content-based restrictions: The U.S. Supreme Court has frequently made clear that at the very core of the First Amendment is the principle that the government may not regulate speech based on its content, and that content-based restrictions are presumptively invalid. (See RAV v. City of St. Paul (1992) 505 U.S. 377, 382.) The concern here, with content-based restrictions, is that government will target particular messages it does not agree with, and attempt to control thoughts on a particular topic by targeting particular messages and regulating speech. “Laws of this sort pose the inherent risk that the Government seeks not to advance a legitimate regulator goal, but to suppress unpopular ideas or information or to manipulate the public debate through coercion rather than persuasion.” (Turner Broadcast System v. Federal Communication Commission (1994) 512 U.S. 622, 641.) As such, these types of content-based restrictions are subject to strict scrutiny (the highest level of scrutiny, requiring that the law be narrowly tailored to achieve a compelling governmental interest, where the government must have chosen the “least restrictive” means of achieving the compelling interest), whereas content-neutral regulations need meet intermediate scrutiny (a less stringent analysis, requiring that the law serve a significant state interest; be narrowly tailored so as not to burden substantially more speech than is necessary to protect that interest; and it leave open ample alternative means of communication). This general rule, however, does not apply to certain categories of speech that are considered unprotected or less protected by the First Amendment, such as obscenity, defamation, or even incitement, despite being content-based restrictions.

A law may be deemed content-based whether it is a subject matter regulation or viewpoint regulation. In other words, to be content-neutral, it must be neither of these things. A law regulating speech is content-neutral if it applies to all speech regardless of the message. A law that prohibits any speech about war would also be content-based, as a subject matter regulation, whereas a law prohibiting anti-war protests would be content-based as a viewpoint regulation. Similarly, a law making picketing illegal, except if it is labor picketing, would be content-based. In contrast, a law prohibiting the posting of all signs on public utility poles would be content-neutral.

On its face, this bill appears to be content neutral because it applies to all speech expressed through bots, regardless of their message. Nonetheless, because it applies to all speech expressed through bots, there does, as EFF has noted, appear to be a question relating to the overbreadth of the bill, which would pose a constitutional defect. (See Comment 5, below.)

1. Potential overbreadth issues and underinclusiveness concerns: First Amendment jurisprudence requires that any statute restricting speech not be vague such that a reasonable person cannot understand what speech is prohibited or permitted, or overly broad such that it regulates substantially more speech than the U.S. Constitution allows to be regulated. While there do not appear to be major vagueness issues on the face of this bill, there are open questions as to its overbreadth.

The major aspect to the overbreadth doctrine that is of relevance here, is that a law must not be substantially overbroad (i.e., restrict significantly more speech than the constitution allows to be controlled). To this end, substantial overbreadth can be demonstrated both by showing a significant number of situations where a law could prohibit constitutionally protected, speech, or by demonstrating great harm to particularly important speech. (See Chemerinsky, Constitutional Law Principles and Policies (2006) 3rd Edition, pp. 944-945, internal citations omitted.)

As indicated in Comment 4, above, this bill does not seek to restrict a particular subject matter or viewpoint; rather, it seeks to enact a restriction based on an underlying assumption that expression of speech through a bot makes it presumptively less trustworthy or more misleading, at least if the identity of that “artificial” speaker is not disclosed. In that regard, however, the bill captures potentially significantly more protected categories of speech than necessary, as it is focused on the perceived speaker (or mode of communication chosen by the speaker), not the nature of the speech itself.

On the one hand, there is an argument that the bill is focused on the misleading nature of the speech, because the user is not presented with all the information (the fact that the speech is being communicated through, if not generated by, an artificial vessel). To that end, however, it is unclear why the bill only seeks to make it unlawful for a person to use a bot to communicate or interact with natural persons in California online, with the “intent to mislead a natural person about its artificial identity.” (Emphasis added.) In other words, under this bill, it would still be permissible for a person to mislead a non-natural person (a legal entity, which falls under the definition of “person” as well) of its artificial identity. This might happen where a person sets up a bot to conduct false purchases online, by way of example.

Furthermore, if the argument is in fact that the bill is focused on the misleading nature of the speech, is unclear why the bill allows a person to use a bot to communicate or interact with a natural person in California online with the intent to mislead that natural person about any other fact other than its artificial identity. In that regard, this bill appears to be potentially both overly broad (in assuming that any speech is misleading if given through an artificial identity without disclosing the status of that artificial identity as such), and too narrow (by allowing for the same type of misleading speech as long as the “person” being misled is not a “natural” person, but rather a legal entity, and by allowing a person to use a bot to mislead others for any other reason unrelated to misleading a natural person about the artificial identity of the bot).

1. Possible avenue to more narrowly tailor the bill: Staff notes that to address the possible constitutional infirmities of this bill, discussed in Comment 5 above, clarification is needed as to whether, at the heart of this bill, the premise is that: (1) the artificial identity of a bot makes the speech inherently less trustworthy and potentially misleading; or, (2) certain communications made by persons through bots are intended to mislead and disclosures of such bot-made or bot-facilitated speech are intended to alert and inform users so that the users can more readily filter out those misleading communications from the non-misleading variety.

Assuming that the author’s underlying intent lies in the latter category, the author may wish to consider an amendment that would more clearly focus the bill on misleading speech in a narrowly tailored fashion. For example, the author could rephrase the prohibition such that it would be unlawful for any person to use a bot to communicate with or interact with another person in California, online, with the intent to mislead the person about its artificial identity for the purpose of knowingly deceiving the person about the content of the communication in order to incentivize a purchase or sale of goods or services in a commercial transaction or to influence a vote in an election. Stated another way, it would be unlawful for a person to mislead another person about their account’s bot identity in order to deceive that other person about the content of their communication for these narrow purposes which are related to commercial and political fraud. Such a bill would address more specific concerns about how political bots (whether created by natural persons or legal entities) are being used to somehow defraud the electorate and influence outcomes of elections, or how commercial bots could be used to defraud consumers (natural persons) or businesses (whether sole proprietorships or corporate legal entities) to elicit purchases or sales. Given their nexus to a more narrowly identifiable harm that the government has a compelling interest in preventing, these communications, if made the subject of the regulations, would presumably fall under a less protected category of First Amendment speech or activity (fraudulent commercial speech, for example) and more likely to survive a challenge in the courts.

The following suggested amendments would accomplish those objectives, and also make clear that a person using a bot would not be liable under this section if it discloses that it is a bot. Notably, this statement is different from disclosing that it is “not a natural person,” in recognition that a bot could be an online account mimicking a company as easily as it could mimic a natural person with the intent to deceive another person. Furthermore, this is not to say that misleading or fraudulent commercial or political activities that incentivize others to make purchases, sales, or votes based on those intentionally fraudulent representations would suddenly be made lawful if remedies can otherwise be pursued under other laws; it is only to say that the person would not be held liable for misleading the user about their “artificial identity” under this particular law. Lastly, these amendments would clarify that a “bot” is an automated online account (to narrow the bill to exclude “parody” accounts) that is designed to mimic or behave like the account of a person (again, as opposed to natural person).

Suggested amendment:

On page 2, line 8, before “online” insert “automated” and after “account” insert “on an online platform”

On page 2, line 9, strike “natural”

On page 2, line 23 strike “natural persons” and insert “another person”

On page 2, line 24, strike “intention of misleading a natural person” and insert “intent to mislead the other person”

On page 2, line 25, after “artificial identity” insert “for the purpose of knowingly deceiving the person about the content of the communication in order to incentivize a purchase or sale of goods or services in a commercial transaction or to influence a vote in an election.”

On page 2, lines 25-27, strike “A person using a bot is presumed to not act with the intent to mislead if the person discloses that the bot is not a natural person” and insert “A person using a bot shall not be liable under this section if the person discloses that it is a bot.”

On page 2, strike lines 31-33

Staff notes that regulating political speech may nonetheless be precarious given the heavy presumptions of unconstitutionality around what is considered to be at the core of the First Amendment and receives the highest form of protection. That being said, by requiring a narrow disclosure for the compelling interest of preserving the integrity of the electoral system in a content neutral fashion, such amendments may have a better likelihood of surpassing strict scrutiny from the courts.

Such amendments would draw a closer connection to what appears to be the underlying goal of the sponsor of this bill, Common Sense Kids Action, which writes:

Sixty-six percent of American teens use Facebook, 47% use Twitter, and 39% say social media is their preferred news source. They are at risk of being targeted with unlabeled bots on those platforms, with an estimated 60 million (Facebook) and 48 million (Twitter) unlabeled bot accounts. These bots are more likely to carry harmful messages: political propaganda, ethnic discrimination, or just false information. Educators and parents are concerned, as only 44% of children say they can distinguish false news online from real.

With so many children and teens online[,] families need confidence that they will be protected from unlabeled accounts that are spreading false information and harmful content.

1. Right of anonymous speech: Another context in which this bill may implicate free speech concerns, is in the context of compelled expression. There is inherent in the right of free speech, the right not to speak. As such, constitutional issues arise when the government seems to be forcing associational activities or when it forces people to use their property for speech by others. Of particular relevance, here, is that the right not to speak also includes a right not to disclose one’s identity when speaking. In 1960, the U.S. Supreme Court declared unconstitutional a ban on anonymous handbills in the case of Talley v. California, writing that “[p]ersecuted groups and sects from time to time throughout history have been able to criticize oppressive practices and laws either anonymously or not at all.” 362 U.S. 60, 64. (See also McIntyre v. Ohio Elections Commission (1995) 514 U.S. 334, declaring unconstitutional a law that prohibited the distribution of anonymous campaign literature. “The decision in favor of anonymity may be motivated by fear of economic or official retaliation, by concern about social ostracism, or merely by a desire to preserve as much of one’s privacy as possible…. Accordingly, an author’s decision to remain anonymous, like other decisions concerning omissions or additions to the content of a publication, is an aspect of the freedom of speech protected by the First Amendment.” Id. at 341-342.) While this bill requires labeling of bot-speech, it does not require labeling of the bot’s user. So the bot could presumably be identified as a bot, but left anonymous. A form of compelled speech, but arguably one that might be upheld in light of the compelling governmental interest, as suggested to be amended in Comment 6.

EFF, however, writes in opposition to this bill that while it may appear that anonymity would be preserved, the bill would in fact “reduce the ability of individuals to speak anonymously online” for the following reasons:

The bill will be hard to enforce in practice without unmasking anonymous human speakers. While merely labeling an account as a bot does not pierce anonymity, platforms may require identify verification in order for a human to challenge their decision to either takedown their account or label it as a bot. Those seeking to unmask anonymous speakers could abuse this system, forcing human speakers to decide between disclosing their identifies and being censored or diminished.

We also expect bot-labeling mandates to lead to demands for verification of whether or not an account is controlled by a bot, which will result in piercing anonymity.

Anonymous speech is not only fully protected by the First Amendment, but it constitutes an “honorable tradition of advocacy and of dissent.” As the Supreme Court has recognized, “Anonymity is a shield from the tyranny of the majority.” As a result, just as government mandates that one speak or publish, laws that infringe on the right to anonymity in “core political speech,” are subject to close judicial scrutiny.

Staff notes that such concerns are heightened by the fact that the bill allows for sharing of such information with the Attorney General upon request. (See Comment 9 for more.)

1. Does this bill compel censorship? To the extent that this bill encourages online platforms to remove bots as an option to address violation complaints, there is a possible concern that this bill compels censorship, with the online platform acting as its proxy by law. Interestingly, if an online platform chose to remove a bot on its own in the absence of a law, this would not be censorship. Of course, frequently, private individuals misguidedly complain about their First Amendment rights being trampled upon by other individuals or companies, but, in fact, the Constitution says nothing about the ability of private actors to regulate the speech of other private actors. Speech regulations, per se, become a constitutional problem when government attempts to regulate speech of private actors (natural persons and corporations alike, since the 2010 U.S. Supreme Court ruling in Citizens United v. FEC (2010) 130 S.Ct. 876.) However, here, the bill would establish a government regulation that arguably compels a private actor to censor the speech of another private actor. That being said, government has required platforms to remove content in other contexts, such as with laws that violate copyright. If amended as suggested in Comment 6, this bill would be more narrowly tailored to advancing a compelling governmental interest in preventing fraudulent commercial activities or political fraud. That is not to say there would not be unintended consequences.

On the issue of censorship, EFF writes in opposition,

SB 1001 would result in the takedown or mislabeling of lawful speech not only as a result of mistakes, but also as a result of abuse. Platforms’ content moderation policies are widely abused in order to censor speech. Those seeking to censor speech have become experts at figuring out precisely how to use platforms’ policies in order to silence or otherwise discredit their opponents on social media platforms. Platforms are, on the whole, terrible at enforcing their content moderation policies, and online trolls and others who wish to silence dissenting viewpoints take advantage of this: online trolls figure out which key indicators will trigger a platform to take something down or flag it as false or unreliable, and then report legitimate posts that contain these indicators. There is no reason to think they won’t figure out how to game SB 1001’s system as well.

The targets of such abuse have been the sorts of voices the supporters of SB 1001 would likely want to protect—including Muslim civil rights leaders, pro-democracy activists in Vietnam, and Black Lives Matter activists, whose posts have been censored due to efforts by white supremacists.

Platforms have proven that they cannot adequately enforce their own self-imposed content moderation policies to prevent abuse and censorship of legitimate speech. Their track record is even worse when it comes to preventing abuse of legally mandated takedown policies. Take, for example, the federal Digital Millennium Copyright Act (DMCA), which requires platforms to remove content accused of violating copyright. Platforms generally err on the side of caution, automatically complying with even absurd takedown requests, rather than risk legal penalties. This has made copyright law a tempting tool for unscrupulous censors. EFF has seen abusive DMCA takedown notices from a would-be Senate candidate, small businesses, and even Ecuador’s President.

Like the DMCA, SB 1001 creates a quick and easy way to make speech disappear from the Internet without clear standards or meaningful recourse, and it will result in platforms acting first and asking questions later. Platforms have little incentive to complete a thorough investigation into whether a reported account is actually operated by a bot—particularly given that (a) SB 1001 requires platforms to complete their investigation in under 72 hours, and (b) taking an account offline or marking it as a bot would quickly and easily fulfill the platform’s legal obligations under the bill.

Supporters of SB 1001 have argued that the bill will not necessarily result in the takedown of legitimate speech, because platforms have the option of labeling accounts as bots. But being incorrectly labeled as a bot could, by diminishing a person’s speech, be just as harmful to someone’s ability to speak as taking down their account or forbidding them from speaking in the first place; people simply may not listen to a human labeled as a bot. (Internal citations omitted.)

1. Suggested amendments to help reduce concerns regarding censorship and loss of anonymity: While the following amendments would not necessarily address all unintended consequences or eliminate all concerns, the author may wish to consider amending the bill to: (1) eliminate the de facto 72 hour limit for online platforms to conduct their investigation to instead allow for reasonable investigations, and make other clarifying changes; (2) require an avenue for persons to challenge the determination that they are an unlabeled bot in violation of the bill and subject to labeling or deletion, prior to being labeled or permanently deleted; and (3) ensure that the online platform does not share the identity of the bot owner with any third party (including law enforcement or the Attorney General) without the prior informed consent of the bot user.

Suggested amendments:

1. On page 2, line 34-35, strike “identify and report bots that” and insert “report if”

On page 2, line 35, after “suspects” strike “of violation” and insert “a bot on that platform to be in violation of”

On page 2, line 36, before “bot” insert “suspected”

On page 2, line 37, strike “expeditiously investigate” and insert “conduct a reasonable investigation”

On page 3, lines 1-2, strike “to disclose that the bot is not a natural person or remove the bot” and insert “a disclosure is necessary that the account is a bot, or remove the bot”

1. On page 3, strike lines 3-4 and insert “An online platform shall notify and provide enable the user of a suspected bot pursuant to this section a reasonable opportunity to respond prior to labeling or removing the suspected bot.”
2. On page 3, strike lines 5-8 and insert, “Any records maintained by an online platform in response to notice received pursuant to this section shall not be disclosed with any third party, including law enforcement or the Attorney General, without the prior informed consent of the person using the bot or court order.”
3. Clear and conspicuous disclosure requirements: Under this bill, a person who uses a bot to communicate or interact with a natural person has to provide that natural person with a disclosure that the bot is not a natural person to avoid misleading that other person about the artificial identity of the bot. The bill specifies that the disclosure that the bot is not a natural person must be clear and conspicuous to all natural persons with whom the bot communicates or interacts. To be clear, this requirement is distinct from a requirement that the disclosure be clear and conspicuous to the average or reasonable person with whom the bot communicates or interacts. If it is not clear and conspicuous to all (i.e., each) natural person with whom the bot actually communicates or interacts, the person will have violated this proposed statute. Of course, this bill does specify the types of statements that will be considered to be made in a clear and conspicuous manner for text and graphic communications, audio communications, and video communications, and also provides that for other types of communications, a statement will be considered to be made in a clear and conspicuous manner if it is made in a manner that is at least as clear and conspicuous as described in those other three contexts. Each of those specific standards, however, while providing helpful guidance, are largely subjective. Accordingly, again, if any single user finds that it was not clear and conspicuous to them, the person providing the disclosure will have failed to meet the standard under the bill.

The following amendment is suggested to address these concerns by inserting a reasonableness element into the standard and removing the requirement that the disclosure be clear and conspicuous to “all” persons:

Suggested amendment:

On page 2, strike lines 28-30 and insert “The disclosure required by this section shall be clear, conspicuous, and reasonably designed to inform persons with whom the bot communicates or interacts that it is a bot.”

In light of other concerns raised by various business and technology stakeholders, led by the Chamber of Commerce, relating to the “clear and conspicuous disclosure” requirements of the bill, the following additional amendments would remove the remaining “clear and conspicuous” disclosure provisions of this bill. These amendments would also correct a drafting error.

Suggested amendment:

On page 3, strike lines 9-31, inclusive

On page 3, strike lines 39-40, inclusive

On page 4, strike lines 1-5, inclusive

On page 5, strike “(d) This section” and insert “(c) This chapter”

1. Related legislation: AB 1950 (Levine, 2018), as introduced, would have prohibited an operator of a social media site with a physical presence in California from selling advertising to a computer software account or user that performs an automated task and is not verified as being controlled by a natural person. That bill was eventually gutted and amended into a bill related to the California Online Protection Privacy Act and was held in Committee.

AB 3169 (Gallagher, 2018) would have prohibited any person who operates a social media internet website or search engine located in California, as specified, from removing or manipulating content on the basis of the political affiliation or political viewpoint of that content. That bill failed passage in this Committee on April 3, 2018.

1. Double-referral: This bill is double-referred to the Arts, Entertainment, Sports, Tourism, & Internet Media Committee, where it will be heard if passed by this Committee.

REGISTERED SUPPORT / OPPOSITION:

Support

Common Sense Kids Action (sponsor)

Opposition

California Chamber of Commerce

California Grocers Association

CompTIA

CTIA

Data and Marketing Association

Electronic Frontier Foundation

Internet Association

Internet Coalition

TechNet

Analysis Prepared by: Ronak Daylami / P. & C.P. / (916) 319-2200

 THIRD READING  

Bill No: SB 1001

Author:

:

Vote:

SENATE BUS., PROF. & ECON. DEV. COMMITTEE: 8-1, 4/16/18

AYES: Hill, Dodd, Galgiani, Glazer, Hernandez, Newman, Pan, Wilk

NOES: Fuller

SENATE JUDICIARY COMMITTEE: 5-1, 4/24/18

AYES: Jackson, Hertzberg, Monning, Stern, Wieckowski

NOES: Anderson

NO VOTE RECORDED: Moorlach

SENATE APPROPRIATIONS COMMITTEE: Senate Rule 28.8

SUBJECT:

SOURCE: Author

DIGEST:

This bill prohibits the use of a bot to communicate or interact with a natural person in California online with the intention of misleading a natural person about its artificial identity, as specified. This bill also requires online platforms to take actions including: (1) enabling users to report bots violating those provisions; (2) investigating and determining within 72 hours whether to act upon reports received by users; and (3) providing details of user reports and internal investigations to the Attorney General (AG), upon request.

Senate Floor Amendments of 5/25/18 address concerns raised by opponents by further clarifying definitions and the AG reporting requirements in the bill.

ANALYSIS:

Existing law:

1. Prohibits any person, firm, corporation or association, or any employee thereof, from making statements that are untrue or misleading in connection with communications regarding the sale of goods or services, including over the Internet. (Business and Professions Code (BPC) Section (§) 17500)
2. Prohibits a person, with a bad faith intent, to register, traffic in, or use a domain name that is identical or confusingly similar to the name of another living or deceased person. (BPC § 17525)
3. Prohibits any person or entity from sending a commercial e-mail advertisement from California or to a California electronic mail address if the e-mail advertisement contains falsified, misrepresented, or forged header information or if the e-mail advertisement has a misleading subject line. Permits the AG, among other parties, to bring action against a person or entity that violates these provisions. (BPC §17529.5)
4. Prohibits a person or entity located within California from using a telephone facsimile machine, computer, or other device to send an unsolicited advertisement to a telephone facsimile machine. (BPC § 17538.43 (b)(1))
5. Requires a person or entity located in California to clearly mark the date and time of transmittal and identity and telephone number of the person or entity sending the message of any communication via a telephone facsimile machine.
   (BPC § 17538.43 (c)(1))
6. Requires individuals or partnerships doing business under fictitious names to file a fictitious business name certificate designed to make the identities of the persons doing business under the fictitious names available to the public. (BPC § 17900)

This bill:

1. Defines “bot” as an online account that is designed to mimic or behave like the account of a natural person.
2. Defines “online” as appearing on any public-facing Internet Web site, Web application, or digital application, including a social network or publication.
3. Defines “online platform” as a public-facing Internet Web site, Web application, or digital application, including a social network or publication
4. Defines “person” as a natural person, corporation, limited liability company, partnership, joint venture, association, estate, trust, government, governmental subdivision or agency or other legal entity of any combination thereof.
5. Prohibits a person from using a bot to communicate or interact with a natural person in California online with the intention of misleading a natural person about its artificial activity and specifies that a person is presumed to not be acting with the intent to mislead if a person discloses that a bot is not a natural person.
6. Specifies that the disclosure required must be clear and conspicuous to all natural persons with whom a bot communicates or interacts with and that the disclosure must include language stating that the bot “is a bot” or is “auto-generated.”
7. Requires online platforms to enable users to identify and report bots that users suspect of not being properly disclosed.
8. Requires online platforms to investigate reports made by users and determine whether or not to disclose or remove the bot within 72 hours of receipt of a report.
9. Requires online platforms to provide the AG with reports detailing notices received and actions taken in response, upon request.
10. Defines the criteria for a statement to be considered to be made in a clear and conspicuous manner for text and graphic communications, audio communications, video communications, and any other communications.
11. Defines that a statement is not made in a clear and conspicuous manner if it is difficult to read or hear or if the placement is easily overlooked.
12. Specifies that the duties and obligations imposed by this bill are cumulative with any other duties or obligations imposed by any other law and do not apply to bots that are communicating on behalf of a corporation, limited partnership, limited liability company, or other form of business entity, with customers of that business who have opted in to such communication.
13. Specifies that the provisions of this bill are severable and do not impose a duty on service providers of online platforms, including Web hosting and Internet service providers.

Background

Bots. Bots are software applications that run automated tasks on a network, such as the Internet, that can interact with computers systems or users. Bots can be purchased or created by individuals or organizations and many online platforms, including social media sites, create bots themselves and encourage users to build bots for use on their platforms. When interacting directly or indirectly with bots online, human users may not be able to discern if they are interacting with a bot or another human. For example, a human Twitter user may assume a Tweet was posted by another human when in actuality the content was generated by a bot. The latter type of interaction has become increasingly common as the number of automated users on social media sites continues to grow. A 2017 study by the University of Southern California and Indiana University found that an estimated 48 million users, or 15 percent of Twitter’s active users at the time, were automated accounts. In November 2017, Facebook disclosed to investors that up to 60 million bots may be on the site.

Utilization of bots online. Bots, particularly those that are automated accounts on social media sites, can be used to amplify the perceived popularity of people, products, and services online. In January 2018, the New York Times published an article, “The Follower Factory,” detailing an emerging market where those seeking to bolster their online presence can purchase followers, likes, and other engagements for their social media accounts from companies that control a large number of bots. One such company highlighted by The Times, Devumi, has reportedly sold Twitter followers to over 200,000 customers, including reality television starts, professional athletes, and politicians. To do so, Devumi draws upon its approximately 3.5 million bot accounts, at least 55,000 of which use the names, profile pictures, and other personal details of real Twitter users, representing “a kind of large-scale social identity theft.”

Just as companies like Devumi use bots to promote their customer’s accounts, individuals and groups can, and have, used bots to spread and amplify divisive views on social media platforms. For example, in February 2018, the U.S. Justice Department charged 13 Russians and three companies with creating fake profiles on social media sites to post contentious comments related to religion, race, and politics and then using bot accounts to like, share, and retweet the posts to help them gain traction during and after the 2016 Presidential election. Information provided to U.S. Congress on October 20, 2017 indicated that Russian agents published more than 131,000 messages on Twitter, uploaded over 1,000 videos to YouTube, and disseminated inflammatory posts that reached 126 million users on Facebook.

There have recently been efforts by other states and the federal government to regulate the use of bots and prevent the spread of misinformation and false advertising by automated accounts. However, there is currently no state that requires the disclosure of bots.

Comments

According to the author, “bots can be – and are often – weaponized to spread fake and misleading news, reshape political debates, and influence advertising audiences…SB 1001 sheds light on the fake accounts that simulate real people and spread waves of false information across social media platforms. On the Internet where the appearance of a mass audience can be monetized, it is critical to protect users by providing the tools to understand if their information is coming from a human or a bot account disguised as one. As long as bots are properly identified to let users know that they are a computer generated or automated account, users can at least be aware of who they are interacting with and judge the content accordingly.”

FISCAL EFFECT: Appropriation: Fiscal Com.: Local:

SUPPORT: (Verified 5/29/18)

Common Sense Kids Action

OPPOSITION: (Verified 5/29/18)

California Chamber of Commerce

California Grocers Association

CompTIA

CTIA

Data and Marketing Association

Electronic Frontier Foundation

Internet Association

Internet Coalition

TechNet

ARGUMENTS IN SUPPORT: Common Sense Kids Action writes in support, noting that teens and children are “at risk of being targeted with unlabeled bots on [social media] platforms” which “are more likely to carry harmful messages: political propaganda, ethnic discrimination, or just false information.” They argue that “with so many children and teens online; families need confidence that they will be protected from unlabeled accounts that are spreading false information and harmful content. SB 1001 allows users to report these unlabeled bots and requires entities behind bots to clearly identify automated accounts.”

ARGUMENTS IN OPPOSITION: The California Chamber of Commerce, California Grocers Association, CompTIA, CTIA, Data and Marketing Association, Internet Association, Internet Coalition, and TechNet write in opposition, stating that while SB 1001 attempts to prevent specific problematic behaviors behind the use of certain types of bots, “aspects of the bill are vague, and the bill places onerous burdens on the operators of websites and applications.” Opponents argue that the requirements of this bill “are burdensome and could diminish the functionality and the speed of impacted websites, while failing to stop bad actors.”

Prepared by:

5/29/18 17:44:34

**** END ****

 THIRD READING  

Bill No: SB 1001

Author:

:

Vote:

SENATE BUS., PROF. & ECON. DEV. COMMITTEE: 8-1, 4/16/18

AYES: Hill, Dodd, Galgiani, Glazer, Hernandez, Newman, Pan, Wilk

NOES: Fuller

SENATE JUDICIARY COMMITTEE: 5-1, 4/24/18

AYES: Jackson, Hertzberg, Monning, Stern, Wieckowski

NOES: Anderson

NO VOTE RECORDED: Moorlach

SENATE APPROPRIATIONS COMMITTEE: Senate Rule 28.8

SUBJECT:

SOURCE: Author

DIGEST:

This bill prohibits the use of a bot to communicate or interact with a natural person in California online with the intention of misleading a natural person about its artificial identity, as specified. This bill also requires online platforms to take actions including: (1) enabling users to report bots violating those provisions; (2) investigating and determining within 72 hours whether to act upon reports received by users; and (3) providing details of user reports and internal investigations to the Attorney General (AG).

ANALYSIS:

Existing law:

1. Prohibits any person, firm, corporation or association, or any employee thereof, from making statements that are untrue or misleading in connection with communications regarding the sale of goods or services, including over the Internet. (Business and Professions Code (BPC) Section (§) 17500)
2. Prohibits a person, with a bad faith intent, to register, traffic in, or use a domain name that is identical or confusingly similar to the name of another living or deceased person. (BPC § 17525)
3. Prohibits any person or entity from sending a commercial e-mail advertisement from California or to a California electronic mail address if the e-mail advertisement contains falsified, misrepresented, or forged header information or if the e-mail advertisement has a misleading subject line. Permits the AG, among other parties, to bring action against a person or entity that violates these provisions. (BPC §17529.5)
4. Prohibits a person or entity located within California from using a telephone facsimile machine, computer, or other device to send an unsolicited advertisement to a telephone facsimile machine. (BPC § 17538.43 (b)(1))
5. Requires a person or entity located in California to clearly mark the date and time of transmittal and identity and telephone number of the person or entity sending the message of any communication via a telephone facsimile machine.
   (BPC § 17538.43 (c)(1))
6. Requires individuals or partnerships doing business under fictitious names to file a fictitious business name certificate designed to make the identities of the persons doing business under the fictitious names available to the public. (BPC § 17900)

This bill:

1. Defines “bot” as an online account that is either automated or designed to mimic or behave like the account of a natural person.
2. Defines “online” as any public-facing Internet Web site, web application, or digital application, including a social network or publication.
3. Defines “online platform” as a public-facing Internet Web site, web application, or digital application, including a social network or publication that has 50 million or more unique monthly United States visitors for a majority of the preceding 12 months.
4. Defines “person” as a natural person, corporation, limited liability company, partnership, joint venture, association, estate, trust, government, governmental subdivision or agency or other legal entity of any combination thereof.
5. Prohibits a person from using a bot to communicate or interact with a natural person in California online with the intention of misleading a natural person about its artificial activity and specifies that a person is presumed to be acting with the intent to mislead if a person fails to disclose that a bot is not a natural person.
6. Specifies that the disclosure required must be clear and conspicuous to all natural persons with whom a bot communicates or interacts with and that the disclosure must include language stating that the bot “is a bot” or is “auto-generated.”
7. Requires online platforms to enable users to identify and report bots that users suspect of not being properly disclosed.
8. Requires online platforms to investigate reports made by users and determine whether or not to disclose or remove the bot within 72 hours of receipt of a report.
9. Requires online platforms to provide the AG with bimonthly reports detailing notices received and actions taken in response.
10. Defines the criteria for a statement to be considered to be made in a clear and conspicuous manner for text and graphic communications, audio communications, video communications, and any other communications.
11. Defines that a statement is not made in a clear and conspicuous manner if it is difficult to read or hear or if the placement is easily overlooked.
12. Specifies that the duties and obligations imposed by this bill are cumulative with any other duties or obligations imposed by any other law and do not apply to bots that are communicating on behalf of a corporation, limited partnership, limited liability company, or other form of business entity, with customers of that business who have opted in to such communication.
13. Specifies that the provisions of this bill are severable and do not impose a duty on service providers of online platforms, including web hosting and Internet service providers.

Background

Bots. Bots are software applications that run automated tasks on a network, such as the Internet, that can interact with computers systems or users. Bots can be purchased or created by individuals or organizations and many online platforms, including social media sites, create bots themselves and encourage users to build bots for use on their platforms. When interacting directly or indirectly with bots online, human users may not be able to discern if they are interacting with a bot or another human. For example, a human Twitter user may assume a Tweet was posted by another human when in actuality the content was generated by a bot. The latter type of interaction has become increasingly common as the number of automated users on social media sites continues to grow. A 2017 study by the University of Southern California and Indiana University found that an estimated 48 million users, or 15 percent of Twitter’s active users at the time, were automated accounts. In November 2017, Facebook disclosed to investors that up to 60 million bots may be on the site.

Utilization of bots online. Bots, particularly those that are automated accounts on social media sites, can be used to amplify the perceived popularity of people, products, and services online. In January 2018, the New York Times published an article, “The Follower Factory,” detailing an emerging market where those seeking to bolster their online presence can purchase followers, likes, and other engagements for their social media accounts from companies that control a large number of bots. One such company highlighted by The Times, Devumi, has reportedly sold Twitter followers to over 200,000 customers, including reality television starts, professional athletes, and politicians. To do so, Devumi draws upon its approximately 3.5 million bot accounts, at least 55,000 of which use the names, profile pictures, and other personal details of real Twitter users, representing “a kind of large-scale social identity theft.”

Just as companies like Devumi use bots to promote their customer’s accounts, individuals and groups can, and have, used bots to spread and amplify divisive views on social media platforms. For example, in February 2018, the U.S. Justice Department charged 13 Russians and three companies with creating fake profiles on social media sites to post contentious comments related to religion, race, and politics and then using bot accounts to like, share, and retweet the posts to help them gain traction during and after the 2016 Presidential election. Information provided to U.S. Congress on October 20, 2017 indicated that Russian agents published more than 131,000 messages on Twitter, uploaded over 1,000 videos to YouTube, and disseminated inflammatory posts that reached 126 million users on Facebook.

There have recently been efforts by other states and the federal government to regulate the use of bots and prevent the spread of misinformation and false advertising by automated accounts. However, there is currently no state that requires the disclosure of bots.

Comments

According to the author, “bots can be – and are often – weaponized to spread fake and misleading news, reshape political debates, and influence advertising audiences…SB 1001 sheds light on the fake accounts that simulate real people and spread waves of false information across social media platforms. On the Internet where the appearance of a mass audience can be monetized, it is critical to protect users by providing the tools to understand if their information is coming from a human or a bot account disguised as one. As long as bots are properly identified to let users know that they are a computer generated or automated account, users can at least be aware of who they are interacting with and judge the content accordingly.”

FISCAL EFFECT: Appropriation: Fiscal Com.: Local:

SUPPORT: (Verified 5/23/18)

Common Sense Kids Action

OPPOSITION: (Verified 5/24/18)

California Chamber of Commerce

California Grocers Association

CompTIA

CTIA

Data and Marketing Association

Electronic Frontier Foundation

Internet Association

Internet Coalition

TechNet

ARGUMENTS IN SUPPORT: Common Sense Kids Action writes in support, noting that teens and children are “at risk of being targeted with unlabeled bots on [social media] platforms” which “are more likely to carry harmful messages: political propaganda, ethnic discrimination, or just false information.” They argue that “with so many children and teens online; families need confidence that they will be protected from unlabeled accounts that are spreading false information and harmful content. SB 1001 allows users to report these unlabeled bots and requires entities behind bots to clearly identify automated accounts.”

ARGUMENTS IN OPPOSITION: The California Chamber of Commerce, California Grocers Association, CompTIA, CTIA, Data and Marketing Association, Internet Association, Internet Coalition, and TechNet write in opposition, stating that while SB 1001 attempts to prevent specific problematic behaviors behind the use of certain types of bots, “aspects of the bill are vague, and the bill places onerous burdens on the operators of websites and applications.” Opponents argue that the requirements of this bill “are burdensome and could diminish the functionality and the speed of impacted websites, while failing to stop bad actors.”

Prepared by:

5/24/18 13:51:38

**** END ****

SENATE JUDICIARY COMMITTEE

Senator Hannah-Beth Jackson, Chair

2017-2018 Regular Session

Bill No: SB 1001 (Author:Hertzberg)

Version: March 14, 2018

Hearing Date: April 24, 2018

Fiscal: Yes

Urgency: No

Consultant: CK

SUBJECT

Bots: disclosure

DESCRIPTION

This bill would make it unlawful for any person to use a bot, as defined, to communicate or interact with natural persons in California online, with the intention of misleading and without clearly and conspicuously disclosing that the bot is not a natural person, as specified. The bill would require an online platform to enable users to report violations of this prohibition, to expeditiously respond to those reports, and to provide the Attorney General with regular reports on such reported violations and the associated responses, as specified.

BACKGROUND

According to data from Internet security firm Imperva, “bots” are responsible for more web traffic than humans, a trend that has remained relatively consistent over recent years. (Adrienne Lafrance, The Internet is Mostly Bots (Jan. 31, 2017) <https://www. theatlantic.com/technology/archive/2017/01/bots-bots-bots/515043/> [as of Apr. 7, 2018].) There are numerous types of bots with various functions. Unfortunately, bots considered “harmful” have the edge over those considered “helper bots.”

Some bots are virtually invisible as they carry out their work behind the scenes of search engines and other Internet Web sites. Some are easier to spot. However, whether or not it is apparent, bots and people regularly interact online. For instance, “[s]pambots show up in comment sections and Twitter bots clog people’s timelines with everything from marketing, to political campaigning, to social activism, to utter nonsense.” (Id.) “Social media make it easy for accounts controlled by hybrid or automated approaches to create content and interact with other accounts.” (Onur Varol, et al., Online Human-Bot Interactions: Detection, Estimation, and Characterization (Mar. 27, 2017) <https://arxiv.org/pdf/1703.03107.pdf> [as of Apr. 6, 2018].)

The widespread use of such bots raises some troubling issues. This bill seeks to address one of the fundamental concerns underlying the use of bots online, the deception of their human counterparts. This bill would make it unlawful to use a bot to communicate or interact with Californians online, with the intention of misleading and without clearly and conspicuously disclosing that the bot is not human. To advance the goal of rooting out deceptive bots, this bill would also provide users the ability to identify and report bots in violation to large-scale online platforms where they are found. Such platforms would be required to expeditiously investigate such notices from users and provide the Attorney General with information concerning the alleged violators and the ensuing investigations and response.

This bill passed out of the Senate Business, Professions and Economic Development Committee on an 8-1 vote.

CHANGES TO EXISTING LAW

Existing law defines “unfair competition” to mean and include any unlawful, unfair or fraudulent business act or practice and unfair, deceptive, untrue, or misleading advertising and any act prohibited by Chapter 1 (commencing with Section 17500) of Part 3 of Division 7 of the Business and Professions Code. (Bus. & Prof. Code Sec. 17200.)

Existing law provides that any person who engages, has engaged, or proposes to engage in unfair competition may be enjoined in any court of competent jurisdiction. The court may make such orders or judgments, including the appointment of a receiver, as may be necessary to prevent the use or employment by any person of any practice which constitutes unfair competition or as may be necessary to restore to any person in interest any money or property, real or personal, which may have been acquired by means of such unfair competition. Any person may pursue representative claims or relief on behalf of others only if the claimant meets the standing requirements of Business and Professions Code Section 17204 and complies with Section 382 of the Code of Civil Procedure, but these limitations do not apply to claims brought under this chapter by the Attorney General, or any district attorney, county counsel, city attorney, or city prosecutor in this state. (Bus. & Prof. Code Sec. 17203.)

Existing law requires actions for relief pursuant to the UCL be prosecuted exclusively in a court of competent jurisdiction and only by the following:

• the Attorney General;
• a district attorney;
• a county counsel authorized by agreement with the district attorney in actions involving violation of a county ordinance;
• a city attorney of a city having a population in excess of 750,000;
• a city attorney in a city and county;
• a city prosecutor in a city having a full-time city prosecutor in the name of the people of the State of California upon their own complaint or upon the complaint of a board, officer, person, corporation, or association with the consent of the district attorney; or
• a person who has suffered injury in fact and has lost money or property as a result of the unfair competition. (Bus. & Prof. Code Sec. 17204.)

Existing law, the False Advertising Law, states that it is unlawful for any person, firm, corporation or association, or any employee thereof with intent directly or indirectly to dispose of real or personal property or to perform services, professional or otherwise, or to induce the public to enter into any obligation relating thereto, to make or disseminate or cause to be made or disseminated before the public in this state, or to make or disseminate or cause to be made or disseminated from this state before the public in any state, in any newspaper or other publication, or any advertising device, or by public outcry or proclamation, or in any other manner or means whatever, any statement, concerning that real or personal property or those services, professional or otherwise, or concerning any circumstance or matter of fact connected with the proposed performance or disposition thereof, which is untrue or misleading, as specified. (Bus. & Prof. Code Sec. 17500.)

Existing law specifies that a violation of this provision is a misdemeanor punishable by imprisonment in the county jail not exceeding six months, or by a fine not exceeding two thousand five hundred dollars ($2,500), or by both that imprisonment and fine. (Bus. & Prof. Code Sec. 17500, 17534.) Existing law further states that any person who violates this provision shall be liable for a civil penalty not to exceed $2,500 for each violation, which shall be assessed and recovered in a civil action brought in the name of the people of the State of California by the Attorney General or by any district attorney, county counsel, or city attorney in any court of competent jurisdiction. (Bus. & Prof. Code Sec. 17536.)

Existing law specifies that any person, corporation, or organization, as specified, which violates or proposes to violate the False Advertising Law, may be enjoined by any court of competent jurisdiction, and that a court may make such orders or judgments as may be necessary to prevent the use or employment of practices which violate the False Advertising Law, or which may be necessary to restore to any person in interest any money or property, real or personal, which may have been acquired by means of any practice in the False Advertising Law declared to be unlawful, as specified. (Bus. & Prof. Code Sec. 17535.)

Existing law specifies that actions for injunction under the False Advertising Law may be prosecuted by the Attorney General or any district attorney, county counsel, city attorney, or city prosecutor in this state in the name of the people of the State of California upon their own complaint or upon the complaint of any board, officer, person, corporation or association or by any person who has suffered injury in fact and has lost money or property as a result of a violation of this law. Existing law specifies that any person may pursue representative claims or relief on behalf of others only if the claimant meets the standing requirements of this section, as specified, but that these limitations do not apply to claims brought by the Attorney General, or any district attorney, county counsel, city attorney, or city prosecutor in this state. (Bus. & Prof. Code Sec. 17535.)

This bill would provide the following definitions:

• “bot” means a machine, device, computer program, or other computer software that is designed to mimic or behave like a natural person such that a reasonable natural person is unable to discern its artificial identity;
• “online” means any public-facing Internet Web site, web application, or digital application, including a social network or publication;
• “online platform” means any public-facing Internet Web site, web application, or digital application, including a social network or publication, that has 50,000,000 or more unique monthly United States visitors or users for a majority of months during the preceding 12 months; and
• “person” means a natural person, corporation, limited liability company, partnership, joint venture, association, estate, trust, government, governmental subdivision or agency, or other legal entity or any combination thereof.

This bill would make it unlawful for any person to use a bot to communicate or interact with natural persons in California online, with the intention of misleading and without clearly and conspicuously disclosing that the bot is not a natural person. Such disclosure would need to be clear and conspicuous to all natural persons with whom the bot communicates or interacts. Language stating the bot “is a bot” or is “auto-generated” would suffice as such a disclosure.

This bill would require an online platform to enable users to identify and report bots that the user suspects of violating the above provisions. Upon receiving such a report, the online platform would be required to expeditiously investigate and determine whether or not to disclose that the bot is not a natural person or remove the bot. Such an investigation and response would be deemed expeditious if it occurred within 72 hours of receipt of the notice.

This bill would require an online platform to provide bimonthly reports to the Attorney General detailing notices received and actions taken in response.

This bill would provide that for text and graphic communications, a statement shall be considered to be made in a clear and conspicuous manner if it is made in larger type than the surrounding text, is in a contrasting type, font, or color to the surrounding text of the same size, or is set off from the surrounding text of the same size by symbols or other marks in a manner that clearly calls attention to the language.

This bill would provide that for audio communications, a statement shall be considered to be made in a clear and conspicuous manner if it is spoken in a clearly audible and intelligible manner at the beginning of the communication.

This bill would provide that for video communications, a statement shall be considered to be made in a clear and conspicuous manner if it is in a written format that meets the requirements for text and graphic communications above and appears for at least four seconds and in an audible format that meets the requirements for audio communications above.

This bill would provide that for any other communications, a statement shall be considered to be made in a clear and conspicuous manner if it is made in a manner that is at least as clear and conspicuous as described in the preceding provisions.

This bill would provide that a statement is not made in a clear and conspicuous manner if it is difficult to read or hear or if the placement is easily overlooked.

This bill would provide that the duties and obligations imposed by this chapter are cumulative with any other duties or obligation imposed by any other law. It would further provide that its provisions are severable, and if any provision or its application is held invalid, that invalidity shall not affect other provisions or applications that can be given effect without the invalid provision or application.

COMMENT

1. Stated need for the bill

According to the author:

Bots can be -and are often- weaponized to spread fake and misleading news, reshape political debates, and influence advertising audiences. Social media companies do not identify the presence of bot accounts, which masquerade as unique individuals. These bot accounts can be purchased to inflate followers for celebrities, businesses or anyone who wants to create the appearance of a large following or have influence online. Social media platforms are highly influential to users, young and old, and can even drive traditional media cycles. This influence has been used to negatively impact our democracy. Bots can be used in large numbers to misrepresent public sentiments and perceptions about topics, or to mute dissenting opinions and distract from current events. On the Internet where the appearance of a mass audience can be monetized, it is critical to protect users by providing the tools to understand if their information is coming from a human or a bot account disguised as one. Bots, when properly identified, can exist positively in the social media ecosystem.

2. Covert armies of Internet bots are deployed online

1. The troubling utilization of bots to drive social media agendas

Although bots can be found throughout the Internet, their interplay with humans is nowhere more pronounced than on social media platforms. According to research conducted by the Center for Complex Networks and Systems Research at Indiana University and the Information Sciences Institute at the University of Southern California, “[i]ncreasing evidence suggests that a growing amount of social media content is generated by autonomous entities known as social bots.” (Onur Varol, et al., Online Human-Bot Interactions: Detection, Estimation, and Characterization (Mar. 27, 2017) <https://arxiv.org/pdf/ 1703.03107.pdf> [as of Apr. 6, 2018].) Their estimates suggest that as many as 15 percent of Twitter accounts are fake. This would mean roughly 48 million Twitter users are bots, automated to simulate real people. Facebook itself disclosed to its investors that the social media platform had “at least twice as many fake users as it previously estimated, indicating that up to 60 million automated accounts may roam the world’s largest social media platform.” (Nicholas Confessore, et al., The Follower Factory (Jan. 27, 2018) The New York Times <https://www.nytimes.com/ interactive/2018/01/27/ technology/social-media-bots.html> [as of Apr. 6, 2018].)

A recent New York Times investigation explored the shadowy underworld of businesses that create social media bot armies and sell their services to the world. (Id.) One company that was highlighted, Devumi, had an estimated stock of at least 3.5 million automated accounts that the company sold many times over. The investigation found the company provided more than 200 million Twitter followers to its more than 200,000 customers, who included celebrities, pastors, businesses, and others seeking to increase their perceived popularity or online influence. Among the company’s customers is even a board member of Twitter. It should be noted that Twitter does not require accounts to be associated with a real person.

One particularly troubling practice is when these bot accounts use the names and/or images of real people. According to the Times’ investigation, at least 55,000 of the fake accounts use the names, profile pictures, hometowns and other personal details of real Twitter users, including minors. This represents “a kind of large-scale social identity theft.”

United States Senator Mark Warner has commented on the issue: “The continued viability of fraudulent accounts and interactions on social media platforms — and the professionalization of these fraudulent services — is an indication that there’s still much work to do.” The United States Senate Intelligence Committee has been investigating the spread of fake accounts on Facebook, Twitter and other platforms.

Most major social media platforms prohibit buying followers. However, their success in the market is largely determined by the number of active users. This creates a perverse incentive for social media platforms to look the other way in the face of these bot accounts. However, there are few if any protections for users from the deceptive tactics of these bots and their human operators.

1. The use of bots in shaping political discourse: “fake news” and political implications

The use of bots are also infecting our political discourse and processes. A study conducted by researchers at Oxford University found that, in the days leading up to the 2016 Presidential Election, an “automated army of pro-Donald J. Trump chatbots overwhelmed similar programs supporting Hillary Clinton five to one.” (John Markoff, Automated Pro-Trump Bots Overwhelmed Pro-Clinton Messages, Researchers Say (Nov. 17, 2016) The New York Times <https://www.nytimes.com/2016/11/18/technology/ automated-pro-trump-bots-overwhelmed-pro-clinton-messages-researchers-say. html?_r=0> [as of Apr. 7, 2018].) The study was based on a collection of approximately 19.4 million Twitter posts. The “chatbots” are “basic software programs with a bit of artificial intelligence and rudimentary communication skills.” According to a sociologist at the Oxford Internet Institute, these specific chatbots were designed “to rant, confuse people on facts, or simply muddy discussions.”

“The use of automated accounts was deliberate and strategic throughout the election,” according to the report, which was published by the Project on Algorithms, Computational Propaganda and Digital Politics. The researchers found that the top 20 Twitter accounts were mostly bots and highly automated accounts that averaged over 1,300 tweets a day, generating more than 234,000 tweets. The study noted that “highly automated accounts — the accounts that tweeted 450 or more times with a related hashtag and user mention during the data collection period — generated close to 18 percent of all Twitter traffic about the presidential election.”

The term “computational propaganda” is used to describe the utilization of these deceptive social media campaigns online. These bots adopt and adapt various retweet and mention strategies to interact with their respective target groups and to more effectively influence them. For example, pro-Trump bots were found to launch into action increasingly earlier during debates between the two candidates and began to “colonize” pro-Clinton hashtags. Additional examples from the presidential election include bots posting embarrassing photos of candidates, making repeated references to inquiries into Mrs. Clinton’s use of a private email server, and spreading “fake news.” The Clinton campaign was found to have deployed “a handful of ‘joke’ bots.” Interestingly, bots were the most effective when distributing negative news reports.

The company discussed above, Devumi, also has fake followers that serve as “phantom foot soldiers in political battles online.” “Devumi’s customers include both avid supporters and fervent critics of President Trump, and both liberal cable pundits and a reporter at the alt-right bastion Breitbart. Randy Bryce, an ironworker seeking to unseat Representative Paul Ryan of Wisconsin, purchased Devumi followers in 2015, when he was a blogger and labor activist.” The company also serves politicians and governments overseas. The Oxford researchers had previously found that the use of political bots played a role in shaping the political discourse in Britain, leading to the infamous “Brexit” vote.

These examples illustrate the troubling consequences of the dramatic surge in bot activity online, calling into question the appropriate role of bots in informing American political discourse and shaping politics in this country and beyond.

3. Providing consumers the tools to sort through the barrage of bot communications

These accounts are counterfeit coins in the booming economy of online influence, reaching into virtually any industry where a mass audience — or the illusion of it — can be monetized. Fake accounts, deployed by governments, criminals and entrepreneurs, now infest social media networks. . . . These fake accounts, known as bots, can help sway advertising audiences and reshape political debates. They can defraud businesses and ruin reputations. Yet their creation and sale fall into a legal gray zone.

(Nicholas Confessore, et al., The Follower Factory (Jan. 27, 2018) The New York Times <https://www.nytimes.com/interactive/2018/ 01/27/technology/social-media-bots.html> [as of Apr. 6, 2018].)

This bill would provide some legal boundaries on the use of bots and their interactions online. One of the more troubling issues with the use of these bots is that people do not know which other users are fake or real. This bill would directly address this concern by making it unlawful for any person to use a bot to communicate or interact with natural persons in California online, with the intention of misleading and without clearly and conspicuously disclosing that the bot is not a natural person. Humans deploying such bots would need to make disclosures ensuring each natural person with whom the bots communicate is on notice of the bot’s nature.

For their part, online platforms would be required to enable users to identify and report bots violating those requirements. The platforms would be required to expeditiously investigate such reports and determine whether to disclose that the bot is not a natural person or remove the bot entirely. In addition, the platforms would be required to provide the Attorney General with reports, detailing the alleged violations and the subsequent actions taken by the platform.

The bill is fairly narrow in that it only addresses those using bots with “the intention of misleading,” arguably the priority targets for regulation. Furthermore, it defines “online platform” as any public-facing Internet Web site, web application, or digital application, including a social network or publication, that has 50,000,000 or more unique monthly United States visitors or users for a majority of months during the preceding 12 months. Therefore, the provisions providing requirements regarding the facilitation and investigation of user-reported violations would only apply to larger, more sophisticated operations. In addition, the bill is careful to lay out thorough definitions of its terms to ensure clarity and enforceability, with a few exceptions.

As pointed out by opponents of this bill, the definition of “bot” may be too broad, sweeping in technology that is not the intended target. In addition, the phrase “intention of misleading” could use further clarity to ensure effective enforcement. In response to these concerns, the author has agreed to take the following amendments:

Amendments

Amend 17940(a) to read:

“Bot” means an online account that is either automated or designed to mimic or behave like the account of a natural person.

Amend 17941(a) to read:

It shall be unlawful for any person to use a bot to communicate or interact with natural persons in California online, with the intention of misleading a natural person about its artificial identity. A person using a bot is presumed to act with the intent to mislead unless the person discloses that the bot is not a natural person.

Amend 17944 to include the following subdivisions:

(c) The duties and obligations imposed by this chapter shall not apply to bots to the extent they are communicating on behalf of a corporation, limited partnership, limited liability company, or other form of business entity, with users or customers of that business, and the user or customer has specifically opted-in to such communications after clear and conspicuous disclosure about their automated nature.

(d) This section does not impose a duty on service providers of online platforms, including but not limited to web hosting and Internet service providers.

Rooting out bots in violation of these provisions will be a difficult task as “deception and detection technologies are in a never-ending arms race.” (Onur Varol, et al., Online Human-Bot Interactions: Detection, Estimation, and Characterization (Mar. 27, 2017) <https: //arxiv.org/pdf/1703.03107.pdf> [as of Apr. 6, 2018].) However, this bill would start the process of drawing legal lines in the sand and provide mechanisms for human users to join the battle.

Support: None Known

Opposition: California Chamber of Commerce; California Grocers Association; CompTIA; CTIA; Data and Marketing Association; Internet Association; Internet Coalition; TechNet

HISTORY

Source: Common Sense Kids Action

Related Pending Legislation: AB 1950 (Levine, 2018) would prohibit an operator of a social media Internet Web site with a physical presence in California from engaging in the sale of advertising with a computer software account or user that performs an automated task, and that is not verified by the operator as being controlled by a natural person. It would further require that such sites determine when an account or profile it hosts on its Internet Web site is being controlled by a computer software account or user that performs an automated task, and if so, prominently indicate that to users of its site and require the account or profile to be linked to a natural person. This bill is currently in the Assembly Privacy and Consumer Protection Committee.

Prior Legislation: None Known

Prior Vote: Senate Business, Professions and Economic Development Committee (Ayes 8, Noes 1)

**************

SENATE COMMITTEE ON

BUSINESS, PROFESSIONS AND ECONOMIC DEVELOPMENT

Senator Jerry Hill, Chair

2017 - 2018 Regular

Bill No: Hearing Date: April 16, 2018

Subject:

SUMMARY:

Prohibits any person from using an automated account, or bot, to mislead a natural person online and requires bots to be clearly identified. Requires online platforms to allow users to report bots violating those provisions, investigate potential bots that are reported, and provide details of violations and investigations to the Attorney General (AG).

Existing law:

1. Prohibits any person, firm, corporation or association, or any employee thereof, from making statements that are untrue or misleading in connection with communications regarding the sale of goods or services, including over the Internet. (Business and Professions Code (BPC) Section (§) 17500)
2. Prohibits a person, with a bad faith intent, to register, traffic in, or use a domain name that is identical or confusingly similar to the name of another living or deceased person. (BPC § 17525)
3. Prohibits any person or entity from sending a commercial e-mail advertisement from California or to a California electronic mail address if the e-mail advertisement contains falsified, misrepresented, or forged header information or if the e-mail advertisement has a misleading subject line. Permits the AG, among other parties, to bring action against a person or entity that violates these provisions.
   (BPC §17529.5)
4. Prohibits a person or entity located within California from using a telephone facsimile machine, computer, or other device to send an unsolicited advertisement to a telephone facsimile machine. (BPC § 17538.43 (b)(1))
5. Requires a person or entity located in California to clearly mark the date and time of transmittal and identity and telephone number of the person or entity sending the message of any communication via a telephone facsimile machine.
   (BPC § 17538.43 (c)(1))
6. Requires individuals or partnerships doing business under fictitious names to file a fictitious business name certificate designed to make the identities of the persons doing business under the fictitious names available to the public. (BPC § 17900)

This bill:

1. Defines “bot” as a machine, device, computer program, or other computer software that is designed to mimic or behave like a natural person such that a reasonable natural person is unable to discern its artificial identity.
2. Defines “online” as any public-facing Internet Web site, web application, or digital application, including a social network or publication.
3. Defines “online platform” as any public-facing Internet Web site, web application, or digital application, including a social network or publication that has 50,000,000 or more unique monthly United States visitors or users for a majority of months during the preceding 12 months.
4. Defines “person” as a natural person, corporation, limited liability company, partnership, joint venture, association, estate, trust, government, governmental subdivision or agency, or other legal entity or any combination thereof.
5. Prohibits any person from using a bot to communicate or interact online with a natural person in California with the intention of misleading and without disclosing that the bot is not a natural person.
6. Requires the disclosure that a bot is not a natural person to be clear and conspicuous to all natural persons and to include language stating that the bot “is a bot” or is “auto-generated.”
7. Requires online platforms to allow users to identify and report bots that the user suspects of being used with the intention of misleading or not being properly disclosed as a bot.
8. Requires online platforms to investigate bots that have been reported by users and determine whether or not to disclose that the bot is not a natural person or remove the bot within 72 hours of the receipt of the notice.
9. Requires online platforms to provide bimonthly reports to the AG detailing reports of bots provided by users and actions taken in response to these reports.
10. Defines the criteria for a statement to be considered to be made in a clear and conspicuous manner for text and graphic communications, audio communications, video communications, and any other communications.
11. Defines that a statement is not made in a clear and conspicuous manner if it is difficult to read or hear or if the placement is easily overlooked.
12. States that the duties and obligations imposed by this chapter are cumulative with any other duties or obligations imposed by any other law and that the provision of which are severable.

FISCAL EFFECT:

Unknown. This measure is keyed “fiscal” by Legislative Counsel.

COMMENTS:

1. Purpose. This measure is sponsored by Common Sense Kids Action. According to the Author, “bots can be – and are often – weaponized to spread fake and misleading news, reshape political debates, and influence advertising audiences.” However, “there is currently no law that relates to the disclosure of the use of bot accounts on social media. SB 1001 sheds light on the fake accounts that simulate real people and spread waves of false information across social media platforms. On the Internet where the appearance of a mass audience can be monetized, it is critical to protect users by providing the tools to understand if their information is coming from a human or a bot account disguised as one. As long as bots are properly identified to let users know that they are a computer generated or automated account, users can at least be aware of who they are interacting with and judge the content accordingly.”
2. Background. Bots are software applications that run automated tasks on a network, such as the Internet, that can interact with computers systems or users. Bots can be purchased or created by individuals or organizations and many online platforms, including social media sites, create bots themselves and encourage users to build bots for use on their platforms. There are a variety of different types of bots that are programmed to perform different tasks. Examples include search engine crawler bots, such as GoogleBot or BingBot, that help search engines discover new content; copyright bots, such as YouTube Content ID, that help identify content posted online that violates copyright law; and chatbots that are designed to simulate conversations with human users, often in order to perform tasks such as booking a flight. Some types of bots are created with malicious intents, including scraping content, stealing passwords, spamming websites, or scalping tickets.

When interacting directly or indirectly with bots online, human users may not be able to discern if they are interacting with a bot or another human. For example, human users on a company’s customer support system may assume they are communicating directly with a human customer service employee, but instead are texting with a bot. Similarly, human Twitter users may assume a Tweet was posted by another human when in actuality the content was generated by a bot. The latter type of interaction has become increasingly common as the number of automated users on social media sites continues to grow. A 2017 study by the University of Southern California and Indiana University found that an estimated 48 million users, or 15 percent of Twitter’s active users at the time, were automated accounts. In November 2017, Facebook disclosed to investors that up to 60 million bots may be on the site.

Bots that are automated accounts on social media sites can be used to amplify the perceived popularity of people, products, and services online. In January 2018, the New York Times published an article, “The Follower Factory,” detailing an emerging market where those seeking to bolster their online presence can purchase followers, likes, and other engagements for their Facebook, Twitter, LinkedIn, YouTube, and other social media accounts from companies that control a large number of bots. One such company highlighted by The Times, Devumi, has reportedly sold Twitter followers to over 200,000 customers, including reality television starts, professional athletes, and politicians. To do so, Devumi draws upon its approximately 3.5 million bot accounts, at least 55,000 of which use the names, profile pictures, and other personal details of real Twitter users. Since a higher follower count implies a higher popularity or importance of the person or product, purchasing followers from companies like Devumi can lead to a real increase in human users engaging with the purchasers account and even lead to endorsement deals from companies seeking to capitalize on a user’s perceived online influence. Social media platforms use similar metrics to determine the popularity of an account, which can influence which accounts a platform recommends or which posts a platform determines are trending.

Just as companies like Devumi use bots to promote their customer’s accounts, individuals and groups can, and have, used bots to spread and amplify divisive views on social media platforms. In February 2018, the U.S. Justice Department charged 13 Russians and three companies with creating fake profiles on social media sites to post contentious comments related to religion, race, and politics and then using bot accounts to like, share, and retweet the posts to help them gain traction. Information provided to U.S. Congress on October 20, 2017 indicated that Russian agents published more than 131,000 messages on Twitter, uploaded over 1,000 videos to YouTube, and disseminated inflammatory posts that reached 126 million users on Facebook.

Many online platforms have their own policies regarding the use of bots, including Twitter which introduced new policies regarding the use of automated accounts in February 2018. While bots are still allowed on Twitter and developers are still encouraged by Twitter to create useful bots that add to the platform’s online community, individuals are now prohibited from using “any form of automation…to post identical or substantially similar content or perform actions such as Likes or Retweets across many accounts”. Twitter also has a variety of enforcement mechanisms in place, including requiring an account suspected of violating its policies to verify ownership with a phone number or email address. Twitter states that this enforcement mechanism helps it “identify violators who are operation multiple accounts for abusive purposes and take action on such accounts”. As its most severe enforcement action, Twitter permanently suspends accounts. Users can appeal permanent suspensions through the platform interface or by filing a report but it is unclear exactly how long Twitter’s investigative and enforcement process takes.

There have recently been efforts by other states and the federal government to regulate the use of bots and prevent the spread of misinformation and false advertising by automated accounts. In 2016, President Obama signed the Better Online Ticket Sales Act, or BOTS Act, (2016; 114^th Congress S. 3183) which prohibited the use of bots to scalp tickets by automatically purchase event tickets online and reselling them on secondary ticket-selling websites at a significantly higher price. This year, legislators in several states, including New York, Maryland, and California, as well as the U.S. Congress, have introduced legislation to regulate the purchasing of social media advertising, including by bots. However, there is currently no state that requires the disclosure of bots.

1. Arguments in Support. Common Sense Kids Action writes in support, noting that teens and children are “at risk of being targeted with unlabeled bots on [social media] platforms” which “are more likely to carry harmful messages: political propaganda, ethnic discrimination, or just false information.” They argue that “with so many children and teens online; families need confidence that they will be protected from unlabeled accounts that are spreading false information and harmful content. SB 1001 allows users to report these unlabeled bots and requires entities behind bots to clearly identify automated accounts.”
2. Arguments in Opposition. The California Chamber of Commerce, Computing Technology Industry Association, CTIA, Data and Marketing Association, and Internet Association write in opposition, stating that while SB 1001 attempts to prevent specific problematic behaviors behind the use of certain types of bots, “aspects of the bill are vague, and the bill places onerous burdens on the operators of websites and applications.” Opponents argue that the requirements of the bill “are burdensome and could diminish the functionality and the speed of impacted websites, while failing to stop bad actors.”
3. Related Legislation This Year. AB 1950 (Levine) would prohibit an operator of a social media site with a physical presence in California from selling advertising to a computer software account or user that performs an automated task and is not verified as being controlled by a natural person. (Status: The bill is pending in the Assembly Committee on Privacy and Consumer Protection.)
4. Prior Related Legislation. AB 329 (Pan, Chapter 325, Statutes of 2013) made it a misdemeanor to intentionally use or sell software to circumvent a security measure, access control system, or other control or measure on a ticket seller’s Internet Web site that is used to ensure an equitable ticket buying process.
5. Policy Question: How should bots be defined? Bots is a broad term for a variety of different types of programs and computer software, including, but not limited to, automated social media accounts with text abilities. Other types of bots may not directly or indirectly interact with human users through text the same as automated social media accounts can, yet may be indistinguishable from human users in other ways, such as through the unique viewer count on a website. It is unclear how bots that do not have a defined account or username, or bots that do not produce text or speech, can clearly and conspicuously be disclosed as bots as this bill would require. Since this bill appears to focus primarily on preventing the spread of misinformation and false advertising by automated accounts on social media platforms, the Author may wish to consider a more narrow definition of “bot” and “online platforms”.

NOTE:

Double-referral to Senate Committee on Judiciary, second.

SUPPORT AND OPPOSITION:

Support:

Common Sense Kids Action (sponsor)

Opposition:

California Chamber of Commerce

Computing Technology Industry Association

CTIA

Data and Marketing Association

Internet Association

-- END –