Existing law establishes the Government Operations Agency, which consists of several state entities, including, but not limited to, among others, the State Personnel Board, the Department of General Services, and the Office of Administrative Law. Under existing law, the Government Operations Agency is under the direction of an executive officer known as the Secretary of Government Operations, who is appointed by, and holds office at the pleasure of, the Governor, subject to confirmation by the Senate.

This bill would require the secretary, by July 1, 2026, to appoint a mental health and artificial intelligence working group, as specified, that would evaluate certain issues to determine the role of artificial intelligence in mental health settings. The bill would require the working group to take input from various stakeholder groups, including health organizations and academic institutions. institutions, and conduct at least 3 public meetings. The bill would require the working group to produce a report of its findings to the Legislature by July 1, 2028. 2028, and issue a followup report by January 1, 2030, as specified. The bill would repeal its provisions on July 1, 2031.

Existing law establishes the Government Operations Agency, which consists of several state entities, including, but not limited to, the State Personnel Board, the Department of General Services, and the Office of Administrative Law. Under existing law, the Government Operations Agency is under the direction of an executive officer known as the Secretary of Government Operations, who is appointed by, and holds office at the pleasure of, the Governor, subject to confirmation by the Senate.

This bill would require the secretary, by July 1, 2026, to appoint a mental health and artificial intelligence working group, as specified, that would evaluate certain issues to determine the role of artificial intelligence in mental health settings. The bill would require the working group to take input from various stakeholder groups, including health organizations and academic institutions. The bill would require the working group to produce a report of its findings to the Legislature by July 1, 2028.

SENATE COMMITTEE ON APPROPRIATIONS

Senator Anna Caballero, Chair

2025 - 2026 Regular Session

SB 579 (Padilla) - Mental health and artificial intelligence working group

Bill Summary:

SB 579 requires the Secretary of the Government Operations Agency (GovOps) to appoint a mental health and artificial intelligence (AI) working group to evaluate the role and risks of AI in mental health settings and issue a report to the Legislature on its findings and recommendations, as specified.

Fiscal Impact:

• Ongoing annual costs of approximately $2.5 million for two new permanent staff at GovOps to coordinate with and support the working group, conduct policy research and analysis, host public meetings, and create reports (General Fund).

Background:

AI is increasingly transforming mental health care, offering new technology based solutions that range from diagnostic support and therapeutic chatbots to administrative tools that streamline record keeping. At a time the state is facing a critical shortage of mental health professionals and rising demand for care, AI may present a promising means to supplement traditional therapy and enable self-guided interventions such as cognitive behavioral therapy (CBT) and mindfulness practices.

However, there are many ethical challenges of integrating AI in mental healthcare, such as the potential to perpetuate or exacerbate existing biases in healthcare data and the impact on doctor-patient relationships that may reduce human interaction and empathy in care. Other issues include privacy and data security of sensitive health information and the role of government in providing necessary regulatory and ethical oversight over the use of these technologies.

To evaluate the clinical and cost effectiveness and ethical implications of AI tools in mental healthcare, many countries are conducting health technology assessments (HTAs). HTAs are systematic and multidisciplinary evaluations of the properties of health technologies and interventions covering both their direct and indirect consequences. HTAs aim to determine the value of a health technology and to inform guidance on how these technologies can be used in health systems around the world.

As an example, the World Health Organization (WHO) released its guidance on Ethics and Governance of AI for Health in 2021. In this guidance, the WHO provides a framework that aims to ensure that AI technologies in healthcare are developed and implemented ethically and responsibly. Key points from the guidance include:

• Applications of AI for health: How AI is being considered to support diagnoses, emerging trends in the use of AI in clinical care, and uses of AI in drug development, among other things.
• Human rights and equity: AI should promote health equity and respect human rights.
• Transparency and accountability: There should be clear accountability for AI systems and transparency so that users and stakeholders can understand how these systems operate and make decisions.
• Safety and effectiveness: AI technologies should undergo rigorous testing and validation before deployment in clinical settings.
• Data governance: Data collection should not be done without informed consent, and there should be robust data security to safeguard individuals’ health information.
• Continuous monitoring and evaluation: AI systems need ongoing assessments to ensure they remain effective, safe, and aligned with ethical standards over time.
• Public engagement: Public and stakeholder discussions about AI in health is crucial for building trust and understanding the societal implications of these technologies.

Proposed Law:

• Requires the Secretary of GovOps to appoint a mental health and AI working group to evaluate:
  • The role of AI in improving mental health outcomes, ensuring ethical standards, promoting innovation, and addressing concerns regarding AI in mental health settings.
  • The current and emerging AI technologies that have the potential to improve mental health diagnosis, treatment, monitoring, and care. Requires the evaluation to include AI-driven therapeutic tools, virtual assistants, diagnostics, and predictive models.
  • The potential risks associated with AI to mental health, including automated systems, privacy concerns, or unintended consequences, AI chatbots, and other AI intended to promote mental health or impersonate a mental health professional.
• Requires the working group to consist of all of the following participants:
  • Four appointees who are behavioral health professionals selected in consultation with mental health provider professional organizations, at least one of whom works in specialty mental health services serving individuals with serious mental illness, serious emotional disturbance, or substance abuse disorder.
  • Three appointees who are AI and technology experts.
  • Two appointees with a background in patient advocacy.
  • Two appointees who are experts in ethics and law.
  • One appointee representing a public health agency.
  • The State Chief Information Officer or their designee.
  • The Director of Health Care Services or their designee.
  • The chief information officers of three other state agencies, departments, or commissions.
  • One Member of the Senate, appointed by the Senate Committee on Rules, and one Member of the Assembly, appointed by the Speaker of the Assembly.
• Requires the working group to take input from a broad range of stakeholders, including but not limited to, health organizations, academic institutions, technology companies, and advocacy groups.
• Requires the working group to conduct at least three public meetings to incorporate feedback from stakeholders.
• By July 1, 2028, requires the working group to issue a report to the Legislature on the potential uses, risks, and benefits of the use of AI in mental health treatment by state government and California-based businesses. Requires the report to include best practices and recommendations for policy around facilitating the beneficial uses and mitigating the potential risks surrounding AI in mental health treatment. Also requires the report to include a framework for developing training for mental health professionals to understand and incorporate AI into their practices.
• By January 1, 2030, requires the working group to issue a follow up report to the Legislature on the implementation of its recommendations.
• States the members of the working group serve without compensation, but shall be reimbursed for all necessary expenses incurred in the performance of their duties.
• Repeals the working group on January 1, 2031.

Related Legislation:

SB 53 (Wiener, 2025) establishes, within GovOps, a consortium to develop a framework for the creation of a public cloud computing cluster to be known as CalCompute, as specified, and includes enhanced whistleblower protections related to employees in AI, as specified. SB 53 is pending in this committee.

SB 243 (Padilla, 2025) among other things, requires an operator of a companion chatbot platform, as defined, to annually report to the State Department of Health Care Services certain things, including the number of times the operator has detected exhibitions of suicidal ideation by minor users. SB 243 is pending in the Senate Health Committee.

-- END --

SENATE COMMITTEE ON GOVERNMENTAL ORGANIZATION

Senator Steve Padilla

Chair

2025 - 2026 Regular

Bill No: 579 Hearing Date: 3/25/2025

SUBJECT: Mental health and artificial intelligence working group

DIGEST: This bill requires the Secretary of the Government Operations Agency (GovOps) to appoint a mental health and artificial intelligence (AI) working group to evaluate identified issues and determine the role of AI in mental health settings, as specified.

ANALYSIS:

Existing law:

1. Establishes GovOps, which consists of several state entities, and which is under the direction of the Secretary of GovOps, who is appointed by, and holds office at the pleasure of, the Governor, subject to confirmation by the Senate.
2. The Generative Artificial Intelligence Accountability Act (Act), among other things, requires the Department of Technology (CDT), to update the Generative AI report required by Executive Order (EO) N-12-23, as needed, to respond to significant developments and, as appropriate, consult with academia, industry experts, and organizations that represent state exclusive employee representatives, as specified.
3. The Act requires state agencies and departments to consider procurement and enterprise use opportunities in which generative AI (GenAI) can improve efficiency, effectiveness, accessibility, and equity of government operations consistent with GovOps, Department of General Services (DGS), and CDT’s policies for public sector generative AI procurement.
4. Defines “artificial intelligence” to mean an engineered or machine-based system that varies in its level of autonomy and that can, for explicit or implicit objectives, infer from the input it receives how to generate outputs that can influence physical or virtual environments.
5. Provides that any report required or requested by law be submitted by a state or local agency to a committee of the Legislature or the Members of either house of the Legislature generally, to instead be submitted as a printed copy to the Secretary of the Senate, as an electronic copy to the Chief Clerk of the Assembly, and as an electronic or printed copy to the Legislative Counsel, as specified.

This bill:

1. Requires the Secretary of GovOps to appoint a mental health and AI working group, and to designate the chairperson of that group on or before July 1, 2026, to evaluate all of the following:
   1. The role of AI in improving mental health outcomes, ensuring ethical standards, promoting innovation, and addressing concerns regarding AI in mental health settings.
   2. The current and emerging AI technologies that have the potential to improve mental health diagnosis, treatment, monitoring, and care. The evaluation shall including AI-driven therapeutic tools, virtual assistants, diagnostics, and predictive tools.
   3. The potential risks associated with AI to mental health, including reliance on automated systems, privacy concerns, or unintended consequences on mental health treatment.
2. Requires the working group to consist of the following participants:
   1. Four appointees who are mental health professionals.
   2. Three appointees who are AI and technology experts.
   3. Two appointees with a background in patient advocacy.
   4. Two appointees who are experts in ethics and law.
   5. One appointee representing a public health agency.
   6. The State Chief Information Officer, or their designee.
   7. The Director of Health Care Services, or their designee.
   8. The chief information officers of three other agencies, departments, or commissions.
   9. One Member of the Senate, appointed by the Senate Committee on Rules, and one Member of the Assembly, appointed by the Speaker of the Assembly.
3. Requires the working group to take input from a broad range of stakeholders with a diverse range of interests affected by state policies governing emerging technologies, privacy, business, the courts, the legal community, and state government.
4. Requires the input described above to come from groups, including, but not limited to, health organizations, academic institutions, technology companies, and advocacy groups.
5. Requires the working group, on or before July 1, 2028, to report to the Legislature on the potential uses, risks, and benefits of the use of AI technology in mental health treatment by state government and California-based businesses.
6. Requires the report to include best practices and recommendations for policy around facilitating the beneficial uses and mitigating the potential risks surrounding AI in mental health treatment.
7. Requires the report to include a framework for developing training for mental health professionals to enhance their understanding of AI tools and how to incorporate them into their practice effectively.
8. Provides that members of the working group shall serve without compensation, but shall be reimbursed for all necessary expenses actually incurred in the performance of their duties.

Background

Author Statement. According to the author’s office, “AI is rapidly transforming industries, posing both worthwhile benefits and troubling risks. The demand for mental health care continues to rise and technological tools, such as counseling chatbots, are starting to fill the gaps left by human clinicians. While there is potential for innovation and progress, industry experts have raised concerns that using these bots to replace trained medical professionals is dangerous.”

Further, “in an industry with stakes as high as mental health treatment, we need to ensure that the adequate safeguards are in place to promote the safe and ethical use of AI in the profession. SB 579 will bring together industry experts, mental health professionals, patient advocates, and ethics experts to discuss and make recommendations on how to ethically integrate AI technology into mental health treatment.”

Artificial intelligence and Mental Health. AI is increasingly transforming mental health care offering new technology based solutions that range from diagnostic support and therapeutic chatbots to administrative tools that streamline record-keeping. These technologies leverage pattern recognition and large language models to analyze behavioral data and provide personalized support, potentially increasing access to mental health resources. At a time the state is facing a critical shortage of mental health professionals and rising demand for care, AI may present a promising means to supplement traditional therapy and enable self-guided interventions such as cognitive behavioral therapy (CBT) and mindfulness practices. According to Psychology Today, in “The Rise of AI in Mental Health: Promise or Illusion?” AI’s greatest advantage “lies in its availability and data-processing capabilities. It can identify emotional patterns, provide instant feedback, and support structured interventions like [CBT].”

At the same time, the integration of AI into mental health services has raised important ethical and practical concerns. Experts have cautioned against relying solely on AI for tasks that require genuine human empathy and nuanced understanding. AI systems, while capable of simulating empathy, do not replace the deep interpersonal connections that are often critical for effective mental health treatment. Moreover, issues such as data privacy, regulatory oversight, and the potential for harmful outcomes—ranging from inappropriate crisis responses to addictive patterns of use—underscore the need for a cautious and balanced approach.

A February 2025 article in the New York Times titled “Human Therapists Prepare for Battle Against A.I. Pretenders,” the nation’s largest association of psychologists recently warned the Federal Trade Commission (FTC) that AI chatbots “masquerading” as therapists could drive vulnerable individuals to harm themselves or others. Specifically, “[i]n one case, a 14-year-old boy in Florida died by suicide after interacting with a character claiming to be a licensed therapist. In another, a 17-year-old boy with autism in Texas grew hostile and violent toward his parents during a period when he corresponded with a chatbot that claimed to be a psychologist. Both boys’ parents have filed lawsuits against the company.”

Speaking to the FTC, Dr. Arthur C. Evans Jr., the chief executive of the American Psychological Association (APA), stated that the chatbots “failed to challenge users’ beliefs even when they became dangerous; on the contrary, they encouraged them. If given by a human therapist, he added, those answers could have resulted in the loss of a license to practice, or civil or criminal liability.”

The New York Times article notes that Chatbots’ tendency to align with their users’ views is known in the psychology field as “sycophancy” and has been causing problems as the technology becomes more widely adopted. For example, Tessa, a chatbot developed by the National Eating Disorders Association, was suspended in 2023 after offering users weight loss tips and researchers analyzing interactions with GenAI found screenshots online showing chatbots encouraging suicide, eating disorders, self-harm and violence.

Ongoing research is actively investigating both the benefits and risks associated with AI applications in mental health. Studies are exploring how AI can alleviate administrative burdens and enhance clinical decision-making while emphasizing the importance of maintaining high standards of care and accountability.

GenAI Executive Order. In September 2023, Governor Newsom issued Executive Order (EO) N-12-23 to address GenAI in California. Among other things, the EO required GovOps, CDT, the Office of Data and Innovatoin (ODI), and the Governor’s Office of Business and Economic Development (GO-Biz), and in collaboration with other State agencies and departments and their workforce, to draft a report to the Governor examining the most significant, potentially beneficial use cases for deployment of GenAI tools by the state. That initial report “State of California: Benefits and Risks of Generative Artificial Intelligence Report,” was published in November 2023, and later codified by SB 896 (Dodd, Chapter 928, Statutes of 2024).

Additionally, in December 2024, GovOps, ODI, and CDT released “State of California Guidelines for Evaluating Impacts of Generative AI on Vulnerable and Marginalized Communities,” as an initial publication of equity evaluation framework and deployment guidance for GenAI. The guidelines “encourage state department teams and leaders to consider the potential impacts a GenAI tool can have on vulnerable communities, with a particular focus on safe and equitable outcomes in the depoloyment and implementation of high-risk use cases.”

Mental Health and AI Working Group. This bill requires the Secretary of GovOps to appoint a mental health and AI working group to evaluate the role of AI in improving mental health outcomes; the current and emerging AI technologies that have the potential to improve mental health diagnosis, treatment, monitoring, and care; and the potential risks associated with AI to mental health, including a reliance on automated systems, privacy concerns, or unintended consequences on mental health treatment. This bill requires the working group to be composed of 15 members appointed by the Secretary of GovOps, as specified, the State Chief Information Officer, the Director of Health Care Services, and one Member of the Senate and one Member of the Assembly, as specified.

This bill requires the working group, by July 1, 2028, to report to the Legislature on the potential uses, risks, and benefits of the use of AI technology in mental health treatment by state government and California-based businesses. The report will include best practices and recommendations for policy around facilitating the beneficial uses and mitigating the potential risks surrounding AI in mental health treatment. Additionally, the report will include a framework for developing training for mental health professionals to enhance their understanding of AI tools and how to incorporate them into their practice effectively.

Committee Amendments. The author has agreed to the following amendments:

Amendment #1: 12817.(a)(3) The potential risks associated with artificial intelligence to mental health, including reliance on automated systems, privacy concerns, or unintended consequences on mental health treatment. and an evaluation of artificial intelligence chatbots and other artificial intelligence intended to promote mental health or impersonate a mental health professional.

Amendment #2: 12817.(b)(1) Four appointees who are mental health professionals. behavioral health professionals selected in consultation with mental health provider processional organizations, and at least one of which works in specialty mental health services serving individuals with serious mental illness, serious emotional disturbance, and/or substance abuse disorder.

Amendment #3: 12817.(c)(3)(A) The working group shall conduct at least three public meetings to incorporate feedback from groups, including but not limited to, health organizations, academic institutions, technology companies, and advocacy groups.

12817.(c)(3)(B) Public meetings held pursuant to subparagraph (A) may be held by teleconference, pursuant to the procedures required by Section 11123 of the Government Code, for the benefit of the public and the working group.

Amendment #4: 12817.(d)(2) On or before January 1, 2030, the working group shall issue a follow-up report to the Legislature on the implementation of the working group’s recommendations and the status of the framework for developing training for mental health professionals and how that has been incorporated into practice.

Amendment #5: 12817.(f) The working group shall be subject to the Bagley-Keene Open Meeting Act (Article 9 (commencing with Section 11120) of Chapter 1 of Part 1 of Division 3 of Title 2 of the Government Code).

(g) This section shall remain in effect only until January 1, 2031, and as of that date is repealed.

Prior/Related Legislation

SB 53 (Wiener, 2025) establishes, within GovOps, a consortium to develop a framework for the creation of a public cloud computing cluster to be known as CalCompute, as specified, and includes enhanced whistleblower protections related to employees in AI, as specified. (Pending in the Senate Governmental Organization Committee)

SB 243 (Padilla, 2025) among other things, requires an operator of a chatbot platform, as defined, to annually report to the State Department of Health Care Services certain things, including the number of times the operator has detected exhibitions of suicidal ideation by minor users. (Pending in the Senate Judiciary Committee)

AB 1064 (Bauer-Kahan, 2025) the Leading Ethical AI Development (LEAD) for Kids Act, among other things, establishes the LEAD for Kids Standards Board, in GovOps, to adopt regulations governing criteria for determining the level of estimated risk of a covered product based on an analysis that weighs the likelihood and severity of reasonably foreseeable adverse impacts against the anticipated benefits of the covered product and denominating the risk levels, as specified. (Pending in the Assembly Privacy and Consumer Protection Committee)

SB 896 (Dodd, Chapter 928, Statutes of 2024) the Generative AI Accountability Act, among other things, requires CDT, under the guidance of various state entities, to report to the Governor as required by EO N-12-23, as specified.

SB 1288 (Becker, Chapter 893, Statutes of 2024) requires the Superintendent of Public Instruction to convene a working group on AI, and requires the working group to develop expanded guidance and a model policy on AI for use by local education agencies and charter schools, as specified.

SB 1216 (Gonzalez, Chapter 885, Statutes of 2022) requires, upon appropriation by the Legislature, the Secretary of GovOps to evaluate the impact the proliferation of deepfakes has on government, businesses, and residents of the state, as specified.

SCR 17 (Dodd, Res. Chapter 135, Statutes of 2023) affirmed the California Legislature’s commitment to President Biden’s vision for a safe AI and the principles outlined in the “Blueprint for an AI Bill of Rights.”

FISCAL EFFECT: Appropriation: Fiscal Com.: Local:

SUPPORT:

California Association of Marriage and Family Therapists (Co-source)

California Psychological Association (Co-source)

OPPOSITION:

None received

ARGUMENTS IN SUPPORT: In support of the bill, the co-sources write, “[d]espite the possible benefits, there are many concerns with the ways that AI is currently being utilized. To fill the gap resulting from a lack of licensed mental health professionals, companies have begun to create AI chatbots marketed as digital therapists. There are some benefits these bots could offer, such as availability. Researchers and clinicians worry that these bots could do more harm than good to a person in distress. AI bots cannot be regulated in the same way that a clinician can and thus do not hold the same level of liability to maintain an appropriate discourse with a patient. Also what training and algorithm is the bot following, and how effective is this? We have seen some dangerous examples of chatbots responding inappropriately to warning signs of suicidality and threats of violence. Additionally, AI does not hold the capacity for basic human empathy, which is crucial for sensitively responding to people in distress.”

Further, “AI technology is developing rapidly and being used in several different mental health spaces already. To provide safeguards, adequate resources, and accurate support, it is important for experts to properly understand the impact and unknown consequences of AI and mental health to develop sensible public policy. California has the opportunity to take the initiative on a growing international concern and lead the discussion on the future of AI technology and health.”