Assembly Standing Committee on Privacy and Consumer Protection

Welcome to the Assembly Privacy and Consumer Protection hearing. We'll be hearing SB 53 by Senator Wiener, which has been referred back to committee pursuant to Assembly Rule 77.2, as the bill was substantially amended on the Assembly floor after we heard the bill in July. To effectively manage our time today, we'll be limiting testimony to two witnesses in support and two in opposition.

Each witness allowed two minutes to present their testimony, and then of course anyone else will be able to provide their name, organization, and position on the bill. As always, you may go to the committee's website to provide hearing, and we will review that as well. I see that we have a quorum. I believe so. Let's call roll.

[Roll Call] We have a quorum.

Thank you. Mr. Wiener, when you're ready.

Thank you. Thank you very much, Madam Chair. And thank you for you and the committee and your engagement with us on this bill. Colleagues, as you'll recall, last year I authored Senate Bill 1047 around AI safety large language models, and that bill was vetoed by Governor Newsom. In his veto message, he established a Joint California Working Group on AI Frontier Models.

That working group worked intensively late last year and for the first half of this year and produced a report. We then incorporated aspects of that working report into SB 53, which moved through this committee with jointly agreed upon amendments with the committee.

After it went to committee, we worked intensively with stakeholders including the administration and took a series of amendments on the floor, which triggered the re-referral to this committee. With recent amendments, the bill's requirements now apply to or limited to models trained above 10 to the 26 FLOP, which is consistent with what we did with SB 1047 last year.

However, the exception to that is that whistleblower protections do apply to smaller models as well. So anytime someone sees something, say something will apply more broadly. The bills also frontier models trained by companies that do not meet the now revenue threshold of $500 million will still have obligations to disclose basic high level safety details.

So even if it's a smaller enterprise, they will still have a more limited obligation. Models trained by companies above the revenue threshold will make more detailed disclosures. Safety disclosures have been streamlined and simplified. The Attorney General will no longer have the authority to issue regulations.

Instead, the Department of Technology will produce an annual report recommending changes to the Legislature. Developers will be required to review and, if appropriate, update their safety framework every year. We establish a penalty of $1 million per violation.

And instead of reporting regular risk assessments of internally not deployed but internally used models, instead of reporting those regular risk assessments publicly, companies will be required to send those reports confidentially to the Office of Emergency Services. And those have not yet been publicly deployed.

Colleagues, AI has such incredible benefit to improve life and to benefit humanity, but there are also risks, and SB 53 will help us get ahead of those risks. I also want to note that we continue to have CalCompute in this bill, which is an exciting new public cloud that will help democratize and expand access to compute for folks who maybe can't afford compute.

Which has gotten more and more expensive. So with that, I respectfully ask for your aye vote on the bill. And with me today to testify is Adam Billen from Encode, one of our sponsors, and Teri Olle from Economic Security California Action, another sponsor.

Thank you, Chair Bauer-Kahan and Members of the Committee, for the opportunity to testify today. My name is Adam Billen and I'm the Vice President of Public Policy at Encode AI. We're a youth led advocacy organization focused on advancing safe and responsible AI development that was founded right here in California.

Just a couple weeks ago, I was up here in Sacramento with 20 high school and college students who came to speak about the sense of urgency we feel for the state to pass AI legislation. What we told you all is that while we are true, true believers in the positive potential of the technology, we can't help but notice that it seems to be going down the same path as unregulated social media.

As AG Bonta recently highlighted in a letter to a large AI developer, industry already simply is not where it needs to be in ensuring safety in the development and deployment of their systems. While many companies have already publicly committed to do basic safety testing and transparency, SB 53 would be the first piece of legislation to actually hold them accountable to those promises.

The consensus around this final bill has grown out of countless hours of thoughtful feedback and deliberation from experts, industry, and academics, in particular Governor Newsom's Joint Policy Working Group on Frontier Models. This most recent final round of amendments is the result of negotiations both with the Governor's admin and stakeholders to address good faith concerns about large developers.

Add new transparency requirements for small developers that do not meet the revenue threshold and more closely align with the working group's recommendations. This final version of the bill is California's chance to take an essential step forward for protecting both the safety of everyday people in the state and the innovation that has defined it. Thank you.

Thank you, Madam Chair, Members of the Committee. I'm Teri Olle, the Director of Economic Security California Action and a proud co-sponsor of the bill. I'm here to speak for just a minute about CalCompute, which the Senator referenced would establish a publicly owned and operated compute cloud, which is critical to building AI infrastructure in the state.

This aspect of the bill remains unchanged from when you voted on it previously and also remains quite popular. CalCompute addresses the extreme concentration of AI compute power in few hands while positioning California as the innovation leader we should be. Compute is the most important factor enabling AI innovation, yet it's controlled by a tiny number of companies right now.

Meanwhile, our startups and academic institutions lag behind. For example, Stanford has merely 300 of the specialized chips, AI chips necessary to develop large models, while Microsoft has 1.8 million chips. California has a proud legacy of transformative public investment in cutting edge research, from stem cell research to world class national labs. We are the home of AI and we should be leading here too. New York is already showing what's possible.

Empire AI, which was established about a year and a half ago, now has about 200 plus researchers that are already building climate models and improving cancer diagnostics. California should join them and show the world how to spur innovation with transparency and safety at the heart.

CalCompute received nearly universal support during the legislative process both last year and this year and was also the top recommendation in the Little Hoover Commission's AI report last fall, which cited a number of reasons for California establishing its own AI infrastructure, including among them the importance of California achieving some technological Independence. With that, I thank you for your engagement on this issue and urge your aye vote. Thank you.

Thank you. Anyone else here in support of the measure? Please come up. Name, organization, and position.

Doug Subers on behalf of the Secure AI Project, pleased to co-sponsor and support. Thank you.

Hi. Emily Pappas, Niemela Pappas and Associates, on behalf of, on behalf of Anthropic. Here today in support.

Thank you, Madam Chair and Members. Meagan Subers on behalf of Omidyar Network in support.

Thank you. And do we have any principal witnesses in opposition to the bill? Come on up.

Thank you. I'll keep it very brief. Thank you. Ronak Daylami with CalChamber. We are still in an opposed unless amended position. We do want to thank the author for all the work done since we last testified on this bill. We do still have some concerns. In the interest of time, they are outlined in our letter. And we do want to thank the committee for really having those represented in the analysis was such a quick turnaround. Thank you for the analysis and thank the committee. Thanks.

This legendary staff always gets it right.

Good afternoon. Robert Boykin with TechNet. Want to line my comments with CalChamber. Thank you for your work, sir.

Naomi Padron on behalf of the Computer and Communications Industry Association. We too would echo the comments made by TechNet and California Chamber of Commerce. Thank you.

Thank you. Oh, come on up. I didn't see you back there.

Alicia Priego on behalf of the Chamber of Progress. Also align our comments with our fellow associations.

Thank you. Seeing no further comments in opposition, we will bring it back to the dais. Yes, Madam Vice Chair.

I thank you, Madam Chair. Deja vu all over again. We've been here trying to thread this needle very carefully without destroying or damaging the very technology, growth, and innovation that we know is important. And that's really my question. We don't know what we don't know where this is really going. I mean, the best brilliant minds may know, may not. But how do we know... Maybe this is really a rhetorical question, but how do we know that we will not stifle innovation?

Thank you for that question. And I've said from the very beginning of last year's effort that we can both promote innovation and promote public safety. And the two are not mutually exclusive. In fact, promoting public safety and safe deployment of these very powerful models is in the interest of innovation.

Because if we don't do anything and something goes badly awry, there will be a strong reaction. And then you could really see the squelching of innovation, which I don't, which I don't want and I think most people don't want. So we can do both. This bill is very light touch.

I think it's an impactful bill, but it's focused on transparency around safety protocols and the ability of someone who's working at a AI lab who, if they see something happening with these models that is dangerous to public safety, they should be able to say something without fearing retribution from their employer. We have... So I don't see this bill in any way undermining innovation. It is about transparency, and we built in protections.

If they have a trade secret, they're allowed to redact it for the internal non-deployed models, they can do it confidentially to the Office of Emergency, to OES. And we are creating CalCompute, which I think will enhance innovation because compute has become so expensive that allowing the small, the small innovators to be able to access that will be very impactful.

Okay, couple, just two follow up questions. You describe and in the, in the analysis you talk about protecting something bad. What is the worst scenario that you've clearly done scenario projecting here. What are we talking about that can destroy the human race, can go do. What can you prevent? Do you know that?

Yeah. And we're talking about very large models here that are extremely powerful. And some of the concerns, and these are all things that have sort of come out from the labs in terms of what. So models that want to create nuclear, chemical, biological, radiological weapons, models that make it easier to shut down the banking system or the electric grid, models that...

And this is maybe a little bit different than this bill. You know, we've seen models try to get kids to commit suicide. Right. And that's catastrophic for that family. It may or may not fall into the definitions in this bill, but that's obviously horrific. But there are some very large harms that could happen. Cyber crimes that affect a huge number of people and create huge damages. You know, there are a lot of things.

However, the people that... Okay, I think those are great examples. I don't disagree. Those are horrific events in world life, human life. But will the bad people, nefarious element of human humanity that wants to go down that path, will they even care about following the law?

So this applies to the bulk of the bill. The most robust aspects of the bill apply to companies with annual revenue of $500 million or more. And these are the companies that are typically deploying these massive models. It's very expensive to train a model that's 10 to the 26 FLOPS. So it's a limited number of labs.

They're creating these models and in some scenarios open sourcing them so other people can use them, which is great. That's really important for innovation. And what we're asking, they've all committed to take safety steps. They've all made those commitments, which is great. This is asking them to be transparent, to just ensure that they have protocols in place. And so that's why when I say it's a light touch bill, it is. It's about that transparency and saying, hey...

The nefarious actors from someplace in the world who are currently doing bad things, and they may be under 500 million. So they get... They are under the radar.

Yeah. So first of all, people who use these large models. Right. The models can be designed in different ways to reduce the ability of people to use them for terrible purposes. But in terms of... It is... We're trying, in this bill, we don't want to place the same obligations on startups that we're placing on say a Google or a Meta or an OpenAI. And so you're always striking a balance.

And yes, you could put more obligations on startups, but then I think some would argue, well, they don't have the same resources that the large companies do. So it's striking a balance. But we think that this provides more simplified obligations on the smaller entities. And so they'll all have some level of obligation. But in terms of...

Yeah, we can always have a situation where someone decides they're going to go rogue and they're going to create something and not follow the law. That's true in all aspects of life and all aspects of law. But in terms of the large models that we know the large labs are creating, this will cover them.

Then one final question, I promise. Why aren't the large platforms taking a position on this legislation?

I mean, why aren't they taking a position? I mean, you, I can't...

Well, you've met with them.

I can't speak for them. Yeah, we've... Yeah, I mean, I can't speak for them, but they weren't, I mean, they weren't here today, so... And as you heard from the folks who were here, they, you know, they still oppose the bill unless amended, but they've acknowledged the work that we've done.

All right, thank you. Thank you.

Thank you, Madam Vice Chair. Ms. Ortega. Okay. Okay. Anyone else have any questions, comments, concerns? Okay. Oh, yeah, sure. Mr. Patterson.

I'm sorry if this very likely this was asked. I know last year it was asked on a different but similar topic, legislation. What just what safeguards are being taken for... And how do we ensure that California's law applies to, you know, I mean, actors not in California or may not even do business in California?

Yeah. So the law applies when doing business in California. So that's why this is not like disadvantaging California based companies because it's triggered by doing business here. They're all doing business here. California, fourth largest economy in the world, or fifth, whatever we are today, goes back and forth.

So I think they're all doing business here as far as I know. I also, I want to be really clear, because some have talked about this. In an ideal world, we would have one federal regulation to apply everywhere. Congress has never passed even a basic data privacy law, so I'm not holding my breath on that.

California can really drive this conversation nationally, not just because of our size, but the role we play in technology and AI in particular. And so I think us passing this very reasonable balance bill into law will send a very, very loud signal around the country. And I think people are taking notice and will take notice.

Thank you. I think that was our last question. Great. I want to thank the Senator for his never giving up on this piece of legislation, despite the many ups and downs he's faced as he's tried to deal with this. I want to point out a couple of things.

One, in response to the Vice Chair's question, which is the whistleblower protections. I think you're right to say that the nefarious actors are not going to raise their hand and tell us who they are. And the question is, are we aware of the risks models pose such that we can keep our communities safe?

And I do think the whistleblower protections, which provide protections for any activities that pose a specific and substantial danger to the public health or safety resulting from a catastrophic risk, allows for individuals in these companies, who I think, frankly, are good actors who want to keep our communities safe, will allow them to come forward and ensure that we are in a position to keep our community safe.

And that has been in the bill, I believe, since the beginning, Senator. And I think that is one of the parts of the bill that is really important and was backed up in the Governor's workgroup report as something that we should be moving forward with. And then I'll also say I've been given the opportunity on many occasions to discuss Senator Wiener's work in this space.

And I've always said that the one piece that we should all agree on is CalCompute. I think it is something that will truly make us safer, but also make AI more equitable. And I think that AI, as I've said on many occasions, poses a lot of risks as it relates to equity in our society. And competition in the marketplace is one of the best ways to combat that. And so there's a real barrier to entry in this field right now.

The cost, the cost of compute and the cost of getting into this. And so having California investing and ensuring that we are democratizing AI I think is a really critical piece of this effort that, again, has been a part of the Senator's work and I appreciate remains as the bill. So I just want to thank you and I'll give you an opportunity to close.

Thank you very much. I do want to say for CalCompute. We've already had more than one college campus in California come to us and say I want to host CalCompute. So there's a good competition happening. I'm excited about it and I really appreciate the Chair's work and committee staff's work. And I respectfully ask for an aye vote.

Thank you. We have a motion and a second. Let's call the roll.

Item number one, SB 53 by Senator Wiener. The motion is do pass. [Roll Call]

The bill is out 12 to 1, and we'll see it again on the floor shortly.

Thank you, colleagues. Thank you, Madam Chair.

We'll adjourn the hearing.