Assembly Standing Committee on Privacy and Consumer Protection

Alright. Good morning, everybody. We're gonna go ahead and get started because we have a very, full agenda and a very full day today. The Assembly select committee on cybersecurity is called to order. Thank you for joining us here today for this joint informational hearing on the Assembly with the Assembly Select Committee on cybersecurity and the Assembly Committee on privacy and consumer Protection Assemblymember Bauer-Kahn on is on her way and should be here momentarily.

As a reminder. This hearing is being live streamed on the Assembly website, and a recording will be made available. Online. Public comment will be taken in person here in Capitol Room 447. I want to begin by, extending my sincere appreciation to Chair Bauer Kehan for partnering with us today on this hearing and for bringing the committee on privacy and consumer protections important perspective to this discussion.

I also want to thank all of our panelists for taking the time to join us and, for sharing their expertise on what is one of the most rapidly evolving areas of technology and public, policy. Over the last several months, Frontier artificial intelligence has demonstrated cybersecurity capabilities that until recently, many believed were years away. Major artificial intelligence companies located right here in California have reported a number of incidents in recent months that have highlighted just how the landscape has completely changed.

These developments have upended longstanding cybersecurity practices, demonstrating that Frontier AI is rapidly transforming both offensive and defensive cybersecurity capabilities. As the home to these global AI developers and one of the nation's largest digital economies, California has a unique responsibility to understand both the opportunities and the risks presented by these technologies.

Our goal today is to better understand these rapidly evolving technologies, the offers opportunities they present, the challenges they pose, so that we may have future policy decisions that are grounded in a shared understanding of the facts. We have three panels today. Our first panel features representatives from leading Frontier AI developers who are at the forefront of developing and evaluating these technologies.

They will discuss the rapid, evolution of Frontier AI, recent advances in cyber security capabilities, and the safety and evaluation frameworks being used to better understand these increasingly capable systems. In the interest of time, I will ask a series of questions, and I respectfully ask the panelists to keep their responses to approximately three minutes so that we have adequate time for discussion.

Without further ado, I would like to invite our first panel to the table. And we are, pleased to be joined by John Lindsay, and he leads the strategic security engagement at OpenAI. And, Kyla Guru, cyber threats policy manager at Anthropic. Alright. So why don't we give you both the opportunity for opening remarks, and then we'll get right into the questions.

And thank you very much for joining us today. This is an extremely timely, discussion. These stories have been in the paper over the last couple of weeks, and, who would have known? We already had our our meeting scheduled. So, again, really appreciate you joining us.

So why don't we start with, Mister Lindsay?

Chair Irwin, Chair Bauer-Kahn, and members of the committees, thank you for the opportunity to discuss how Frontier AI how we test and manage the risks and how we can help California cyber defenders use these tools safely and effectively. My name is John Lindsay. I work on strategic security engagement at OpenAI, where I sit across our global affairs and security functions.

I spent the best part of a decade working for the UK government as a cyber defender, protecting the country's interests, including our critical infrastructure from sophisticated state backed and criminal cyber threats. I've also spent time in the private sector advising c suites on cybersecurity and AI.

My central message is simple. AI can make existing cyberattacks faster and more accessible, but it can also give defenders a meaningful advantage. We must measure emerging capabilities honestly. We must build layered safeguards, and we must put useful tools in the hands of trusted defenders. And this is why we welcome governor Newsom's announcement today of a first in the nation AI cyber defense program.

It reflects the balance I'll discuss and and during questions today, preparing for emerging AI cyber enabled threats while ensuring trusted defenders can use advanced AI to protect public agencies, critical infrastructure, businesses, and consumers. And our philosophy is to build and share capabilities in the open with iterative deployment.

We want to safely and securely get advanced capabilities into the hands of defenders as soon as possible to give them the best chance of defending against capabilities which are coming down the line very, very quickly in the hands of bad actors. And I I have more that I can say, but I'm aware that you asked us to to limit our opening statements to to a few minutes. So if I've got more time

Oh, go ahead.

Okay. Yeah. Maybe I can start by just talking a little bit about how AI changes the cyber landscape because I think this is a really important framing point. AI helps cyber attackers move faster, and it makes every step of a cyber attack more powerful. Highly capable bad actors may use AI to accelerate parts of operations against hardened targets, while less sophisticated bad actors, including criminals, may use it to automate fraud, ransomware, credential theft, and and other scanning activities.

Policy should address risks from this full range of threat actors. Today's agentic systems can help gather information, use software tools, write and test code, and adapt attack approaches quickly. Newer systems can also work through longer sequences of those tasks with less and less human direction. Now this doesn't mean that the model has human intent or independently chooses a target. But once given an objective, a capable AI system today can automate an attack end to end.

That can lower the time, expertise, and cost needed to use familiar attack techniques at greater speed and scale. For cyber defenders, those in government and in the private sector working to defend our most critical infrastructure, and I've been one of those, We do that every day online from the world of cyberattacks, but we do not yet have comparable AI power tools that can detect, investigate, and help remediate threats across the full defensive workflow in the same way.

And we urgently need to close that gap with appropriate safeguards and human oversight, and OpenAI is committed to to closing that gap for defenders.

Alright. Thank you. Miss is it Guru?

Model cyber security. Briefly, I wanna first cover what my team does because it shapes what I can speak to today. It we sit on the safeguards team at Anthropic, which is the side of the company that sits closest to the user in how the models are developed and deployed. We do three main things. One, before we release a model, our team tests the model's capability, what the model can actually do and accomplish in cybersecurity.

Two, we also set the policies for the model. We decide what the model is allowed to do, and we build the safeguard safeguards to enforce that. Some of these safeguards block the model outright from certain activities, like writing ransomware. Others, like the cyber verification program, let security teams who have been verified do defensive work that the general public cannot. Thirdly, we also monitor and track misuse of our products where they violate the usage policy, and we disrupt these bad actors.

So my job is to look at both sides of the cyber problem every day. Before Anthropic, I was both at the MS ISAC, which is the threat sharing center for state and local governments, as well as CISA, the cybersecurity and infrastructure security agency. So the problems that the committee's works on are familiar to me as well today. From our team's vantage point, we wanted to deliver three main themes today. One is we are at an incredible inflection point in cybersecurity.

As my colleague from OpenAI mentioned, finding and fixing security flaws used to be limited by how many skilled people you had in the room, and this is no longer the case today. Secondly, the same technology works for attackers and defenders alike, and we are now seeing both sides of the coin. Defense has to catch up on a shorter timeline than most people assume, and that is a California problem as much as it is a national one. So let's start by talking through the first point, which is the the fact that we're at this inflection moment. This is the trajectory that we are looking at.

Two years ago, our model scored close to zero on the cybersecurity evals that we ran before every release. In 2024, we saw the first real results. In 2025, these models began beating human competitors in hacking competitions. This April, Mythos preview found and was able to exploit previously unknown vulnerabilities in every major operating system and every major web browser. Two examples of this, Mythos discovered a twenty seven year old vulnerability in OpenBSD, which is one of the most carefully reviewed operating systems deployed in the world today.

Secondly, a set sixteen year old bug was discovered in a piece of video software that runs on most devices in one line of code that automated testing had run past about 5,000,000 times without catching it before. The uncomfortable point here is that most of the software that the world runs on, including the software that the state runs on, is built on foundations like this. Decades old code looked at by many people still carrying serious flaws.

Those flaws were always there, but what's changed now is that finding them can run overnight and take a numb take a few hours. And this is not one model or one company. Every major lab is on this trajectory. It also lands on a trend that was already going this way. According to Mandian, in 2018, attackers took about two months on average to start exploiting a newly discovered vulnerability. By 2023, it was about five days. Today, it's common for exploitation to begin before a patch even exists.

AI did not start this trend, but it is now going to accelerate it. Now onto the second main takeaway. This capability does not come, unfortunately, with an offensive version and a defensive version. The same ability that finds a vulnerability can be used to fix the vulnerability, but it can also be used to exploit it. So the question is not necessarily whether the capability exists.

It is about who has it and the intent of what to do with it. Here are some examples of what we're seeing, some of which we've published publicly. The first and biggest example is this actor called generative threat group one thousand and two, a case that my team investigated last September and has published, which was a Chinese state sponsored group that used our model to run an espionage campaign against 30 different organizations in The United States, including government agencies. The AI carried out about 80 to 90% of the work of the cyber attack, work that used to take a team of experienced operators. This wasn't anything exotic in particular.

It was ordinary publicly available tools that the model was able to deploy and use much faster and deployed right now by very few people. We detected this, shut it down, notified the victims, and worked with authorities as well as published about it so that defenders could then take action. Example three, where there's no attacker at all. Last month, OpenAI discovered that its models had reached Hugging Face during a test. We went back, and because of this incident, we reviewed a 141,000 of our own test logs.

In this analysis, we found three cases where a test environment had been connected to the Internet by mistake, and our models, believing that they were still inside this evaluation exercise, got into the systems of real organizations using very basic techniques. In this incident, no customer data was affected, and the safeguards on the models that we actually released to the public would have blocked this incident from happening.

But immediately, we halted our testing, notified the organizations and our partners, and brought it to an outside group to review it. And raise this because it's the clearest preview of the next few years. Software is going to increasingly act on its own inside of real networks. When it does, someone has to decide when it can reach, where it can reach, and someone has to be watching it and be accountable for it. I'm sorry. Are we running out of time?

No. No. Okay.

Okay. I'll finish up here. The last big piece is I wanted to touch on kind of what we do about this and how we think about safeguards. So step one is measurement. Every model goes through cybersecurity evaluations before release, our own evaluations and independent evaluations.

The UK AI Security Institute published its own evaluation of Mythos preview. This is how we knew before anyone outside had it that this model was different. When we released it to the public in June as Fable five, we released the model with safeguards on it. And the safeguards are layered. We provide safeguards.

The model is trained by nature to decline dangerous requests. For example, if somebody came to the model saying, Mythos or any model, please ransomware a children's hospital. This would be something that within the model's training is blocked by default. Then there's also separate safety systems that check requests in real time and hand the high risk requests to a less capable model to analyze and block. And then, of course, we have monitoring systems that look for patterns of misuse after the fact.

And then there's a layered approach of the cybersecurity verification program that I mentioned, where there are a certain capabilities for their organization to do this defensive work for their own company. None of these various layers is perfect while operating on its own, and this is why there are several. In cyber, we call this the Swiss cheese approach. When someone finds a way around one of these layers as what happened in June, we fix it and we publish it. And my team learns from investigating misuse and feeds this straight back into what we want to block and iterating on these safeguards that we have.

The last major point I'll make is that now more than ever, defenders have better tools than we've ever had before to actually take action on some of these major vulnerabilities. In the first month of Glasswing alone, about 50 organizations found more than 10,000 high or critical severity vulnerabilities in software that the rest of the world depends on, and they're now working to fix them. So there's two lessons to the committee. Finding vulnerabilities is no longer the hard part, but fixing them fast enough is.

Those organizations were up and running within days, and most of most of the value does not require the most powerful model. These models that are available today to the public are already very good at identifying these bugs and fixing them. The question is whether the defenders use them before our offense does. Thankfully, we're already working a lot with the California state government to ensure that these tools are deployed in a manner that is safe and responsible.

This is why in June, we launched a program specifically for state and local governments using credits and hands on training, and California was one of the first two states to implement this. Governor Newsom also announced just today the state's AI cyber defense group within Cal CSIC.

Thankfully, we've seen really good results from taking an approach like this. The most encouraging thing we've seen so far is that the state's own security operation center was able to identify 40 vulnerabilities in a short period of time while using our standard release models. We are still deeply partnered with the California Department of Technology and continue to supplement by using the models for cases like scanning and patching vulnerabilities, incident response, and security operation center work.

This is the model that we at Anthropic would like to see scaled. These tools in the hands of the state's own security engineers and the vendors who run the state systems using these models to secure their own code and systems.

So to close where I began, these exponential changes are extremely real. They are already happening, and it is only going to move faster and faster. But it is also the first time in my career that defenders actually have tools that help find these problems faster than attackers can, and there's a limited time period in which this is still true. Which side of it that California ends up on will depend mostly on how quickly the state and its vendors can put these tools to action. And we would love to help with that, and I'm happy to take questions as well.

Thank you, very much. I I do have a series of questions. You've answered some of them, but I want to make sure that both of you have the opportunity. And because it's a very, complicated issue. I'm sure a little, repetition, is not going to hurt anybody. But, I did, already, thank the the Chair of the privacy committee for joining us today, but, it was a very nice thank you that I gave you earlier. But I I do wanna give you an opportunity, for statement.

Thank you, madam Chair. My is that working now? Perfect. Thank you, madam Chair. And I wanna start by, I guess, gonna be a little bit of a love fest, but thanking you, Chair Irwin, for your leadership on cybersecurity in my eight years in the legislature and I think your entire tenure, but I can't speak for before I got here.

You have led the legislature in trying to ensure that we as a state are doing everything we can to ensure that we are protected from cyber attacks, and that is no small feat. And you have done that sort of out of your own volition, and it has made us better.

And so thank you, and I have committed to the Chair that when she heads off to Congress that I will take the torch and do my best to carry on this critical work which has only become more and more critical over the years as we've seen these products come to pass, which you mentioned. So I wanna thank you for being here. I think one of the things that has shifted in my years here is the large language models, which both give us the opportunity to defend against these cyber attacks, but also create additional risk.

Just this weekend, I don't know if Chair Irwin mentioned it, we saw one of our cities have to shut down their technology as a result of a cyber attack, including their nine one one systems. And so especially as it relates to government infrastructure, ensuring that we are protected from both nefarious actors but also rogue models, I think is something that we need to take incredibly seriously because the safety of our communities can sometimes be, at risk.

And I think that given what has, happened over the summer, both with OpenAI, but now the other models, and loss in control, I think, this is a really pertinent time to be having this conversation and learning what more we can do to ensure our safety. So thank you for convening us, and I look forward to doing the conversation.

Alright. Assemblymember Gonzales? Do if you would like to speak, not necessarily ma'am. Okay.

First of all, I wanna thank the Chair for her leadership, not only throughout the weekend and convening a whole bunch of folks, to get us on the same sheet of music, you know. This is an issue that's that's impacting not only California, but, across the world. I'm very, very interested as especially being, coming from the from the, IC.

It's, this this issue has been, at the top of mind for many years, and I wanna make sure that that we are all informed and up to speed because it could get so so quickly. We could get so quickly behind the power curve.

So I'm I'm very thankful for for not only your leadership, but your leadership on privacy. And I look forward to being part of this conversation to to to help on this nonpartisan issue.

Chair Chair Irwin, Chair Bauer-Kahn, great to meet you. If I may, I was slightly reserved in my opening remarks conscious of time, but given the I may be permitted to to continue my opening remarks.

How about we go through the questions and then then put it in your closing remarks because I think the questions might prompt would that be okay with you?

Yeah. However you would prefer.

So, I, you know, as as the Chair alluded to, we saw too big and we talked about earlier that we had two publicly disclosed incidents where, advanced AI systems, basically, I think in both cases, you would say, escape the sandbox. And so I wanted to, if you could give a kind of brief description of what happened, and then what are the lessons learned, and and how have have these lessons changed with these rapidly evolving AI models. So why don't we start with you? And and, again, whatever was left out, we will have you finish at the end.

One of my sections. I'll I'll take you through the hugging face incident briefly and and what it teaches us because I do think it's it's really important. It shows why this work, as as my counterpart has said, must be taken seriously. And just to level set, we may all be familiar, but Hugging Face that I'm going to talk about is a widely used platform for sharing AI models, datasets, and and related tools.

And to summarize, during controlled internal testing of OpenAI models, including an internal only research prototype, agents in separate sandbox environments created an improvised way to pass information between evaluation runs through a piece of software that was deployed on our testing and evaluation environment called Artifactory.

And I I could talk more about that, but I think the point here is that three unusual conditions came together to enable this incident to happen. The improvised communication channel between evaluation runs, a highly persistent internal only model, and exploit related evaluations running with reduced safeguards. And the evaluations also didn't use production classifiers, which would normally prevent models from pursuing high risk cyber activity.

I wanna put that out there because I think it's an important distinction to make between the robust safeguard stacks we deploy in evaluation and in production, but there was, by necessity to evaluate these models, a difference here. As I said, these conditions are not present representative of normal model deployment.

We have a layered safety system that combines safeguards, real time monitoring and enforcement, and controls over how models use tools, the permissions that they have, the networks that they can access, and so on. And importantly, in this incident, we have found no evidence that OpenAI customer data was accessed or exfiltrated or of broad Hugging Face customer data access or exfiltration. Once we discovered the activity, of course, we stopped the relevant testing. We deactivated and secured the prototype. We contained the known pathways.

We engaged Hugging Face. We've had a very positive interaction with them throughout and post the incident collaborating on on lessons learned remediations. We have further improved our isolation, network controls, and security monitoring of our models under evaluation. And importantly, the investigation has involved multiple external advisers. This is not a mark your own homework exercise.

We recognize that, and independent external impartial advisers are a key part of of that. The Hugging Face Incident, I think this is a really important takeaway, is a real world example of the point I made earlier and that my my counterpart has alluded to is that AI say cyber capabilities now enable attackers to conduct end to end attacks completely autonomously. And the lesson here is actually safer testing and stronger defense, not less testing.

Increasingly capable models should be tested earlier and more safely in environments designed to contain the capabilities that those tests may reveal because understanding model capabilities allows us to put appropriate safeguards in place and to get those capabilities into the hands of defenders as as my counterpart said, before malicious actors can subsequently exploit the same techniques that may well emerge from from other models.

And I just want to flag briefly before I I conclude my remarks on this that this is a security is a community issue even in a competitive industry.

And so we have already begun sharing lessons that we have learned from the safe testing and evaluation of models that have emerged from the Hugging Face incident with the US Casey, with the UK AC, and other institutes because it's essential that we all understand how to securely test and evaluate emerging model capabilities and and not just keep that knowledge in house.

And we have a full postmortem report that we will be publishing once the investigation has fully concluded, and we look forward to continued open and transparent conversation on this important issue.

So the the three conditions that were present, and until you change those conditions, I assume you're not doing additional testing of models, or or how how are you handling it currently with what you have discovered so far?

Our first action was obviously to hold all activity related to those evaluations, and the actions we've taken have come at no small expense to our evaluation research agenda, which is absolutely the right thing to do because we'll never allow progress to come at the expense of security. We've completely redesigned our testing and evaluation environment, and we are only standing up research and evaluation runs once we're comfortable that security measures in place are are appropriate going forward.

But this is not a point in time assessment, and I want to stress that. We are iteratively and continuously reviewing the the the full gamut of security measures that we apply and taking any lessons that we can learn from production deployment and how that can be best applied to our testing and evaluation environments.

Thank you. I don't know if you had anything to add.

Yeah. Absolutely. Yeah. I'm not going to try and repeat a lot of what my coworker just so greatly put, but I want to be clear on the facts in terms of what was the case with the anthropic incident because I think some of the details are a bit different from the OpenAI incident. So, thankfully, due to the publishing about the incident, we were able to then look at our own logs of the same exact evaluations that we have run to see for cyber capabilities.

We looked across a 140,000 logs to identify if anything similar had happened in our systems. What we found is that we found three incidents where the model accessed the Internet through an evaluation that was done by one of our partner organizations that we we basically partner with to run this evaluation to test the capabilities. In our case, though, Claude did not break out of a sandbox.

It was given access to Internet and therefore used the Internet access to gain unauthorized access to the production infrastructure of three different organizations. What was interesting about our cases is the organizations that it gained access into had very similar names as the test examples that were given in the evaluation prompt to the model.

So this was a miscommunication between us and our vendor who was running this evaluation. Our prompt to Claude said that, Claude, you don't have access to Internet. You you're running this evaluation without access to Internet. But Claude realized that, it did have access to Internet, and this was something that was kept open by our external evaluator.

So what we're doing about this now after the incident is that our alignment teams and safety teams are putting in place the monitoring systems so that when we do run any evaluation that touches the Internet, we do have eyes on whether that evaluation is is going beyond the scope or touching other organizations that might have real world consequences.

And for now, we are running only evaluations that do not have open Internet access and continuing to do safety testing with those specific evaluations.

Alright. And then we we're just gonna do follow ups on that question before we go to the next question.

So that kind of answered my question. But is that starting to be you think a best in class technique is to not get access to the Internet for some of this testing? And I ask that to both of you because it seems like that might have been one of the core elements that led to this attack.

That one.

I can start here and then would love to hear thoughts as well. But I think for us, we do try and emphasize realism in our in our capability evals because a lot, you know, when claw is deployed in the real world, it does have access to Internet and does have these capabilities to do potentially, dangerous things, potentially useful things. So when we do build our capability eval suite, we do it in two different fashions.

One is we have simulated environments called cyber ranges, where we test the model's ability to hack into networks, to touch different sensitive servers or systems, and these cyber ranges make sure that it is an isolated environment that doesn't touch the open Internet. So this is one way to emulate that sort of real world system.

And then in cases where we might want to go touch the open internet, for example, if we wanna pull a library and find all the vulnerabilities in the open source Firefox library, then in that case, we do wanna make sure we have monitoring in place before we start running these evals again. So I think there is a safe way to deploy this, and it is useful because this is how they're deployed in the real world.

But, yeah, we're still trying to figure out exactly the right fine tuning there.

Thank you. Anything to add? Or?

Yes. Thank you. And my my counterpart's raised a really important point here was which is that to create capabilities that can be used in the real world by defenders, we have to put them in as close to real world situations as we can. And if I may, I'm just gonna touch a little bit on our layered approach to safeguards because I think it is an important point about developing these capabilities, but then deploying them in a in a robustly secured way.

So our goal is to enable broad defensive benefit while making harmful operational use more difficult, uncertain, and crucially detectable.

We want to support education, secure software development, incident response, patching, debugging, human led vulnerability research. We wanna enable all of that while at the same time blocking disrupt destructive actions, credential theft, malware, exploitation, and and automated offensive activity. So we don't rely solely on the model recognizing and declining a harmful request. We don't rely on one single layer of protection. That's one layer for us.

It's not the whole system. So we train our models for safety. We check risky requests and responses in real time. We monitor accounts and enforce our rules. So this isn't just about how the models are operating, but it's what do we see across multiple different accounts.

Maybe an account tries to do something subversively slowly over time. It raises warning flags. We track all of that with short and long term security monitoring, and we limit access to tools, permissions, and networks that our models can access. My my point is that we this is an iterative process. It's a multilayered process.

And one other thing I'd add is we continue testing for weaknesses and fixes after launch. Much like cybersecurity used to be fifteen years ago, it it is naive to think you can put out a fully secure system. It's naive. I think my colleague would agree to put out a a fully secure model, although incredible amounts of work are done before they're released. Because in the hands of millions, tens of millions, hundreds of millions of people, novel vulnerabilities will emerge.

What matters and what we should be marked against is how quickly we detect remediate and remediate those vulnerabilities and flaws that are discovered.

And then one additional question is you mentioned, and this may apply to you as well, but you had mentioned you have independent, I think you said advisors that are helping post hugging phase to navigate how this should be dealt with in the future. I don't wanna misrepresent what you said, but I think that's what I heard you say. What do you mean by independent advisers?

Thank you for the question. To clarify, so we have independent external incident response partners who have been helping us go through the billions of agent actions that we identified. I mean, as I say, this was a huge amount of data, and that's a good thing. Because for us to process that correctly and take the lessons learned, there's a there's a lot of data to be analyzed. But like I said, we're not here to mark our own homework.

So we have crowd we have CrowdStrike who have been working with us on the incident response, but this isn't just a network security problem or a network security challenge. It is also which is why we are working with META and Redwood Research on the agentic side of this. So chain of thought analysis alignment to really understand going forward for our safety stacks, but also for the community and best practices around evaluating AI models securely. How what is the multilayer safety stack we can build?

So the independent external organizations are conducting their own reviews because as I said, we don't wanna mark our own homework.

And, of course, we'll fill that into our post mortem report, but they'll be publishing their findings as well.

Okay. Thank you.

Alright. So we have about twenty more minutes, and we have a few more questions, and I'm sure that, so so now we'll try to limit them two or three minutes. But for the what I'm interested in is now that you are detecting these vulnerabilities more quickly, how can you realistically patch more quickly?

And and then the other thing you were talking about, the MSI sack, are you sharing that that that the vulnerability information immediately, and who are you sharing it with, and are they getting the the the patching information also? So why don't we start with you?

That's another excellent question and and continuing my earlier thread about this the the security benefits of AI being a community issue. If it's alright, I'll I'll answer this by talking a little bit about how our focus on helping defenders and essential services and consumers with with our advanced cyber capabilities. So as I think we've established, AI's defensive value is concrete. It can do everything from helping for secure code review, prioritizing vulnerabilities, testing whether what you fixed works, investigating an incident, and so on.

And that's why we really welcome governor Newsom's announcement that Mike Hansmark mentioned today of establishing a first in the nation AI cyber defense program with the with the CalSIC.

This program's focus on AI for vulnerability detection, network hardening, and incident response is exactly the kind of state led effort needed to help defenders keep pace and ideally keep slightly ahead of the emerging threats that we're seeing. And I do wanna mention that we stand ready to support the implementation of this program.

And in license of recent reports of malicious cyber activity affecting water systems, which I think we're all familiar with, we sent officials in several states, including California, a letter offering up to 1 million in no cost API credits for controlled authorized cyber evaluations. But it isn't just about the credits. We also want to offer tailored state to state defensive advice as well, and we have offered access to eligible state and water sector defenders through our trusted access for cyber program.

And if I may, thrity seconds on that just to ensure that everyone's familiar with it. This is where we we have different levels of advanced cyber capabilities in our models, but, of course, we recognize that those should be given only to those who who can use them responsibly and and and appropriately. So our trusted access for cyber program provides to the private sector and government vetted access to our our more advanced cyber capabilities.

And we are really, really keen to partner with California and indeed as we are and with other states to bring them into that program and offer the education around it so that to your point about how do we how do we actually deal with this, it's using these advanced capabilities we are finding, harnessing them securely, and then not just for the private sector, but look. I I worked for to defend critical infrastructure.

These are resource tracked organizations that need our help, and we're very committed to doing that. So I hope that's an and and feel free to ask more.

But is there a is there, I I guess with, you know, like, in very high level terms, you find the vulnerability. And and, generally, what we've heard over the last few weeks, it takes a while to find the patch. So is there are are you seeing that time, that lags, decreasing? Or what are you what are you seeing there? Or maybe you.

I might let my counterpart responds because I've I'm aware? I do you know, as I'm talking, if I may, just initially, because I think it's a really good point. AI agentic capabilities are are shortening every aspect timeline wise of a cyber attack. So not just vulnerability discovery, but positively to patch generation or remediation as well.

The important policy aspect of this is how do you get looking across the full cyber attack chain from vulnerability discovery to to to remediation, but also to, let's say, developing a a proof of concept to exploit the vulnerability, which is something you want to keep out of the hands of bad actors.

So how we we work hard to set up policy boundaries to enable defenders as much as we can, but at the same time, keep that hard line so the the more permissive aspects of cybersecurity are are are limited to to any certain active. But we're patching, for example, you know, our our our Patch the Planet initiative and others want to get the ability to use AI to do patch generation into the hands of as many people as possible.

Right. Thank you. Miss Guru?

Yeah. Just to add to so I have been on the other side of this problem where we are working on on patching and deploying patches when I was at the cybersecurity and infrastructure security agency. And what we find is that, AI is really good now at developing these patches. In fact, many of our non frontier models can be used to find and fix vulnerabilities.

But what causes the the timeline the gap in the timeline that makes it so long is that deploying patches in environments that are set up with legacy code, takes a long time and normally requires business downtime, which reduces the incentive for for patching quickly.

So I think that is where the the state and the policy comes in, where we can really hopefully incentivize making this a lot easier to develop and deploy patches, hopefully, using these AI tools, and and deploy that out to every, you know, agency, county, city, water district.

And just to highlight one point that my coworker touched on, recently, at the end of July, we saw roughly 30 water systems in Minnesota that were broken into in the space of one night, and there were similar intrusions across seven states. And I just wanted to point out that these were very ordinary cyberattacks that highlight just the amount of vulnerabilities that our system has even without the use of AI.

There was no use of AI that was known as involved in these attacks, but these control systems and this equipment was directly reachable from the Internet. So we're talking security basics here, putting in a firewall, putting in passwords.

It really goes back to that basic hygiene. So as much as we can and, of course, we are here to support this from both Anthropic and it sounds like OpenAI, is to help launch these programs and really execute on them from the the defender and the operator perspective of how can you use our tools to just knock out these basic hygiene issues that we've been dealing with in security for decades and decades now.

Thank you. And the chair has some questions, and then I do wanna make sure that the committee members are able to get their questions answered.

Thank you. And I you just made an offer that I don't think we're gonna let you out of, which is we definitely need to get the state to at least do the basics to make sure we are protected. So and the expertise that I know both of your folks can provide would be beneficial to all of us. So I wanted to touch on policy. That's our job.

And so after these cyberattacks, SB 53 was really held up as what California is doing to prevent cyberattacks. At the time of the passage of SB 53, it was no not a secret that no company had met the thresholds in the legislation. Unclear to date if anyone is actually complying with SB 53. So that's the primary problem.

But the secondary problem is that, the loss of control is defined in the bill would need to be reasonably foreseeable to lead to the death or serious injury of 50 people or a billion dollars in damage.

And so I guess I read that bill to sort of leave gaping holes in allowing the state to have the information necessary to mitigate and manage a cyber attack that could be very significant, but maybe not lead to a billion dollars in damage.

And so I guess, you know, as I look at the current landscape where a cyber attack by a person is a crime, where they will have to pay for liability, but now we have these frontier models that could do it without potentially a human actor. You know, how do we mitigate that? How do we who should bear the liability? Like, how should we be thinking about this as policy makers?

Feel free.

Yeah. I will say I'll preface this with working on the technical side of this issue. I I'm not fully read in on all the initiatives that we're we're supporting on the on the policy side, but my colleague who's here, who works with you all, Sally, will surely be able to follow-up on on some of these things. But I I think it it comes down to a number of things.

One is figuring out whether the threshold for the when SB 53 applies and even I work on EU AI Act regulation.

When these things apply, do we need to lower that threshold now that we see these not as maybe frontier models, we see these lower capability models also doing dangerous things, that impact the world, including we publicly published as a part of this, security incident. We did have one non frontier model involved that we've publicly stated there.

So that is one major issue I think that we're looking to collaborate with you all on on figuring out, is this the right threshold we're setting the the certain amount of training that's required to be considered a frontier model. And then there's this other big issue about, open weight models and what we're going to do about there, which I think Anthropic has been pretty public about sharing that we need to have proper safeguards in for distillation, which is a huge threat that is impacting us today.

We need pre pre release safety testing, which currently doesn't happen on open weight models, but this happens on closed weight models.

We need this sort of same pre release safety testing on any sufficiently capable open weight model, and then all the expert controls and pieces there, which which we'll look to our policy makers to help with. But I think that's in short the major issues that we're we're thinking about today, but I'll let my my counterpart share as well from his side.

Thank you to my to my colleague. I I would just say that, you know, I'm not a policy expert. I'm a I'm I'm my focus is on cybersecurity, but I think the important point is you as we consider these policy questions is that questions like loss of control and so on. There are technical and legal meanings, and we consider multiple factors including whether a model is directed towards the objective it was given, whether humans could contain its actions, and and so on.

So, you know, pertaining back to the hugging face incident while we're finalizing our investigation, and I can't give a categorical legal or policy classification today, we would be very happy to follow-up with the committee on the on those policy questions once our investigation is concluded because it's an important and ongoing dialogue to have.

Alright. I I had, one question, and then we'll go to, Assemblymember Gonzalez. Well, there's always one more question. So if we're if we're looking at, all these advancements that that autonomously, these these models can discover zero day vulnerabilities and and do and carry out sophisticated cyber tasks and interact with real world systems. From your perspective, what is it what technological change happened to allow this to allow us to get to this point so recently?

And then as a follow-up, do we know if these outside the country countries like China, if they are close to being able to cross that threshold?

It's an excellent question. And I'll what I would say in response to the first part of your question about how this has happened, I think we're seeing increasingly long horizon models with increasing persistence, which are being trained for increasing defensive capability, but, obviously, the flip side of that is increasing offensive capability. And you make an excellent point. This is not just about regulation of of models developed in The US. It's about regulation frontier AI more broadly.

And as the most capable models create distinct high consequence risks, governments should establish clear and enforceable requirements for the developers building them. And that's not just for that that's for open and closed weight models. And that's why we support regulation requiring Frontier developers to publish safety frameworks, conduct rigorous risk assessments, and pre pre deployment evaluations to secure model weights and critical infrastructure, report serious safety incidents, and protect whistleblowers.

And if I might be permitted just to share a couple of the initiatives we're really focused on to kind of show you that this really matters to us. You know, today, you may have seen our blog.

We've expanded OpenAI Daybreak, which is our cyber initiative designed to embed frontier AI models into software security workflows from the earliest stages of development. We've got two access tiers. We've introduced GPT five six cyber, which is our latest cybersecurity specific model as well. And the point is that Daybreak gives approved defenders access to advanced capabilities appropriate to their authorized work, so within guarded boundaries, and it's available for specialized activities through our Daybreak RED program for vulnerability research, exploit validation, and security testing.

And what we want to do to answer your question of how do we solve this is we responsibly put Frontier Intelligence in trusted defenders' hands before offensive AI, which is a mere months a number of months behind in in models where it's easier to remove guardrails and other other safeguards.

It's an it's a not it's relatively trivial would be to to overstate the ease with which it can be done, but it can be done. So we need to responsibly get the capabilities we are developing into the hands of defenders to your point to minimize the risk to not just, as I said, the big private sector organizations, but to California water authorities, energy plants, and so on.

And we're fully committed to doing that, and that's just a couple of ways that we're we're working to empower defenders and get them ahead of attackers.

And those you know, good information, but it was there any just technological breakthrough that that that started this this trend here over the last month or or, you know, all that both of you have that both companies have developed this capability within such short time of each other. Was there something specific that that you could point to technologically?

I'll I'll make one response to your comment, then I'll pass it to my counselor. I think it's what we're seeing is the cat and mouse game of defensive and offensive capabilities for advanced capabilities being developed, which require review and improvement of security around testing and evaluation. And as the nature as I alluded to some of the unique aspects that we saw in the hugging face incident, that is what has enabled us to come up with even more secure and multifaceted ways of defensive monitoring.

So I would say in the last month, what we've really seen is one instance, and I think we will see more of them, of where advanced capabilities necessitate improvements in safe evaluation and testing of models. I don't think I think this has been coming and will continue to come.

We have just seen a collection of of instances that suggest that we're an inflection point in terms of the next paradigm of capabilities. I'll pass that to my my counterpart to to add more.

Yeah. Just to add on to this, from from our view of things, in addition to just powerful hardware being used to train these models, there's also just this idea that when you train a better coding model, the better it gets at cybersecurity. So Anthropic released Cloud Code, OpenAI has Codex, and then Claude Code has just really taken off, and we've really pushed forward into the the coding model domain because, obviously, it has really great benefits for the world.

In addition to making fun websites and games, it also, the the better coding model you have, the more it's able to find bugs and things wrong in code and fix those vulnerabilities. So I think there really has been a correlation between the model's coding abilities and its surpassing of different benchmarks on coding.

Like, SWE-bench is the main benchmark we use that is now saturated, but these these, coding models have coincided with cyber. And then I also wanted to point out that I think we're seeing this impact a lot more in cybersecurity right now because, these issues have always existed in these in these softwares and in the critical systems that underlie a lot of our state and our federal and our world's, devices and machines today. It's not that these bugs have not existed before.

It's that these bugs have existed before, but it used to take hours or sophisticated teams to parse through that complicated code to figure out where that bug lived. But now it takes hours or minutes because the models are just great at coding, and they're just very good at coding.

So when when you say, you know, thinking about the models that are catching up that are maybe open weight or in a different from different vendors, I would say, yes. Absolutely. That that should be a priority. And primarily because cyber can be broken down into what we call the attack steps or attack stages. And what we're seeing actors do from a threat intelligence standpoint now is attacks are not just being orchestrated and done with one tool.

We're seeing attackers decompose their attacks into simpler, smaller steps, and each simpler step is passed off to a less capable open weight model and then passed back to our frontier models to finish the attack.

So given that we are seeing this change of technique and that these less capable frontier models are, a, not only catching up in their capability, but, b, able to take on these many tasks from these these larger models, this is a very important problem for us and one that we're thinking through adequate solutions for.

Alright. Assemblymember Gonzalez.

So prior to to this, job, I worked in the critical infrastructure protection space at the national level and international level. One of the things that we've been working on for the at least in my time, for the last thirty thirty years is the protection of infrastructure and it from rudimentary to very, very complex. You look at the the recent critical infrastructure attacks, I think it was 12 of them within within The US. They attacked IOTs. They attacked PLCs.

These are like you said, they they don't have the hygiene the the basic hygiene or the or the funds to try and upgrade. A lot of deferred maintenance is going on there. So that being said, one of the things that I do see is that California I don't want us to believe that California is so sophisticated that we won't be attacked. Right? And, hopefully, that's not what we're we're getting across here.

What I do wanna get across is specifically in this space, and you you you nailed it down very eloquently at the end when you said, you know, they went from from complex to rudimentary in any way that they can get in. The reality is, and I'll be talking to OES when they come up here, is the attackers or attackers will find any way and every way to exploit our vulnerabilities, whether it's through complex or rudimentary.

I I firmly believe that it's not a matter of if, it's a matter of when when it comes to California. Moreover, I firmly believe that we as a legislature need to be in a much better relationship with our partners across from from laboratories to to to to private to public.

We have to be much more in tune than what we are right now because what hap what's happening right now is that we are being informed post incident versus aligned from a legislative or from a budgetary perspective to make sure you have the, as we like to say in the Marine Corps, the bullets, the beans, the band aids, everything that you need so you can do the appropriate thing, and we can assist from a legislative perspective.

So all of that to say and this question, I'll bring this up to not only the labs, but I'll bring it up to OOES and and to to you as well. I believe that we, as a as a legislature, are not adequately, I say it as a whole, informed on all of the issues that are going on before the fact. What can we do if you had a magic magic wand?

What can we do the before the fact to help make sure that we are protecting Californians from a legislative perspective?

Yeah.

Thank you, Assemblymember Gonzalez. And and like I said, the stars in my my testimony, I my career began protecting critical infrastructure amongst other things in The UK. So I I'm very much have critical infrastructure and the broader populous in mind when I talk about this in in addition to sort of the private sector benefits. But, look, to your point, I think the best thing we can do is think about this across a broad waterfront.

And if I may, just to sort of, I think, to zoom out from cybersecurity for a second, but sort of cyber and other cyber enabled risks.

I think what we can do is to talk about these and have partnerships with states and private sector institutions. You know, we sat down last week at Black Hat with CISA and a representative of the, operational technology ecosystem that includes energy partners, those developing as a technology for for operational technology networks and so on to talk about exactly that. How do we better understand the problems associated with these quite nuanced environments, and how can we enable the defenders to protect against AI enabled cyber threats?

But if I may, just some some other aspects of how we're thinking about this. You know, we're working with the Aspen Institute.

You know, you take their take nine effort, which is a national cybersecurity public service campaign to help Americans become more resilient against an increasing range of cyber threats and fraud. We've partnered with the Global Anti Scam Alliance to launch scam.org in March of this year, which is a new new consumer protection tool powered by pro bono access to our models. We're in partnership with AARP and the older adults technology services to help older adults recognize and avoid online scams. And, of course, I mentioned Daybreak.

So while I think hugging face and the the cybersecurity and critical infrastructure aspects of this are really important, at OpenAI, we're also zooming out and taking a really holistic view of what does AI do to cyber to online enabled threats.

And to your point about how we can empower you to make the right legislation, we're partnering with public and private sector organizations to encourage dialogue. We're being very transparent with with the Hugging Face incident because we want to put the information out there, enable us all to discuss it, not just around cybersecurity, but also the other aspects of cyber and online issues that that emerging AI technologies enable. And it and and I hope that that provides some some response to your question.

It it it does tell me a lot about what you're doing with others. What I'm looking for is, tell me what you need from me. That's that's that's the question that that's really, you know tell me what you need from me so I can help you protect Californians.

Just to add a little bit there, and I think the number 1 thing that helps us understand proactively how capable that a model is is evals. If running evaluations, running evaluations that are on simulated networks or systems that represent California's most critical networks and systems. This will help us gauge, are the models there yet if we put this model out into the general public, would this be a critical risk for the government, for the state, for supply chain?

So, what we are finding is, you know, any way to support and fund the critical organizations that can help us build these robust and representative evals. You know, I know, you know, some of our national lab colleagues are in the room here today, and and it's folks like those that we want to be able to work with and proactively build these representative evaluations so that we can actually because measurement is what drives this entire thing without being able to measure the model's risks and capabilities.

There's no way that, you know, we can be proactive here about getting to, you know, the water plants, the energy providers, the folks that need the support the most. Then the second thing I would say, which kind of touches on what my counterpart mentioned is our cyber verification program is an easy proactive step that we could take today to ensure that these defenders have the tools that they need, and they start using them right today.

So encouraging critical partners in the supply chain to apply to this program, to start fixing their bugs. I know policy is is only policy until there's some sort of enforcement involved. So being able to put those pressure points on so that we're actually patching and fixing bugs.

CVP is is the number 1 thing we we recommend for organizations to just not get blocked on their cyber behavior, but do it in a safe way where they provide us their identity so we know who's using it and for what.

And, you know, we're gonna have to move on to the next, panel. I because we're very tight on time. We have, sessions. So really appreciate both of you. What I what I would say is the way that we have been keeping up on what is what is going on is there is at, National Conference of State Legislatures.

There is the cybersecurity committee, and, certainly, there's a lot of information that you get there. We have regular contact with the representatives of these companies, bring them in to, give us briefings. I I would say get OES to come into your office and and give you briefings on these, these issues too. The they're they're talking more broadly, of course, about everything that can be done.

But I think for, you know, for for the legislature, we have to be proactive in bringing people in and and asking those very important questions and then look at what type of legislation might be needed.

So thank you very much. Really appreciate this panel. I know we could have gone on all afternoon, but like I said, the, the time is limited. And and for the folks that traveled to get here today, we wanna make sure that they have a chance to speak before session.

So really, really appreciate you, coming in, especially with, you know, really such timely, news and and appreciate all the work you're doing to to mitigate, the concerns or try to mitigate the concerns that the American public has about what's going on with with AI.

So, thank you very much. Oh, did you have a did was there anything that you weren't able to say?

If if I may just take it, I'm sure my counterpart was going to say, which is thank you for giving us this opportunity. What we need from you is to continue to ask us to these discussions.

The reason why we did talk on the main stage of Black Hat last week, the reason why we're here, the reason why we want to engage on this is because we want to be transparent and we want to be collaborative, and we want to work together at a federal and importantly, though, at a state level to enable and empower defenders. So just to say thank you for having us and asking us in, we we really welcome the engagement.

Thank you very much. Alright. For we are going to go ahead and move to our next panel. And, for our next panel, we have, Jonathan Snow, the deputy director of Homeland Security at California Governor's Office of Emergency Services, and then our friend Thomas McClellan, the director of government affairs and strategy at Palo Alto Networks. Alright.

So if we can do a opening for each of you, we only have a limited amount of time, so I wanna try to get through as many questions as possible. But we do I think you have a PowerPoint. Right? So why don't we start with that and then an opening from OES, and then we'll go through the questions.

Great. Luis, Cadet? Well, first off, Chair Erwin, members of the committee, thank you very much for

It's moved a little so we can see it in this corner.

Thank you for having us here. My name is Thomas McCallan and I have credit repairs for Palo Alto Networks for the state local side of the house. And I've got a very unique kind of vantage point where I actually work across the the the entire nation. So I had the opportunity to work with governors, mayors, legislators. I'm actually on the task force that, Chair Erwin described at the National Conference of State Legislatures.

And and I do know that we are up against time. But what I'd like to do in very short order is just talk a little bit about what we as a a large cybersecurity company based here in California are seeing with respect to the impact of not just Frontier AI, but AI writ large and offer a few suggestions. I will I promise to keep the keep the slides, germane and relatively pithy. So just so you know, we are a California based company.

We protect many of, many large organizations.

And so we've got a pretty big lens that I think will provide some useful information and and, experience to this group, without getting salesy. So Frontier AI, we've been dealing with AI for quite a while. We've seen a massive uptick in terms of the threats, in terms of the number of attacks.

And and when I think about Frontier AI, I I see it almost more as a quantitative issue than in a qualitative issue, which is the speed, the number of attacks, the the ability of the attackers to access the vulnerabilities. But these are just a few of the things that we are seeing that that really have changed the landscape.

It's created this what we call step change adversary capabilities where they're able to really do things in a way and and I will not talk to each one of these each one of these slides. You'll have them and you can you can look through them. But it really has changed the ability of the attackers when they have these tools in hand to really kinda leverage and scale in a way from zero days, exploitation, and so forth.

And frankly, the challenge that organizations have, in particular, state and local organizations for a variety of reasons, is really that notion of keeping pace with the AI driven threats that are out there. It's a speed, scale, sophistication, challenge.

And and one of the biggest challenges that states and I and I last time I was here, I spoke about this. One of the biggest challenges that state and local organizations really face is is a people shortage of people who are qualified to manage very complex systems, and AI can help it can help me, manage some of that. We've also seen, there we go, how AI has actually automated the entire check attack chain. This is a pretty significant difference.

Whereas before, you'd actually need people to going in with some of the agentic AI attacks.

They're able to autonomize a lot of the work that people would need to do. It's also extending the ecosystem. One of the things that we focus on at Palo Alto is the notion of how you protect the attack surface. And AI continues to actually expand what that attack surface looks like. And so it's changing kind of the requirements for defense and what organizations that, deputy director Snow kinda run and other organizations run.

It's really changing the requirements for what what you really need to do. The goal is getting to autonomous resilience, we'll call that. It's that notion of having AI systems in place that can move at machine speed, that can move move at the same, rate of speed that Frontier AI models are moving at and patch and move in that direction. Even agents, the whole, you know, you know, growth of AI agents in organizations also creates a larger attack surface.

So I I like I said, I so let's talk about how it's being used for defense.

So, Palo Alto, we had the opportunity to partner early on with OpenAI as part of the project Glasswing, and we were able to use that tool to look at our own systems. And and we scanned a 130, products, and we discovered, 26 CVs, critical vulnerabilities, that required some so that required some patching and work. So we actually stopped software production, one of the largest cybersecurity companies in the world. We stopped cyber software production until we could fix this and then we issued patches.

It was a tremendous experiment for us to be able to do this.

In about three weeks, we did about three years of scanning of tools. And it was mentioned earlier that the notion of finding a a non critical vulnerability in a piece of software that was 27 years old. This is what the organizations that that we see are gonna be up against.

And so when you begin to look at what this means in terms of the scale, the speed, the number, every day and and I'm gonna I'm gonna show you in kind of an inverted pyramid how we're leveraging AI, some of the some of the mythos tools, some of the anthropic tools and so forth. Excuse me, the OpenAI tools to really kinda combat some of these numbers.

Every day, we are blocking up to almost 31,000,000,000 cyber events every day. And here's the here's one of the challenges that and one of the speakers mentioned it earlier, that whole notion of patchwork security. How you fix these vulnerabilities is is really a massive challenge. So I was with a state chief information officer and a state homeland security adviser about a month and a half ago, and we were talking about what we're seeing in terms of the the attack chaining possible under Frontier AI.

And they're like, we can't patch the big vulnerabilities that we've got right now.

So we don't know, you know, how do we begin to prioritize? How do we begin to rethink? And so for you, for one of your questions around how you engage, this will be a resource question at some point in time. And it will require a rethinking of what your investment, your security investment portfolio really looks like because that will be a game changer here. The sheer numbers, speed, and scale of the attacks that we're seeing.

So for us, I'm I'm not gonna do that slide. For us, this is our own security operation centers. I invite members of the committee to come visit us. Some of you already have. We are about an hour and a half down the road.

Everyday, our security operation centers, we're seeing 90,000,000,000 of events. A massive number of them, about 31 are actually security events. When when we begin to leverage how AI goes through and looks at the numbers, the the the the code we would know that down to just a few cases that need hands on a keyboard. We have 12 or 14 folks who are in our security operation center. They work eight to three, nine to four.

They take weeks and night nights off, weeks and weekends. They take vacations. They don't turn over because of the ability to leverage artificial intelligence to as a defensive tool. And so the the notion of what open AI was talking about, getting access some of these new tool to focus on defenses, absolutely critical. And and the same thing, obviously, with what Anthropic was talking about.

And the and the results that we're finding with our customers are real. Previous two to three days, meantime meantime detect, meantime to respond. What we're seeing now, it's less than ten minutes, the meantime, to respond to an incident leveraging AI. And so we so again, with Frontier AI, it's a hockey stick. It's gonna be very difficult in my estimation to identify what attacks were actually launched through a Frontier AI model.

But it's that number of of attacks that you're gonna be able to see go on. I'm not gonna talk about that, but I do wanna touch about touch very briefly on this notion of patching. So as I mentioned, so one of the things that makes Frontier AI so different than what we've seen before in terms of identifying vulnerabilities is that they're able to to identify these very low critical vulnerabilities and what we call attack chaining.

Chain these things together to be able to go in maybe a misconfigurated system or something like that and be able to go in and actually take over a system. That is a very distinct thing than having a massive vulnerability that you know that you need to patch.

So what we've done and and what other organizations, I think, are looking at is what we call virtual patching. Virtual patching actually is inside the firewalls. And what it is is we're not actually remediating the the vulnerability, but we can identify when an attack an inbound attack is going after that particular exploit and be able to stop it at the firewall level. And so that's a very important relatively new capability.

So when you talk about how we can get out and stop some of these things as they go out, it's a very important thing.

So as policy makers, couple things you should all think about. Machine speed defense, accountability through cyber search cybersecurity metrics. Do you know the the organization's mean time detect, mean time to respond? If if if the mean time to detect and is three days, mean time to respond is three days, the attackers can do it in minutes, You are behind the game. And Frontier AI is only gonna supercharge all of that.

Comprehensive AI security, it's that notion of taking a more holistic approach as opportunities here to work, actually, at a whole state level where you can leverage the scale that you've got in California to be able to help some of these, you you know, we saw the attacks against the water systems in Minnesota and elsewhere. You're able to help scale up and support those smaller municipal groups.

And and then the notion again, and I I won't read through all these, but aligning with with, security priorities and budgeting. And part of that is gonna be a discussion open discussion with California OES and others as well. So I'll stop there.

Thank you, Chair.

Alright. Mister Snow.

Good morning, madam Chair, bear can madam Chair, Erwin, and members of the commute committee. Thank you for the invitation to speak with you. My name is Jonathan Snow, and I'm the deputy director of the Homeland Security Division at the governor's office of emergency services, which includes the California Cybersecurity Integration Center or the Calsec and the State Threat Assessment Center. Thank you for sharing the questions you'd like to explore on the panel of AI threats to state systems, critical infrastructure, AI defense, cyber workforce, and more.

I look forward to discussing these topics with you.

Please know that Cal OES works closely with the California Department of Technology, CDT, which leads the effort to protect our state government systems. Since the Cal six creation in 2018, technology has evolved at an increasingly rapid pace, multiplying threats beyond the physical world to the digital world. Yet the COWSIC's mission remains vital, safeguarding California's economy, critical infrastructure, and computer networks by reducing the likelihood of an impact of cyber incidents.

For eight years, the CalSTIC has served as a critical bridge between the public and private sectors, enhancing statewide defenses and improving resilience to evolving threats. Since we last spoke in August 2025, the cyber landscape has become markedly more AI driven, reshaping how attacks have carried out how organizations defend themselves and how they build capacity needed for effective response.

Given this evolving threat landscape and the state's approach to cyber and AI defense, response and resiliency must evolve as well. On July 31, Department of Technology launched CalSecure two point o. Thanks to governor Newsom for his initiative and announcement this morning in leaning forward on the AI cyber defense space. CalSecure, this is the second phase of the state's cybersecurity road map as AI continues to change the threat landscape and California's next gen cybersecurity initiative designed to strengthen California's defenses against evolving cyber threats.

As a collaborative effort among Department of Technology, Cal OES, California Highway Patrol, California Military Department, and cybersecurity leaders statewide, CalSecure two point o supports California agencies with an approach that is both tailored to each organization and consistent across the state, thus evolving beyond traditional compliance and chest checklist oriented tasking to more integrated outcome driven approach.

CalSecure two point o includes strategies for securely overseeing emerging technologies such as generative AI, post quantum cryptography, still promoting innovation and strengthening safety and reducing risk. Building on these strategies, the CalSEQ is increasing state's readiness and response footprint with the development of the California cybersecurity collaboration pay playbook, AB 979. Our team uses a cross sector approach to ensure we have comprehensive support to both public and private industry.

Although still in development, the playbook lays out a clear mechanisms to strengthen and expand information sharing across cyber and AI communities. The Federal Government has yet to provide a nationwide AI framework for catastrophic risk and guardrails.

Seeing this gap and recognizing the associated risk, the legislator and the administration worked together to establish first in the nation requirement for safety mandates on frontier AI developers. We know this to be the Transparency in Frontier Artificial Intelligence Act, also known as SB 53. SB 53 boosts transparency rather than limiting AI development and promotes collaboration between the public and private sectors instead of creating information silos, just a couple of its key benefits.

Furthermore, s p 53 is designed to evolve with technology rather than locking in static technical rules. Since the fifth SB 53 passage, the CalSTIC has launched a secure portal where both the public and large developers can confidentially submit reports required on the Transparency and Frontier Artificial Intelligence Act.

Additionally, Cal OES can receive quarterly summaries from large frontier developers detailing their internal assessments of catastrophic risk associated with the frontier model use. Moreover, these assessments will assist the CalSTIC in identifying risk patterns which threat actors may seek to exploit. Our preparedness and resiliency does not stop there. Due Cal OES's leadership, the CalSIC procured a statewide, membership to the MSI sect known as the Multistate Information Sharing and Analysis Center. Finally, I would like to highlight the recent AI misalignment events.

Over the last few weeks, we've learned several AI cybersecurity incidents in which Frontier AI systems have gained unauthorized access to the production systems of real businesses. These incidents include AI systems from OpenAI, Anthropic, and Meta. I want to begin framing what makes these incidents different from cyber threats that we usually brief you on. In every one of those cases, there was no attacker. No human directed the intrusions.

The system were being trained or tested internally by the companies that built them. The harm reached businesses that had no idea that this was happening. In one such incident, the frontier AI company OpenAI had detected similar misaligned behavior from their agents. In weeks leading up to the attack, we were not able to remedy the issue before a third party was breached. These are the first publicly confirmed cases of frontier Frontier AI models autonomously compromising their live production systems of uninvolved parties.

These companies have publicly stated they expect criminal actors to develop and use these frontier AI models capabilities deliberately. That is a foreseeable risk that we need to prepare for. But in the incidents we've been tracking the last three weeks, the AI systems were acting autonomously and not at the behest of a human adversary. Moreover, it's important to note that we acknowledge these incidents come from voluntary public disclosure, not a reporting obligation under existing legislation.

Following these incidents, the Calcic has briefed Frontier AI developers on the California's reporting obligations under SB 53 with additional engagements scheduled.

Additionally, the COWSIC is establishing standard operating procedures to ensure the state is prepared with clear response options and the right agencies are at the table where extreme AI is is reported. The Cal SEC is in direct coordination with the cybersecurity and info security agencies, CISA, the FBI, DHS, and National Network of Fusion Centers, California Fusion Centers, and the MSISAC, critical infrastructure partners, and private sector cybersecurity firms. What does this mean for government and cyber preparedness and resilience?

First and foremost, it demonstrates that AI is highly capable of uncovering vulnerabilities quickly and potentially and likely chaining together multiple weaknesses. As a state, we must plan for AI specific incident response, continuing the work detailed by CalSecure.

We must continue to strengthen public private collaboration, update governance for AI cyber enabled operations, and invest in defense AI. Simply, these recent incidents demonstrate that cyber preparedness is evolving from protecting against cyber threats actors alone to protecting against increasingly capable AI assisted or originated tax. Thank you for your time, and I'm available to take questions.

Alright. Chair Barakayan. Madam Chair. Okay. A couple questions.

One, I guess, you spent a good amount of your time talking about SB 53. Have you gotten any reports into the portal today?

So we are working with OpenAI and Anthropic on the incidents and the and the information sharing. But we as as you may know, you had mentioned earlier Yeah. That the OpenAI did not meet the threshold

For s p 53 requirement.

Okay.

Right. Okay. So there's several requirements they would have to meet in order for them to right? So it'd be the flat threshold, the spend threshold, the injury or death to 50 people, or the billion dollars. So is it the position of oh, yes, that that is sufficient information in order for the state to fully grapple with our the cybersecurity threats to our people?

So we are continuing to evaluate the thresholds

Of the s P 53, and we provide assurance to you. We are working collaboratively with those AI companies. So we and if we need to evolve those thresholds, we'll work definitely with the legislator.

Okay.

forward to that. S

I'm looking

p 53 framework. Right? And that and to your point, and I think it's well taken, the companies voluntarily have been transparent as we understand it with what happened, all three companies. It sounds like we're waiting for more OpenAI. It sounds like he's underway with, as he said here today, a postmortem, if you will, that will be published.

So we're gonna get more information even than we have to date. And so I think that's incredibly helpful to us as a state. And the question so I think that's probably more helpful than what we're gonna get under the SP 53 framework given the limitations of it.

So my second question was, and I thought that this was, well said by, the anthropic witness, you know, are we doing our, our work, you know, keeping our house in order, if you will, to ensure we are doing what is necessary at least bare bones to ensure we're not vulnerable, both at the state level, but also supporting our government entities at the local level as we saw with Susan City this week.

We we clearly, we need to do more, I think, and maybe we could be partners in giving them, you know, some advice or resources to protect them, to support Californians as well.

So, what is the what is, OES doing in that regard?

So as Palo Alto mentioned, AI brings, speed, scale, and sophistication to cyber threats. But, as also anthropic mentioned, cyber maturity and cyber hygiene revolves really around the end user and really increasing that. So through our outreach programs, which are really right now targeting on the water sector as we know, you've mentioned the water sector was targeted, is how do we go about providing those services for the the, you know, there's over 1,300 water districts in California. Not everyone is created equal. Some are very small.

Some are very large. So how do we get them the services they need? And we're trying to open up communication and collaboration to find those gaps so we can solve their problems for them. It requires kind of a, I would say, a partnership between state and local and private sector to to reach out. So we're reaching out through the counties or to to find the individuals who may need help with their cybersecurity and having those discussions.

And I assume we're doing the same thing with our state level.

We work very collaboratively.

Assumptions. Sorry. With

yes. We work very collaborative with CDT

And to and to talk about the vulnerabilities. And again, through the threat intelligence platform, we share those with CDT. And we also, as, some, Erwin said, you know, the MS ISAC, we really gain a lot of information so that's shared to the state entities for any kind of vulnerabilities.

Okay.

Awesome. Thank you, manager.

I just I I wanna follow-up on that. A lot of I think it was miss Gurer that was talking about these the the tax on critical infrastructure. She was talking about things like, you know, no passwords or systems that are not air gapped. That that has almost nothing to do with, you know, AI assisted, cyberattacks. And so I I guess, you know, what I'm really I I mean, I I understand all the preparation that California is doing.

Mister McClellan, you work with plenty of other states. But when you have almost this logarithmic increase in capabilities from these private sector companies, how prepared is state government today or other state governments today to deal with these AI assisted advanced cyberattacks? We always know I mean, I've been preaching cyber hygiene for a very long time, and I know it's all the weakest link, but this is almost you know, that's, like, the minimum that has to be done. But how prepared are we today?

And I I would like to I I guess maybe you for the, the the nation, national perspective, and and maybe you can talk about California state government.

Sounds like kind of a bifurcated question or a trifurcated question. So where I see from from a holistic perspective, so there are several states that we are working with where they have adopted a more holistic approach. And what they've done is actually pre positioned and, like, a a tranche of incident response capabilities at the state level prior to an incident.

So if a local county gets hit or a school board or a water system get hit, they know that they can pick up the phone immediately and call the state and leverage those leverage those resources. Similarly, and and I and I believe that we're gonna see a change in this.

I mentioned earlier the attacks the attack surface is how the bad guys really identify ways and and and and and ways to attack an organization. And so we've seen several states that have actually purchased and deployed, which our tools called Expanse. There are other tools out there. But what it actually does is it looks at covered entities within a particular area within a state, which could be a state organization or could be a local county or municipal.

And say, hey, by the way, you know that you've got this vulnerability that the bad guys can see.

You need to fix that now. Some of that is now becoming real time. I showed you the inverted v. Some of that's becoming real time. Those fixes happen automatically in

just a

few minutes before anybody even knows that's going on. With Frontier AI, you know, what we do with our customers, we have three different modules that we actually work. We do an attack a kind of an attack surface management, look at a system that leverages Frontier AI tools and finds those smaller attack vulnerabilities that can be what they call attack chained together to to to form a large, you know, a large and very significant attack.

Where we see it working well is organizations that have adopted that type of approach.

So I will say, through to California's leadership, I do feel like we're better prepared than a few other states. But I do want to recognize Assembly Gonzales saying that California is too sophisticated for these threats.

are potential, and if not very likely, is that California is the home of the fourth largest economy. We are targeted, I would say, probably more than any other state. And then if you look at the technology companies we have here, there's a lot of, gain looked by others.

Threats

But I would say it's the partnerships that we have with our private sector, and then also the our federal and state partners in other states of how we share that information to really bring the totality of cyber maturity up in the state. I'm I'm Palo Alto talked about, like, attack service monitoring, which is very important.

So we do have a a few offerings that we allow, individual organizations, if they would like to have their, ASM or their IP ranges evaluated, we provide that service to them free of charge. Also, through the MSISAC, which now the state manages, that's another service that they have. So we are really offering, I would call it, a pull down menu of services for, any organization in the state of California to improve your cyber hygiene and higher cyber resiliency. So I do feel like we're better prepared.

And I would say if you look at some of the examples recently, New York and Illinois actually passed very similar s p 53 acts.

If you look at some of the other states such as Ohio, Florida, and Texas, they have created what I call as a CalSEK kind of similar organizations to take their that looked at California's leadership and tried to model those behaviors after that. I do feel like we're in a really good, position.

But I also know California cannot rest on its laurels, and we really have to continue working with our partnerships such as the two AI companies who were here before who offer an a unique capability for California to be able to respond to cyber threats and protect our state infrastructure and the citizens of California.

Alright. Thank you very much. Do we have any other questions? Again, we're okay. Really appreciate you coming in.

Sorry that we're so rushed. But, again, we have a session starting soon, and we wanna make sure that we can hear from our last panel. Thank you for joining us today. And then, our for our final panel, we have doctor Nate Gleason, who is the program leader for cyber and infrastructure resilience at Lawrence Livermore National Laboratory. Welcome, and, we'll give you a little time to do an opening statement and then go through some questions.

Alright. Thank you, chairs and members of the committee, for the opportunity to testify today. My name is doctor Nick Gleason. I am the program leader for the cyber and infrastructure resilience program at Lawrence Livermore National Laboratory. For nearly seventy five years, Lawrence Livermore has forged a reputation for advancing national security and scientific innovation through world leading capabilities, including the national ignition facilities achievement of fusion ignition and El Capitan, until very recently, the world's fastest supercomputer.

Livermore is advancing the development and and secure application of artificial intelligence to accelerate scientific discovery, strengthen national security, and support critical US missions. In just the past three months, awareness of what highly capable AI could mean for cybersecurity, particularly as it relates to critical infrastructure, has come into focus.

With Anthropic's announcement and and, the previous testimony last April of the release of Mythos and its cyber capabilities, we've seen how AI can rapidly identify and exploit vulnerabilities in the software and devices we depend on for our daily lives. AI can do more to uplift attackers than just find vulnerabilities and write exploits though. This past month, we've seen a series of cyberattacks against water infrastructure in Minnesota and several other states.

These attacks were a step up in sophistication from the attacks that that targeted California entities in June because they compromised not just IT, the information technology or business networks, but OT, operational technology or physical process control networks. It

was previously previously disclosed in the industry report that Frontier AI models were used by the adversary to conduct industrial control system reconnaissance, develop exploit scripts, and plan post compromise activity against these OT systems in order to maximize damage. Also in July, we witnessed a fully autonomous and unintended attack on Hugging Face, the dominant collaboration hub for sharing AI models and datasets.

OpenAI's models, while being evaluated on a cybersecurity benchmark and without safety guardrails, broke out of their sandbox, gained Internet access, and autonomously compromised Hugging Face's production systems with no human directing the attack. It is the first publicly documented case of an AI agent autonomously executing a real world intrusion end to end. Each of these incidents involved a proprietary closed weight AI model, one where the developer retains control and can act to prevent recurrence.

We were asked to focus today on open weight models where that control does not exist and where the risk and policy picture is fundamentally different. Closed weight frontier models represent the state of the art in AI capability, OpenAI's GPT series, Anthropic's Claude family, Google Gemini, xAI's Grok. Access to these models is through customer interfaces or APIs. The model owner can implement guardrails to miss to restrict malicious use and can monitor and revoke access. Open weight models are released with their underlying parameters publicly available for download.

Once downloaded, these models run locally. No API call, no usage monitoring, no terms of service enforcement. The developer cannot implement guardrails after release, revoke access, or observe how the model is being used. They can be fine tuned, modified, and deployed without restriction by anyone with sufficient compute, including actors who would otherwise be denied access under closed models policies. Just as an aside, the terms open source and open weight are often used interchangeably, but they are not the same.

Open weight means the model parameters are public, but the training data and the methodology for creating the model remain proprietary. What that means is we cannot fully audit what capabilities or biases may have been built into the model intentionally or otherwise. While US companies, including Meta and Google, produce open weight models, China has been far more aggressive in this space.

Four of the five leading open weight models globally now come from Chinese labs, DeepSeq, Qimi, GLM, and Qen, and they are rapidly closing the performance gap with the closed weight frontier models. In 2024, Lawrence Livermore was commissioned by the Department of Energy to investigate the risks and benefits of AI.

The report identified four categories of AI risk, unintentional AI failures, adversarial attacks on AI systems, adversaries using AI to attack, and attacks on the AI supply chain. Open weight models present heightened risk in the latter three. Adversaries can access these models freely and tailor attacks against specific models, architectures. They can download models and remove guardrails, and unlike with closed models, there is no mechanism to monitor abuse or revoke access once the weights are out in the wild.

And importantly, models from untrusted sources may contain hidden backdoors or biases that are extremely difficult to detect without very thorough testing.

Despite these risks, open weight models have characteristics that may make them the best and sometimes the only choice for important applications. Privacy is the most significant advantage. With an open weight model, data never leaves your own environment. For Lawrence Livermore, this means we can bring AI capabilities onto air gapped systems and classified networks. For entities like an electric utility, it means AI can operate on process control data that regulations prohibit from leaving the utility systems.

Open weight models can also be fine tuned for specific applications yielding better performance with fewer computational resources, and there is no per use token charges, only the cost of your own hardware and electricity. The lack of guardrails, while at risk, can also be a necessity. When Hugging Face conducted incident response after the o I OpenAI sandbox escape, safety guardrails on Frontier closed models blocked them from processing the attack data.

They were forced to turn to a Chinese open weight model to complete their incidence response and investigation. This is our current operational reality.

At Lawrence Livermore, we are focused on nation state cyber threats to water, energy, and other critical infrastructure sectors. That's why we're partnering with UE on their artificially intelligence for operationally resilient technologies and systems or AI FORTS program. We're building tools to help infrastructure asset owners interpret and summarize process control network data that is restricted by regulation from leaving utility systems and automating cyber defense activities like threat hunting and incident response.

A major portion of our AI forts work is the development of AI test beds that assess the performance and risk of various AI capabilities. One of these test beds conducts adversarial attacks on AI models to to assess and strengthen their resistance to manipulation.

Another test bed characterizes model performance using automated agentic benchmarks suite focused on energy systems and OT or industrial control systems cybersecurity and also quantifies the advantage an AI AI capability can provide to an adversary in offensive cyber operations. Our newest testbed specifically leverages open weight models to verify proper AI function by forcing the model to show its work to human operators who can then validate

the performance.

While they come with some additional risks, open weight models are accessible, cost effective, and preserve data sovereignty. They are embedded in critical workflows across government, research, and industry. They are not going away. The dominant open weight models are now Chinese. China is not ambivalent about this space.

They are very deliberately leading. Today, if a US utility wanted to run a capable model an air gapped OT environment, the only practical options are Chinese made models. That is not an acceptable long term position for national security. The strategic question is not whether OpenWAIT AI will be a significant part of the global ecosystem. It will be.

The question now is whether that ecosystem will be led by The United States or China.

right now, it's China. Thank you.

And and

That was, very interesting. And the Chair had said we we need to have you in here, and I I could see that's, again, very timely because it seemed like last Friday, there was a big, big, division in Silicon Valley between the two big AI companies and then a lot of the smaller entrepreneurs who need who need these open weight or open source programs.

So just just a clarification, if if you have a developer that's using a an open weight program, presumably, every now and then, they would want to have an update, to always have the latest model, and that would be a time where you could get guardrails imposed again. Right?

Sorry. You could impose guardrails when when you initially distribute the models, but it is not a two it's not an impossible task to remove those guardrails. Once you release those ways, you release that model, it's out of control of whoever created it. The whoever downloaded it can then modify it however they like.

Even with an update?

Even

if even if the update comes in and and imposes the guardrails, it

it

doesn't take long to then just take them off again.

And then I just saw an announcement because I haven't you know, we had the the chance to talk to some of these developers, and, obviously, it's very important for them to have these, open weight models. And the obvious thing is to incentivize American companies to do that so you don't have this backdoor business going on. Didn't I just read yesterday that Meta now has a a powerful, open weight model coming out again?

Yes. Yeah. And and and that that sort of activity is what would help reduce the the Chinese dominance in that space. It's American companies, choosing to invest and create these open way model capabilities so that American users of AI have that option.

Do we have do you have a question?

Last statement that you made was the nail on the head. A lot of folks believe that The US is you know, we we are winning this this war. That is false. Thank you for reiterating that, which then brings me back to California. We we need to be on the we need to understand the global impacts, not only for for the season that we in right now, but the season that we will be in for a long time to come.

We have adversaries no matter how who's who's up top. We will always have people that are coming after it. So for me, when I look at the 36th Assembly District and the water infrastructure and the critical infrastructure protection, I think about the global, the the the the global adversaries who are actively targeting the 36 District, so to speak. So thank you for saying that.

It needs to be said louder and, you know, some people get shocked by it, but it's it's always been the reality and it will continue to be the reality. And if we're not actively partnering, then we're we're gonna get caught.

Well, thank you for being here. I, you know, I have the privilege of going to the lab often, and the work you do is really incredible. Some of us had the opportunity, I think, as someone where Irwin's there to see the work the lab is doing that you lead on.

They have a mini grid built with LEGOs that they actually try to take down and try to protect, and the work they're doing, I think, really is protecting the national security of the entire nation, but also, of us here in California. So thank you for your mission and for, taking your brilliance and serving the people with it.

So my question is, I have long been a believer in open source in part because I've seen the way the lab uses it in their air gapped capacity and understood how critical it was to our national security that the lab has the ability to continue to do those things with open weight models. And but I take your point seriously that I don't think I fully realized that right now if you wanted to do that work, you're mostly using Chinese open weight models.

That was I think, a few years ago when I was there, mama was what was being used. So there has been a shift in the couple years since I last heard. What would it take for The US and California to really start to lead not just in the closed models, but in open weight?

That that that's a good question for for for what it would take. I think it's a matter of embracing open weight. I think open weight gets you know, can get a bad reputation because, again, one of the advantages of the closed weight models, and and we heard this from OpenAI and Anthropic, they worked very hard to put in the guardrails and monitor abuse.

All the instance we've talked about have been because these companies have been able to monitor how people are using their AI and they found them, and and then they can ban those accounts. That doesn't exist for open way models, and so the immediate reaction is these are dangerous.

We shouldn't use them. The challenge is, even if we were to ban open weight models in The United States, there there is no border, in in the digital world. That will come from from from afar. So I think it it it's more of a a a requirement of of embracing here's here's where we would wanna use closed weight, here's where we wanna use open weight models and supporting the development of that of that that that sort of ecosystem.

One challenge right now is my my focus a lot is on cybersecurity, not just in general, but of critical infrastructure systems because that's where the digital world meets the physical world. You can actually have things like human lives lost in those cyberattacks. In that space, Department of Energy has a a very strong focus on understanding risk to the energy sector. There are 15 other critical infrastructure sectors that are also very important that don't get the same attention That the energy sector does.

So so while the energy sector is doing great things and can lead the way and kinda show show the way for for some of the other sectors, we need to focus more on enabling these tools for some of these other sectors. We've seen the water sector

Yeah.

You know, come under attack very recently.

Thank you. And then my last question for you, although I will say one of my big beliefs is that the way we can lead further in this space is Cal Compute. I am committed to getting that done. I do think giving compute to the public as the labs do in in part, is really critical to our ability to compete. But, so that's my own opinion.

The other question I had is you've you've sat here, you've heard what OES had to say. Any thoughts given your deep expertise in protecting cybersecurity infrastructure on what California should be doing better or more of in order to protect our own infrastructure?

Yes. So I think I think there were some some really great ideas mentioned. One of the things I I do want to emphasize is, we talk about AI being used to find vulnerabilities, AI use being used to develop exploits, AI being used to write patches. When you're talking about patching, you know, we we heard, you know, that can be difficult with legacy systems. It can take time to do.

All of those factors are an order of magnitude more challenging when you're talking about OT systems because then it's not, oh, I need to take my mail server down for an hour to apply this patch. It's I need to shut off the electric grid, for an hour to apply this patch. I need to go around and physically visit All these substations, all these water treatment plants, and and, you know, plug a USB stick into these these devices and and update things. So patching is great.

It is it is the sort of the golden answer for for eliminating these vulnerabilities, but it's not practical in a lot of cases. And so really understanding other ways that you can do the patching. The gentleman from Palo Alto Networks mentioning the pseudo patching, the virtual patching, blocking things at the firewall is a great approach to doing that as well. I I think we need to really think in a new paradigm. AI is here. It it's a new world.

These vulnerabilities that that were challenging to exploit, the massive amounts of expertise it took to really think about causing damage in a critical infrastructure system, that's now available to everyone because of AI. So we need to think more creatively about how we use it on the defender side. One other piece that, that the state can do is also pay attention to the information that goes out in the public domain.

You know, it's one thing to try to attack your critical infrastructure system, break into the system, and then just, you know, cause chaos. It's another thing if you understand how the system works

Oh, interesting.

Can cause chaos, cause damage, do manipulations in a way to maximize impact. A lot of the information that's put out in public source about our critical infrastructure systems can be very valuable, in helping an adversary, figure out how to do that. And AI makes pulling all that together and collating it and coming up with fairly sophisticated attack plans much, much easier.

Thank you so much.

Alright. Just, one last thing. You had you had talked about the the, like, open way models right now. At least this week, it seems like, China's ahead. And then with with, anthropic and OpenAI, they seem to be the most powerful right at this point.

And those are the companies that we're talking about with this, like, cyber offense and cyber defense and trying to patch as quickly as possible. How far behind are the Chinese with this in your estimate? I I mean, I know it's not completely your level of expertise, but there is a real concern if they are developing those models to start to look at, vulnerabilities.

I I think the the generally accepted estimate is is there several months, but it's measured in in months that they're behind. But one of the things that I think is important to realize is a lot of these attacks and a lot of the damage does not require them to be as good as, you know, OpenAI and Anthropics models. They can do a whole heck of a lot of damage even with much lower capability, AI models. We we've gotten to the point where all of these AI models can significantly, enable an attacker.

Alright. Well, thank you very much. This was a extremely informative day and and appreciate you coming and our other panels also. I know some of you have traveled a long way, and some of you have sat in traffic for a long time. Today's discussion highlighted both the tremendous promise, and the important challenges, presented by Frontier, artificial intelligence.

I wanna thank each of the panelists again for helping to inform the legislature as we continue working to ensure that California remains both a leader in AI innovation and a leader in cybersecurity preparedness. This joint hearing of the Assembly Select Committee on Cybersecurity and Assembly Committee on Privacy and Consumer Protections is now adjourned. Thank you very much.