Assembly Standing Committee on Privacy and Consumer Protection
- Chris Ward
Legislator
Alright. Well, good morning, everybody. We're gonna go ahead and get started with our informational hearing this morning. My cochair is, I think walking into the capital as well and be able to join us in a few minutes. And of course, we certainly welcome other members of the Assembly Select Committee on Biotechnology, Medical Technology, as well as the Assembly Committee on Privacy and Consumer Protection, to join us here for this hearing this morning in Room 126.
- Chris Ward
Legislator
I'm really excited, about this hearing. I know we just got back from our summer recess. So yes, I've got my batteries recharged. But this is something that absolutely has been evolving very quickly in our lifespan.
- Chris Ward
Legislator
And certainly for this industry, we're really excited both of these committees, to learn a lot more about the snapshot in time, about where we are here today, and certainly your insight on where we're headed to be able to better inform us in the work that we're doing up here as we're making critical policy and budget decisions that hopefully will align this so that, yes, our our companies, our job sector and other attributes that will be in good shape here in California.
- Chris Ward
Legislator
But I think most importantly, that the advancements, the outcomes, the health care, the discoveries, the cures, that you're seeking are closer within reach, and I think that is certainly the goal of what we're like, we're seeking to see. Today's hearing is titled, artificial intelligence in the life sciences from research and development to patient care.
- Chris Ward
Legislator
And we know that California life sciences industry here is really generates close to $400,000,000,000, to our state's economic impact, supports over a million jobs, and that's because of a lot of the companies that choose California, from our start ups to industry giants. They have life changing advancements, but also are deeply invested in our local communities, and they fuel economic growth. And they of course, attract significant venture capital here to the state.
- Chris Ward
Legislator
And we know at the same time that artificial intelligence is really shaping the future of our economy and our health care industry. California Life Sciences are using AI in drug discovery and manufacturing medical technology, and I think that's what makes it so interesting is I have a deep and and long background working in biosciences and in medical technology.
- Chris Ward
Legislator
And, you know, I can remember the time thirty years ago that, you know, life behind the bench was, you know, pretty pretty pretty difficult, pretty slow, pretty tedious.
- Chris Ward
Legislator
Years go by and your experiments would fail. You had to go through an array of substance and product, to be able to maybe find that kind of discovery, and see if your theories actually worked.
- Chris Ward
Legislator
And so AI now coming into that mix and everything is gonna rapidly accelerate, our ability to predict, what could be successful, really be able to get to those successful experiments and those, those successful, drug discoveries a lot faster. And so it's just a really exciting moment in time that we're here. And so, to that end, we're hoping that this hearing today is gonna be able to provide us a lot of information on how AI is being is changing, the landscape for biotechnology industries.
- Chris Ward
Legislator
We know a lot of positives that AI can bring to our health and quality of life. But, of course, there are gonna be some big privacy and consumer protection questions and challenges to ensure, that AI isn't being used to further exacerbate any kind of growing health equity gap, or any potential bias.
- Chris Ward
Legislator
We need to know about the safeguards and best practices in place because we wanna make sure this is being used responsibly, what checks and balances are there since we're dealing with a real growing amount of potential for using personal health care information. So I think that that's something we're gonna be able to talk about a little bit more at this information hearing today.
- Chris Ward
Legislator
I wanna welcome and thank, the collaboration here with the Assembly Standing Committee on Privacy and Consumer Protection and invite Chair Bauer Kahan to be able to say some opening remarks.
- Rebecca Bauer-Kahan
Legislator
Thank you so much, Chair Ward. Thank you for convening this hearing in this important conversation. I wanna thank all of our panelists for attending and participating today in advance and of course, thank both Chair Ward staff and our staff at the Privacy and Consumer Protection Committee, rules committee, our sergeants, and everyone who supports these important hearings.
- Rebecca Bauer-Kahan
Legislator
So I think this hearing is really exciting because as we have a conversation about AI and its promises, often health care is what we talk about when we talk about the promises of AI. I was watching recently, an interview with one of the leading AI companies.
- Rebecca Bauer-Kahan
Legislator
And when asked about, the perils and promises, they just kept saying, well, AI will cure cancer. And the truth is, I think it is where promises might be the largest. I don't think that that was an exaggeration and each of you is at the forefront of seeing that promise come to fruition. But it's important that we balance that promise as noted with ensuring that we don't also exacerbate the perils. And I am someone who believes we can walk and chew gum at the same time.
- Rebecca Bauer-Kahan
Legislator
That we can achieve the promises, we can protect privacy, and we can protect our communities. And I know that each of you are working hard to ensure that that is the future we face. So thank you for that. And it's also critically important that we look at California's role in ensuring that these benefits are fully realized.
- Rebecca Bauer-Kahan
Legislator
I will use this moment to plug one of my favorite projects that California has been discussing but not achieved to date, which is Cal Compute. We know that one of the things that makes California's life science industry what it is is our university system. Whether it's UCSD or UC Berkeley, we are creating incredible talent and brilliance.
- Rebecca Bauer-Kahan
Legislator
Our labs on our campuses are helping to partner with private industry to create the future of biotechnology and science and ensuring that we are the first in the world, I believe, to provide public compute power to put AI into the hands of more people because that will make us realize more promise.
- Rebecca Bauer-Kahan
Legislator
I think it's something that California should be committed to and we should work to realize in the years to come. Should work to realize in the years to come. So I'm really excited for the conversation today and all that we will learn and how we can better leverage both the talents in private industry, as well as the public sector to give Californians the future they deserve, which is a long, healthy life where they can thrive and enjoy the beauty of our great state.
- Chris Ward
Legislator
Well, thank you, Chair Bauer Kahan. And I really appreciated that example as well on Cal Compute because I think exactly where we're gonna kick off is hearing here today with our first panel, from, you know, representatives of just of a few of our great institutes here in the state of California, from UCSF, from Parker Institute, and from Lawrence Livermore Laboratory.
- Chris Ward
Legislator
I'd like to invite up the three panelists, Doctor Ida Sim, who is a Professor of Medicine at UC San Francisco on Computational Precision Health at UCSF and UC Berkeley, and Dr. Karen Knudsen, the Chief Executive Officer of Ford Parker Institute for Cancer Immunotherapy, and Doctor Shankar Sundaram, the Deputy Principal Associate Director for Mission and Engineering and Director of the Bio Resilience Incubator at Lawrence Livermore National Laboratory. Welcome. We're really looking forward to your presentations.
- Chris Ward
Legislator
This, I think will be a good theme about developing AI technology in the life sciences, and, we'll begin with Doctor Sim.
- Ida Sim
Person
Great. Wonderful. Good morning, everybody. Mister chairman and members, thank you for the opportunity to join you today. My name is Doctor Ida Sim.
- Ida Sim
Person
I'm a professor of medicine at UCSF and also cochair of the UCSF UC Berkeley joint program in computational precision health. I'm also a practicing primary care doctor. As you know, the University of California is the research arm of the State of California, and you see research as a major driver of the California economy as we've heard, and produces life saving treatments for patients in California and indeed around the world.
- Ida Sim
Person
Today, I'm pleased to describe and discuss with you research from my lab and those of my faculty in computational precision health and how we're advancing AI and medical technology, which I'm gonna discuss actually is digital health. Certainly, there's a lot of work in biotech.
- Ida Sim
Person
There is this growing area of digital health, which is technology for patients and sensors, and I will be focused on on that aspect of medical technology. First, a note about computational precision health or CPH. We are UC's only fully by campus program, that brings together UCSF's top notch clinical care and research with Berkeley's world renowned leadership in computer science, statistics, and public health. It's really a marriage of two amazing institutions.
- Ida Sim
Person
CPH shows how the University of California takes creative leadership in emerging areas of high value research like AI and health.
- Ida Sim
Person
So let's start. If I asked you where is the frontline of health care today? What would you say? I think many people would say the bedside, nurses, the clinic. As a practicing primary care doc with over thirty years of patient care, I would say the true frontline of health care is daily life.
- Ida Sim
Person
You are the frontline of your health care. It's 24/7 at work, home, and play, and yet traditionally, we think of health care is only what happens behind the walls of the clinic and hospital. With AI and digital technologies, however, like smartphones and sensors, those walls can now come down. I've been calling the medicine that we can do that's beyond the Clinics Edge Medicine.
- Ida Sim
Person
It's at the edge, where as a Doctor, I can for the first time get a window on patients disease state between visits when, you know, between times when they come see me.
- Ida Sim
Person
And for patients, AI can now provide information and support between visits like never before. And the walls between the health care systems and the edge must come down. 90% of US's $5,000,000,000,000 in health care starts at the edge. It's chronic diseases. As you see listed on the bottom here, heart disease, cancer, chronic lung disease, stroke, and so forth.
- Ida Sim
Person
75% of Americans, three quarters of Americans have at least one of these chronic conditions, and over half have two or more of these chronic conditions. And chronic conditions are exactly that, they're chronic. They're 24/7 at home. We cannot and should not wait for people to come to healthcare systems when technology now allows us to go to them. So let me present, and this is the I will present today on three main areas of research in my lab in CPH.
- Ida Sim
Person
First is on sensors, making sense of data, and then connecting care between the edge and traditional healthcare systems. So for sensors, it turns out that half of Americans have high blood pressure. 40% of people who have high blood pressure don't know that they have it.
- Ida Sim
Person
That's 1 in 5 Americans walking around with high blood pressure of hypertension and don't even know it. And of the ones who do know that they have high blood pressure, only 35% are controlled.
- Ida Sim
Person
So tremendous amount of prevention that we can do. And firstly, to address the control of blood pressure is that we need to get your blood pressure rating, and yet that is actually really hard. And one reason for that, of course, is checking your blood pressure at home is actually quite a pain.
- Ida Sim
Person
When I ask my patients to check their blood pressures at home, they come to me with numbers scribbled on a piece of paper, And then I manually enter that data into my electronic health record in 2026, and that's ludicrous. So I am working with a California based company called CareX to test an amazing new technology.
- Ida Sim
Person
CareX uses your phone camera to sense ultra fine grain color changes on your face to estimate the stiffness of your blood vessel and from there to infer your blood pressure. This is technology and engineering that's gone on forever. They've been able to now make that real, and it works for patients of all skin colors in various kinds of lighting. And in fact, it works on Zoom. So I can do a virtual visit with my patients and get their blood pressure showing up on the screen.
- Ida Sim
Person
And that's the kind of outreach to patients that doesn't require a blood pressure cuff. And once you set it get it set up, you know, people even with low technical literacy can use it. We've just started to integrate this into our clinic at UCSF. Initial patient responses have been very enthusiastic. Some of the problems that we face, is the challenge of bringing this data into our electronic health record just technically.
- Ida Sim
Person
And also there are many companies doing remote blood pressure monitoring. And so how do we buy these services and how do we integrate them in a way that doesn't you know, generate a lot of silos and a lot of confusion. It's very exciting. Next, making sense of health data. You know, health care generates tremendous amounts of data.
- Ida Sim
Person
AI is crucial for helping, patients and Doctors make sense of that fire hose. An example then is to take a lot of data, squish it down into one actionable item. Ziad Obermeyer, from our CPH department just came up with a ECG biomarker for elect for your twelve lead electrocardiogram that predicts your risk of sudden death. Now that means you have to come in and get a twelve lead EKG, but now we don't have to.
- Ida Sim
Person
Alex Schubert, one of our students in CPH, has now taken this model into using just a single lead EKG, which you can now get from their sensor like an Apple watch.
- Ida Sim
Person
So you can imagine how we're getting data from the edge and making sense of it and making predictions. Now to get the most out of digital health, it is critical that this kind of health data has to be able to be combined together to give a full picture of our health. The data should not be tracked here and there in different health systems, different apps, and different sensor companies.
- Ida Sim
Person
Here's where the connection between UCSF and Berkeley really mattered. For 20 years, I have been working on open source software to make data flow between wearable sensors and electronic health record.
- Ida Sim
Person
It's been hard. At the same time, Professor Fernando Perez from UC Berkeley built something called Jupyter, which is a world changing scientific computing platform that has powered entire fields of science. It's really one of the star discoveries and inventions out of Berkeley. It has changed the fields of astronomy, chemistry, and geology.
- Ida Sim
Person
In 2021, Nature, a top ranked science journal named Jupyter, one of the top 10 computer codes that transformed science. But in healthcare, nobody had heard of it.
- Ida Sim
Person
So in one hour, one single one hour conversation, Fernando and I got together, put two and two together, and now we're bringing Jupyter, then the power of Jupyter to the 18% of GDP that is healthcare. We are building Jupyter Health, which very simply put take silos of individual solutions, which is what we have these days, the solutions and sensors. They're all siloed from each other.
- Ida Sim
Person
Like the early adopters we have here, many of you can't read the screen but we have pilots go with UC, Duke, and Cornell, and Columbia. Take individual solutions and AI models and metaphorically flip those silos on their side and create on the bottom an open standardized pipeline so that data and AI can flow, connecting healthcare, traditional healthcare and the edge and then back again.
- Ida Sim
Person
We have secured over $10,000,000 in philanthropic support to build Jupyter Health. We are transforming digital health research and digital health investing, which in 2025 was a $14,200,000,000 for digital health starts up, most of that in California. So in closing, the front line of healthcare daily life is at the edge.
- Ida Sim
Person
Digital health is a frontier that is made dramatically possible by new sensors and AI. The University of California's researchers are blazing the path of course, in sensors making sense of data, technology, infrastructure.
- Ida Sim
Person
These discoveries and inventions will touch every Californian. It is not a tomorrow thing. It is a today thing. These technologies will allow us to reach underserved communities in their language, at their level of language, whatever Spanish, English, Chinese. Many of my patients are Chinese speaking.
- Ida Sim
Person
This digital health is a huge opportunity that California cannot miss, and investments in UC research will ensure that California continues its leadership. I would be like delighted to answer questions. Thank you.
- Rebecca Bauer-Kahan
Legislator
I just have to say this is why the UC is the envy of the world. Just had to say it.
- Chris Ward
Legislator
Thank you. No. I'm already starting to generate. I know it's well, Rep. Bauer Kahan, you know, some pretty interesting questions. But we're gonna go ahead and hear from all of our panelists and then maybe take questions, like you know, as a group.
- Karen Knudsen
Person
Okay. Yeah. Karen Knudson. I am the CEO of the Parker Institute. Agree with everything that was said.
- Karen Knudsen
Person
Obviously, I'm a big believer in wearables. But what I want to talk about is cancer. In the case of cancer, which I really think is the use case poster child of AI for good, not taking jobs from individuals, but allowing us to escalate getting science to people, and patching long standing holes in health care systems.
- Karen Knudsen
Person
And I can say that having run oncology for one of the largest health systems in The US. So cancer is by definition complex. It is two it's not diabetes. It's not Alzheimer's disease. It's 200 different diseases that all arise due to different reasons and require different mechanisms for cure.
- Karen Knudsen
Person
As we sit here today, 2.1 million Americans will hear you have a new cancer this year, and 600,000 will die of their disease. Of those 2.1 million that hear that they get a cancer diagnosis, 200,000 of them live here in California.
- Karen Knudsen
Person
And those are just those with a new diagnosis. We also have 19,000,000 cancer survivors here in The United States, and every one of those cancer survivors will have a higher than average risk of developing an additional cancer in a recurrence but also an additional second cancer as a result of having been a cancer survivor. So cancer is a major problem.
- Karen Knudsen
Person
It is the number 1 cost of care for all mid and large size self employed companies here in The United States, and that's a new fact. We also know that cancer is coming earlier and plaguing those under 50 population, which is something that we hadn't expected.
- Karen Knudsen
Person
Again, I was also the former CEO of the American Cancer Society, the rights cancer screening guidelines. We actually had to drop our guidelines to a younger age because of this early onset cancer for which we actually have no preventative strategy. So how are we going to win against cancer using AI?
- Karen Knudsen
Person
I think there are three major use cases. One, I think you're going to hear quite a lot about today, so I won't expand quite a lot on it, and that's on accelerating drug discovery.
- Karen Knudsen
Person
We know much of that work has started here in California where, including at our own Parker Institutes here in California, where AI is allowing us to test scientific hypotheses instead of in the ones and twos as you talked about, Chairman Ward, back in the day but testing them now by the thousands and by the minute to do the right experiment the first time.
- Karen Knudsen
Person
This is really important in the context of cancer as we develop new curative strategies which are coming through immunotherapy.
- Karen Knudsen
Person
Right now, the time from discovery to FDA approval for a new oncology agent is between 12 to 15 years. AI is poised to shorten that time frame. Right now, the cost of bringing a new oncology drug to market exceeds $2,000,000,000.
- Karen Knudsen
Person
AI is poised to make us, more effective and cost effective as we make the smarter choices using data.
- Karen Knudsen
Person
We know that many of our, our own portfolio companies at the Parker Institute like 3T Bio, a Bay Area based company, is actually leveraging AI to learn from someone's own immune system and fine tune off the shelf cancer therapies and are about to enter into that clinical phase for advanced colorectal cancer, which is one of the major issues, the second leading cause of cancer death, for in total men and women here in The United States as well as in California.
- Karen Knudsen
Person
So the accelerating drug discovery by essentially synthesizing complex scientific datasets as you've heard about is poised to allow us to go faster. But I don't think that's going to cure the cancer problem. It's the other two.
- Karen Knudsen
Person
The second is that we need to learn from every patient. Cancer is by definition complex as we talked about. But if you take any individual single cancer patient, you have their pathology, their genomics, their family history, their lab testing, imaging that they've had, their response to therapy.
- Karen Knudsen
Person
And so what we can't do at present prior to the time of being able to utilize AI is to look at very deadly cancers like uveal melanoma, a melanoma of the eye, which is unfortunately incredibly has a very high mortality rate, and it's very rare. So it's hard to learn from these patients.
- Karen Knudsen
Person
AI is going to allow us to take all of the information from these patients, and then determine what worked, what didn't work, why not, what are those next hypotheses that we can go after. And, you know, there are as well other ways that we can look at preventing cancer through using AI by learning from every patient.
- Karen Knudsen
Person
California grown companies like Color Health, again Bay Area, is using that go to employer strategy to ensure that employees of these mid and large sized health companies actually have access to cancer screening, are navigated to care, get guideline concordant care, and then again poised to learn from every patient with their virtual cancer clinic. So these are one of the ways that AI is not a promise.
- Karen Knudsen
Person
It's actually happening now while we're sitting here And our ability to learn from every patient and then do better for that next set of patients with uveal melanoma or any of the rare cancers for which cures have been elusive.
- Karen Knudsen
Person
And then finally, clinical trials. One thing that we've known from oncology from the beginning is that clinical trials are the most advanced form of care. Full stop. Unfortunately, and I'm 56 years old, in the entire time I've been in oncology, we have been bemoaning the fact that only single digits. Right now in The US, five percent of oncology patients go on clinical trial.
- Karen Knudsen
Person
Why is that? AI is helping us. Some of it's regulatory, but some of it is just having AI help us get out of our own way. So we know that 76% of oncology patients want to be on a on a clinical trial. So there's a pretty good big gap to fill.
- Karen Knudsen
Person
So again, California homegrown companies like Paradigm Health, are poised to do that leveraging AI right now across the world actually but also here in California to determine should a particular hospital system open the clinical trial. Do you have the right patient population? Do you have what would have to be true with this trial? Is there something that would need to be tweaked in order to make it more suitable for your patient population?
- Karen Knudsen
Person
The second component of actually matching patients to study, taking away that big gap that we have in a workforce system of actually identifying which patient is eligible for which trial.
- Karen Knudsen
Person
And then third, which plagues every cancer center director, it's how do you actually pull down the data? Taking all that complex data set from somebody on study is a brute force, very difficult job that's been manual my entire career that now AI is helping us fix.
- Karen Knudsen
Person
So you have really confidence in groups like Paradigm that are able to make that accessible, and we're enjoying that as well at the Parker Institute as our portfolio companies get into the phase of clinical testing. They've been great partners for us. So in some, I think California has led in AI.
- Karen Knudsen
Person
I think California can lead in AI for health that California absolutely is, as we talked about, starting to lead in AI as we tackle the cancer problem of everything from prevention to detection and cure. Thank you.
- Shankar Sundaram
Person
Good morning. Chair Ward, Chair, Chair Bauer Kahan, and members of the committee, thank you very much for this opportunity to talk to you all on this important topic. Brief introduction. My name is Shankar Sundaram. I am the Director of the Bioresilience Incubator at Lawrence Livermore National Laboratory, one of the DOE national laboratories located in the Bay Area.
- Shankar Sundaram
Person
Prior to this, in '23 and '24, I served in the White House as the Senior Director for Global Health Security and Biodefense. For more than 30 years, I've been working at this intersection of biology, computing, and national security. So briefly, Lawrence Livermore, why am I here? Why is Lawrence Livermore here? Most people know Lawrence Livermore from the nuclear security and sort of the stunning result on ignition fusion.
- Shankar Sundaram
Person
But less well known is its foundational role in a lot of biotechnology, particularly focusing on another AI for good topic is a public health and biosecurity.
- Shankar Sundaram
Person
So I'm showing a couple of examples starting from sequencing human chromosome 19 as part of the original human genome project, ultimately leading to the development of foundational diagnostic technologies, BioWatch, which is still in use today, as well as companies that spun out of of some of that foundational work, including Cepheid and Quanta Life, companies that, you know, leverage some of the technologies of founders coming out that started data generation companies like Ten X Genomics over in Pleasant in California or Second Sight and Neuralink that blazed first, you know, of its kind neural interfaces.
- Shankar Sundaram
Person
A lot of them owe either technology or founders back to Lawrence Livermore. And now Livermore is again, pushing the frontier again using AI and high performance computing to advance, you know, AI for public health and biosecurity.
- Shankar Sundaram
Person
One example is using El Capitan, which until couple of weeks ago was the top supercomputer in the world to help develop OpenFold three, which is basically the protein folding AI model that is similar to AlphaFold, which everyone has heard of, but out in the open helping the development of novel cures and understanding disease.
- Shankar Sundaram
Person
So with this as background, I just wanted to offer a couple of observations to start off with. First, you know, I'm an engineer. Right? But it takes no I mean, like it's particularly exciting for me to say that biology is becoming an engineering discipline. Chairman Ward, I relates to what you said before.
- Shankar Sundaram
Person
That's why initially, I chose engineering as a quantitative science rather than biology because it was more just observational and describing. And I think it's just changed completely now and no minimal part due to AI that we can actually move to designing rather than just describing. And I think the benefit is very, very clear. We can diagnose diseases early. We can design cures.
- Shankar Sundaram
Person
We can deliver or deploy them really fast as well. So I just wanna illustrate a few examples from sort of the larger team at Lawrence Livermore. Working with Kaiser Permanente Northern California and act and working with them on their electronic health records, the team showed that analytics could predict outbreaks about up to three weeks earlier than when you pick it up using syndromic surveillance. And this was about one third, right of the outbreaks that they tracked.
- Shankar Sundaram
Person
Just shows you the power of what you can do by harnessing the data and applying modern algorithms.
- Shankar Sundaram
Person
Couple this with things like wastewater, not just the PCR but genomic tests. And we're already showing that you can actually not only pick up, but also you you don't have to know what disease you're looking for. You can actually pick up diseases that you do not know are spreading in the community or potentially incident in the community.
- Shankar Sundaram
Person
And before several patients are walking in the clinic and spreading in the community, you can actually go to the source and try to kinda stop it. So that's the power of of bringing some of these approaches.
- Shankar Sundaram
Person
Other examples during COVID 19, I was privileged to be part of the program where folks use high performance computing to be able to redesign a clinical antibody from AstraZeneca. There was a frontline clinical antibody that lose that lost potency against Omicron in just three weeks using our then supercomputer, Sierra. Right? We were able to redesign it to to regain the potency.
- Shankar Sundaram
Person
And more, even more impressive than that, the same team was able to then design these antibody therapeutics to be future this protective. Right? 44 future variants that could emerge, some of which did emerge.
- Shankar Sundaram
Person
So that's the power of using engineering as opposed to observational biology. Similarly, other groups have used same high performance computing to attack what was considered undruggable in cancer, KRAS, with first in class therapeutics that BridgeBio, a California based, you know, biotech company is commercializing.
- Shankar Sundaram
Person
Even more recently working, leveraging some of the VA data, working with our Stanford colleagues, right? teams published, that they could identify 18 common drugs that enhance survival in ALS.
- Shankar Sundaram
Person
Imagine I mean, like, just things like those kinds of insights are locked in our datasets. So that's the power, and that's the experience that some of our teams bring to the table. And the other observation is, you know, I think, Chairman Ward, you mentioned the 400,000,000,000, you know, 1,000,000 jobs. California is the big player. Right?
- Shankar Sundaram
Person
Even compared to UK or China or other companies I mean, other states. But that is under threat. Right? I think you've seen the headlines last year. One third of the inlicensed drugs, right, from US companies was coming from China.
- Shankar Sundaram
Person
So I'm worried about the state of innovation. And then the reason I'm worried is that when innovation moves overseas, then manufacturing, clinical trials, supply chain, training, workforce, everything follows afterwards. So it becomes more of a a hollowing out of our the ecosystem that's been built painstakingly over decades. It's not gonna be a sudden collapse, but, you know, we've seen this movie play out before.
- Shankar Sundaram
Person
So I'm really glad that the committee is taking a look at what can be done to protect our ecosystem in here.
- Shankar Sundaram
Person
So I just I wanted to not all to mingle, but I wanted to acknowledge that, you know, California is a leader for a reason, including, you know, the institutions my fellow panelists represent, UCs and CSUs and institutes like Parker Institute and IGI and and Scripps and so on and so forth. The company, the life science ecosystem, San Diego, Bay Area, elsewhere, the tech and private sector, we have unmatched assets to bring to bear but they are desperate.
- Shankar Sundaram
Person
So, you know, having the components is not sufficient. They need to work together as a connected whole. And so looking at it, I had you know, kind of three thoughts.
- Shankar Sundaram
Person
And this is the last one. So I I just wanted to kind of highlight three areas for your consideration, right, if you want to maintain California as a leader. I think the first one, I was very happy to hear about the data problem being, you know, so eloquently and personally testified to. We have to unlock and unleash the data. Our strength is really our diverse populations.
- Shankar Sundaram
Person
Right? The individual initiative and effort compared to certain other adversarial or or strategic adversaries, if you will. And so bringing these datasets together is, I think, going to be really important. Of course, you have important privacy and trust considerations to balance as you do that. But doing that I mean, I'm just the UC health system, which we've had the privilege of working with, unmatched in terms of, the the the the depth and and and variety of data records.
- Shankar Sundaram
Person
And so, again, Kaiser Permanente, which is, you know, one of the leading providers was able to come to us and say, like, here, what can you do with this? So being able to kind of put all of these assets together with the public health CDPH and other datasets, I think, is gonna be really important. And there are important solutions that are coming up.
- Shankar Sundaram
Person
It's it's not like we're moving all the data to a central place where it's gonna be, you know, access will be an issue or or or or things that could be vulnerable to cyberattacks. There are novel solutions beyond my scope.
- Shankar Sundaram
Person
I'm not an expert, but I'm sure, right, that folks can tell you about how, you know, models can travel and data can stay put. And the second thing, I was really happy to hear about CAL Compute. Right?
- Shankar Sundaram
Person
And that was gonna be a a recommendation in the sense that what I see and observe is just like in the mainline mainstream AI, in the AI bio world as well, our faculty and small startups are getting locked out, right, of being able to participate in the front edge, right, simply because it requires a lot of compute or it requires, in addition to compute, also laboratories to be able to design and test what you're making so you can close that feedback loop really, really fast.
- Shankar Sundaram
Person
We can actually do this. We have shown this in days compared to months and and and and years. So thinking about it as an ecosystem compute as well as the Automated Bio Labs. Right? Then now California is gonna be leading in agentic AI, in self driving labs.
- Shankar Sundaram
Person
Right? But not just to you know, but making that available to our public institutions, to our start ups, and our ecosystem so that they can participate in an equal footing in this in this revolution and spawn a new generation of of of technologies.
- Shankar Sundaram
Person
And and last but not least, I'm just looking back at the EV example, electric vehicle example, right, about how California led the way in terms of of of setting standards, if you will, right, and letting the marketplace compete to come up with the right solutions. So I'm just imagining in a similar way, right, in my own neck of the woods and looking at public health or biosecurity, for example, which suffers from lack of a buyer, right, unlike maybe cancer or neurodegenerative diseases and so on.
- Shankar Sundaram
Person
You know, California or others could consider, you know, sort of sending, you know, a visible signal of demand that, hey.
- Shankar Sundaram
Person
You know, if you were to build these early diagnostic technologies, we want to push wastewater surveillance. We want to push wearables. We want to push, you know, electronic health record access and so on with the with the view of, you know, not just infectious disease, but looking at sort of environmental exposures. Right? So a lot of our population lives with exposure to pesticides on one side or toxins on the other side and brush fire and whatnot.
- Shankar Sundaram
Person
Right? So thinking about how we can leverage all of this to make breakthrough discoveries in that, I think, could be a a very useful thing for us to do. So but really sending that visible one other example. I was in DC in '24. In about August, we were watching or we were managing the h five n one outbreak.
- Shankar Sundaram
Person
Right? And I remember when some of the first cases, the first two or three California firms came in as, like, farms, right, as as having h five n one. We're worried about who's exposed to it, and then it rapidly grew. You know, within two months, you had about 600 dairy farms that were affected, right, in unknown farm workers. Think about those are the kinds of situations where we need to have these kinds of systems in place before the crisis hits.
- Shankar Sundaram
Person
So overall, finally, I just wanna say that, you know, the California biotech ecosystem is a crown jewel, but it was built by, you know, discipline, by sustained attention. And AI is clearly transforming the entire area. And if we want to kind of maintain the leadership, you know, some of the the these areas and what you heard from the panel, I think, are are really fantastic places to start. Thank you.
- Chris Ward
Legislator
Great. Thank you all for your presentations. I'll start off with Chair power of hands, see if you wanna kick it off with a couple of questions.
- Rebecca Bauer-Kahan
Legislator
Okay. I think that was such an amazing way to close. I have conversations with my colleagues often on how California has this crown jewel, and we assume we can never lose it. And I will say the way the state has treated the r and d tax credit is proof that we are not convinced that it can walk away from us. And I think your point is very well taken that it can, and we need to continue to make sure it thrives.
- Rebecca Bauer-Kahan
Legislator
And we are doing a lot, and I think you highlighted a lot and like a proud, you know, legislator. For those who don't know, the lab is located in my district. It is one of the
- Rebecca Bauer-Kahan
Legislator
Constituent. And what they are doing and ten x genomics is a perfect example of the ecosystem that the lab has created through a public private partnership that is really creating innovation in a way that I think a lot of people don't understand. And one of the things you didn't mention was the open campus, which is one of the newer features of the newer in my tenure features of the lab. The lab, as mentioned, does a lot of nuclear science.
- Rebecca Bauer-Kahan
Legislator
So the so the lab did an open campus so they could do more private public partnership. And they are doing, to the point that was made, work around advanced manufacturing and things that really are changing the game for bioscience in a way that allows for that collaboration that I think is so important. And I will note that at one point many years ago, California engaged in this.
- Rebecca Bauer-Kahan
Legislator
Go biz, put some money into an incubator in the community that would allow for this ecosystem of behind the gates, the open campus, and then an incubator for companies like ten x genomics that now are ready to launch out of the lab. Those investments have not continued, but I think those are the type of ecosystems that truly make California what it is.
- Rebecca Bauer-Kahan
Legislator
And I will note that another example of that outside of my own pride is Baker the Baker Labs at UC Berkeley. We are seeing and one of my constituents, got a desk at Baker. And for those that don't know, UC Berkeley now has this incredible world class facility that allows for wet labs that a startup could never afford.
- Rebecca Bauer-Kahan
Legislator
And you can come in and you can start that science in a small scale, and then you can prove your case, and then you can spin it out into a California company and change the world. And these are the types of investments that I will tell you when I said that the UCs are the envy of the world.
- Rebecca Bauer-Kahan
Legislator
About six years ago, the EU opened an office in San Francisco. And one of the focuses was why is California the ecosystem it is? And after a couple years, I said, well, what have you learned? And they said, it's your campuses. And it's not just the world class education.
- Rebecca Bauer-Kahan
Legislator
It is that you are focused on how your education translates into innovation, into business, and into changing the world in real concrete ways. It is not just education for the sake of education. It is education for what it can do for people. And I think these are the examples of how that has. I will also say the supercomputers at both Lawrence Livermore and Lawrence Berkeley, are available.
- Rebecca Bauer-Kahan
Legislator
And you compete. You compete. If you have the best idea, you get access to that compute, and you get to create something. And I think it is that competition of ideas and us supporting them through real infrastructure and, resources that will change the world, but we need more of it, which is why I'm such a believer in Cal Compute. And I know that UC Berkeley wants to be our partner in that, so we need to keep that alive.
- Rebecca Bauer-Kahan
Legislator
So thank you for highlighting all of that. In addition to the procurement piece, we're doing it on insulin. We have created a market for insulin that is driving costs down. There is no reason we couldn't do the same thing as a state for these, more advanced technologies. And I love the idea of using our procurement power as one of the largest procures in the world. Next to the
- Rebecca Bauer-Kahan
Legislator
US government, I imagine California is up there as one of the largest procurement entities in the world, and we can use that power to really make a difference. So I did you know, and I do think the privacy point came up a little bit. I think under undertones, I will say one of the examples you gave was color. I will say I did color, and you know I did color. It was one of the few gene tests that was HIPAA protected, and I looked into it.
- Rebecca Bauer-Kahan
Legislator
And so there was an example of a company that got me to give them my data by saying we are going to protect it. I would not have participated otherwise. And so I want to make the point that privacy isn't always antithetical to the goals we wanna achieve. It can actually make people join the party, and it can get us more data, and color did exactly that.
- Rebecca Bauer-Kahan
Legislator
So I think it's an important point to make about because some companies are like, we don't wanna protect data. We'll do better. It's not always true. And so I do think HIPAA and CMIA are important part of this. But I think, you know, conversations I've had, and I don't know if Roche is in the room, they have a presence in my district as well.
- Rebecca Bauer-Kahan
Legislator
So I've had conversations with them about how do we take this data, de identify it, anonymize it, and then use it. So we are protecting people, but we're also using the power of data, and both of those things can be true.
- Rebecca Bauer-Kahan
Legislator
And I think anonymizing and de identifying is an important tool to keep in our back pocket because, you know, as a member of our rare disease caucus here, as someone who's been touched by rare disease more than once, the only way to solve these rare diseases is to take data and find it. But we don't need to know who the person is. We don't need to know their health past.
- Rebecca Bauer-Kahan
Legislator
We just need to have their data. And so I think that's, really important because you can see how quickly you can get from the example that was given about my watch being able to tell you the likelihood of my heart attack to me not being able to get health insurance. Right? That trajectory is so clear. We lived it, those of us that are older than the Affordable Care Act.
- Rebecca Bauer-Kahan
Legislator
And so we do need to make sure that these things are protected. It is helpful. It will save lives. It will improve lives. But I do actually believe that privacy will get more people to buy in and will get us better outcomes.
- Rebecca Bauer-Kahan
Legislator
Some questions I had were around that. I had two questions. So and I'll just give them both now so I can be quick. One is, how are we and I've had this conversation with Kaiser, which is doing a lot of these using technologies to work. And I know that when Kaiser procures these tools, my understanding was that they then are protected by HIPAA and CMIA under the and that's how color was protected as well.
- Rebecca Bauer-Kahan
Legislator
It was through a doctor order. And so how are we making sure that these technologies that we're using are protected? The second is around the health records point you made, and I think both of you made this. I think is the giant in the room, and I think we should just acknowledge that. I've heard from many people that it is hard for startups to really gain ground because of Epic's market control over health records.
- Karen Knudsen
Person
Yeah. I mean, I think the epic one is is it is challenging. It's very difficult for startups in the health care services side to tap in, but there are, I think, unique ways that people are starting to be able to do that with patient permission, back to privacy, like full disclosure and consent from the patient. Can they actually get access to that record by, having the patient actually grant access on behalf of service x.
- Rebecca Bauer-Kahan
Legislator
I really love what you're doing with Jupiter, but how does that interface with this problem we have of the single player that's trying to get into this market but really has such market control that it can negatively impact, I think, competition in the marketplace? So those are my two questions.
- Karen Knudsen
Person
And so that's another way in that doesn't require you and the health system to agree for it to tap into your instance of Epic. It's actually a much lower bar.
- Karen Knudsen
Person
So I think there are different points of entry that are happening now. And, you know, when it comes to protecting protecting privacy, Aye, you know, I think you articulated it perfectly with color. They thought that through from the very beginning. You're holding on to someone's genetic information, which according to GINA law, you cannot be discriminated against the level of employment or health care, but you can be discriminated against for life insurance, which actually prevents many men from going forward genetic testing.
- Karen Knudsen
Person
So ensuring that, you know, whomever that is it was working with you has a tight hold on their on the data and how it will be utilized.
- Karen Knudsen
Person
The data won't be sold is, you know, an incredibly component important component, but there has to be some legislative patches in this as well. You know, the challenge is further exacerbated by the fact that as whole genome sequencing comes down in cost and becomes very quick and will appear in your health record, that, you know, how much information do you actually need about you before I know it's you without knowing your name? Have your genetics, have your family history.
- Karen Knudsen
Person
It starts to so those are those are some of the new world issues that we're going to have to really face. But when it comes to connecting people with rare diseases, like many cancers, almost every advanced cancer becomes a rare disease.
- Karen Knudsen
Person
You find that with full consent that people actually understand what are the pros and cons of sharing that data that at least in the oncology space, people are very willing to share that data with the hope that what's learned about them will prevent someone else from dying of their disease.
- Ida Sim
Person
Can I address? Yeah. Yeah. Those are excellent questions for someone to address the issue of HIPAA protection and some of these sensor companies, for example. If the market for direct to consumer sensors and in fact direct to consumer care that's outside of what we would traditionally be covered under HIPAA is dramatically growing.
- Ida Sim
Person
Many consumers are now giving their data to companies that are not under HIPAA. So that whole regime of legislation and protection in this rapidly growing market is critical critical to address. The connection with EPIC is is a very challenging one, but there have been substantial federal legislation that makes the technical connections to EPIC and other health records mandatory. So there's something called FHIR standards, h l seven standards, data interoperability standards that are open, that are mandated, that the health technology industry already uses.
- Ida Sim
Person
For example, you can pull out your phone right now, whether you have an Android or an Apple, and get your electronic health record from any health system in the country onto your phone.
- Ida Sim
Person
What you're talking about It's already there. Yep. Why is there no market around it? Well, it's partly because the data comes in and then it doesn't go anywhere. There's not that infrastructure that we need.
- Ida Sim
Person
So what we're building with Jupyter Health is really a layer on top of Epic. You can think of Epic as having sort of stubs that you can connect to. So Jupyter Health is that open source layer just like the Internet is an open source layer. A technology layer that interfaces allows you to connect to Epic. I practice.
- Ida Sim
Person
I know all my billing, all my work transaction are on Epic. It is not going to change. Same thing with CERN and others. But we can layer data, we can layer intelligence, we can layer AI on top of it, and from that coupled with legislation around privacy, I think will open a whole new ecosystem of investment and advances of, you know, both of my panelists. There's a lot to be done there.
- Ida Sim
Person
I'd love your your point about we need to connect the data. We need to connect all of what we're doing. But I would like to see us doing it if an open source layer that then allows private companies to build on top of it as opposed to private companies building lots and lots of silos that aren't adding up for our patients.
- Chris Ward
Legislator
It makes perfect sense. Yeah. I wonder could we go back really quick to some of the earlier, points you were making around, I liked your example about the blood pressure and being able to look at, like, you know, just, like, really, you know, fine details within an image. And is that as, like, a case study, but more generally, like, validity. Right?
- Chris Ward
Legislator
How is that being tested? Obviously, when we find a new technology, a new way about, you know, sort of, like, there is either as aggressive, you know, sort of peer reviewed, work, that is making sure that we're we're getting this right. Not to say that, like, you know but but you could be right all along. And if you've found that discovery, you wanna be able to utilize this.
- Chris Ward
Legislator
How is it going sort of in that that that sort of existing process, or are we reforming that process right now, to be able to test validity?
- Chris Ward
Legislator
And I think to that end as well, you had mentioned about, the AI, the human statement, can be able to help to speed up, the FDA process. And Aye, in my mind, understand how AI can help us get to, you know, kind of pass the the theoretical and and and and and to, a discovery that we believe works, but then you submit it for that process.
- Chris Ward
Legislator
And so where are we seeing sort of alignment or hopefully any kind of response from the feds to be able to reform their process?
- Ida Sim
Person
Yeah. The regulatory process is is challenging. It is changing almost on a week by week basis. Currently, the FDA has what we call an acceleration as posture. They're really taking the brakes off and letting letting innovation flourish.
- Ida Sim
Person
I think for particularly for the example that I gave, it it is a challenging question because what is the gold standard? We have patients right now who are using, you know, your standard Omron or eye health cuff at at home. And, like, there's a difference, but we know, actually, when we bring people into the clinic, the standard cuffs that you use at home are kinda all over the place. The FDA rules have a you can be plus or minus 15 millimeters of mercury, which is huge.
- Ida Sim
Person
So I think what is what is gold standard, I'd love to point about clinical trials and how we can deploy learning as part of what we do in health care, not as a separate thing.
- Ida Sim
Person
But we need to learn, we need to test, we need to monitor AI on an ongoing basis. That's not the kind of science that we've been used to doing, but we can get there. Now AI is gonna allow us to do that. And in fact, AI must help us do that because it is something that we need to do. I think the regulation and the validation, in my mind, almost proceeds in parallel, but but, you know, regardless of regulation because that's what we need.
- Karen Knudsen
Person
Yeah. Could agree with everything that you said, and I'll I'll just pick up on the other thread. You know, I totally am am in favor of the idea that AI will help us be in a consistently learning health care ecosystem. Right? Learning from every patient and then putting that back into the drug discovery or device discovery engine.
- Karen Knudsen
Person
But when it comes to regulatory at the federal level, we were we're starting to work very carefully with FDA because we do see gaps and slowness of not being prepared on the not on the device side, but for us on the drug discovery side for platforms to be, at the level of approval. Right? It wasn't so long ago that we had the first drug approval that wasn't for breast cancer and for prostate cancer. It was for a molecular driven cancer. Right?
- Karen Knudsen
Person
PARP inhibitors, for cancers that are related to BRCA one or BRCA two deficiency. That was a big hurdle for the FDA to get to because now it's not about tissue of origin. It's about the molecular underpinning. Now we're in this new era in the drug discovery side where platforms will be discovered. So here's an example.
- Karen Knudsen
Person
Right now but, well, let me let me back up just quickly. We actually have really phenomenal cancer vaccines that prevent cancer. Right? Vaccination against HPV, vaccination against hepatitis b. So we've been in the cancer vaccine space for a long time in the in the in the oncology world, but it's been on prevention, which is the gold standard.
- Karen Knudsen
Person
But we're now entering this entire new era, which we are beginning to spearhead on therapeutic vaccines. You have a cancer at surgical resection. We take out your cancer. We look at unique proteins that decorate the surface of this cancer and develop a bespoke vaccine for you to prevent recurrence. And the clinical trial data have been spectacular for very difficult to treat cancers like pancreatic cancer, renal cancer, melan metastatic melanoma.
- Karen Knudsen
Person
So but those approvals won't come on a specific drug. It will come on the back of a technology of if we do this, what is the process by which we find the unique proteins decorating the surface of the cell? What is the unique manufacturing strategy by which we can create a personalized vaccine for you, and how are we going to deploy? That's the future of cancer cure, and we're seeing it happen now. And I don't think I'm overstating based on the clinical trial data that's happening.
- Karen Knudsen
Person
So we have to get the FDA prepared for those types of decisions, which are not at all the typical way that you think about drug discovery. Right. And, you know, the the thirty four percent decline in cancer mortality that we've seen since 1991, which by the way was the high watermark for cancer mortality rate in The US. You can point to research as the answer. And that first set of declines was coming from prevention.
- Karen Knudsen
Person
We know what to do for prevention. It's getting people over the hurdle of of implementing. But the second wave of reduction in mortality is coming from immunotherapy, and those approvals are going to look fundamentally different than drug off the shelf. And so how do we get the FDA ready for that is something we're working very hard on and would enjoy working with others who are like minded.
- Chris Ward
Legislator
Yeah. Not speed on that. We all have heard for a long time that we need to sort in that window. And I know we've seen some progress on that, especially on the experimental level when something is working or there's enough evidence to suggest that it is.
- Shankar Sundaram
Person
Chen Ward, I just wanted to make a quick comment if possible on that. Just, endorsing it, just from a an example. Right? I mentioned to you about antibody design. Right?
- Shankar Sundaram
Person
Redesign. So take an FDA clinically approved antibody and make four modifications. Right, that makes it much, much better. Now do we go back in the starting line, right, to start all over in terms of phase ones and phase twos? This was the question that we post to FDA.
- Shankar Sundaram
Person
They're working on it. Right? And so this just highlights the other thing about the necessary for shared compute and sort of a public side of it outside of companies, people who don't have, you know, a vested interest. Right? Helping the regulators understand, right, what is right and wrong and push it because it's you know, you really need to get various drug developers to come together and share their data.
- Shankar Sundaram
Person
They're not gonna share it with each other. Right? How they share it with the government in the trust that you can do it to issue guidance back to them that makes their lives easier, be more certain of of approval or not. Right? So that thing, it just goes back to modernizing that regulatory infrastructure.
- Shankar Sundaram
Person
I don't know if it is still within the purview, right, of of states, but that is a very crucial one. I I said we redesigned the antibody in three weeks. It'll still take nine months to get it through, FDA approval. Right? The drug discovery process, you know, I think you mentioned takes twelve years.
- Shankar Sundaram
Person
Right? The first four to six years is preclinical discovery. Now I can do this in months to
- Shankar Sundaram
Person
To a year, but the rest still remains. So that's a a real challenge that we have to address as a nation.
- Chris Ward
Legislator
I I appreciate you. I really address that. You know, I I have a few other questions that I might have to kind of, like, you know, consolidate. We're thinking about our future or or our last panel here on privacy and data protection interests and everything. But I also have to balance, I think, in the interest of time and and us and and the room space that we have here today.
- Chris Ward
Legislator
But I I did wanna close with, just one open opportunity to see are there any state level barriers or state re budget, or other considerations for some of, the work that your institutions are doing right now that you see as, needing to draw our attention to.
- Karen Knudsen
Person
So as as outstanding as universities are here, and they are. Right? And some of our Parker Institute sites are here in California. I actually don't think we have a discovery problem. What we actually have is a translation problem.
- Karen Knudsen
Person
You know, there's too little invested in the strategy of getting new concepts, be it a device or a drug into clinical trial. We've talked about some of the potential fixes for that. But it's also within what happens when this good idea is ready to be capitalized and take that next step. You know, the venture community has moved a little bit more to start to act more like biopharma where they want to see a clinical signal before they're going to invest. So we've stepped into that space.
- Karen Knudsen
Person
But I worry that we're starting to lose our accelerator engine power here in California of investment into really good ideas that need this little little piece of clinical testing or clinical information that could then allow the venture community to really get behind it and let it sing.
- Ida Sim
Person
Thank you. One issue coming back to the issue of evaluation. These AI models that are being deployed often from private companies are very proprietary. I'm a physician. I use an AI scribe.
- Ida Sim
Person
The AI scribe companies hold their model proprietary, which I can fully understand. But the data that is collected, for example, the audio that is recorded and then the first draft that is generated, that is not publicly available, which makes it almost impossible for us outside of the proprietary company to say whether it's even accurate. And so what we're having is individual companies are saying our mock thing is accurate or our thing is good.
- Ida Sim
Person
And as a system with our responsibility to our patients, we cannot access that data. That is true across The US.
- Ida Sim
Person
That is not true in Canada. That is not true in UK. And I think California could play a leading role in thinking carefully. We do wanna foster innovation, but we must have some openness where a neutral party can evaluate the quality and accuracy of these models and provide a public signal, which I think actually will improve the technology and in in the end, improve investment and improve profits and improve impact for our Californians.
- Shankar Sundaram
Person
As a national lab, we are federally funded. So but what I would make a remark on is, you know, looking either us or others that are developing in universities, technologies, and solutions, you know, that needs to be ported over to make impact to the agencies, to the front line. Right?
- Shankar Sundaram
Person
So making sure that those folks are resourced and and and and mandated, for example, California Department of Public Health, right, to make sure that they have the resources and the mandate to be able to take advantage of this unique ecosystem, work with the universities and the institutes, and to be able to kinda take those methods and apply it. Right?
- Shankar Sundaram
Person
We can't get their data together, get that mission together, work with them to show the impact so that you can go back and say, like, this is what we delivered for for the state.
- Chris Ward
Legislator
Great. Well, I wanna thank you all for being here. Any final comments? Of course. Yeah.
- Rebecca Bauer-Kahan
Legislator
So something just came up that I think is probably the hottest conversation being had in AI today, which is the question of open source
- Rebecca Bauer-Kahan
Legislator
Versus closed and safety and security. And I think you made the point, which is really helpful, and it's why I call in the lab a lot. You know, you don't have a financial interest in the question. You are a federally funded lab, neither does UCSF. And my sense is that ensuring open source remains available is critical to some of the successes we're hoping to see coming out of this, but I know that that's a fight that we're seeing play out in real time in Washington.
- Rebecca Bauer-Kahan
Legislator
So I just wanted to sort of touch on and give you an opportunity to give your perspective because it will come up in some of the fights we're having here in the capital.
- Shankar Sundaram
Person
Yes. In fact, I want to draw your attention to another lesser understood sort of AI or foundation models, which is speaking the language of biology, which is, you know, the a's, g's, c's, and t's or the 20 letter alphabet of the protein, which have been developed. A lot of them, EVO two, came out of ARC Institute in California.
- Shankar Sundaram
Person
Yes. In fact, I want to draw your attention to another lesser understood sort of AI or foundation models, which is speaking the language of biology, which is, you know, the a's, g's, c's, and t's or the 20 letter alphabet of the protein, which have been developed. A lot of them, EVO two, came out of ARC Institute in California.
- Shankar Sundaram
Person
Right? ESM, the the Meta or or Chan Zuckerberg, Biohub, OpenFold three, as I mentioned.
- Shankar Sundaram
Person
Right? We helped, develop and launch it. So that is an important class of biological foundation models, models, which are critical to our line of work, including future ones in electronic health records and wearables that as they come and as they start to interplay with these sort of more large language models, that is sort of where I think the whole future promise as well as the peril lies.
- Shankar Sundaram
Person
And so making sure that those innovations are accessible, right, to our research community is going to be an important if that becomes completely locked in, then I think, you know, there may be some we may have some issues. But so that's why having the Cal compute, having those kinds of, you know, laboratory infrastructure and these kinds of partnerships set up between research institutes and and and faculty and students and workforce is gonna be important.
- Ida Sim
Person
Yeah. It's gonna be an ecosystem of fully closed proprietary bottles, open source bottles, open weight models. I think thinking about a way where all of those different models can serve the market needs and and public needs where there are very carefully thought through public private partnerships that can sort of wall off risk to IP and also address privacy risk and also validation risk. I think that is something that can be done, whether that should be done at a state level. Could think about that.
- Ida Sim
Person
We've been thinking about it at the UC level. But I think this is really an area that's right for public private partnership. I think if we're looking at federal regulation, we've heard about it. It's too slow. Yep.
- Ida Sim
Person
It is just too slow. We need to be very nimble. We need to work very closely together. And, you know, we need to bring up the issue of China, the those open models and open source models. We need to have a strategy against that, and it can't just be the public sector or the private sector.
- Ida Sim
Person
We need to be creative together. There are lots of advanced methods in in sort of distributed computing. You know, my colleagues at Berkeley, I have a lot of thoughts about that. I had a lot of thoughts about that. That is something that is absolutely worth looking into more.
- Chris Ward
Legislator
Thank you all very much. Well, I wanna be mindful of time, but also I didn't wanna, you know, miss the chance to welcome a Senator Lowenthal, our colleague for being here as well to see if you got any questions for this panel. Great. Okay. Maybe we can thread them into the next two panels.
- Chris Ward
Legislator
I wanna thank you for your testimony and all the work you do. We look forward to our continued engagement with you, as, everything, accelerates on this issue. So thank you for being here. And we'd like to welcome up our, four panelists for panel number 2, doctor Viber Gupta, the CEO of Pangea Data. Ginny Hu, the senior director for regulatory affairs at Dexcom.
- Chris Ward
Legislator
Doctor Danjuma Aqualas, who is the senior director of AI innovation at, Lilly Ventures and Eli Lilly. And doctor James Diggin, the VP for policy and biosecurity at Twist Biosciences. I think, some great representatives of the, companies that we have here in in the Golden State, who are using AI to be able to drive, a lot of innovation in the life sciences. Doctor Gupta, maybe we'll begin with you, and we're looking forward to all your presentations.
- Vibhor Gupta
Person
Thank you very much. Very good morning, everyone. And thank you very much, Chair Ward, for this opportunity and Chair Bauer Khan for extending us this opportunity over here to share what we are doing, with a focus on AI in health care. My name is doctor Vibhor Gupta. I'm the founder of a company called Pangea Data.
- Vibhor Gupta
Person
As you can tell from my accent, I'm from London, and a recent migrant to California from about two years ago. I've been visiting the state over the last twenty years, and and I would just like to thank you again, to your state and to your country for welcoming me over here. What do we do? As you can see, on the screen over here, we are in the business of closing care gaps.
- Vibhor Gupta
Person
The previous panel talked extensively about the existence of care gaps, meaning that patients not receiving the right treatment or not receiving the right testing or not being referred to the right trials in spite of information in their records.
- Vibhor Gupta
Person
And therefore, we are very motivated by this problem. I've been working in medicine and computing for about twenty six years, battling this problem in various different avatars and most recently at Pangea for the last eight years.
- Vibhor Gupta
Person
So I'm here to talk about what we have done in this regard today over the next five to seven minutes, and more importantly, what it means not just for enterprise health care, which is what we hear a lot about from, but also for rural health community, which is where a large proportion of our population is treated and where we see a large proportion of these care gaps occurring.
- Vibhor Gupta
Person
So with that said, you know, a lot of people ask me, you know, what does it mean when we talk about care gaps? How big is really this problem?
- Vibhor Gupta
Person
And if you can see on the slide, this problem extends across chronic, rare, metabolic, oncology, and several other areas of medicine. And this problem is not because Clinicians are fallible or, you know, or that we don't have the systems in place. As you all know, we have invested heavily in EHR systems, in technical infrastructure, in cloud systems, but these problems still persist.
- Vibhor Gupta
Person
And the reason is simply because when you have ten minutes with a patient, it's very hard to glean everything that might exist in their electronic health record, which might extend up to hundreds of pages, and at the same time as a human, permutate across all those various guidelines that you have to think about in the context of that patient's trajectory.
- Vibhor Gupta
Person
As a result of which, these patients fall in these care gaps where they might you might have thought about having done the best for them, but yet they show up in the clinic after a little while, be it in three months or six months with a condition that you could have prevented or could have spotted earlier on.
- Vibhor Gupta
Person
I'm sure we know of several of the stories from our friends and family. Now if you think about it, it's because of this partly plays a role in increasing the cost of reimbursement or increasing the cost of care and therefore insurance premiums. And you multiply that with the fact that a large population, especially here in America, is going to fall off payer programs given some of the new legislation that is coming in.
- Vibhor Gupta
Person
This problem is going to only compound on itself, especially in rural communities and in communities would perhaps are not as privileged as some of us in the cities. And so, therefore, what I bring to you is just some analysis that this problem is clinically and economically important to solve.
- Vibhor Gupta
Person
And, certainly, people are making several different efforts in order to solve this problem. We've heard about from the previous panel about the use of imaging of various sorts, be it imaging facial imaging or imaging of the eye, which is what a large program called the AI, for eye screening is trying to do. We're looking at images from the eye, they're trying to detect diseases like chronic kidney disease, neurological disease like multiple sclerosis.
- Vibhor Gupta
Person
And then, of course, there's the aspect of genetic testing, which we heard about again from the previous panel, where Clinicians, patients, community groups are coming together to make sure that patients are screened or populations who are at risk are screened well in advance so their cancers or other relevant diseases are spotted earlier and they can be taken to the relevant care pathway.
- Vibhor Gupta
Person
What we at Pangea started our journey with was focused on electronic health records or patient records, not necessarily just based in electronic health record systems, but in systems like your laboratory information management systems, your packs, which is where your images are, so that we could make sense of all of that data and make sure that that intelligence is available to the clinician at the point of care and also to all the administrators at a population health level so they could do something about it.
- Vibhor Gupta
Person
What you see on the slide are results from some such work that we have done in the context of chronic kidney disease and COPD, which is another chronic kidney disease sorry, which is another chronic disease. And there, we have shown how closing of care gaps in those disease areas has led to a very positive financial outcome. In the context of just a few thousand patients as you see on the slide, we were seeing hundreds of thousands of dollars being saved.
- Vibhor Gupta
Person
And, of course, you can imagine how that translates when you apply this across millions of patients. So that's that's really the sort of the overlay.
- Vibhor Gupta
Person
But, of course, when you get under the hood as to how we achieve this, you can imagine in the day and age we live, the the expectation was that just through the use of large language models, we could achieve this very quickly. And why don't we just make available those large language models within our existing electronic health record systems and magic happens and we can achieve all of what we are describing over here?
- Vibhor Gupta
Person
The reality is far from it because in clinical practice, as some of my predecessors mentioned, when we think about clinical decision making, it's not just about generating the next word or looking at just the language of what has been written in the electronic health record.
- Vibhor Gupta
Person
It involves temporal analysis, understanding what the patient went through at certain time points, understanding certain levels of information that might exist in their data, such as their lab values, and also understanding what truly is being referred to when we say simple things like the patient was having a headache or the patient, you know, was not very, you know, what not very coherent when they were speaking to us during the consultation. All of these things could mean different things.
- Vibhor Gupta
Person
And, therefore, expecting a large language model to be able to crack all of these things that we would normally do in the clinic, we found that that was just not possible. Whereas what was possible was if we broke down the clinical decision making process into different components and determine using mathematics what the best model was for each of those components. It could have been a large language model. It could have been some other kinds of AI models.
- Vibhor Gupta
Person
And then compounding the outputs from all of these different models to come to a clinical decision, much like what we would do in the clinic.
- Vibhor Gupta
Person
And using that approach, we found the accuracy to be much greater, but more importantly, to be repeatable. Because I'm sure you all we all here have tried large language models. Even if we prompt the same way, it would give us a different response.
- Vibhor Gupta
Person
But the fact that we are able to break the clinical decision process into different components and come to a repeatable and a validated outcome each time was much more trustworthy for us at least in the clinic and, therefore, something that we could follow-up with. So we did this in anger across several different disease areas, right from chronic to rare to metabolic, and the results speak for themselves.
- Vibhor Gupta
Person
And as some of my previous panelists mentioned, it was very important to publish this in peer reviewed journals to demonstrate not just the accuracy of this approach, but also the usability of this approach in the clinic. Because as good as the AI or the mathematics can be, it has to be usable, Meaning that we have to be able to access it within our workflows, within the clinic so we can act on them. And thirdly, it has to create that value.
- Vibhor Gupta
Person
Meaning that it has to either save us time and it has to be able to demonstrate improvement in patient outcomes. And that's really what we focused on, you know, when we were building this piece of technology and deploying it, which was accuracy, usability, and value.
- Vibhor Gupta
Person
And with that, we started working with some of the, of course, the large centers of excellence and health systems starting with here in California and more broadly The United States. And today, I'm proud to say that we're deployed at some of these health systems that both large and small where we have integrated within their existing systems and workflows to achieve what I have described.
- Vibhor Gupta
Person
Having said that, we have also noticed that while these enterprise level health systems are privileged enough to be able to do this and recognize the value and, most importantly, pay for it, the same is not true in rural health care. In rural health care, we find these community practices might not even have an EHR system. Right?
- Vibhor Gupta
Person
Or they might not even have the budget to pay for the compute that is needed to run these kinds of technologies because the cost of those compute can be quite high, and some of that was alluded to in the previous panel. So what we are seeing now, interestingly, is grants being made available, to the rural community and some of these sort of, institutions who could benefit from them in order to do more of this work.
- Vibhor Gupta
Person
But where we see a gap is that these grants are not solely focused on demonstrating value in the context of improving patient outcomes, wherein they should. And we have some examples here from our neighboring states in in or or relatively neighboring in Texas and in North Dakota where we are seeing Clinicians and these health systems wanting to demonstrate that value through the use of these technologies in the context of specific clinical areas which are of high priority in their patient populations using these grants.
- Vibhor Gupta
Person
If you demonstrate value like we have done, how do you reimburse for this? If you're using AI to demonstrate that you can close these care gaps and more patients can benefit from screening or treatments or trials and that the overall cost of care is reduced and that the patient, you know, quality of life is better. Well, how do you and if you've shown that in a small population, how do you extend it to a larger population?
- Vibhor Gupta
Person
Where do you get the funding in order to do that? And how do you reimburse for this kind of AI technology?
- Vibhor Gupta
Person
And if I was to make a request over here to the committee today, it would be to please have a look in some of those reimbursement frameworks that can be more value driven, that can allow adoption and utilization of such technologies at scale so that they're not limited just to the enterprise health systems. And finally, of course, drawing a leaf out of the book of biotechnology, which is where I come from originally, you know, I I feel that this is very much akin.
- Vibhor Gupta
Person
If you have that framework which can, you know, unlock that value at scale, you're practically creating a similar kind of industry that we've seen in biotech, where you're demonstrating value, you're creating new jobs, and most importantly, it's financially sustainable on its own without having to rely on these kinds of sort of small grants of funding. So with that, I would just like to thank you all for your attention, and happy to take questions later.
- Ginny Hu
Person
Thank you. Chair Ward, Chair, Barak Khan and members of the Assembly, thank you so much for the opportunity here. I'm Ginny Hu, senior director of regulatory affairs and regulatory science at Dexcom. Really appreciate the opportunity here to tell the members of the Assembly about what we're doing in terms of incorporating AI technology into the medical device product that we we manufacture. Dexcom manufactures a device called the continuous glucose monitoring device as as shown here.
- Ginny Hu
Person
It's a small wearable device that you can wear on the back of the upper arm. It senses glucose data every five minutes from the interstitial fluid. This is a transformative technology for individuals with diabetes over, the past two, three decades or so. We're a California company, founded in 1999 in San Diego, California, still headquartered in San Diego, California. And the mission of the company has always been to empower individuals for people to take control of their health.
- Ginny Hu
Person
And this is for individuals with diabetes to help manage their diabetes, for individuals who are on insulin therapy to help manage their glucose so that they can make the right treatment decisions, and also for a broader population to help maybe people with prediabetes or people who have a risk of comorbidities to to to help manage their risk, to understand their risk of potentially developing prediabetes or diabetes. K. So we've been oops. Back one slide here. We're the pioneer really in the biosensing industry.
- Ginny Hu
Person
A lot of first here from Dexcom. I would like to just take a minute to emphasize the field that's relevant to the topic here today about incorporating AI, from a software perspective. Dexcom was the first, manufacturer to gain FDA approval of sending glucose data remotely from our wearable device to a mobile application. And this was done, in close collaboration with the FDA.
- Ginny Hu
Person
And not only are we able to send that data directly to the mobile app on the user's, mobile device, but we're able to also, through FDA approval, send this data remotely through the cloud to a caregiver's mobile device.
- Ginny Hu
Person
So this mean for, for example, for a child with type one diabetes, they're able to manage or monitor their glucose, in real time for for from their own device. But their parents, their their caregiver, their diabetes educator, maybe this is a school nurse, maybe it's their endocrinologist. Somebody in the health team for that particular child is able to help manage their glucose information in real time. This is really transformative and life saving and life changing for a lot of our users.
- Ginny Hu
Person
This this means a child could could go to school, would have a normal day of activity as much as possible, not having to worry about having hypoglycemic event going low on their glucose anytime.
- Ginny Hu
Person
So there's really a care team that that can help remotely monitor their glucose levels for that for that child. Another example is more recently, we've been working very closely with the FDA to achieve approval of the first over the counter CGM device. So our CGM device now, the bow the Stell O Biosensor System is available over the counter to a broader population. Individuals, who are not on insulin, who may, have an interest to understand more about glucose health in general, to help manage their metabolic health.
- Ginny Hu
Person
So these devices are now available over the counter to a broader for a broader use case.
- Ginny Hu
Person
Relevant to the topics that we're discussing here today, we were the first to really be able to introduce generative AI technology into our software product, into the mobile app as part of the Stell O biosensor system. And I'd like to also talk a little bit more about that here in a minute.
- Ginny Hu
Person
Before kind of talking more about the the biosensor system that Dex produces and how we incorporate AI into our software, I'd like to spend a few minutes on the FDA regulation of AI in medical devices. We heard a lot from the previous panel on the kind of the process of drug discovery, what the regulation is like.
- Ginny Hu
Person
I think from the medical device side as one of the manufacturers that have been working very closely with the FDA throughout the evolution of this landscape and working with the FDA to find the right way to incorporate AI technology where where it's safe and and effectively into our product.
- Ginny Hu
Person
We've really kind of enjoyed the privilege of that close collaboration and and really appreciate FDA's innovative thinking on this front from our CDRH branch within the agency. So in general, FDA has a long standing kind of a history of establishing a total product life cycle approach. This me and as you can see kind of on the circle here, this includes AI. It's a wheel. It's not a a process that has a beginning and an end.
- Ginny Hu
Person
It's a really a continuous process. What FDA really emphasizes is that throughout this process, manufacturers use a comprehensive risk management approach. This means anything we do when designing a product, including software, including AI, we make sure that we consider any of the foreseeable risks. This could be clinical harm to the user. This could be as little as as a nuisance to the user.
- Ginny Hu
Person
This could be maybe a distraction. Give an example, if our app has a notification that's supposed to pop up on your phone that says your Google's level has reached the threshold that HCP or the user has set for that particular patient. What if another notification comes up on that phone from social media or for something else? Is that going to really distract the user? Are we gonna be able to make sure that the user really sees their Google's notice?
- Ginny Hu
Person
Because it's really important for them to see, to be able to act on it. So that's that's an example of a foreseeable risk that we really consider this throughout the life cycle of that product. And this is really FDA's clear guidance and requirements for for us as a medical device manufacturer. This also encompasses cybersecurity risk management. Security testing is not really for the sake of testing.
- Ginny Hu
Person
What we do is that we really look at what is the potential result of a cybersecurity related risk. If there is a threat of vulnerability, if there is an issue, if there is the presence of a malware, what is the potential harm to the user? What is the potential nuisance? What is the potential impact of that situation to that user?
- Ginny Hu
Person
And we then assess the overall risk and the appropriate mitigation either from a design perspective, additional security control perspective to make sure that all of those risks are mitigated to as little as possible and to an acceptable level.
- Ginny Hu
Person
Another concept that I'll touch on a little bit here is performance verification and validation. I think there are some some discussions here from the previous panel, from from Chair word, Chair, Bauer Cotton here. What is the right way to test? So we from an FDA perspective, the framework is is very robust in terms of software related verification and also validation. These are very two two different concepts.
- Ginny Hu
Person
Sometimes that they're they get mixed together, but they're actually both critical steps of risk assessment and testing for software as a medical device. Performance verification of a software means that, you know, we test what we we say the software does. We say the software is going to provide a notification when a glucose level hits a predefined threshold. Did it actually provide that notification? So that is an example of how we do verification testing.
- Ginny Hu
Person
Validation is really ensuring that in totality, the functions that we build for our product meet the user needs. If we say that our glucose CGM device needs to be accurate enough so that the users can safely manage their insulin dosing decisions for that intended use population or help them manage their health in general from a metabolic perspective. We have robust clinical validation studies to actually demonstrate that and provide that evidence.
- Ginny Hu
Person
So two different aspects of testing, but they they work hand in hand for any medical device software including AI in in that technology. The last element I'll touch on here is post market monitoring.
- Ginny Hu
Person
This has always been a cornerstone of FDS regulation and becoming even more important in the AI field and for software as a medical device in general. Because as I mentioned, the the life cycle of that product, there's no beginning or an end. As a manufacturer, we're always looking at signals through our design, through our testing, through real world use after the software is released. It's it does not stop there.
- Ginny Hu
Person
We're constantly monitoring to ensure that any issues, any changes that's needed from a post market perspective, from real world user needs are taken into consideration when we make device changes.
- Ginny Hu
Person
So that's kind of a high high level overview of the framework that we follow from a medical device software regulation and AI total product life cycle perspective. So how do we put that into practice? I like to maybe show a few examples here. This is our, STELLO Biosensor Glucose Management System. It has been available in The United States, for about two and a half years.
- Ginny Hu
Person
It's the o the first over the counter CGM device. We were able to introduce a lot of generative AI enabled innovation into this mobile app for the STELO BOW sensor system. Over the past few months, we're able to introduce a really transformative redesign of the user interface in its totality and incorporated AI enabled insights throughout the user experience. One example here is that there's a lot of data that we're able to provide to our users. Our system measures glucose levels every five minutes.
- Ginny Hu
Person
That's 288 data points per day. And when you wear the device for fifteen days, there is a a rich history of data. A lot of a lot of data points in there. What does it actually mean to each individual user? What we do is that we include AI enabled contextual interpretation of that data to each user.
- Ginny Hu
Person
It might mean as simple as helping the user connect the dots of their behavior. Maybe they locked a meal at around lunchtime. You can actually take a photo, and the AI helps to recognize what kind of food is in that photo. I help to estimate some of the nutritional values, and then give the user some additional contextualized information. An example could be, hey.
- Ginny Hu
Person
You know, you ate a a pretty balanced meal here. It looks like there's some chicken, half of an avocado, and this is generally, you know, a a balanced meal here. It looks like you take a you took a walk around this time as well. Overall, this meant that your glucose trend was, you know, had had a had a little increase, but then it went down within an hour after that.
- Ginny Hu
Person
So this really just provides some contextualized information that the user is able to see by themselves anyways as they're part of the user experience.
- Ginny Hu
Person
But really just helps to increase the stickiness of their experience. Another feature that we've introduced is an AI enabled health coach. This is a LLM enabled functionality where the user can ask questions and receive some answers. So you might be thinking, is this really like, is this just a chatbot that the user might be able to access? It is enabled by the same technology.
- Ginny Hu
Person
However, we add a lot of really important guardrails to this exchange. I'll touch on that those guardrails a little bit later. The user is able to ask any questions. However, we'll look at those questions and determine what are the questions that our health coach is actually able to answer. Those have to be the questions related to glucose data, related to glucose health, and related to information that we're able to provide user with some helpful feedback on.
- Ginny Hu
Person
So it's not just any information. One of the guardrails that we put into place because the Stela biosensor system is is intended for individuals not using insulin. It's a very important mitigation is that we make sure the chatbot, the health coach is not giving any insulin dosing advice or decisions. In fact, that is one of the things that we we've really made sure that we put in the safety guardrail and test it very, very thoroughly internally before we release this feature to to the public.
- Ginny Hu
Person
Some of the examples of the guardrails that I have mentioned for our AI enabled technology, I'll I'll mention a few here.
- Ginny Hu
Person
The first is quality control. Any software as the medical device is regulated by the FDA here in The United States and AI enabled software function are included, must go through a rigorous quality management system. So so this is no exception for any AI enabled technology that's released as a medical device. And this QMS system is in compliance to all relevant IEC standards, so there is a lot of international harmonization on this topic, and also other relevant regulation and FDA guidance along on this topic.
- Ginny Hu
Person
I I mentioned a little bit of about governance, which means what are the guardrails that we have put in into our AI enabled function to make sure that we give the right feedback, and we're not going to have AI act as a health care provider.
- Ginny Hu
Person
In fact, we we've put in this guardrails to make sure that the chatbot is not giving any medication adjustment advice, is not responding to those questions that the users may be asking about adjusting their medication. The AI is not able to give insulin dosing decisions advice, and also is now making diagnostic decisions. Since the re release of the feature, we've actually seen the users in the real world that they will ask. The users would say, hey, I wore this lens for fifteen days based on this.
- Ginny Hu
Person
And our AI coach is able to say, I'm an AI function here. I hear that you're interested to understand a little bit more about your data. I'm not able to make that determination. If you're concerned, you you might wanna reach out to your health care provider to have a discussion about the data. They might recommend some additional lab testing that will help you understand if you have diabetes or not.
- Ginny Hu
Person
So we see that really robust guardrail in place. The user sometimes we all also heard earlier that when you ask an LLM a question five different ways, they might get different answers. We see that as well. The user will persist. Some users get actually a little upset to say, just tell me.
- Ginny Hu
Person
Just tell me. Do I have diabetes or not? And we're actually very happy to see that our coach save those guardrails are held in place. So we're giving that feedback, but in an empathetic way, we're saying, you know, we it it sounds like it's a little frustrating for you that that, that you're you're not able to get the answer right now. But I'm I'm an AI chatbot.
- Ginny Hu
Person
I'm not able to give that diagnostic decision. If you're concerned, it it might be a really good idea to talk to your health care provider. So we've seen some really good real world evidence on those guardrails and on those controls being very robust. As I mentioned, another element of having those really robust guardrails and safety measures in place is to make sure that there's always human oversight.
- Ginny Hu
Person
These guardrails that we we put in place are called counting guardrails, and those are always reviewed by our subject matter experts, our our health health care providers on staff, or consulting from a subject matter expert perspective.
- Ginny Hu
Person
And we also don't do that once, and this is part of FDA's requirement as well. It is an ongoing basis. We do this periodically to to ensure that there is always that human layer of oversight that's for periodical review and for content generation. I've mentioned a little bit about cybersecurity controls.
- Ginny Hu
Person
There are very robust security related standards and guidelines that we adhere to, including relevant ISO standards, twenty seven zero zero one, SOC two standards, and all those controls are in place not just for for the sake of having controls in place, but they're in place to ensure that they mitigate any risks that could be presented to the user, either from a clinical harm perspective, a user convenience perspective to make sure that this device is safe, is effective, it's usable, and it's really beneficial to the end user.
- Ginny Hu
Person
And lastly, cannot really last but not least, on the post market monitoring element, for any software medical device and especially with AI enabled technology, this is critical. We're putting a lot of emphasis here. The FDA is putting a lot of emphasis here to make sure that once the feature is released, it's not just out in the wild. We're really monitoring the the performance of those guardrails that we put in place. We're monitoring that the function is still usable to the user.
- Ginny Hu
Person
It provides the benefit that we intended to. And it's a continuous monitoring assessment process to make sure that we make the best for for our users, and use AI the right way, to help increase engagement, help increase contextualized information understanding of what the device is able to provide. That's kind of yeah. Thank you so much. Happy to take questions later.
- Danjuma Quarless
Person
Thank you. There's there's no slides for Lily, but good morning, Chair Ward, members of the committee. Thank you for having me. My name is Danjuma Quarless, and I'm currently a senior director of biotech innovation at Lilly Ventures, the strategic venture arm of Lilly, Eli Lilly. I live in my in Oakland with my family right down the street from Baycar Labs.
- Danjuma Quarless
Person
I work in South San Francisco. I'm a scientist by training with a PhD in biomedicine and computational stats from UC San Diego. I also hold an MBA from Northwestern's Kellogg School of Business. And, ultimately, I joined Lilly about nine months ago to help one of the world's leading biopharma companies enable AI to improve patient lives.
- Danjuma Quarless
Person
The focus of today is on the use of AI drug discovery, but first, let me focus on on Lilly's ethos because I think it forms the foundation of our ambitions for AI and drug discovery.
- Danjuma Quarless
Person
Lilly is a 150 year old medicine company, and for that entire history, our focus has been to create treatments for challenging diseases. Since 2014, we've brought more than 20 new medicines to patients, and the most notable being tirzepatide for weight loss, which is very prominent right now. And we've worked to reduce that time needed to develop medicines sometimes in half over that over that window. What makes Lilly different isn't a slogan, but it's a core strategy.
- Danjuma Quarless
Person
We strive to be bold in all that we do and to solve challenges with passion.
- Danjuma Quarless
Person
We hold a simple scientific principle as truth, to enable the strategy, which is the fastest way to the right answer is to eliminate the wrong ones as mentioned previously. This means we strive to fail fast. In drug discovery, most ideas fail. That isn't a flaw in the process, but that is the process. And our organization is designed to achieve this as fast as possible.
- Danjuma Quarless
Person
Every compound we rule out brings us closer to one that works for patients, so getting to know as quickly how we get to the right yes. And this is where we think AI is gonna have the biggest impact because this is what AI is extraordinarily good at. Sheer complexity of human biology is the constraint for drug development, and there will never be enough scientific expertise to explore all diseases by hand. This is a problem AI helps us solve directly.
- Danjuma Quarless
Person
First, let me address a major misconception about AI in our industry.
- Danjuma Quarless
Person
When most people hear AI, they think chatbots or LLM usage or software that writes text are primarily used for text. To us, that's a small part of the technology as it relates to drug discovery. Most of Lilly's compute power isn't used to process language. It's used to build molecules. And our chief AI officer writes, we're moving from AI simply as a tool, but to leverage it as a scientific collaboration collaborator in many instances.
- Danjuma Quarless
Person
Here's a bit more of what that looks like. AI as an instrument or as a supercomputer. Earlier this year, Lily partnered with NVIDIA to enable the most one of the most high power supercomputers owned and operated by a biopharma company. We call it Lily pod and it went on operation this summer. We built it with our neighbors in mind.
- Danjuma Quarless
Person
The computer lives in inside existing facilities, require no new construction, no constraint on local land or utilities, and it operates with our commitment to renewable electricity and carbon neutrality. It's a serious scientific instrument that gives Lilly access to high high performance compute, and it's a California story too given that the co innovation lab, one of the the collaboration with NVIDIA is being housed, is gonna be right here in South San Francisco.
- Danjuma Quarless
Person
On designing medicines, Lily runs AI model schools models trained on decades of internal experimental data, which is billions of data points. These models help scientists design entirely new medicines across every drug modality, not just small molecules, antibodies as we heard, but emerging modalities such as peptides, RNA, and gene therapies. AI increases the number of chemical designs human scientists can attempt, and it doesn't replace the human.
- Danjuma Quarless
Person
It works with the human because as a as a regulatory element, you always want the human in the loop. We can use AI computational methods to identify, optimize, and validate the most promising drug candidates. What does that mean for the ecosystem? We don't aim to keep these advancements to ourselves through a platform called Lilitune Lab.
- Danjuma Quarless
Person
Lilit gives smaller biotech companies, including those right here in California, access to all of our AI models built on Lilitate data without anyone having to expose a proprietary data in a method that's called federated learning, which is something that could be critical to the the privacy aspect.
- Danjuma Quarless
Person
And we used it to scale up our whole innovation ecosystem across California and globally. Taken together, the impact isn't a measure in computing power alone. We believe these efforts will have tangible impact on California and and beyond. However, we need to be clear about what AI is as a technology. AI doesn't replace scientific judgment or reasoning, and it won't invent new medicines magically.
- Danjuma Quarless
Person
It's a tool that enhances great scientists and great thoughts, which brings me to the point I'd like to address with this committee. First, with respect to pharma r and d, we think federal unified AI regulations preferable to state regulation, A patchwork of different state rules that biotechs have to navigate and govern related to AI usage could slow innovation with patients as need. We respect with respect to pharmaceutical research, the FDA remains the gold standard for regulation.
- Danjuma Quarless
Person
Second, it's critically important to keep investing in California's universities and the programs that produce scientists and researchers of tomorrow. These are what's gonna help us build the workforce, and together we'll build it, the medicines that constituents are seeking.
- Danjuma Quarless
Person
Finally, the r and d tax credit, which was brought previously, the this program is critical to state innovation and strengthening the ecosystem in California, and we thank you for, efforts there. And with that, I'll yield. So thank you.
- Chris Ward
Legislator
Great. Thank you, doctor Croles. And then our final panelist, for this panel is doctor Degan.
- James Diggan
Person
Thank you very much, chairs Ward and, Bauer Camp for the opportunity to speak today. My name is James Diggan. I lead policy, biosecurity, and trade compliance for Twist Bioscience. I also Chair the International Gene Synthesis Consortium, which is an industry association whose members manufacture the large majority of commercial synthetic DNA. Twist was founded in 2013 in South City and sitting near San Francisco.
- James Diggan
Person
It's a synthetic DNA manufacturer based in The United States. Our headquarters remains there. We manufacture our materials there. We also have a manufacturing facility outside of Portland, Oregon. We've grown to more than a thousand employees.
- James Diggan
Person
We've grown to more than a thousand employees, and we have offices and staff around the world given that we serve a highly international market. In the past year, we produced nearly a million unique gene sequences for our customers, so we operate at quite a large scale. And to understand what we do, you can imagine a a bridge between the digital and physical worlds.
- James Diggan
Person
So our customers send us text files that consist of a's, g's, t's, and c's, representing the individual chemical basis of DNA. We have a unique silicon based manufacturing process that you can see sort of depicted in this video here. This is a classic California innovation story. We've taken the technology that's been available since the eighties to make DNA and shrunk that that down and miniaturized it.
- James Diggan
Person
So in the footprint where most companies make one g, we make about 10,000, and we've reduced the amount of chemicals necessary by 99%.
- James Diggan
Person
And that allows us to reduce our costs, which we can then pass on to our customers. This is really about a a volume, capability. There's a tremendous benefit that comes from gene synthesis technology. Synthetic DNA is widely used in biotechnology applications, including health, energy, agriculture, biomanufacturing. You've heard today it's used to design and test new antibody based therapeutics.
- James Diggan
Person
We can program bacteria that sit at the roots of plants and provide nitrogen, allowing farmers to use less fertilizer. Synthetic DNA has also been used to program yeast to make spider silk. That spider silk is stronger than steel or Kevlar and can be used to produce ultra strong parachutes or enhanced body armor. There are also microorganisms being programmed with DNA that aid in rare earth element recovery and separation.
- James Diggan
Person
So there's an incredible, breadth of applications all going back to DNA as sort of the language used to program these cellular systems.
- James Diggan
Person
I would actually add one point that sometimes it's easy to overlook that this is all domestically manufactured. So when a US pharmaceutical company begins developing a new drug, the synthetic DNA that's at the core of that research and development process is manufactured in a US facility and ordered through a California company like Twist. So that's a part of the pharmaceutical supply chain that is one of the places where The United States has real genuine strength.
- James Diggan
Person
In terms of how AI is impacting work at Twist, there are two major areas of impact. The first is that AI has substantially changed what biologists are able to design.
- James Diggan
Person
I think you've heard a little bit about that from the panelists today. There are structure prediction models like AlphaFold and genome models like EVO two that was mentioned earlier. And these allow production of sequences that perform some intended function while looking very little like anything existing in nature. So in June of this year, a group at Stanford and the ARC Institute posted a preprint describing a system they called proto.
- James Diggan
Person
But let's a researcher describe the biological function that they want in ordinary English, and then AI agents write design programs that then generate candidate DNA sequences that would produce proteins to satisfy those English language requirements.
- James Diggan
Person
And when they tested this, they designed synthetic gene regulators that shared less than half of their sequence with any natural protein, and yet roughly half of them were still functional when tested in the laboratory. So we're at the precipice of being able to design sequences that don't exist in nature and yet fulfill requirements that humans are setting at the outside of their scientific work.
- James Diggan
Person
And what this means in practice is that these these researchers can now explore an enormous space of possibilities in protein molecules, but the cost of doing that is also coming down at the same time. So work that would have required a year of laboratory screening can now be a few weeks of computation and a single round of building DNA sequences and automated testing. The second development, I think, is less visible, but may arguably more may be more important, commercially for California.
- James Diggan
Person
And that is that these biological design models require data for training as you just heard from my colleague. Vast amounts of experimental data describing the relationships between a given sequence and its function, and that data does not exist on the Internet. It has to be generated in a laboratory usually via laboratory automation to keep costs low. And this has created an entirely new category of customer for Twist.
- James Diggan
Person
Companies training biological design models come to us for very large libraries of sequences rather than just for buying individual genes, which is where our business started.
- James Diggan
Person
This is hundreds of thousands of designed variants that we build, we express as proteins, and then we characterize in our high throughput laboratory. And in many of these cases, what we deliver back is a dataset rather than a physical product. So the customer never handles a wet lab reagent at all. We run the experiment. They receive the measurements back.
- James Diggan
Person
They use that to refine their model. The model produces new experimental designs that they then order from us, and this goes around and round in a design build test learn cycle. And that ability for large scale DNA synthesis is available at only a handful of organizations in the world, Twist being one of them here in California. And we should also be clear that DNA synthesis and many of these AI design tools are considered dual use technologies.
- James Diggan
Person
So the same models that can produce a better therapeutic antibody can also help redesign a toxin.
- James Diggan
Person
And for that reason, every order Twist receives is subject to biosecurity screening. The sequence itself is compared against databases of dangerous pathogens and toxins. Customers are screened against sanction lists and to ensure that they're legitimate scientific actors. We've done this voluntarily for as long as we've been in business. The industry first developed these practices through the International Gene Synthesis Consortium, the IGSC, in 2009 and were most recently formalized under a White House framework for nucleic acid synthesis screening issued in 2024.
- James Diggan
Person
There's a an Assembly bill 1864 that was introduced by Assembly member Berman. This would require California gene synthesis providers explicitly to carry out this kind of screening. Twist already performs, and we're very supportive of the goals of that bill. We don't consider screening a competitive burden. This is really a crucial part of our business in making sure our customers use our products responsibly.
- James Diggan
Person
We also have ongoing efforts to determine whether screening of this kind is maintaining its defensive capability against sequences that are increasingly engineered by AI. So Twist recently worked with Microsoft Research and a group of academic and government scientists to ask whether current AI protein design tools could redesign a toxin so as to preserve its structure and function while evading the screening systems that we use? And the answer, in theory was that this this was so.
- James Diggan
Person
And so over roughly ten months prior to publication, we developed a corrective patch distributed to synthesis companies worldwide. And so that by that time by the time the vulnerability was disclosed publicly, the defenses had all been updated.
- James Diggan
Person
And this approach this is similar to the approach that the computer security field uses. It has used for thirty years, and and we would encourage the committee to regard this as a standard that we should be holding ourselves to in biology as well as these sorts of vulnerabilities begin to be more common. That laboratory bottleneck I described a few minutes ago, is also where I think the legislature has real opportunity for impact. You all mentioned CAL Compute earlier. We're also huge fans of Cal Compute.
- James Diggan
Person
We hope we hope that the legislature appropriates funding for that that resource. But in most fields, access to computing is a the major or a major binding constraint. In biology, that's less so. Biological models are really limited much more by experimental data, and generating that data requires laboratory capacity that academic groups or three person startups can't really afford.
- James Diggan
Person
So we fear that how compute that is only access to GPUs could do relatively little for the life sciences, whereas one that also underwrite some degree of laboratory measurement and public datasets would give California researchers a decisive advantage.
- James Diggan
Person
As we said, we're supportive of of AB 1864. But to be clear, we really hope the state continues to reference federal standards rather than coming up with California specific requirements. We serve customers in all 50 states and divergent state level requirements would add cost to our business. The measurement problem I mentioned where states could fund datasets to help us build defensive models, I think is one area that really would would be incredibly valuable.
- James Diggan
Person
One last thought we might also suggest that the outputs of design models should remain free of regulatory oversight.
- James Diggan
Person
A proposed digital sequence is by itself not dangerous. It's that transition from the digital world into a real molecule where now the danger, you you can think of, as being posed. So keeping the your intervention point at the point of manufacture, I think, is much of why the design side of this field has been able to move as quickly as it has, and we'd like to see that stay that way.
- James Diggan
Person
So with that, I'll thank you for your time, and I'll be glad to take any questions.
- Chris Ward
Legislator
Thank you very much, Ajju. All all four of our panelists. Some I think a lot of variety here, I think, in the work that we're doing, how we're applying AI to any one of your kinda areas of specialty. I wanna welcome Senator Pappan as well from South San Francisco. Any questions come to mind for you?
- Diane Papan
Legislator
Keep talking. I'll come up with some. Alright. But I do wanna welcome the people from my district. I I sit here with enormous pride
- Diane Papan
Legislator
That, South San Francisco is, obviously, at the forefront of these things. But I appreciate the Chair putting this hearing together so that we have guardrails in place. But at the same time, we're not interfering with tremendous advancements that are really changing the health of people and, their survival rates for very rare diseases, and, I'm absorbing it all. I am too. Questions, but they were so general.
- Diane Papan
Legislator
And I thought I think most of it's been been addressed, but I'll let you take over too.
- Chris Ward
Legislator
Let me let me open it to any of our panelists as well. You know, we talked in the first panel kind of about these shared resources and the the need possibly for using AI technology generally in a more open source format.
- Chris Ward
Legislator
But you maybe could any one one or more of you sort of talk about how you are acquiring technology to then be able to more specifically improve the efficiency of your work or the, you know, Kinda get to, you know, the deliverables that that that is your mission. But when you're acquiring that technology, right, you've got you're running against a proprietary interest. Right?
- Chris Ward
Legislator
You wanna make sure that, like, you know, the work that you're doing rightfully is also, you know, sort of kinda kept in lock key for the investments that you're making. So how do you really thread or what's what's the what's the landscape around those issues where, yes, not everybody needs to acquire their very own $10,000,000 supercomputer and do that over and over and over again when shared resources, you know, could, be able to more effectively help everybody out.
- Chris Ward
Legislator
But you wanna make sure that, you know, your data and your information, your work is, you know, responsibly being, managed. Sure.
- Chris Ward
Legislator
I can I can speak to that from the perspective of the biotech ecosystem as far as company formation, company creation? So what often happens is open source models are are widely available. Academia is incredible at releasing these models and and maintaining these models.
- Danjuma Quarless
Person
And what's happened what typically happens with company formation is these open source models are then leveraged, which have been released probably to the public, with private sources of information or or experimental data that a company then may acquire venture funding to supplement or attack a specific problem that they're going after in drug discovery. You have to think of drug discovery is not just this one contiguous system.
- Danjuma Quarless
Person
It's biology is very fragmented and very bifurcated, and I don't think there's one model that just owns them all or rules them all. So I think that's open source is always going to be access or a a a element of that ecosystem, whether you argue it's a laggard by four months, six months, a year in innovation, that's kinda trivial. It's always gonna form the basis or underpinning of how these companies are forming, especially right here in California and San Francisco.
- Ginny Hu
Person
Yeah. I can add a little bit to that maybe from a slightly different perspective. As we use these models in our products that's released to the end users, there's definitely an element for us to consider with the right model to be able to provide the right guardrail as the right experience to the user. There's also another element to make sure that these models, whichever model we choose, is able to work seamlessly with the other cloud based infrastructure.
- Ginny Hu
Person
All of those are supporting a user experience in the in the regulated medical device kind of an ecosystem.
- Ginny Hu
Person
So there's actually a lot of work. I think we enjoy the proximity of being close to Silicon Valley to some of those tech companies that offer the the the infrastructure and and also may offer some of those models to be able to work with them. There's actually a lot of synergy, a lot of work with those tech companies to make sure that the operating system, the models, and and any of the infrastructures that they provide to as this medical device. We're a customer of theirs.
- Ginny Hu
Person
We're a developer on their on their platform to make sure that we are able to work seamlessly with them.
- Ginny Hu
Person
Sometimes we do have to work on issues like making sure that what priorities that we have as a medical device product does not get overshadowed by something that has very high high user usage. For example, a social media, mobile application, or something else. So there's a lot of work there, but a lot of a lot of great collaboration. But that's a unique element as a regulated device product that we encounter to make sure that the model that we choose works with the ecosystem.
- Chris Ward
Legislator
Okay. Great. Thank you. You got me thinking I had a chance to visit the Dexcom facility in San Diego region as well too. But just just, you know, really opened my eyes about how amazing a time that we are in That, you know, when you think about those that are afflicted with diabetes or thirty years ago, twenty years ago even, you know, had a lot more difficult time, you know, with their active with daily living.
- Chris Ward
Legislator
And but I'm wondering as you're as you're using this technology, you know, you're also on the individual level kinda getting, you know, individualized feedback as well. And so how are you integrating that that personal protection for that information?
- Chris Ward
Legislator
But also, maybe in the aggregate, you know, some of that information on your own device, your own products, your own software is helping to inform yourself about how to make improvements in the next generation of devices that that you're that you're, you're so would you would you would you sort of process that information sort of in the aggregate?
- Chris Ward
Legislator
Are you is AI, software or or technologies that you're using also looking for individualized variabilities that that can help, like, kinda look for new observations that you've never even thought about. Can you talk a little bit more about the use of AI for, maybe the individual's experiences and anybody else as well, that is, sort of seeing now this this once you've acquired it and you're applying it to the work that you're doing, all of a sudden, you know, there's these new tangents.
- Chris Ward
Legislator
There's these new, like, you know, bits of information that you're getting off of that. And and how is that going with respect to, data privacy?
- Ginny Hu
Person
Yeah. That's that's an excellent question. Definitely something that we we think a lot and discuss a lot about internally. We we started, like you mentioned, Chair word with by providing this using this technology to enable, individualized insights to help to make sure that the information that we provide to each end user is contextualized to their own experience. But like you said, there's a richness of that data at a population level.
- Ginny Hu
Person
Something that we're considering, actually working with the FDA on this very closely as well through a new initiative called the temple program. Dexcom was the the first company that was selected to participate in this pilot program with the FDA and also in collaboration with CMS and CMMI to pilot a new digital health technology that can incorporate AI enabled kind of clinical decision support type of tools for Clinicians.
- Ginny Hu
Person
So that at aggregate level, not just for us to do this internally, but also be able to use AI to help provide the right type of insights for our HCPs. HCPs use our portal to look at all of their patients and aggregate how are they how how do we help them make their time more efficient? How how do we help them identify really the right the right patient and the right right treating treatment paradigm for that thought process?
- Ginny Hu
Person
So that's something that we we're starting to work on, and there is great kind of FDA framework for us to do that.
- Vibhor Gupta
Person
I think a great point was made. Two good points were made by the previous panel.
- Vibhor Gupta
Person
The number 1 was around making the training data available and, you know, or also making the results of the validation available for some of these models, which, of course, goes a long way, especially for companies like ours who are using them and for users of our end users who are going to be using our products because then you can really show them what's under the hood and, you know, how you got to the final result that you're presenting.
- Vibhor Gupta
Person
The second so so, you know, I I would fully endorse that if that was possible through legislation to make sure that companies who are building these models, be it open sourced or be it the ones which are proprietary, can at least share some of that training data information and how they got to where they've gotten to.
- Vibhor Gupta
Person
Because as I'm sure you've read in literature, you know, right now in clinical decision making, there are publications, you know, galore which talk about, you know, this tool is better than that one or this model is better than that one.
- Vibhor Gupta
Person
And and it's very hard to know, well, what is the repeatability of it? How do you know in six months' time this is not gonna change and the rankings change again? And that can only be ascertained if you do have that source data along with it. And the second part of it is which is, again, comebacks to creation of value because all of this has to be attached to some kind of value creation. You know?
- Vibhor Gupta
Person
And if you can demonstrate how you got to that value and and work backwards as to, okay, how are we going to now scale the determination of that value through a reimbursement framework or something that can be applied at scale and can be paid for at scale, then you can truly increase application. Because as you can appreciate, otherwise, a lot of the apps, including ours, end up, you know, in this you know, one of the thousands apps that exist on the App Store.
- Vibhor Gupta
Person
But how do you know that you can use them until unless you have a very clear mechanism to pay for them or somehow benefit for them? And this brings to the question that you were asking, Chair Ward, which is from a patient's perspective, that's what's most important. You know, even today as a citizen, if I had to access a lot of this technology, there is not a straight mechanism for me to do that.
- Vibhor Gupta
Person
I'll probably go to one of, you know, the usual suspects, web search or an open source large language model and ask some questions. But I don't know if I can trust that information. And frankly, I don't even know if what to do with that information. I might have a semblance of an idea, but doesn't mean that it really connects the dots for me. So I think there is a work to be done over there, both from a legislative standpoint and from a technical standpoint.
- James Diggan
Person
Maybe, something to consider when, discussions arise around AI governance and guardrails. We use models in many use cases that are sort of defensive. You know, we're asking, is the sequence that's coming in the door? We're asking a model to say, could this thing be used to cause harm? And that triggers the guardrails on most of the models, and they refuse to answer.
- James Diggan
Person
And so coming up with a way to think through who are trusted parties that should be given access to models where those guardrails are lowered to some degree so that they can use them in appropriately defensive applications. And and right now, some of that is going on with the Frontier Labs, but a lot of that decision making is somewhat ad hoc in terms of who gets access to what and how they make those decisions.
- James Diggan
Person
And so coming up with some sort of framework for how to think through that, I think, would would be very valuable in the long term.
- Chris Ward
Legislator
Thank you. I know that's gonna be a good segue for, I think, the discussion in our next mail when this happened. Any questions for
- Diane Papan
Legislator
Oh, quick question. How how do you charge for your services? So we Do you just download the app or you
- Vibhor Gupta
Person
Yes. So we it's ours is an enterprise software product, and currently, it is paid for by enterprise health systems for use within their own health system. And the way they measure the usefulness of the product is alignment with their own KPIs, be it around value based care objectives or be it around, you know, improving patient quality and safety and so on and so forth. But you this is the point that I was making.
- Vibhor Gupta
Person
If somebody had to use this in community practice or if a patient had to use it, what would be the framework that would allow them to use a product like this and pay for it?
- Diane Papan
Legislator
I know how Italy works. I probably have a good idea how you work. You, I'm not so sure about.
- James Diggan
Person
By the base pair. So if you want an a and a g and a t and a c, we charge you for each of those base pairs. So the longer the sequence, the more expensive.
- Diane Papan
Legislator
And if I may ask one more which isn't necessarily related to AI, but I'm just curious. The synthetic DNA, is it as accurate as perhaps, you know, regular old human DNA that you might be working with? What's your accuracy like?
- James Diggan
Person
But but accuracy well, that's a good question. That is one of the key differentiators throughout the industry is the the degree to which a molecule will have the fewest number of errors versus what the customer requested. And so we make very high quality DNA. We have data to back that up. Many of the leading companies do the same.
- James Diggan
Person
So the error rates are are very low in terms of, you know, if you order a 500 base pair piece of DNA, we're gonna deliver to you the 500 base pairs you asked for.
- Danjuma Quarless
Person
Quick quick aside on that. AI does have the ability to incorporate what's known as unnatural products or unnatural amino acids and things of that nature to the synthetic piece that can unlock aspects of biology that wouldn't be natural with canonical amino acids or or natural amino acids. And that's a burgeoning area of AI innovation across it.
- James Diggan
Person
Yes. And to to to my colleague's point, we do have customers who request that we use non natural base pairs or other chemicals in building those DNA molecules because it gives them certain advantageous physical properties for use in diagnostics and things like that.
- Diane Papan
Legislator
Okay. I wish you bad speed. Thank you for asking my new five questions.
- Chris Ward
Legislator
Thank you very much. Thank you, our panelists, for your participation here today. We'll certainly look forward to continued engagement as well and gonna see where the universe has taken us. Thank you. We would like to welcome up our final panel.
- Chris Ward
Legislator
This is gonna cover an area that is important to all of us for privacy and consumer protection perspectives. I'd like to welcome doctor David Magnus, who is a professor of medicine and biomedical ethics and a lot of other things at Stanford University, and Christine Von Rayfield, who is a board member and a community liaison for the light collective. Welcome. And thank Doctor. Magnus.
- David Magnus
Person
Okay. Thank you so much, chairman Ward and the committee for inviting me here. I'm gonna talk a little bit about, some, not all, but some of the ways in which we're trying to have oversight and governance of the use of AI at Stanford both in clinical care and in research. So for see. For the governance of clinical applications at Stanford Healthcare, some colleagues have developed something called the firm assessment process, which stands for fair, useful, reliable models.
- David Magnus
Person
So the idea is you you've already heard a lot today about some of the different things that have been developed and all the things about how great all these different things are. But are they really great? Because a lot of these the data behind them are proprietary. We don't know if the patient populations that that was served as the training data for some of the models will fit for ours.
- David Magnus
Person
And so, we wanna really see whether or not the different, available tools that are potentially to be introduced really work very well.
- David Magnus
Person
And so this framework has been developed as a way of doing it. It's actually quite extensive and expensive. The starting point, which I can't emphasize enough, is really starting with the point of, like, what is the problem that the AI model is trying to solve. Because a lot of the times what we're seeing are models being produced by companies or by Epic, that are just sort of tools looking for an application. And, and this so I think that's the important starting point.
- David Magnus
Person
Is is there a real need for things, to happen? And this process starts with that, with interviews of of the stakeholder. There's, part of the process for this includes, there's an article that lays this out and it's in your in your, slides. But it really they really look at what happens when you actually implement this, look at the workforce implications. But from my point of view, the most important thing is also the ethical assessment.
- David Magnus
Person
So some of my colleagues carry out ethical assessment and it includes interviews with stakeholders that include patients who are gonna be impacted by this. So having that patient perspective is critically important before implementing any of these things. This happens actually pretty rapidly. Let's see. Is it moving forward?
- David Magnus
Person
Oops. I just lost the slides. There we go. Okay. So next next slide.
- David Magnus
Person
Here we go. So in six to eight weeks, all these interviews are conducted, models are developed to sort of run through and see what will happen. And doing this gives people a good sense of, like, what the actual impact of this is gonna be on our health care system. And a whole set of recommendations often come out of the ethics ethics review process. And that ethics review then gets read viewed by another committee that I'm on.
- David Magnus
Person
So I've got get to I've been able to review every single enterprise wide AI tool that has been proposed for implementation at Stanford Health Care, And we can really start to see not just how how it's likely to work and what might work, but also where the gaps are and where we think we may need to learn more and model things and watch over time to see how it's working and to see if it's functioning well. Oops. Sorry. I seem to be there we go.
- David Magnus
Person
So every single AI system that's proposed for enterprise wide deployment at Stanford goes through a firm assessment.
- David Magnus
Person
They cost about $300,000 per assessment. So we're doing that regularly for every single attempt. There's an ethics team carrying out interviews with stakeholders for every enterprise wide AI tool to be implemented, and then a committee that reviews all ethical assessments and all the interview summaries. Let me turn so that's what we're doing sort of at the implementation stage in terms of implementing health care, at at the hospital level.
- David Magnus
Person
Let me turn to the research space because I think that's in some ways, builds on, especially the some of the things that came up in the last panel and some of our really novel approach to to oversight there.
- David Magnus
Person
And some of our approach we've developed here has started to be adopted by the EU. We were able to present some of this at the White House during the last administration, and we're trying to have this approach taken up more. Part of this comes out of the recognition of the limitations of our current human subjects protection system.
- David Magnus
Person
A lot of people think that if you're doing research and it involves human subjects, institutional review boards give oversight in between the FDA and IRBs, we're all set for regulation and oversight. That is not the case.
- David Magnus
Person
IRBs are actually very narrowly focused on research participants. If you're doing research on something that's safe for the participants but will destroy democracy or lead to many deaths outside of the subject, that is none of the IRBs business. And in fact, the IRBs are prohibited from considering that a research risk and their evaluation of the risks and benefits of the research.
- David Magnus
Person
So this is a huge gap because downstream consequences of research or any research that doesn't involve human subjects and falls outside of FDA purview are for largely unregulated. Okay.
- David Magnus
Person
Let's see. Next slide. Okay. So what IRBs don't do, as I said I just mentioned this, and it's actually in the regulations themselves that IRBs don't do this. Next slide.
- David Magnus
Person
Okay. So we've tried to fill that gap by developing something we call the ethics and society review process or ESR with where we try and fill that gap by getting researchers themselves to reflect on the ethical and social implications of their research. And essentially, as they're applying for funding, consider these things so that we can so that we can then assess them and see if we can do some work on mitigating them in advance. Next slide. Okay.
- David Magnus
Person
And this is sort of what the process looks like. This has been so far tied to funding efforts internal to Stanford through our human centered AI Institute. And so what happens is people apply for grant funding. It gets a scientific review. If it's going to get approved from a science review perspective, then the all the researchers who are applying have to produce an ethical and societal review statement where they assess all the writ downstream consequences of their work that they can consider and mitigation strategies.
- David Magnus
Person
We then review those. We have a big interest funding group that reviews those things, and then we often iterate with them back and forth and say, like, how are you changing this? Or you may not consider this risk or maybe you haven't mitigated this successfully. And all that has to happen before recommendations funding. I would say up until now, we've always been able to work things out with all the researchers.
- David Magnus
Person
We just had our first case where we said no to something that it's crossed the boundary where we said we we can't can't do that. Next slide. We did some when we first launched this, and this is based on a publication we published in the proceedings of the National Academy of Sciences, we wanted we asked researchers what they thought about going through this process. And you can see that it was generally I thought it was actually impactful on what they did.
- David Magnus
Person
And the ones who iterated where we actually gave them more feedback found it more useful than than others.
- David Magnus
Person
Next slide. We also conducted qualitative research where we interviewed the researchers who'd gone through this process. And these are some of the quotes. You can see the ESR requirement led me to engage with my co co PI. As a psychologist, I wasn't aware of some of the potential ethical implications Sam and I have, and it helped me to engage my co PI as part of this requirement.
- David Magnus
Person
Next slide. In fact, we might flip our whole research approach to being about privacy. The pretty strong reaction from the ASR made us rethink to lead with privacy. We don't have answers yet, but it's definitely helped us think about a better way to approach the research, how we're doing it and how we're talking about it. So this has been a really value it seems that researchers themselves have said how valuable this process is for really impacting things for for bringing about change.
- David Magnus
Person
And one of the reasons why this is important is the way research one of the things we saw very early on in this process is that when you talk to researchers about the downstream implications, they tend to think about that as not their problem, but something that's gonna happen later downstream. So, okay, we're not worried about the application. We're just doing this upstream research.
- David Magnus
Person
By the time you get downstream, the people who are are actually implementing things say, well, if you wanted to avoid these problems, you should have thought about mitigation strategies further upstream that could have been put into place. So we're trying to really do that upstream, getting researchers to put those mitigation strategies in place.
- David Magnus
Person
Next slide. And here are some of the common issues. Dual use, which you heard some a little bit about earlier, which I'm gonna talk about a little bit more, as a very common issue or soon as it's referred to as reasonably foreseeable misuse. Issues around representativeness of data, stakeholder involvement and design.
- David Magnus
Person
One of the things we try and do is hold people accountable to making sure they're engaging with patients or people who are impacted by the work research that they're gonna do even if they're fairly far upstream, as well as potential harms to subgroups, which we know in terms especially in terms of equity and and of the of the results of the use of some of these models.
- David Magnus
Person
Next slide. So I wanna just very briefly end by talking about one of the things that came out of this around the dual use issue or reason for foreseeable misuse. You just heard a presentation in the last panel about how some of the work that they're doing could potentially be misused. They develop strategies to try and do screening to avoid that. AI can get around that screening.
- David Magnus
Person
They said they figured out patches. Turns out AI can get around most of those patches, and there's all this constant back and forth that takes place as things are as the potential for misuse. The potential for misuse here, especially when you have AI integrated with biotechnology, is very, very real and very, very serious.
- David Magnus
Person
The New York Times a few months ago, had some of my colleagues, David Relman and some other colleagues showed how they could actually use chat GPT and other widely available models to design bioweapons that, actually figure out how to do it, how to deploy it. I mean, if you don't know what mirror life is, you should learn what mirror life is because it's something that keeps me up at night.
- David Magnus
Person
But there's a all life has a chirality handedness, and now we're starting to develop bioengineered reverse chirality. And that has lots of benefits, unite. So it can get around antibody resistance by creating antibiotics that are reversed chirality. But, of course, life on Earth evolved with a certain handedness, which means if you have a bioweapon or an accidental release of a pathogen that has reversed chirality, it is not at all clear that hue that life on Earth will have defense mechanisms against it.
- David Magnus
Person
So the potential for some of these weapons or just accidental release of something is quite quite frightening.
- David Magnus
Person
So next slide. Sorry to scare everybody, but we are literally talking about lots of great up science for AI might lead to the end of life on Earth. So one of the challenges that we found when we were doing our ESR of researchers whose work had this kind of dual use bioweapons potential was, that there's nothing to help them. There's tons of stuff. If you have privacy issues, there's tons of stuff to help them.
- David Magnus
Person
If there's worries about bias, there's worries about inequity, tons of stuff out there. Very little on what to do about biosecurity for researchers who are trying to mitigate these risks.
- David Magnus
Person
So we spent a couple of years working on developing at least an initial framework for how to think about this that we published in Nature Machine Intelligence to try and really give this is just it's it's just a starting point for dealing with this issue, but this is to give researchers tools to try and address these problems. And this was work was funded by the Human Centered AI Institute. Next slide.
- David Magnus
Person
It's a very busy slide, but it kind of goes through what we recommend researchers do. That includes engaging stakeholders so you can talk with the people who are gonna be implement impacted by the research that's being conducted, doing a risk mapping. And then on the right side of the screen, you can see three different mitigation strategies that we've developed. I've given you three slides, and I'm not gonna go too into detail here.
- David Magnus
Person
But there's three extra slides that really explain in a little bit more detail a whole range of mitigation strategies that are there to try and mitigate risks downstream as you're building a building models for researchers to consider.
- David Magnus
Person
When they're done thinking about working through their mitigation strategies, then they can look at whether what the level of risk is that's left over and whether or not the benefits outweigh the risks.
- David Magnus
Person
The goal here is not to make risk go away because you can't do that, but to really build a deterrent strategy where if you make the risks the mitigation strategies robust enough, it's easier for bioterrorists to just go buy gasoline and fertilizer than to go with some of these kind of bioweapons, and you can reduce the risks of accidental release enough that it's that the benefits outweigh that. Thank you very much.
- Christine Von Raesfeld
Person
Perfect. Chair Ward and and Chair Bauer, Cahan and members of the committee, thank you for allowing me to speak today. My name is Christine Von Raesfeld. I'm a patient and research advocate, and I serve as a board member for the and, sorry, and community liaison with the Light Collective. As I prepared for this hearing, I kept coming back to one thought.
- Christine Von Raesfeld
Person
Millions of Californians now use technologies that generate health information without realizing that information has become part of a much broader landscape. When most people hear the word biotechnology, they picture a scientist in a lab or a pharmaceutical company developing a new drug. But today, biotechnology has become a part of everyday life. It's in the devices we wear, the apps we use, the genetic tests we purchase, and the choices we make without ever thinking of them in that way.
- Christine Von Raesfeld
Person
Those consumer technologies generate health information that AI can analyze, connect, and transform into insights that increasingly influence research, product development, and patient care.
- Christine Von Raesfeld
Person
We tend to experience these technologies as separate decisions. Increasingly, they are not. AI can now connect information from those different experiences in ways that weren't possible just a few years ago. What begins with everyday consumer technologies can now influence every stage of the product life cycle from research and development to that patient care. I've had a unique perspective on how we arrived at this moment.
- Christine Von Raesfeld
Person
I was 14 years old when I received my first certificate recognizing my participation in my participation in advancing research. It sounds pretty cool, but the reality was very different. My parents and doctors were just trying to keep me alive. At the time, we didn't I didn't know what biotechnology was or how it was shaping my care. I only knew that research represented hope, and that is what a reality that many Californians face today.
- Christine Von Raesfeld
Person
I'm 51 now, and I'm still searching for answers. But for the first time in a while and because of AI, I have new reasons to be hopeful. What began as an effort to save my life has become nearly four decades of participating in research and contributing to science. During that time, I've watched the field move beyond laboratories and become part of everyday life. Today, we're watching AI accelerate that progress in ways that are changing how discoveries move from research and increasingly into our care.
- Christine Von Raesfeld
Person
The advances we've heard about today are extraordinary. AI is accelerating discovery, improving clinical trials, identifying new therapeutic targets, and bringing us closer to treatments that many families have waited decades to see. As someone who's is still searching for answers, I want that future as much as anyone in this room. But progress also changes the questions we need to ask. When information from research, health care, wearable devices, genetic tests, health applications, and other consumer technologies can all contribute to AI.
- Christine Von Raesfeld
Person
The question is no longer simply who collects our information. It's how that information is being used. Do people understand what they're agreeing to? And if AI generates new health insights that influence their care or coverage, how do they know those insights exist or challenge them if they're wrong? As AI becomes part of every stage of the life sciences, the people whose information makes those advances possible should become less should not become less visible.
- Christine Von Raesfeld
Person
They should become informed and more engaged. When someone entrusts an organization with information about their health, whether through research, a genetic test, a wearable device, or an AI enabled product, that organization isn't simply collecting data. It is accepting responsibility for information that can have real consequences for people's lives. The information we're talking about comes from real people, real families, and real experiences. And when those insights can influence someone's care, people cannot disappear from the process.
- Christine Von Raesfeld
Person
We deserve transparency, the opportunity to ask questions, and meaningful ways to remain involved in the decisions that affect us. AI works because it learns from patterns across many people. That makes collaboration essential. When information is unnecessarily locked away, we limit the shared learning that allows research and innovation to benefit everyone. These questions become even more important with genetic information.
- Christine Von Raesfeld
Person
A direct to consumer genetic test purchased by one individual can reveal information about parents, siblings, children, and future generations who never made that choice themselves. As AI expands what can be learned from beat this data, thoughtful stewardnesship thoughtful stewardship becomes even more important because the consequences extend far beyond the original consumer. Public confidence is essential to scientific progress.
- Christine Von Raesfeld
Person
When people lose confidence in how their information is used or stewarded, they're less likely to participate in research, adopt new technologies, or contribute in the discoveries that make better care possible. California has built strong protections for medical information inside the health care system, But many of these technologies people rely on every day exist outside those traditional boundaries.
- Christine Von Raesfeld
Person
As AI generates new health insights, our approaches need to evolve alongside the technology so people remain protected as these innovations become part of our lives. Around the world, governments, researchers, health care systems, and industry are working through many of the same challenges. As a global leader in life sciences, California has an opportunity to help shape what responsible progress looks like. I've spent nearly four decades saying yes to medical research because I believe progress could improve lives. I still believe that.
- Christine Von Raesfeld
Person
And as AI accelerates this progress, I believe California has an opportunity to focus on four things. Transparency, so people understand how AI is being used. Stewardship, so organizations remain accountable for the information they hold, meaningful agency so people can ask questions and correct inaccuracy, and collaboration because no one sector can solve these challenges alone. My hope isn't that we slow innovation.
- Christine Von Raesfeld
Person
It's that our approach to consumer protections and governance evolve alongside it so people remain informed, participants in future and participants in the future they're hoping to build.
- Christine Von Raesfeld
Person
The decisions we make today won't simply shape the future of technology. They will shape whether people continue to believe that participating in research and scientific progress is worthy of their time. Thank you.
- Chris Ward
Legislator
Great. Thank you both for your presentations. Turn to my colleagues as well, see if any questions come to mind.
- Rebecca Bauer-Kahan
Legislator
Well, I'll just say I I had to step out because I had a meeting with some constituents who are
- Rebecca Bauer-Kahan
Legislator
here advocating on behalf of, pediatric brain cancer awareness month. And one of the things one of the moms who lost her son at 13 said was she had donated, his brain to science, and could we help figure out a way for that data that is being used to help save other kids.
- Rebecca Bauer-Kahan
Legislator
And I mentioned this hearing, and I think the patient voice and the survivors' voices in that case are so critical to our understanding of how we can best navigate this in a way that protects people's privacy and protects their dignity and also does what, patients want most, which is to find more cures and more help for more people. And so I wanna thank you for that advocacy and to the constituents that showed up to do the same.
- Rebecca Bauer-Kahan
Legislator
And thank you for showing us sort of I think that it is true that many institutions are trying to model how to do this in a way that is responsible, and that's really critical.
- Rebecca Bauer-Kahan
Legislator
And so making sure that we as policymakers hold everyone to the highest standard, I think, should be our goal, and learning how people do that is critical to that work. So I really wanna just appreciate both of you for being here. And I'll turn it back over to the Chair.
- Chris Ward
Legislator
And I wanted to kinda build on the two. I also really, impressed, that, you know, we're thinking sort of critically about how to what we're looking for when we're talking about the ethics of these questions right now, and that there's some general, although not everyone agreed, general positive response from researchers themselves that wanna see themselves hold to such high standards. Now how do you see that translating into, you know, kind of actually the standard?
- Chris Ward
Legislator
It feels like research at this point, must correct me if I'm wrong, that, you know, I like maybe this is a precondition for public funding or maybe this is a precondition for certain other kinds of approvals. What are you seeing as the response from government officials?
- Chris Ward
Legislator
And I'm gonna add on to that. How does that compare to response from the EU or from other international systems?
- David Magnus
Person
Yeah. So we did present the awesome of our work to the EU, and they now have made something like the ESR process of a mandatory part
- David Magnus
Person
Of EU funding for for research. So it is starting to spread. It's challenging to make this scalable. Right? So it's very, very time intensive for a relatively small number of reviews.
- David Magnus
Person
And we've been trying to figure out ways of making this more scalable without building a huge infrastructure and analogous to the kind of compliance functions of institutional review boards. So we are trying to figure out ways of making this more scalable. Tying into funding mechanisms helps a little bit because everybody's incentivized to do this. California certainly spends a lot of money on research, and you could make an ESR requirement for the things that are are funded. They you don't currently do that.
- David Magnus
Person
And we did present this at the White House as a way of thinking about this, but that that was in the previous administration. And I don't know that it would have been picked up anyway, but that's, I think, what we have to do is really it it has to be tied to funding across the board.
- David Magnus
Person
And I will say one of the things that's been interesting to me as a group that you would think not sensitive to that as industry, but, actually, industry has been very responsive to the idea of doing this, partly because they're worried about what might those happen if they have produced things that have catastrophic downstream consequences. So there are actually a lot of companies that are starting to build their own ESR processes, and that we've been talking with them as they've started to do this.
- Chris Ward
Legislator
On average, I've recognized it very it would be different for different situations. But, like, how cumbersome really would an ASR review be?
- David Magnus
Person
I I don't think it's a it's it's getting the group together to think about it, talk about it, and a couple of hours. Okay. Something in that ballpark. Yeah. They tops.
- David Magnus
Person
And then and then if there's iteration and back and forth, then it takes longer.
- David Magnus
Person
One of the things we're trying to do is figure out tools that can make that process more robust and require less oversight. So one of my colleagues got a grant from NIH to build essentially, it's an empathy tool because one of the things we found is that when computer scientists are working in in the health space, they just think of this as data. They're not thinking about the people behind them.
- David Magnus
Person
And so she's built a bunch of tools that an exercise is really for for for data scientists to start to transition their thinking to recognize the human people that are behind the data. Other people, Katie Shelton, the University of Maryland has built an ethics identification tool.
- David Magnus
Person
We've built some ways of looking at these things. Ultimately, actually, AI itself may give us guidance because it may very well be that as we're learning more and more and doing more of these things, and we're starting to build a pretty good set of data about what our reviews look like, we may be able to train a model to help essentially look at these things and make it more scalable by getting the feedback and giving the kinds of suggestions that our committees would give anyway.
- David Magnus
Person
So all of these are efforts to try and find ways of making this more scalable. Ultimately, though, they won't be universally implemented if there's not. Right now, it's all all carrot, no stick.
- David Magnus
Person
And except for the fact if you want the research, that's the that's both the carrot and the stick. But ultimately, there needs to be both both carrot and stick in order for this to become universally applied.
- Chris Ward
Legislator
Yeah. I can appreciate that. And, you know, kind of answer my next question, but, you know, I know we're always trying to be able to sort of balance, you know, making sure we're doing the right things, against, you know, the need for speed. And innovation is going fast. Competition, of course, is going really quick.
- Chris Ward
Legislator
And, you know, these things aren't necessarily if we're not, you know, sort of being held to the same standards whereas the EU or China or or or other other markets as well too. It's just that's that's the tension there. But, you know, it's probably something that I think is more widely incorporated into the the work that any of our companies or institutions are doing right now. I think we'll we'll kinda see that.
- Chris Ward
Legislator
It's a a standard that really does need to be in place for the the the reasons that you're you're bringing up here.
- Chris Ward
Legislator
You know, we do not wanna be able to see an unintended consequence or worse, you know, somebody that is taking that kind of information and then twisting it in a way. I haven't heard about chirality since, organic chemistry in college. So thanks for bringing that back as well.
- Chris Ward
Legislator
But it does, I think, highlight, you know, the potential, you know, catastrophic, and and nefarious opportunities that are out there right now that I think, this kind of review and, you know, at least just taking that moment to to to think through what we are inventing, what we are what we are we are, Kinda putting out there right now, you know, different ways that that could be used, for for benefit or or for potential harm and safeguarding against that.
- Chris Ward
Legislator
So appreciate, you know, the, I think, the leadership that you're providing to, this area of, you know, kind of, of ethics review of necessarily bringing, that perspective in these questions, into the, rapid advancements that we're seeing in real time, think, is a fundamental thing.
- Chris Ward
Legislator
So we'll think about what we can do here as well for our California institutions to make sure that we are the gold standard on the subject. I wanna thank you for your participation here today. And with that, we do are gonna we're gonna open up to public comment for any members of the public that are here for the, I guess, either of our committees. We'll go ahead and bring the microphone up if anybody wants to offer. Seeing none.
- Chris Ward
Legislator
Okay. Great. Well, I appreciate all of our panelists that are here. I know we're a little bit over time, and we have other meetings that members had to get to as well.
- Chris Ward
Legislator
But critical information that I think is gonna set for the like I said, is a is a good snapshot in time for where we're at on these questions around intersecting AIUs and biotech biotechnological advances that will, I think, help tee us up for where we're gonna be in 2027 is, you know, we wanna make sure California continues to lead in this space, that we're we're doing so in a way that's gonna be helpful, I guess.
- Chris Ward
Legislator
Do we have public comment that's lining up or no. Okay. Great. Great. Well, with that, a lot of information covered here today.
- Chris Ward
Legislator
I wanna thank you all for your participation, and we are adjourned.
No Bills Identified
Speakers
Legislator