Hello, and welcome to the NVIDIA AI Podcast.
I'm your host, Noah Kravitz. Our guest today is no stranger to NVIDIA, nor to the podcast.
Anima Anandkumar joined the show nearly three years ago to talk about her then-personal record of having seven of her team's research papers accepted to NeurIPS 2020.
To say she's been busy since then would be a bit of an understatement.
Anima is a Bren Professor at Caltech and Senior Director of AI Research at NVIDIA.
Her work developing novel artificial intelligence algorithms enables and accelerates scientific applications of AI. including scientific simulations, weather forecasting, autonomous drone flights and drug design.
She's also a fellow of the ACM IEEE and Alfred P. Sloan Foundation. has received best paper awards at venues such as NeurIPS and the ACM Gordon Bell Special Prize for HPC-based COVID-19 research.
And she's part of several world-renowned foundations and councils.
We'll get to all that in a moment. I could go on with a list of accolades, but I think we'd all rather hear from Anima herself.
So let's get right to it. Anima Amin-Kumar, welcome and thank you for coming back to the NVIDIA AI Podcast.
Thank you, Noah. It's a pleasure to be back and great to see all the amazing things we're doing at NVIDIA and in the broader AI community.
It's an exciting time to be sure we were speaking briefly before we hit record.
And I think that last conversation was in the midst of the pandemic.
Lots has obviously happened since then. And let's start, obviously, with AI in the world, generative AI, which as we record this is... all over the news these days.
And my non-technical friends and family members have been asking me all about Wait, what's this AI thing?
Haven't you been paying attention for the past 20 years?
So let's dive into it, starting with your recent trip to speak with the President's Council of advisors on science and technology.
You spoke that of AI's potential to make a huge impact on science, but obviously, AI's potential impact, your own purview, not to discount science by any means, but is larger than just the scientific community.
So maybe you could just kind of take the reins and talk a bit about the moment we're in, the President's Council, Gen AI, and we'll kind of go from there.
Yeah, it was such an honor to be part of the three speakers whose spoke about AI and science to PCAST or the President's Council on Science and Technology.
This was a virtual talk and panel, in fact, that's available online for everybody to go and watch.
And to me, what really came out of that was this aspect of how generative AI is an inflection point in our lives.
And how can we harness it for all the benefits to society and to humanity through scientific applications?
So first of all, for the public here, you've heard of the term generative AI, right?
But what is so... exciting and different from this previous decade of AI developments is this ability to generate de-noble or from scratch entire paragraphs of text.
Right. Very realistic looking images or creative images.
So we are seeing those that are about generation, whereas the previous decade was about discriminative AI where given an image we would reason about oh is this face belonging to the person we're who's claiming that identity or so on.
So it's more about discrimination. What's in the image?
What's in this piece of text? rather than generating it.
And generation was always considered to be very hard because it's much more high dimensional.
You're specifying some very few properties you're saying, oh, generate an image with a cat, right?
There are so many possible ways the cat can be and all the variations that we need to capture to have that ability to sample from that whole distribution.
And we're able to do that. So that's an exciting time.
And so what I delved into in my talk to the Science Council was to ask, how can this be harnessed for scientific domains, and what is there beyond text and images? because text and images are also present in science.
We can harness scientific knowledge through text, right?
But there's so much beyond just the English language.
And one of the examples I started with was showing how instead of the thinking about generating paragraphs of English language or any other natural language, why not think about the language of the genomes?
We took all the DNA data that's available, both DNA and RNA data of virus and bacteria. that are known to us, about 110 million subgenomes.
We learned a language model over that. and then we can ask it to now generate new genomes.
This was also during the pandemic where we said, or let's now focus on coronavirus and get to generate new variants of concern And we were able to predict that before they emerged.
And we are also able to ask, which are the dangerous ones with strong binding?
And how can we reason about that and be better prepared with vaccines and drugs before they emerge?
And so that's one kind of a direct application of what text Language models can be now brought into genome language models and we can get the benefits immediately.
You mentioned that the English language obviously is not the only thing we can input and output in these generative AI models and LLMs.
And we've had a couple of podcasts. I say this, we've recorded them recently. to the listener.
Not sure when they've been out. If they're not out yet, stay tuned.
But a couple of podcasts with folks working on AI models using LLMs for code, for writing computer software.
Again, it's a language and it's maybe for a lot of people, it's not the first thing they think of, but then it's kind of an easy mental leap to think, oh, right, of course, that's a language.
It's text and symbols and such. When you're talking about the language of genomes, and feel free to extrapolate into other scientific areas as well, How similar or different is the process of training and working with an AI model using the language of genomes as opposed to English or another written and spoken language.
Yeah, that's a great question. And that really depends on the domain, right?
In our case, this was still discrete. So it's now you have the nucleotides, you have ATTC instead of write like English language letters.
So that way we could like kind of, you know, Think of like taking in these discrete tokens or think of other abstractions like that.
But the bigger challenge is that the genomes can be very long.
We looked at bacteria and virus. Think about going all the way to humans.
Exactly. Extremely long sequences that our current models are not really able to take in that kind of context.
And some of the latest work is really looking to asking, can you take very long contexts?
And so what we did was to think about these long range dependencies being captured in the latent space of these models and that way create more realistic generation of valid genomes, right?
The kind of the Whole idea is by looking at all existing virus and bacterial genomes, we are implicitly encoding fitness and other functions. because these virus and bacteria survived.
They probably thrived. They created havoc.
So we're taking that examples and asking by forcing this into a language model learning bottleneck. can you come up with that encoding and can you come up with functionalities, right, of different genes and proteins?
So that's how we can get insights from the model, even though we don't have explicit aspects of fitness and other functionalities of what these genomes are doing.
You mentioned working on the coronavirus, or I should say working against the coronavirus.
And predicting, and forgive my lack of precise language, but predicting new variants and which ones might survive.
And then obviously drug discovery as relates to. coronavirus.
Taking that sort of a step broader to thinking about some of the other global challenges, extreme weather, is obviously a big one.
I'm sitting in California in the middle of July right now.
And you know, whatever direction you look, there's wildfire threats, there's flooding, it's all around us, and I think we're all more and more aware of it.
Can you speak a little bit about the potential or even the progress, and we can even use that 2020 recording as a milestone if that's helpful, but The potential and the progress of leveraging AI to tackle some of these big global scale issues.
Yeah. Absolutely. I mean, there are so many scientific challenges we are facing today, right?
I mean, going back to even just the coronavirus example that I'm talking about, Yes, we can generate new variants of concern through this language model trained on genomes.
But to reason what do these new genomes do, we have to look at how do they bind with molecules in our body. for instance, right?
Like if there is strong binding, they may have a bigger impact on our health compared to when there isn't.
But those kind of like molecular binding dynamics is a highly multi-scale process.
You know, you can think about effects all the way from quantum scale, right, to, of course, our full body, right, in all of those different levels of organization that all come into play.
And that's why immunology is a very complicated subject.
And for us to even make sense of, OK, you know, looking at a specific combination, right, a protein and a molecule, how well to their mind.
I know it's not trivial. People think about docking, but also the aspect of Now the protein backbone can change in many cases as the binding is happening.
So it's a highly dynamic process. And that involves multiple scales.
And the same is true with the extreme weather forecasting that you talked about, right?
There are very small scale phenomena that happen in terms of how the clouds you know, have turbulence or precipitation is something what people call microphysics, right?
So this is very complicated and not fully understood even today. and definitely not having the scale even in our biggest supercomputers to be able to faithfully replicate all of this.
And that was kind of my segue to saying text and image models are great, but so much of our scientific domain data cannot be captured just by that because text limits to discrete tokens.
These are continuous processes. And image models that we use it for say applications like mid-journey where you generate say new images, it's all like limited to a fixed resolution, right?
And mostly in the natural image generation world, You're thinking about bigger objects and you want to mostly get the shapes of the objects right and some nice looking texture.
Whereas if you just go stare at a hurricane, you can't tell how it evolves, right?
None of us have that ability. So it's not just about human visual perception.
And we certainly cannot predict where the hurricane will go in the next week or so.
So longer term prediction is even harder and chaotic.
There's even inherent uncertainty there.
And those are the aspects we've been working at NVDM you know, at Caltech in collaboration with many other organizations as well to say, How do we capture these multitude of scales that are present in the natural world?
And even when we don't have data for all the scales, it would be just impossible to... be able to simulate through current numerical methods.
And also we may not know the full physics or the full equations that govern them.
But with the limited data we have, can we hope to extrapolate to finer scales?
Can we hope to embed the right constraints and come up with physically valid predictions that make a big impact.
So hold me back in if I go kind of the wrong way in asking this question.
But I know that a big part of your role at Nvidia is kind of leading the charge on developing next-gen AI algorithms.
And in the five or so years that I've been lucky enough to host these podcasts and talk to all kinds of people and all kinds of stuff, One of the things that's kind of emerged was that there was sort of originally this drive for more compute that a lot of people were talking about.
And it feels like, and I'm speaking very broadly again, so I'm eager to hear your response.
It feels like more recently, there's been a little bit of a shift towards, okay, a lot of folks have enough compute and now we're looking at developing the next gen of whether it's algorithms or going from these large general LLMs to fine training, maybe smaller ones, but that are, you know, more fine tuned or trained on more specific data sets for kind of task specific things.
Your world, I imagine, is somewhat similar and probably quite different from what a lot of folks are doing out there with LLMs or other AI tools.
I guess how much of your work is sort of gated or accelerated by hardware and compute availability and kind of always, are you always just waiting for the next gen of compute to be available so that You can really push towards the next big breakthrough.
Or are we at a point where actually the focus, even for someone as... working as in-depth as you are, let's put it that way, is also kind of more focused on, okay, I don't need more giant models that take huge resources to train.
We're actually looking at sort of a finer scale tweaking of things, to put it that way.
I mean, at NVIDIA, we are lucky to see the full stack.
Of course. Right? always even think about the next generation hardware before they emerge, like Grace Hopper, for instance.
And that's very well suited, in fact, for many of not only large language models, but also weather models and other scientific domain models that require memory.
So we're always mindful of those hardware constraints and efficiency gains that we can get and design algorithms with that in mind.
And scale is important with the weather forecasting work.
All the audience here should go and look at our CEO, Jensen Wong's berlin talk it's the eve summit it's already about a million views on youtube within a few days and yeah and there he puts it really nicely right so there is uh the now the uh you know, the next generation hardware that is going to really make these models more efficient.
But you do need algorithmic design as well, you know, at least...
Hopefully, we are not out of work anytime soon.
I know there is the notion of like a bitter lesson that, oh, just throw in more data, throw in more compute and out comes magic.
We don't need to think. Thinking, in fact, is harmful. to this process, right?
And I've seen these kind of comments on social media.
I would say in the realm, especially of scientific domain, this will get us nowhere.
I mean, it'll get you somewhere, but not a long way.
To give you an example, you know, for weather forecasting, there is a wealth of historical weather data available and it's re-analyzed, meaning the observations are assimilated and you kind of add in through physical assumptions, right?
Any kind of gaps that are in observations.
And we have that for several decades and tens of terabytes of such data openly available.
So you can consume and build models and people have done vision transformers similar to the image models.
And for what we call medium range forecast, meaning one to two weeks, you can get some reasonably good forecast with that models.
But the distinguishing features from those general purpose image models versus the models we use for scientific domains, what we call neural operators and four-year neural operators, is that we are in a principled way able to predict at any resolution.
So the idea is we have this say weather data that's available for training only collected at 25 kilometer due to the constraints of observation and the data assimilation.
But we know the real world is continuous.
So we need to incorporate the aspect that, oh, can we have the model also adaptively interpolate and come up with a zero-shot super resolution prediction.
So you want the flexibility to be able to predict at a different resolution consume training data at multiple resolutions if available, because that's, Ultimately, the underlying phenomenon we want to predict is continuous.
And so that's the aspect that our models incorporate.
And also in this case, because we are predicting on the globe, which is a sphere, we also incorporated the fact that the Fourier transform should be on a sphere.
So this is what we call an inductive bias, right?
So the more of the inductive bias of the domain we bring in, what helped this was not only to get really good forecasts in a deterministic way in the short term, but to also get a much longer term stability, meaning even if you now run this for several months, you have models produce something physically valid, meaningful, and be stable.
And this is because we are embedding the right properties into the model.
And so this is what I call in the realm of extrapolation is where you see the differentiation.
Right, so in the sense if your predictions are pretty much in the same distribution as the training data?
Mm-hmm. A lot of models that fit to the data will do reasonably well.
But in scientific domains, so many times you are looking for extrapolations.
You want to look at longer-term seasonal to sub-seasonal forecasts.
We also want to think about extreme weather events.
One of the demos that Jensen showed in his Berlin summit talk that's available on YouTube is the ability to predict extreme weather like hurricane and heat waves.
And for this, you need larger set of ensembles.
Our speedups enable that. We have speedup of tens of thousands of times over current weather models.
So we can do much more richer set of ensembles, but also by correctly calibrating that. meaning we're not thinking of this as just one deterministic forecast for the next few days, but probabilistically asking, what is the risk assessment?
This is very critical for tail events for extreme weather events, right?
What matters is the tail. And again, the models that do not capture the right physics will fail to extrapolate to tail events because that's like the margins of the distribution, right? machine learning tries to cap fit to the data of normal events.
So I think that's where, especially in scientific domains, I think the better lesson doesn't hold We need to be much more mindful in building the domain knowledge, the domain constraints.
For instance, in some of the other examples say, you know, when we are looking at fluid dynamics and turbulence, we know the equations.
We know Navier-Stokes as the equation we want to satisfy.
So bringing that into the framework will also really help us really capture the fine scales well.
And our neural operators are able to do that because even if our data, we are forced to get only core scale data because it's very expensive to generate it. through simulations or the observations we collect is only coarse resolution.
We can embed the physics in a finer resolution. all directly while training the model in a seamless way.
And that's the benefit that neural operators have.
We're speaking today with Anima Amankumar.
Anima is a Senior Director of AI Research at NVIDIA, And as we've been talking about, her work spans the world of computer science and AI, and also the scientific domain, which extends into many arenas.
And she's also a Bren professor at Caltech.
And I wanted to switch gears slightly and ask you a little bit about your work at Caltech and specifically your work in the tensor lab there.
Could you share some of your current work and its alignment with the broader conversation around AI's potential?
Yeah, I mean, to me, Caltech is a great place where a lot of interdisciplinary work happens It's a small community, but very tight knit.
And yeah, the neural operator work started here by collaborating with Andrew Stewart, Kaushik Bhattacharya.
Victoria, experts in numerical methods, Kaushik in the realm of material modeling.
And so, again, this aspect of building the right model architectures and algorithms by looking at all the wealth of knowledge that has been developed. right, for almost a century with how partial differential equations are solved. and incorporating that because, again, it's very easy to go wrong in these because if you don't capture the fine scales, you're completely off, right?
So really building that into the model came about here.
You know, I also really like Richard Feynman's quote, who was at Caltech, right?
Perhaps one of the most famous professors was at Caltech.
And he says, what I cannot create, I do not understand.
That is so apt for this era of generative AI because I really think generative AI is bringing us to this realm of both scientific understanding, really domain understanding.
And I'm really happy Jensen also used this quote in his Berlin Summit talk as well.
But to me, you know, that quote is like a showcase of The kind of long-term thinking that's been here at Caltech, that's very much the fabric of this university's tool. think about some of the hardest challenges and how to frame it in a way that we can make headway towards that.
You know, I get to work with with scientists across multiple domains.
I mentioned material modeling, in chemistry, working with Francis Arnold, who is Nobel Prize winner here, thinking about, right, how directed evolution and machine learning can really enable us to have the next breakthroughs there.
Thinking about seismology, You know, Caltech has had a long history of being able to predict earthquakes so well in advance and understanding the physics.
And there again, these tools can be very effective because we don't know the ground truth.
We don't know what happens underneath the earth, right?
We only have sensors. on the ground. So our ability to do very fast reconstruction and inverse modeling, what we call, what is the probability of the source of the earthquake at different locations.
It's directly, right, is the one that impacts our safety.
And so again, our models are tens of thousands of times to hundreds of thousands of times faster than current simulation methods.
And with that speed up, we can do now a much greater set of ensembles.
So we can get the right risk assessment, we can do what we call inverse modeling and And also inverse design.
Another example is collaboration with Kira Daraio here at Caltech. where we design a better medical catheter that reduces bacterial contamination by about two orders of magnitude.
Oh, wow. Again, a great example of interdisciplinary research where we used neural operators to model the fluid flow and how the bacteria tend to swim upstream. to the flow, right?
Bacterial density as a function of the flow.
And if we can build like triangular shapes inside the catheter pipe, then how much does it stop the bacteria from swimming upstream?
Very simple. Right, right, yeah. Right, but the optimization doing that directly with AI in the virtual realm, you know, with the physical experiments, It was a huge time saver and then the collaborators went and did those 3D printed experiments in the lab. saw this benefit.
And I think that's also this broader aspect of what AI can do.
It can speed up simulations. It can speed up modeling.
We can get better probability estimates as a result of that.
And we can also invert because these AI models are differentiable.
So we can explore the design space and the space of different hypothesis much more effectively, think of it in a virtual lab before bringing that onto the physical labs.
We mentioned that PCAST earlier, the President's Council, you're also a member of numerous other advisory councils and networks and such, one of which is the World Economic Forum's Expert Network.
We're talking about Gen AI. We've been talking a lot, or as you say, you've been talking a lot about the... need and problems associated with longer term predictions.
So this is kind of a long term and short term thing, at least from where I'm sitting.
The explosion of Gen AI into kind of the mainstream consciousness and the LLMs that have been available, text and image, and people have been using them. have sparked, let's just say, a lot of both positive and imaginative thoughts in the general public, and then a lot of worry and consternation over the future of job markets, economies, humanity as a whole.
From your view and sitting on these different councils and specifically, again, the World Economic Forum's expert network, What, if any, key policies do you see the need for and to be prioritized if we're going to kind of continue advancing the field of AI in a responsible and constructive way that hopefully benefits all humanity, or at least kind of, you know, rising tide, lifting boats, as opposed to negative outcomes.
What do you see as kind of the big things to be discussed and acted upon?
Yeah, I think we should always be thinking about the downstream impact of these models.
And I think it's very hard to think about regulating and trying to stop these models.
I feel like any such attempts Usually, because the system is so complex, usually puts more power in the hands of bad actors, right?
So kind of the saying, when something is outlawed, it's the outlaw that becomes the law.
Exactly. So we have to be- They're not going to stop.
Right, of how to operationalize the good intent.
And I do think Focusing on the end applications is the key because think of LLM being used for mental health diagnosis or counseling or any kind of health advice.
We should be much more mindful of what it says and where, you know, how it's being exposed to patients, if at all, right?
And how should the human be in the loop and all that. compared to using LLM for poetry, right?
So I think the use cases matter and I understand like the Questions of misinformation and others are indeed very hard to tackle because we can have these models generated at scale.
But again, right, I feel like bad actors with enough resources would always have access to do this if we try to limit open source and if we try to limit... the broader public from being able to access and do meaningful things with the model.
You know, to me, the AI revolution so far has been a lot of open source revolution, right?
And I do see the challenge with very large models, how much of it should be open source or not.
That's to be determined. But I feel like first we should think about also strengthening the existing laws for all kinds of downstream application, right?
And that'll really help us get started on what could be the most dangerous Use cases and how to limit the harmful effects based on that.
I also think, you know, going back to research and trustworthy AI and best practices is so important at NVIDIA we launched Model Cards Plus Plus, which is an add-on to the Model Cards that was proposed by Meg Mitchell and Timnit Gibru.
Could you actually just for the audience kind of explain, I was going to do it, but you'll do a much better job, what a model card is, because I think it's a fascinating and very simple as a lot of these solutions tend to be in their base.
Idea to help with all of this. Absolutely, right?
Model course is all about transparency that you want to say what was the training data that was used, right?
What is the intended use? So if this model is now used in a scenario where it's not intended, then that's already a red flag.
It's all this list of things and we added many quantifiable metrics.
How fair is this model? How private is the model if you care about privacy of like the bar? say, health records on which this is trained?
Can we give metrics like differential privacy?
So we added a number of such metrics. I mean, this is available online and some of the NGC models launched by NVIDIA have this, right?
So I think this is a great effort that is started, but we need to put this on steroids for these large language models for health.
And some of the research we are doing is also asking how do we automate the testing of various models?
Because Hugging Face, for instance, has a huge repository. source language models, right?
But how biased is one model compared to the other?
And that also depends on the use case. Maybe you have very specific requirements, say, for healthcare.
Where you may want some of that differentiation of different demographics, but you don't want bias.
So what is undesirable and what is not? And so we are now developing tools that really gets users who are not machine learning experts put in what social groups would they like to test the bias on?
What are the attributes and dimensions along which they want to do the testing?
And I think we really want to see more and more of such tools that promote transparency and explainability.
So we don't have a lot of time to dive into your background.
And it's fascinating merging of, merging is quite the right word, but the scientific and the academic and industry and government and NGO things and all of that.
But a question popped into my head earlier when you were joking about, we do need now and hopefully we'll continue to need people thinking about how to design the algorithms.
I read something earlier this week, which kind of harkens back to earlier this year when I, as a writer and content creator by trade, had my own sort of excitement and fear at the same time of what are these LLMs going to do?
And, you know, they get a Am I going to become super creator or out of work and how fast?
I was reading something recently where somebody was talking about giving advice to his own child.
And originally, whatever it was they said they wanted to be when they grew up, you know, the advice was we'll go to school, you know, learn all you can, get some experience, et cetera.
And then they started taking an interest in technology and computer science specifically. okay, well, go to school for computer science and learn the fundamentals and learn the languages that are in vogue at the time and you can get started that way.
And now with the explosion of LLMs, this person was saying they're now at a moment of not knowing what to tell their kid to do because Will there even be a need for software coders by the time the 14-year-old is ready to join the workforce?
So not to throw all this on your shoulders, but I'm going to anyway.
Given your career path And now kind of the point where you sit and having, you know, sort of this deep dive hands-on view as well as kind of the broader scope of everything we've been talking about.
What advice might you give to a young person who feels right now like they're interested in a career, in technology, in science sort of broadly, but they're also aware of these advancements that feel like they might actually be accelerating the rate of change in these domains.
Is the advice still to go study the scientific disciplines, the... bases of computer science and how to build software and all of those kinds of things.
Or has it actually shifted because of the developments with AI that you've seen and been a part of?
Yeah, I think AI will have a huge impact on education, right?
So I was at TED where The Khan Academy founder, Salman Khan, talked about Khan Mego and having... chatbots that are very personalized and really help people kind of wade through their any barriers in thinking and really learn.
And to me, I think learning will never stop.
And in fact, the best advice I can give is be a lifelong learner.
You know, that holds true for language models, that also holds true for humans.
You really need to keep updating, otherwise you're out of date, right?
And I think that's much more important in future than even now.
And to me, Computer science isn't about, oh, will it be Java or C or an AI-based software. model, right, that's coding, but it's really about algorithmic thinking.
So, you know, we still need to frame like what are the right specifications and what would we like AI to help with or even maybe do some of the tasks, but we still are the thinkers.
And just our notion of thinking will change.
I mean, for instance, recently we released a framework called Lean Dojo that... uses lean framework for theorem proving with language models.
You're kind of instructing this lean theorem prover to go and try different premises, try to prove a theorem and ultimately...
You're guaranteed that the theorem is always going to be right as opposed to language morals that may hallucinate.
So this ability, right, means in all the mathematicians just completely obsolete.
I don't think so. I think these... really help maybe prove very laborious theorems, right?
To kind of like make sure they're correct.
And maybe over time may even help aid how they work together, right?
The humans and the machine together. And I think we'll just go on to solving harder and harder problems this way.
And I think that will be true for every scientific domain. neural operators and other models we're building is really enhancing our understanding of different phenomena.
It's not just the intuitions of a domain scientist, it can help and aid and complete that. by saying, oh, let's just look at a broader set of design possibilities.
Maybe you never thought of it before, but...
I can just very quickly rule it out or optimize through it and come up with something better.
But we're still humans. Then what are we left to do?
What we are left to do is to think about what problems to solve.
This is also the research advice I give to everyone.
The most important thing is the question, not the answer.
Because the answer is always 42. Now I'm joking.
Exactly. Well said. question and take all the time to debate.
Are you asking the right question? Are you solving the right problems?
Don't just go by trends. I mean, the trends inform us something, but really ask what are the unique contributions that you can make?
And so that holds true for research, that holds true for education. focusing on learning and the pleasure of really understanding something, I think that'll never go away.
Fantastic. I'm going to carry with me what you said.
The LLM needs to keep learning and so do you.
I love it. Anima, this is fantastic. We could talk for hours, but you clearly are working on and singing about a million different things and have have other places to get to.
So we appreciate you taking the time to join us.
The PCAST full recording, as you mentioned, is available on YouTube. as is Jensen's Berlin Summit address, which we encourage everybody to go check out.
NVIDIA Research has a homepage at nvidia.com slash research.
Where else, if anywhere, would you direct folks who want to know more about the work that you and your various teams are doing?
Yeah, I mean, those are great resources.
You know, sometimes I'm on social media, my website at Caltech, TensorFlow Lab. you can find online.
But yeah, it's really a great place to be here today and thank you all for Oh, thank you.
And let's do it again maybe sooner than two and a half years if we can.
Look forward to that. Thank you.
