Hello and welcome to the NVIDIA AI podcast.
I'm your host, Noah Kravitz. Clean, cheap fusion energy would change everything for the better.
Our guest today, Bill Tang, has spent a career at the forefront of that field.
He's currently the principal research physicist at the Princeton Plasma Physics Laboratory, And he's also one of the world's foremost experts on how the science of fusion energy and high-performance computing intersect.
Now he sees new tools, deep learning and artificial intelligence in particular, being put to work to enable big data-driven discovery and key scientific endeavors. such as the quest to deliver fusion energy.
Bill, thank you so much for joining the podcast.
My pleasure. So I have to admit, when you were booked as a guest and I got the nose and I started looking into your background...
I sent my mom a text and as part of the text, I said, I'm not entirely sure what plasma physics is.
My mom sent me a Wikipedia link back, which I thought was perfect and kudos to my mom.
But maybe we could start out with you kind of explaining, you know, For the smart layperson, what plasma physics is?
That's an excellent question to start. My home department at the university... for which I'm a faculty member, is the Department of Astrophysical Sciences.
And you wonder what's plasma physics got to do with that?
Well, if you take a solid object and you heat it, it becomes a liquid object, right?
If you heat it some more, it becomes a collection of molecules and such.
If you heat that some more, it dissociates into a collection of charged particles, positively charged ions, negatively charged electrons, And this kind of a gas of charged particles is a plasma.
And it's sometimes called the fourth state of matter, you know, solid, liquid, gas, and then plasma.
Right. And it comprises over 99% of the visible universe.
And that's why it sits in the Department of Astrophysical Sciences as discipline.
Right. And we work with a hot thermonuclear plasma.
Because the greatest source of clean energy that's possibly... if you will, is fusion energy.
And no less an authority than the great Stephen Hawking, in a 2016 BBC podcast or broadcast.
When asked about what do you see for the future, he said the hope for mankind is fusion energy.
It's a limitless supply of clean energy.
It just needs to be harnessed, but people should focus in this area.
It's almost unfair to ask you this question in the context of our short little podcast on AI, but before we get to AI...
How does fusion energy work? Okay, fusion energy is a form of nuclear energy, similar in a basic way to fission energy, which you know for peaceful purposes, produces energy in nuclear reactors.
And fission energy is more desirable because it doesn't have a lot of the issues with nuclear energy, which you already know about, you know.
When you have an accident like Fukushima, they're still cleaning up after it.
Japan, as a country, is totally traumatized about investing in fission or nuclear energy as they know it.
And so they're turning more towards investing more aggressively in fusion also.
I won't get into details of that, but fusion, on the other hand, is really very different from vision, which is the the kind of reaction that happens in nuclear reactors.
There you take heavy bodies, like the isotopes of uranium, And these are rare elements, so it's harder to get these things.
And as the name may imply... Vision means that you split like an atomic pile of this very, you know, very, very highly energetic energy uranium product, and it dissociates.
And it also follows Einstein's rule about the constituent products that come out of the reaction has less mass than when you started out with.
When people try to total things up, they said, where the hell did the mask go?
You know, E equals MC squared nuclear power is released at a very expansive rate.
Mm-hmm. And so that's how that works. But fusion works the same way.
Only here you take... the elements uh the isotopes of hydrogen for the most part which is like you know i hydrogen deuterium and tritium and uh... there you create a medium very hot gas but a very low density so that it cannot cause any accidents.
You know, if you're a terrorist and you fly an airplane into a fission reactor, all hell breaks loose.
But if you fly it into a fusion reactor, the reaction just ends.
Nothing happens. Okay. It's also harder to do.
But in that kind of a reaction, the byproducts, if you will, what you start out with, with the reaction, what you end up with, if you see how much these constituent products weigh, they have less mass.
And again, the mass disappeared... according to Einstein, E equals MC squared, right?
Right. And there's a huge multiplication of energy, like a ratio of 450 to one.
Wow. And so the dream of harnessing fusion energy is a real one, and it's being pursued now because there has been progress.
When I was in grad school and such, you know, they used to joke saying, yeah, yeah, it's the ideal energy source of the future and it always will be.
Right. But then, as we've seen from my early days at Princeton and such, where we had the first deuterium-tritium experiment there in this toroidal fusion test reactor.
And... The magic ratio is how much energy do you get out compared to how much you put in.
And so at Princeton, we achieved a ratio of 20%, which was a huge deal compared to everything that happened before.
We were all over the front page of the New York Times and so on.
And those days, fusion was more favorably looked at by the U.S. government and such.
It was very enthusiastically embraced. Times change, political situations change.
Then over time, what's happened is that this dream in the international community did not slow down much.
And the Europeans, the Eurofusion community, Euratom and such, put together a larger, more aggressive project. in the UK.
And they call this a joint European Taurus.
And that's still the high water mark now.
Because in that case, Scott, you know, they were able to hit about break even, which is energy out almost equals energy in.
So on the basis of that, they said, okay, it seems like the scientific foundations now will strongly motivate going forward to this big $25 billion project in the south of France. that involves the investments of uh... governments representing over half the world's population right and the goal there is to reach a factor of 10 to 20 above break-even.
And you might say, well, so what? What does that mean in terms of power on the grid and such?
Uh-huh. And the best way I can answer that is that would be analogous to in aviation about where the Wright brothers were at Kitty Hawk. the damn plane will fly.
But if you stuck this microphone in front of the Wright brothers, they wouldn't have begun to tell you Is this now competitive with trains and boats and things as a form of transportation?
So similarly, you know, we haven't even reached the plain yet. flying.
So I can only speculate that this will have a huge impact into the future.
So how does big data and tools like AI and deep learning, how does it help advance the cause?
That's a really excellent question. Because, you know, we've made a lot of progress in understanding through more conventional what you call hypothesis-based first principles physics using exascale class supercomputers and so forth, right?
But there's one very elusive problem That involves validated huge amounts of data that are accumulated in these sites. fusion plasmids, you know, like the JET project that has over a half petabyte and still growing of a database.
Right. And so that's truly big data. And moreover, there are More advanced experiments than JET, just not as powerful using superconducting materials to extend the pulse of the discharge to make it last longer.
So all of this technology means that the data is more and more plentiful.
And then the challenge for any hypothesis-based theoretical model is to what extent can you predict things that can be validated against the observations, which means measuring things, right?
And that is the coin of the realm in any area of science.
And I like to tell this to my students at Princeton too.
Don't think that experimental verification validation is, oh, well, it's a mundane exercise.
It's not. Because if you just think back, not that long, you know, when at CERN they found the Higgs boson, which they called the God particle.
Right. Well, Peter Higgs discovered that theoretically 50 years ago.
He's an old man right now, and he could care less about the hoopla.
But it was a huge deal because that validated that theory.
And then even more recently... the discovery of the gravitational waves was a huge, huge breakthrough because it provided information experimental observational validation to Einstein's theory of general relativity.
And so three Nobel prizes popped out immediately.
And there's a big hoopla. Why is that? That's experimental validation.
So this you cannot run away from, no matter how you know, head in the clouds, theoretical you want to be, you have to face mother nature.
And so getting back to your question, about how does big data come into play.
I started to state that because the database for the fusion-relevant results that you want to be able to predict better comes out of this very large database.
Moreover, arguably the most pressing problem that faces fusion, and that's why we're very excited about deploying deep learning AI because you can't do it with just first principles based.
Even if you have an exascale supercomputer, you can't solve the problem.
You have to have a combination of first principles knowledge together with this big hammer that is AI, you know, to help you solve the problem.
And that's why we're so excited about this right now, because this is not, you know, in my mind, I've been around for a bit now and I look at this and This is not incremental evolutionary progress.
This has the potential to be revolutionary progress. because arguably the biggest problem you have to solve to deliver magnetic fusion that's using a big magnetic bottle or trap to keep the hot plasma confined.
You can do it with proper design of electromagnetic fields because remember we said the hot plasma was made of charged particles.
Right. And so you don't have to contain it with a material.
Okay. So you can contain it. with the electromagnetic field.
Exactly. But, you know, Mother Nature is always going to be very challenging for you, right?
Because Mother Nature tells you from the fundamental laws of thermodynamics that if you want to try to keep a very hot gas confined for a long period of time...
And you have natural sorts of characteristics of that hot plasma, like gradients and the pressure and things like that. that tells you you're not in the lowest energy state because the lowest energy state means everything is flat.
You know, everything is diffuse. And so you have these big sources of what we thermodynamically call free energy. and it finds its way to escape sometimes very rapidly out of the trap and doing damage also if you have a lot of thermal content in the plasma.
Right. And you can damage this very expensive machine or it can escape more slowly.
And there's other processes I'm not going to get into.
But you better solve this, what we call a large-scale disruption problem first, because Some of my colleagues may not like this emphasis because we've worked on pure first principles related approaches before.
But if you don't solve this disruption problem, which means predicting accurately and quickly and then doing something about that by combining that with methodologies for controlling how the plasma behaves.
You go into the system and figure out how you run the machine.
I'll give you an analogy. You can make a great prediction for your Ferrari running across the field.
It's going to hit this wall. And so your good predictions are going to tell you, okay, you've got 10 minutes, 20 minutes, a half hour, whatever. but ain't gonna do you a lot of good if you don't have control, right?
And so these are the two things that combine, and this is what AI, and deep learning methods are gonna start to bring you.
And why am I so confident about that? Because the joint European tourist people they had the most powerful machine in the world.
And so they cared about their continuing experiments leading up to ITER.
Now, they don't want their machine damaged and such.
And so they put a lot of effort into using what they call shallow learning methods for dealing with their big database and trying to figure out how do you predict the onset of these disruptive events.
And I'll try to make it as accessible as possible.
Sure. So, you know, why is Google, Alibaba, all of these big companies, Amazon, Why are they so tremendously enraptured by big data analytics, machine learning and such? because they always want to predict some kind of a simple outcome, a binary outcome.
Will the customer buy or not buy the product, right? but then they have this huge database you know whatever part of the human population you want to consider And then they have the pre-binary event classifiers, age, demographic, sex, previous vibing patterns, and so forth, right?
And so, in analogy with that, the fusion problem has, you have this big database of measurable disruption-relevant quantities.
And then you have the observables that, well, you observe these pre-disruptive events, physics events.
And that's your workflow to the binary decision.
If you cannot resolve some of the physical features of the data.
And the data is inherently multi-dimensional.
But all they could do, because it's a big problem, is just deal with scalar representations of the data.
What does that mean? that means that if you're measuring three-dimensional information about the density, temperature, and so forth, You can only work with 0D time traces.
That means you only take the magnitude of those quantities that are measured.
You do a volumetric average or something. and you follow the evolution of it in time.
And they made good progress with that. In fact, they were able to move well beyond anything that the first principles codes could do.
But then, you know, they said, but we need to do better.
And if you're going to make predictions that will make a big impact on this $25 billion international facility, You got to do better.
This is where the deep learning that emerged about three years ago before it was, you know, a lot of it was, uh, Also the basic educational material and all that, that was all on GitHub and publications like that.
No standard texts. And now you're starting to see some really good standard textbooks come out.
But the big thing in the injection of... our software that we're able to do is we started out with a shallow learning thing.
So we had a guide, you know, a pedagogical guide. here's the data jet used, here are the classifiers, here are the answers they got.
Now let's use the Keras cookbook, you know, which is like the deep learning cookbook.
And you engage some very bright young people to work with us on this, to which this is not... alien stuff and then say your homework problem is to try to you know adapt this new technology to the same logical problem and uh And this young team is really, really amazing, the work that they did in a little over a year's time.
And so they were able to bring into play these stacked neural nets, you know, Three layers of stacked neural nets, including the very key capability of convolutional neural nets.
This means that you can resolve... three-dimensional images, you know, your face, a rabbit running across the field, so on and so forth.
So if you think about that, it's a Well, gee, you know, where we've struggled like hell, you know, working with the three-dimensional features in a physical system, we can certainly adapt this material in time. to giving you higher fidelity representations of the physics.
And it was amazing how fast this all moved. and uh... the were able to do that now and so we're starting to see unprecedented very very faster predictions and more realistic predictions.
And so we're off and running. And do you feel like the pace you're running at is... still increasing and going to keep increasing in the near future?
Yeah, I do. That's why I think it's so exciting.
And the other part of it that I'm so excited about is that, um, we're attracting some of the best and brightest young people at all different levels.
You know, that's the part that's really stimulating Noah.
Cause, uh, There are people that come over from other departments at Princeton University, which is a top-rated university.
And they come over and I don't even have to advertise the problem.
Right. They somehow find me. And then they say, well, we want to do... In Princeton, you have to do a...
Junior year thesis as an undergraduate, and then also senior level thesis, which, you know, it's... not a simple thing.
And so they say, we want to do it in machine learning. applying it to a practical world impactful sort of problem.
So we'd like to do our projects in this area.
So I have two students from The physics department, which is, you know, all due respect to my home department, it's a higher rated department. whole raft of Nobel laureates and such in there.
And these are young theoretical physicists that are really good.
One is in her junior year, one is in her... his senior year, and they're both doing really good work.
And it's just really stimulating. Really stimulating, though, to see these bright young people just engage and go forward.
And there's also a good level of international interest.
And so... People from abroad, they're saying the bureaucracy in terms of receiving money from other people, they can give you compute time.
That's not a difficult thing. But to send you money to support personnel is not easy.
And so they asked me, well, what else can we do?
And I said, if you want to train some good young people, we'll get them in gear and so that they'll be productive sooner than later.
So everybody benefits from that. And the other thing that I say to everybody is, You know, it's crazy if you don't leverage the investments in this area.
You know, all the big companies are pouring tons of money into this area.
And they're doing it for different purposes.
Their data is not necessarily shared. We're in an application domain where the data can be and will be shared.
Moreover, there's cross-disciplinary, cross-cutting opportunities.
You mentioned cancer research before. A colleague of mine... who's a very influential person at Argonne National Laboratory, Rick Stevens.
He has a big project in cancer research.
But not to get into too many of the details, but if you look at some of the workflows associated with it, you know, including some of these hyperparameter tuning sort of workflows, which is a little hard to explain quickly.
You see some real good similarities. And so the opportunity for cross-cutting collaborations is also high, you know, that you don't have to reinvent the wheel in your separate silos.
So all of these things contribute, I think, to a really exciting environment right now.
I can see it in your face as you're talking.
The listeners can't see it. They can hear it coming through, though.
I can only imagine. So what's next? Where do you see the technologies accelerating?
And it sounds like there's progress being made on fusion energy.
There are... roadblocks, so to say, maybe perhaps more.
Well, I say more, but there are human roadblocks, there are technological roadblocks, I'm sure.
Next three to five years, beyond working with the people and working with the students, what's got you excited about moving forward?
Well, it's the fact that the technology is moving forward so fast and that's part of the reason that I'm here.
Sure. Okay, and the products here, the GPU products, are truly exciting.
Right now, there's no match for it. The Pascal P100s already were more powerful than any other processors out there.
Is that what you, what are you using in your work day to day?
You know, we feel very fortunate. Uh-huh. because as people are more aware of what we're doing and the fact that once we got into the deep learning software, we found that the GPUs were just a perfect match for these dense matrix operations.
And not only did we get an immediate speed up with just our local Princeton University cluster of K20Xs, the scaling was just perfect.
It was already faster, one GPU to one CPU, But then the GPU scaled right up.
And when I showed those results to the people at Oak Ridge and to NVIDIA themselves, I was pretty amazed.
The access was really good. You want to try something bigger?
You want to try something faster? And we passed the mark on that.
And the young team of collaborators that I have are just amazing because they just... dove into it and they adapted to the more powerful processors and the larger the system, the better.
You can imagine that could be really stimulating.
Yeah. We can get on these supercomputing systems not just in the U.S., but also in Japan.
Satoshi Matsuoka, who's like the present king of kings in AI in Japan right now.
He gave us access to their biggest supercomputer that's running now, the Tsubami.
And Tsubame has well over a thousand Pascal P100 GPUs, which is more than anything in the US right now. that soon to be superseded by Oak Ridge's next big machine, Summit. which features the next events, which are the VoltaBoxes.
And so it's just very, very exciting, you know, because you can see the speed of the predictions improving And the fact that you can do things so much faster, that's the other big part.
You can try different creative ideas. You can say...
Well, you left out this kind of possible pre-disruption classifier because you couldn't do it before because you couldn't. put profiles in there.
Now you can, okay? Try it out. Does that make things better?
Well, you have these things that they call ROC curves.
These are in any deep learning application.
It's a plot. On the vertical scale, you have true positives.
And on the horizontal scale, you have false positives.
And you can't fool that, you know, you can be the best politician and the most eloquent guy.
And if it isn't working, it's going to tell you this isn't working.
So I love that, the fact that you don't have to debate the politics very much.
Right, right. And the young people love that too.
You can imagine, right? Yeah, that's beautiful.
That's great. Bill Tang, if people want to find out more about the work you and your young team of colleagues are doing at Princeton and elsewhere, perhaps, is there somewhere online that's a a good place to start.
Yeah. The first of all, you can Google my own name and there's plenty of references to that.
And NVIDIA was very generous in giving me their 2018 Global Impact Award.
So there's an associated, you know, hour-long presentation.
I was coached to try to not make too dense.
I had to rein things back a little bit because I was just last week in Japan giving a much more detailed technical talk about AI.
But this one tries to – it starts out with what I was mentioning earlier.
You start out by saying – Why is this important?
Because it's a marriage of the grand challenge of delivering clean energy. together with this explosion of the technological capability of AI.
Right. Okay. And then you motivate that and then you quote Stephen Hawking, you know, his famous quote and all this.
And that gets people into thinking, OK, well, then what's the big problem?
And then we walk them through. Well, you better solve the disruption problem and tie that to control.
I think, you know, not to over advertise the talk itself, but that'll be generally available. and the references cited therein, so people could look at that, and hopefully that isn't too much of a heavy sledding for anybody to get through.
It's worth putting a little time into it.
On their end. Thank you so much for your time.
Congratulations again on the award. Well-deserved.
And it's really been a pleasure speaking with you, listening to you and getting...
A little bit of a glimpse and just the energy that comes off of you when you talk about your work.
It's fantastic. Thank you so much. I really appreciate your time.
And it's a pleasure talking to you. You're a great interviewer.
I appreciate that you make it easy. Thank you.
Thank you. Thank you.
