For your first trip in a self-driving car or your 99th trip for that matter, you won't find a better person to ride shotgun than Neda Zviadich.
She can give you a turn-by-turn commentary on what an autonomous vehicle is thinking.
Svjetich heads up product management and product marketing for NVIDIA's self-driving car software, and she has a passion for helping people understand what AI is doing under the hood.
In 2019, she launched and hosted NVIDIA's Drive Labs video and blog series explaining how self-driving cars work.
To date, more than half a million people around the world have watched these videos on YouTube and Nvidia's website.
She's also passionate about helping pave a broader road for women in technology.
And we'll talk about both these areas. So, welcome to the AEI podcast.
My name is Rick Merritt. I'm a staff writer for NVIDIA, and welcome, Nita.
Thank you, Rick. It's great to be here. It's great to have you here.
So your most popular video to date You took an autonomous vehicle around the Bay Area.
Explain for people who haven't seen that, and basically anybody who has a driver's license should go find this video and go watch it.
But for those who haven't seen it, what was it like?
So this video is like going on an autonomous drive with us in NVIDIA's self-driving car.
We share with the viewer exactly what the car is doing and how it's doing it in real time as the car is driving itself.
What are the AI modules? How do they work together?
How do they work with different systems in the car?
It's basically real-time technology play-by-plays that are meant to bring to life in a very clear and understandable way exactly what is happening as the autonomous vehicle is driving itself.
So why do you do this? So I want every driver out there to feel that they understand AI, to understand how AI works in self-driving cars. and feel empowered by that understanding.
Not to feel intimidated, not to feel alienated by it, It's not some sort of inaccessible mystical thing that we're not meant to understand.
No. It is an engineering solution among other engineering solutions.
It's state of the art in many ways. And for that reason, we really treat our viewer as a colleague. in these videos.
We focus on clarity of description, but we still use terminology that is used in state of the art AI research.
We want our viewer to know those terms, to feel comfortable with those terms.
And so we explain every aspect. from what type of AI model the car is using to do different functions to what the engineering visualization means. what all the different colors are, what all the different boxes are, what are the deep neural network outputs.
And I think this is important because, frankly, AI and technology are becoming something that's prominent even in things like politics. and election campaigns.
You have candidates now that involve technology and AI in their election platforms.
So being versed in that I think is important, just in general.
Okay, so what's it like for you to make these videos?
One of my secret dreams has always been to be a sports journalist, like an ESPN sportscaster, yes.
And doing this drive was kind of exactly like that.
It was like a play by play, but an autonomous driving maneuver.
So instead of that backhand down the line, I'm all right, that lane change is coming up, you know, getting close to the 280 exit and you have to do it in real time.
You have to use the exact industry terminology and you have to make it so that the audience absorbs it as it's happening.
For me, it's like I got to live out my dream of combining sports journalism and electrical engineering together.
The Formula One moderator. Yeah. Sort of.
Actually, pretty much, yes. Autonomous Formula One, in-car rally, supercomputing technology, casting.
And the French Open. Yes. And it strikes me that, you know, the self-driving car is kind of like a Ouija board to the average driver these days.
You're both kind of drawn to it. and kind of afraid of it.
I think so too. I think so too. And I think that what we want to do is demystify that.
AI is an engineering solution. It's a mathematical model that is meant to take in raw sensor data.
It is modeled after the operation of our human brain. after the connections between neurons and synapses in our own brain.
And that's why AI is so closely related to neural networks and deep neural networks.
AI is unique in the sense that it learns on its own in some way by being trained on lots and lots of data.
So the function it does Other solutions and other models can do that too.
The main difference here is that it's able to learn in some sense on its own by seeing a lot of examples, by seeing a lot of data.
And that's where the artificial intelligence term really comes from.
Yeah. And I think if you like a doctor talking through an operation just calmly.
Yeah. using the proper terms to say this is what's happening now and this is what's happening. and making sense of it.
Okay, so I opened the hood on my self-driving car, my autonomous vehicle, I opened the hood What's in there?
How many different neural networks are running around and who's coordinating them all?
There are three key pillars inside the brain of a self-driving car.
The first one is perception. So this is basically the car taking in data from its senses or sensors, if you will, to a car.
We have senses, a car has sensors. For example, pixels from a camera image and converting that raw data into semantic information, so detecting pedestrians, cars, detecting lane lines, detecting traffic lights, intersections and so on.
And especially in this area at NVIDIA, we use a lot of AI in the form of these deep neural networks that you mentioned.
Somebody told me there's like 20 neural networks in the car.
Is that right? Yeah, we have dozens of deep neural networks and they are trained on thousands of hours. of real-world driving data and they give a lot of rich perception information to these other two pillars in the car.
And so the second one is called localization and it's closely related cousin mapping.
To localize a car means to place it on a map.
So localization is always with respect to a map.
And the way that a car can build a map is by using that perception information we talked about.
Where are traffic lights? Where are lane lines?
Let's say that you drive the same commute every day. it sees these landmarks in the same place every day.
And so it can build up its own map over time and use that information to drive.
And the third pillar... once you have all this information, whether it's from perception, AI perception, or localization and mapping, is planning and control.
So planning takes in this information from perception, from localization and mapping, and plans out the actual execution of an autonomous driving maneuver.
So for example, how does the car stay centered in its own lane? planning takes care of that.
It literally plans out a center trajectory.
How to do a lane change. It literally plans out a trajectory for how to get from the center of your current lane into the center of your target lane, how to take a left turn in an intersection.
It plans all that out. And once the plan is in place, then the control layer is is what issues the physical acceleration, deceleration, and steering commands to execute the maneuver.
So it all sounds very analytical and scientific.
And we know that... So many crashes that happen every day on U.S. highways, but human error is the cause of them, right?
So it should be. safer, better if we're using autonomous technology to do this for us, but there's There's some concerns that, you know, there's so many corner cases.
You're driving down the road and, oh, my God, there's a mattress in the middle of the highway or some jaywalker runs out in front of the road.
Are cars smart enough to understand these corner cases?
How do they deal with these corner cases?
As a human, it's really interesting. We actually, whether we realize it or not, we have a lifetime of data collection experience, right?
Even before you were driving, you were watching other people drive.
You were looking at what a car looks like from every different angle in so many different weather conditions.
You were exercising your perception deep neural networks for years. before you got behind the wheel.
We have so much of this data that allows us to anticipate corner cases.
Based on this analogy and based on the analogy of neural network software following the operation of the human brain, I think a lot of this really is a data problem.
We talked about how AI fundamentally learns from data.
And so our challenge here is producing a high quality, large data set that emulates these types of corner cases in the real world in a way that they actually occur in the real world.
In terms of statistical distribution, in terms of weather conditions.
And what makes this really challenging to actually do is that there are so many permutations that you can have, like you mentioned.
And so we have to both gather a large data set and we have to find the right data in that data set.
And so this is why AI is really closely related to a lot of data infrastructure questions as well.
In AI, the data writes the software ultimately.
And so garbage in, garbage out. So there's a matter of time there and just accumulating that data and accumulating that experience.
The computer is getting that experience.
Absolutely, and it's a fascinating question because there's the aspect of gathering data in the real world.
There's also the aspect of using powerful simulation tools as well.
So we have the NVIDIA Drive Simulator that seeks to create some of these corner cases that might take years to actually observe, that long tail of events.
And training the car on these before it even gets on the actual physical real world road.
Computers teaching computers. Yes, computers, teaching computers.
That's what's great about being in a high-performance computing company when you're doing self-driving cars.
Where do you think we're at in this journey to educating the average driver about what's really happening here and that it's safe?
One of the most exciting things for me when we started this NVIDIA Drive Lab series, oh, and that video that we talked about, by the way, is literally called Ride an NVIDIA Self-Driving Car. aptly named.
So check that out. You know, not a lot of videos had commentary.
You see the car doing stuff. but you don't necessarily understand how it's doing it.
And so what was super exciting to me is being able to take them a step further, take them inside the mind of the computer, because I think it's incredibly important as a consumer to really understand what the car is doing, to feel comfortable in it.
You've got like a new job here of the AV interpreter.
The AI whisperer. Yeah. But I love that.
I love that. So I got to confess, I'm a real Luddite here and I went on vacation this summer and I rented a car that's newer than mine.
And I was out on the highway and I was on cruise control and I got behind somebody and it started to slow down and I freaked out at first.
And then... like the third time it happened, I said, wait a minute.
This is like the coolest feature ever, dynamic cruise control.
Okay, so probably everybody in the world has figured this out but me, but this summer I thought this was just super cool.
While we think about and read about and talk about a lot this getting to the self-driving car, it struck me that there's probably 57 different cool things on the way to that that might even... be more interesting for the average Luddite like me.
I completely agree in the sense that We get a lot of questions on the NVIDIA Drive Labs also, what if there's rain, or what if there's snow, or a tornado, or what if there's a zombie apocalypse, or all of these things together.
Especially the zombie apocalypse. Especially.
But if that happens, you've got other problems.
You know, I think you've got other things you need to really take care of first.
From personal experience, I really can say that even autonomous driving in low speed stop and go traffic is pretty excellent.
It takes off so much of the stress and you absolutely, I absolutely loved it.
Or on a long highway trip, let's say you have long stretches of California highway.
I mean, they've got lane markings. The weather is good.
It is awesome to have the car stay in lane for a long time, be able to take care of lane changes for you.
I think we ought to be offering free drive-alongs with you to help educate more people about it.
That's what we're trying to do through the magic of the internet.
So you hear that it's safer to fly somewhere in an airplane than to drive in the car.
And we all kind of relax when you get in a plane.
But really, the reality is, isn't it that most planes are highly automated with pilots doing very little?
They're on automatic pilot or whatever. And we don't worry about the plane, but we're still scared of the car.
So what's with that? It's interesting. The aviation industry, I think particularly in terms of regulation, is a lot more mature.
I think the first commercial flight in the US was something like 1914.
And since then, we've learned a lot. The industry has learned a lot and put in place best practices and safety processes.
And actually here at NVIDIA on our self-driving car team, we borrow a lot. from these pre-flight checklists and safety processes to develop our own safety processes for every time before we get in the car, what we do.
I think that's one thing. It's more mature in some important ways.
There's also fewer corner cases, right? Planes fly above the weather.
That's a big deal. And then I think the third thing is that we have the comfort of knowing that there is centralized air traffic control.
There is somebody who knows at all times where the planes are and who's coordinating these planes.
And actually, that last one part is under discussion by folks in the industry, especially for these level five fully self-driving use cases.
What would that look like? So that would look like, for example, let's say that a car is in a situation where it doesn't know what to do and it's connected to the internet.
This is the idea of the connected car. and he calls a human operator, and the human operator, through the sensors on the car cameras, for example, can see what the situation is and use their good judgment to help the car figure out what to do.
Kind of the nexus of FAA and IT support.
Yeah, basically. Cool. Okay, so... We talked about helping average drivers understand that self-driving cars are safe, but how do you teach cars themselves to be safe?
From the engineering perspective on that, it's very important to design diversity and redundancy into every system and subsystem of a self-driving car.
So for example, that is why you have multiple cameras on the car with overlapping fields of view.
So if one can't see for whatever reason, the other one sees the same thing.
And then on top of that, you have radars collecting information on the same kinds of things.
Algorithmically too, we have different DNNs that do the same thing in different ways.
So for example, our first NVIDIA DriveLabs episode on Path Perception Ensemble In that one, we have three DNNs trained in three completely different ways to perceive the drivable path ahead.
Traceability is also really important, especially we talked about the importance of data for AI. data quality, data integrity, making sure that the data is labeled in a way that's accurate, that is checked and traceable.
Understanding how changes in the DNN model affect other modules and submodules, having traceability in that way.
And the third pillar I would say is validation.
So being able to validate safety of the intended function, being able to validate literally everything along the way, that is another reason why simulation is so helpful because it helps you validate these algorithms before you get on the road.
Okay, so I recognize that software is your area of expertise, but can I ask you a hardware question?
Have people figured out yet what combination of cameras and LIDAR and radar or whatever else is needed as inputs?
What we know for sure is that we need to have 360 degrees around perception.
And in this way, cameras are superior to a human because we literally can't see from the back of our head.
And the car can, if you put cameras in the right place, if you put radars all around the car as well, it can do that.
In our case at NVIDIA, we have both surround camera perception and surround radar perception and the ability to fuse these two diverse and redundant sensory modalities into an even more robust solution.
Yeah. So do the cameras ever argue with the radar and they have...
I didn't see that. Yes, you did. Yes, absolutely.
So, for example, we talked about weather conditions.
There are situations like rain or fog where...
One sensor is having trouble, but the other one is not, simply because of the laws of physics.
They don't affect that sensor in a certain way.
Definitely that agreement and disagreement analysis between multiple modalities is how we are able to compute confidence metrics on outputs. which is really, really important.
Yeah. Yeah. LiDAR sees everything. LiDAR is a cool sensor.
It's a little expensive. But very cool.
3D. It's season 3D. Yes. Neda, you have a fascinating story, a personal story about how Nikola Tesla led you to the smart car.
Tell us your story. Yes. So I grew up in Belgrade, Serbia, and I grew up across the street from a very large statue. of Nikola Tesla, who was a Serbian American electrical engineer.
And he invented the AC induction motor and polyphase AC power distribution.
That's a mouthful. I know, and also, as if that is not enough, right?
Wireless radio transmission. And so since I was a kid, he was a superhero to me.
Because it was a very concrete and tactile example of electrical engineering and technology literally changing the world for the better.
Now, I am also a Serbian-American electrical engineer.
I guess I really followed through on this childhood inspiration.
And there was a part of the story that I don't know if he, Nicola actually led you there, but there was just a irony about part of your job search. leading to smart cars.
So one thing that was really painful to me because I knew about Nikola Tesla so early on is that For years and years in engineering school and engineering graduate school, nobody talked about him.
He was like some sort of P.S. By the way, here's Tesla in so many of our textbooks.
And then when Tesla Motors started evangelizing this name out there, I'm like, this is pretty cool.
So getting a job there was how I came out to Silicon Valley.
And it meant a lot to actually have that name now be in the world in this cool way.
So let's talk about non-technical stuff.
Before you got into self-driving cars, you've been in a lot of things.
You've been a professor at Columbia and you've been a researcher in optical networking at NEC.
You've been in a lot of different areas of engineering.
Does this new area of AVs and AI seem any more welcoming or open or diverse than the other fields of technology?
I am very, very proud of the fact that NVIDIA is a proactive leader in welcoming women into the AI field.
Our GPU technology conference has actually been named by Forbes as a top conference for women to attend. to further their careers in AI, which I think is excellent.
The number of women attending and speaking at the conference has doubled in recent years, which is also great.
And we also have women in AI initiative that includes early career accelerators and invitation only programs designed for those who want to get started in AI.
We have our Deep Learning Institute Fundamentals of Deep Learning and Computer Vision Workshop as part of that as well.
And so that kind of proactive approach is something that, again, I feel really, really proud of.
And I think it's setting just the right tone for women in the AI space.
How would you characterize where you think our industry is overall and being welcoming or what do they need to do to be more inclusive?
I think it's an interesting mix of big initiatives, like the ones we talked about, like the women in AI and the conferences as well. but also sometimes it's really the daily or grassroots kinds of things.
Something as simple as providing a team t-shirt in both men's and women's cuts instead of just men's cuts because there are just a few women on the team or a leader in the company, whether it's an engineering leader or in a different area. wearing the T-shirt of the company's women employee resource group in front of his peers, in front of his team, I've been almost shocked at how much of a real concrete welcoming impact something as simple as this can have, actually. literally, visually appropriates the space in a way that's empowering using something as simple as a t-shirt.
Yeah. So what kind of advice would you give to young women today thinking about AI or technology as a career?
The first thing I would say is that in the U.S., women start businesses at twice the rate that men do.
Interestingly enough, yes. And they're coming up with brand new products and services in this process.
And having more women do that in this incredibly creative AI space, I think would be so powerful and compelling, in my opinion, both from the financial opportunity perspective. and frankly, a positive social change perspective.
And the advice that I would give is to build and nurture a close inner circle of female friends, colleagues, advisors that you really trust and that you really feel comfortable talking to about all kinds of things from how to handle email to, uh, company culture to openly discussing compensation with them.
There are studies now, both from the National Academy of Sciences and Harvard Business Review, I think, that show that women with this tight knit circle of friends, female friends, female contacts are more likely to be successful.
And that has been my personal experience as well.
It feels like there's more sort of female rock star engineers bubbling up in the AI world.
And I remember like I used to cover semiconductors and it's just. really dense male-dominated industry by contrast.
It feels like things are changing, but then maybe that's just my perspective.
When I think back to my school days and my early career days and compare it to how things are now, I think that we as a society are definitely learning to prioritize being more welcoming to women and minorities than we used to.
There are studies now that that literally show that diverse teams make for better business.
That was not really around before. It wasn't really. a topic of conversation.
And then I think also technology and social media and millennial culture have brought on this beautiful transparency and real-time fact checking that I absolutely love in this context.
I mean, I think it really celebrates what is right out there, and then it helps highlight and oftentimes correct might not be ideal yeah well thank you so much for doing this work not only for diversity but for getting the self-driving car on the road so It's great talking with you.
My pleasure. Thank you so much, Rick. Thank you.
