Welcome to NVIDIA's AI podcast. We are here at the mothership of NVIDIA with this summer's, and it's the coolest name, Jetson Interns.
Jetson interns, introduce yourselves. I'm Mokshith Vudarla.
I'm from Cupertino. I'm Mark Theus. I'm from Los Altos.
I'm Isaac Wilcove. I'm from Los Altos. Representing Silicon Valley.
Like to see it. Let's start with the name.
Why Jetson interns? I know it's not because of an old cartoon.
Well, I mean, the Jetson and Vitia's Jetson is named after the old cartoon, the Jetsons, but we work with the Jetsons.
The Jetson boards. Yeah. And these are the brains that allow you to do what?
A lot of different kinds of deep learning and neural networks, which...
Help robots become smarter. Now, you guys are all in high school.
Is that right, Moksha? What grade are you in?
I'm going to be in 11th grade. And Mark?
I'm going to be in 12th. Isaac? I'm going to be in 10th grade.
10th grade. And you guys all have a background in robotics.
I know that. And I think are all of you part of robotics teams?
Is that correct? So the summer's project has been what?
What has been the goal of the summer? And how did you even end up here?
Mark, let's start with you. Like with many other interns, We were approached actually by NVIDIA at a robotics competition during the main season in the spring. since many of the students that come to these robotics competitions, they already love robots.
NVIDIA asked kids who you know would potentially be interested in not only using their background in robots but to apply artificial intelligence into what they already know so that hopefully this coming year, the interns who were using artificial intelligence can then bring what they have learned into FIRST this coming year.
You three all are into robots. You're on robotics teams, so clearly you like it.
How did you get into it? And Isaac, you're in 10th grade.
Let's start with you. Where did your knowledge come from and even just your interest?
I started programming, first of all, probably way back when I was eight-ish.
And I just loved it and I kept learning languages and different stuff.
And then when I got to high school, I learned that they had a robotics team and that kind of aligns because they have a programming team. on the robotics team.
And so I joined the robotics team and I mean, it was really fun.
Can I just ask when you were eight, what were you programming? really simple things eight ish i know i don't want to i don't want to exaggerate like eight ish python and scratch and like and you were writing programs to do what small games like, you know, dice rolling games and like the type of stuff that entry level programmers would find interesting and Or entry-level gamblers.
Kidding. Moksha, how did you get into it?
Because both my parents... work for technology companies.
I was kind of exposed to a bunch of different technologies that like all these companies were developing and just, I guess, robots in general.
And I just Every time I program a robot or build something myself, I just find it really, really cool. cool after it starts working or after it starts moving on its own or doing something like that.
So I mean, just that kind of fascination and interest.
Do you have an intention? Do you build these robots since you were younger?
What's the dream? What do you want them to do?
Well, I mean, when I was small, it would probably be just like, like I think I built this like Lego alligator thing, like with mind storms.
And I mean, It's not really specifically a goal that I have right now in building a bunch of different robots, but I guess I just find it to be that.
I mean, I'm a lazy person and I don't think I would want to maybe go to the kitchen or get something for myself.
You sound like every teenager. You're building a robot for every teenager on the planet.
I guess one goal for my robot to make... to satisfy my lazyness.
We're going to get into the rub that you've been building and the AI you've applied this summer.
But Mark, how did you get into this? I'd always been fascinated by robots, but I didn't know much about it.
So for me, it was kind of just just jumping in to see like, cause I knew the term robots and I knew that, all right, I'd always seen as, you know, as a young kid on the TV shows, these cool robots, but I never thought that,
I'd actually be building them. And when I had the opportunity to join my team's robotics team, it was amazing. it was just amazing with you know how much you know with just a few machines and you know a great set of young minds, you know, a team of kids can just build these amazing robots and I just loved it as soon as I joined.
Right. And so, like you say, you've been in these competitions and those competitions, robots have to go through a series of tasks, picking up things and moving things and doing other things.
That's how you guys build them. But here at NVIDIA and being a Jetson intern, I'm just going to say that a lot because I like to say that.
Using the Jetson board and applying AI, how did that change what you were able to do?
And where did you start and where has it gone?
Maybe Isaac, let's start with you. I guess the robots we're building right now are kind of similar to what we do in FRC, but a bit different.
So we're working as a team. in teams of four.
Just to clarify, FRC is First Robot Competition.
Yes. So we're working in teams of four. And first, you work in probably bigger teams, but you're still working in a team here.
And we're all working on one robot. Mm-hmm.
And just like in first, we're splitting up tasks and doing different things and trying to work together to make one goal.
But what's really different is that kind of the resources and the people here that we can talk to are like a lot different than first.
Cause I mean, first, all the mentors and everything they're from everywhere and like but nvidia like a lot of people here know exactly like how the jetson works and what it is and all that stuff and so it's really helpful And so let's get into your individual projects and your goals, because in each case, you're applying deep learning. you know, methods to a task or a series of tasks.
So Moksha, what is your team building? And is it a robot?
Is it what and what does it do? We're building a delivery robot.
Oh, that's right. You don't want to go to the kitchen.
And what that does is just... You give it a map of a building and it can move from place to place to just deliver objects or like food or whatever to people.
And number one, we have to drive the robot around the building because we want to create a map so the robot knows how the building looks like.
So you're training the robot on the building.
The robot is learning basically the contours and the outline of the building. it's the robot.
That's like, it's not specifically learning that it's just given that data.
Um, The learning part comes where when you put an object on top of the robot, then it knows what the object is.
So like, say you put a pen on the robot or say you put like an orange or an apple, it'll know what that is and then it'll be like like it'll know if it's there or not so like say it's in the middle of a trip from like my cubicle to my manager's cubicle and like I'm trying to deliver like a flash drive and someone just chooses to take that flash drive off the robot, then it'll know and it's going to stop and. potentially even sound an alarm or something.
I see. So and if it takes my Apple, same thing?
Yeah, same thing. Okay. Mark, what are you guys working on?
My team is building a humanoid robot that detects and trash and it will then walk up to the trash. pick up the trash, and throw it away at the proper place, such as a trash can.
Hopefully, it will just eliminate the need for people in the future to worry about where they throw their empty coffee cups.
You guys know this, right? You don't throw them on the floor.
Well, with robots and AI, that might not even be a problem.
I see a laziness theme here in the robots so far.
What did it have to learn and what were the problems that you had to solve to get it to recognize trash and then do something with it?
We had to train the robot how to pick out from an office space what is considered trash.
And so right now, our definition of trash, we started with just cups.
Cups that are on the ground. And so did you show it images, you know, thousands of images of cups on the ground?
It's exactly what we do. It's almost just it's almost just like training a young kid how to recognize like a certain car or Like a certain, um, a certain type of animal, you show the robot, the images of what you consider trash in our first case, it was cups.
And you tell the robot, all right, this is trash.
And then after so many iterations of doing that over and over with, you know, thousands of images, it can then on its own, you can deploy it on this robot and it will then pick out from a scene, all right, this is trash and this is not trash.
So that way, if you drop your phone on the floor, it's not going to pick it up and try and throw it in the trash can.
But if you drop some... Or Moksheth, when they bring me a flash drive or an Apple, it's not going to toss it in the trash.
Exactly. As we have continued... We've seen that cups is a very limited definition of trash, so it's possible to expand what the definition of trash is and train it further so that it can pick up more than just cups on the ground.
Right. Isaac, what's your team working on?
So we are... in essence, building a self-driving car or a mini RC car that can navigate through an office building and find anybody who would be trapped inside in the case of an emergency.
Like when somebody is in danger and you have to have somebody risk their life to go in and see if everybody got out and is okay or if anybody is really in danger. then you can just be causing more lives lost than you would otherwise.
Give me a scenario then where this robot wheels into action. let's say there's a fire in the so then you can deploy the car and it'll go through the building, navigate through all the cubicles, and basically check every nook and cranny, and will detect any people that are inside, and let firefighters know, so they know where the people are.
In your case, it's image recognition. It needs to know that's a person, not a laptop lying on the floor.
But it also needs to do what? What were the problems that you guys had to solve, and where did the deep learning come in?
In total, we're running four neural networks.
One of them is, like you said, for the person detection.
So image recognition. Yeah. Yeah. And then the other three are for steering.
So to drive, we use our LiDAR and camera. as sensors that we can use to train a model to recognize like a hallway, for example, or a turn and know how to react in those situations.
Is it working? I want to ask you all, like, what's been hard?
I mean, I walked in, I think, a week ago or so, and I saw, Isaac, you guys were working on the driving part, and...
I think it was banging a left turn into a cubicle.
Has it been easier or harder? And what are some of the hard things that you guys have had to overcome?
Moksha? Well, I mean, I guess in general, um, Sometimes we've faced issues for a really, really long period of time facing the exact same issue for like maybe like days in a row.
And I mean, I guess that's probably the really hard part because we know what issues we have and what we still have to tackle.
So I guess. persistence through a lot of the issues that take a really long time to solve is like But so give me one of those issues.
What's been hard that you worked on for days and days?
One issue was getting, I think— On our robot, we have two different ways we can figure out where we are in a map.
So odometry, which is how the motors like how far motors go and also the lidar.
And we had a really hard time doing was connecting that to the rest of our navigation around a building and getting the sensor data at proper times.
I guess in general, like just integrating the whole system was really difficult and, um, That was something we had to spend days and days on.
That's like one of the issues we had to...
To work it out. Was it interesting how your team worked together?
Did you guys have tension? Did you, you know, were you all crying in joy when you got it fixed?
Well, yeah, we were really happy after we got it fixed and after it was actually... moving around the building on its own.
But a lot of like most of the time we've as a team been able to delegate tasks to each other and kind of split up certain things we're working on.
So we're really effective as a team. So like that issue in specific, it wouldn't really be that effective for all three people to be focusing on that because only it was only possible for one person to be working on that issue at once.
So like at the time I was actually working on the neural network that, um, figured out if like the object on top of your robot is like an apple or an orange while another one of my team members was fixing that issue.
Well, how many things can I recognize now?
I think 15 different objects. There's like pens that can recognize apples, oranges. mice, like keyboard.
Oh, not rodents. That would be the trash people's problem.
Maybe not. I'm sorry. Not to say that rodents are trash.
I take that all back. Mark, when you describe just even defining what trash is, that's kind of a hard, subtle problem.
Yeah, so we we kind of just picked something that we saw on the office that We saw it would just be a good place to start, so we picked Cups because it's... fairly distinct from the color of the floor and it would be they're often tall enough that we can pick it up without having to move the robot in an extremely funny and complex way.
And so we started with that and initially we were having trouble understanding how to create a proper neural net to be able to detect the cup because we're using our own data and it was just Initially that was our kind of big challenge. we realized later that the way that the software was working, it was different based on where we deployed it. uh, based on different backgrounds, based on what the kind of floor.
Oh, so the image recognition was, was more subtle or tougher than you thought. yeah exactly and once we figured that out that I'd say the kind of the second biggest challenge was being able to train on multiple types of trash after we had gotten cups kind of settled down we we moved on to things like paper balls and other kind of classic ideas of trash in the office.
And it turned out to be even more difficult to train these neural nets to be able to detect multiple things because as we, as we would train it to, um, find more types of trash such as paper. it would kind of forget the cups, right?
So it's almost like a person where Right?
Like if you're memorizing for a test and... It only needs... Exactly.
As you memorize for... you know, the test for your final, you might forget stuff from the midterm.
Here's what I think you should do. Paper balls, you know, there's the classic, I'm going to throw the paper ball into the way But when I miss, your robot runs over, gets the ball, and brings it back to me so I can shoot again.
Isaac, what's been hard? One thing that's been difficult is getting everything to work together.
So one thing that I really hadn't had experience with before is getting a lot of multiple kind of small projects and putting them together into one.
And... Getting those to work together fine is... more difficult than I thought it would be.
So we've had to like go through trial and error, fix just lots of bugs and problems and get it all to work together.
I mean, you guys are working on really super cool stuff, and it would seem like it would be more complicated for anyone.
But do you have a sense of what's possible because of what you've been able to do so far this summer?
I think really because of the deep learning.
I haven't really used it very much before, and I know that most first teams don't use deep learning at all before.
When I came in here and we said deep learning, I thought it was just like, you know, maybe an extra thing we're going to add on.
But it ended up being very, very helpful to making it work. for the project that I'm doing right now, it's necessary.
Because of that, it's kind of changed the way I think about deep learning and I think I can use it in the future for many projects.
You're going to use it to win some competitions, aren't you?
Yes. I can see it now. pardon the interruption but we have a favor to ask leave us a review on itunes google play music or whatever your favorite podcast platform of choice is It helps more people find us.
As always, thanks for listening. Now back to the good stuff.
What would you say to students or people for that matter who don't even – they don't have your backgrounds and they haven't been coding since they were eight-ish in your case.
Isaac, how does this become a part of their lives?
And how do you think people should think about approaching it?
And also, is it approachable for most people?
I think it is possible for people that don't have a background in robotics or in any deep learning to to jump in and get a... at least a basic understanding of what deep learning means.
And I think actually NVIDIA is doing a great job of trying to reach out to most to like people who have never done artificial intelligence before to that, you know, So that they can learn.
Because... There's all these great, often free programs online now beyond just NVIDIA that are trying to educate the vast population about what deep learning is so that so that people can learn how applicable it is to so many industries beyond just robotics and technology.
There's been a lot of discussion about AI, and it means different things to different people.
And there's those people who are steeped in it and understand it, and those people are on the outside often who are a little bit nervous about it.
So like, where do you guys stand on that?
And how do you explain it to people who may not be so technical?
I guess I could be like one of those people.
Just six weeks ago, I was one of those people on the outside. and I didn't really know about it.
And maybe thinking about some things like it can detect your face and recognize you sounded a bit scary.
But then now I see how the technology works and what it does.
And I've kind of learned that it doesn't really... necessarily mean that just because it detects you doesn't mean it's going to find you and track you down because the neural network is just trained to detect you or find you.
And I think people who are using it use it with the best intentions.
Being able to detect your face is like a good thing.
For example, like on social media, you can see, instead of having to tap someone and type their name to tag them, it just shows up. for example, and so deep learning in that, I think it's just, helpful and I don't see now that I know how it works I don't think The neural network itself isn't like a hive mind brain who's trying to.
And you're too lazy to tag people yourself anyway.
I know that. Moksha, what's your view on this and how do you describe it to people who maybe don't understand it as deeply as you guys do now?
At least right now, it's not like we have developed something that's going to screw over humans or something.
I just think that in general, everything that we've developed, anything that any AI researcher has developed has helped people.
I guess that's how I would explain it. Spin this all forward.
What are your robots, when they're perfect, what are they doing?
Well, so the delivery robot, what it's doing is it's always on and it's going from place to place when people message it on Slack.
So, someone asks it like gives it tells it if they want to get something from someone or if they want to deliver something to someone and then says the name of the place they want to go and the robot will first come to the person if they want something from them and then go to the other person.
So that's... what happens in a perfect world.
Yeah. Mark, what about yours? The trash former will, uh, be able to, um, detect various types of trash.
It will and then it will also be able to like kind of walk stably for long periods of time just kind of gazing through the office and it will be able to pick up the trash, throw it away and then continue just walking around and looking for trash.
And so that, you know, eventually, People just – they're just not going to worry about where they're throwing trash.
I still think people should throw their trash in the trash can.
But if they miss – The trash former will come and pick it up.
Isaac, in a perfect world, your deep learning robot is doing what?
Ideally, it would be sitting in a dock. maybe in the corner of a building, for example, and maybe a fire alarm is pulled or something.
And the fire marshal would say, okay, go.
And it would drive off and go through the building, weaving in and out of rooms. and such and just find all the people and report all those people who may be stuck or in danger to firemen and images of those people. so that firemen can go help them and know where they are.
You don't have to answer this if you don't have a real answer, but what do you guys want to do with your careers or the time that you spend doing everything but sleeping?
I think I've known, I think since like kind of the majority of high school that I whatever I want to be, whatever I do, I just want to build things.
I like. I mean, I started when I was really young, building with Playmobiles, and then that developed into Legos.
And I really loved Legos because I could just, you know, even if you don't follow the instruction set, you can still build whatever you want.
And that has developed now into building all sorts of cool gadgets with various electronics that you can find now but also to things like woodworking and just um I just love building and when I can incorporate technology into that, it makes, you know, it always puts a smile on my face.
So I just. I plan in the future to just be building in whatever industry I end up in.
Moksha. I just want to build. some technology or make some technology that helps a lot of people around the world.
I mean, that's really, really broad, I guess, but In general, that's all I know that I want to do.
Right now, like I don't really have a specific way right now which I want to help people, but I know that I want to help people with technology.
Well, you have time, and that sounds pretty good.
Isaac? I guess I want to do some sort of engineering, probably software engineering.
Um, like computer science. Um, and I'm not quite sure what I want to do in that yet, but I think something where I work on something hands-on like a robot.
Something mechanical is more interesting to me because I get to see the product right there and I get to see what I'm working on in real space.
Right. So you guys are all going to go out there and make stuff in some shape or form.
And hopefully we'll all get to use it too.
Jetson interns Mokshith Vudarla, Mark Thies, and Isaac Wilcove.
Thank you guys so much. Thanks for having me.
That was so heartfelt. Sure, yeah. Oh, my God.
I don't know.
