Welcome to NVIDIA's AI podcast, where we discuss all things related to artificial intelligence.
Today, we're going to talk about DIY AI.
Try and say that five times fast. Applying AI to everyday things and not needing an engineering degree to do it. joining us to discuss the harebrained and not so harebrained applications out there, our Make Magazine executive editor,
Mike Sinise, and NVIDIA chip engineer, Bob Bond.
I know Bob Bond is an engineer, but his engineering expertise is not in deep learning.
And it turns out he's applied it to all kinds of things.
Let's start with this. Bob, I know you might not be your average bear when it comes to machine learning, but you used machine learning and a sort of a deep learning approach to solving a problem that you have in your own house.
So walk us through the initial process. problem and then how you thought that this kind of technological approach could help you solve it.
Sure. So I got this idea that I could drive an RC car around and that I could completely control the RC car just by looking at it with a camera.
I could use deep neural nets to you know, to decide whether the car was going forward or backward.
And then I could use that to steer the car.
Well, I got to working on that and I did a lot of neural net training and I got things kind of up and running.
And in the meantime, my wife said, well, gee, you know, I've got these gardens and she's an avid gardener and we have a problem with The neighbor's cats.
The neighbor's cats seem to come into our yard and they do what cats do.
Now, I like cats. I mean, don't... Well, yeah, I do like cats.
We had cats when we were kids and our cats roamed free.
That was just kind of how my mom brought me up.
And so I didn't want to go have a big confrontation with the neighbors.
That didn't seem like the right thing to do, but I did want to discourage the cats from hanging out in our yard for obvious reasons.
And so here in Oregon, there's a lot of rain.
So we thought, well, a little more water for the cats really wouldn't hurt anything.
Just as a side project to this car thing, I decided that I would stick a camera in the front yard and then I'd use the net that I'd already trained for a bunch of other reasons.
The network that I was using was trained on cats.
So I said, well, we'll just try this. We'll hook the camera. so that it sends pictures to the computer.
The computer will watch for cats if the computer sees a cat.
Then what the computer will do is send a note to a little box.
It's actually a photon, a little single user. the things board that I stuck in the sprinkler box, and we'll have the computer turn on the sprinkler when a cat comes in the front yard.
So back up a second. So you trained data, you trained your sort of data set to recognize cats as they were walking across the lawn or the garden, correct?
Well, it actually turns out that when you install on neural network, I use cafe.
And when I installed that on my Jetson board, it knew how to recognize cats.
It knew what they looked like because the model that I picked up had been pre-trained as part of the ImageNet competition to recognize cats.
Well, of course, cats, as they say, power the Internet, so there's probably all sorts of cat... cat recognition thing built into everything, for that matter.
I'm sure your driverless car can recognize cats, too.
So it recognized cats and it turned on the sprinkles when the camera said, hey, here's a cat.
In just a bit more detail here, the camera setup to just not do anything, Unless something changes in the front yard so if if somebody walks by or you know the wind blows or something then the camera will start start snapping shots and at the rate of about one a second.
It FTPs those into the Jetson board Then the Jetson board runs a little loop that watches for these pictures to arrive, and when they arrive, starts looking at the pictures to classify them.
So it can classify people or cats or dogs or animals. a number of other things that came with it.
The way it started, it actually wasn't very good at it.
It turned out that it would recognize maybe 30% of the cats that wandered into the yard.
So is it a false positive or a false negative?
Were neighbors getting sprayed or were they not getting sprayed?
A little of both. What actually happened was...
The cats didn't get sprayed. The 30% number is 60% of the time the cats were not being sprayed.
So that was actually really easy to fix it.
It turns out after looking at the pictures on the net that a lot of the network pictures are just what you'd expect.
You know, the cat laying on the owner's lap and looking really cute. and the camera down right in the face of the cab.
So what I ended up doing was training was just taking some of the pictures I had of cats that I picked up from the security camera.
I fed those to the network to augment the training.
And after two or three rounds of that, it actually is up to about 90% now or 95%.
It does a really good job of recognizing these cats slinking around in my bushes. or kind of crossing the yard.
Well, I'm going to ask you about dogs and small children dressed as cats in a minute.
Mike Sinise, I want to bring you in. This is a, it sounds like at least a sort of classic adaptation of fairly sophisticated technology. to kind of fun or interesting everyday problems.
Are you seeing more of this out in your community maker world of people kind of deploying this kind of sophisticated technology for, as Bob has done, more fun everyday things.
One of the big things that's happening and it's happening in real time right now is that the technology is finally hitting a point where we as consumers have access to this type of capability.
We recently worked on a project that's actually pretty similar to what Bob did.
We did a project in partnership with NVIDIA, in fact, using... their Jetson board, the TX1, which I think is the newest of their Jetson boards.
And it's a powerful, powerful machine. It can process hundreds of photos in a second to do computer vision processing and understanding and really gather information about scene.
The one that we did, again, pretty similar to what Bob built, but we did a... But you went after dogs, didn't you?
We... We actually did cats as well because there's a great cat data set. that that's already built um and you know like you said cats power the internet so there's there's a lot of cat information out there but we did a cat project that was um a laser pointer You know, imagine you take this supercomputer prototyping board, and what do you do with it?
You build a cat toy. Yeah. Of course. But we built something that would detect a cat like Bob's did, but once the cat gets detected... uh laser would uh would power up laser pointer and connected to a servo and it would move in random patterns around the room so you your cat could be entertained for hours until it decides it's not entertained anymore.
The cat leaves the system. realizes there's no cat anymore and turns the laser off.
And that's actually something we have a write-up on the Make website about this, if people wanted to to reproduce that themselves.
But that's the thing, that board just came out.
There was another NVIDIA board, that came out about a year and a half back.
I think the TK one, which is also a really powerful board that's focused on image processing. and really starting to leverage this machine learning world that's That's got a whole lot of capabilities to it.
Bob, both you and Mike kind of know your way around technology, but it sounds to me that it's getting more and more.
Is it getting easier? And if it is, what are you starting to see out there in the world in terms of application for this kind of stuff?
I do think it's getting easier. It's been traditionally in an academic environment and I think It still helps if you can read academic papers, if you can still understand the math in it.
But you really don't have to do that to get started.
Really, I did. was download the CAFE setup, download the Python lives and things that go with it, and fire it up.
I think it really would take maybe half a day to install this on a capable machine.
As Mike said, the TX1 is a great one. I had a similar network on the TK one.
It was a smaller network and what it did was recognize things other than cats It recognized ants in this particular situation.
But that was easy as well. Whoa, hold on a second.
It recognized ants and then what happened?
Oh, look, there's Ant. Oh, I wanted to see if I could...
So I built a little contraption that has a couple of servo motors and a couple of mirrors and a laser.
And when an ant crawls through and it recognizes an ant, it tries to hit it with a laser.
This is like Reagan's Star Wars project.
Maybe actually it's more successful than that too.
One of my friends said that I was the first person to arm an AI, but I don't think so.
So, yeah, so that was really straightforward as well.
And that I actually trained on a Jetson board.
I think the technology, to come back to the main question, I think the technology is approachable and it's getting easier as time goes on here.
Along those lines, there's something that we've been watching over... a little over a year now, a colleague of ours at Make, Jim Burke, he runs this racing series called the Power Wheels Racing Series.
It's been going on for a few years now. We're People take their kids discarded power wheels, ride-on toys, like the Barbie Jeeps, and they soup them up, and they bring them to the races, which tend to happen at Maker Faires.
And they have, it's practically a sanctioned race series at this point.
Last year, they added a new class into it, which was autonomous power wheels racing.
Two different groups entered that. This year we had, I think there were two or three more that came to New York.
We're talking about DARPA challenge type results but with 500 to $1,000 budgets.
This is something that when I think back to that first DARPA challenge, No car got further than a couple miles before it got tangled up in bushes.
It's mind blowing, you know, that this is now something that I could build.
I could build this in my driveway. And a lot of it's using, some of these are using the TX ones and the TK ones, and there's a couple others.
But the combination of the processing powers, The sensors, and that's a big part of this machine learning world, sensor capabilities and sensor accessibility has just exploded.
And now we're doing wild stuff. And some of it seems, you know, racing power wheels.
It sounds... frivolous in some senses, but some of these guys that were racing last year have already gotten jobs at Tesla and other big corporations.
And I think there's a recognition that this is something that you can do, but it's also turning into a gateway to do bigger things as well.
For you, Bob, how do you think about that?
And where do you think this stuff heads next?
Well, I've seen a lot of work on speech recognition, and I think that's coming quickly.
So I think people are deploying chatbots, I don't think it's very hard for a person like me to say, well, I'd like to do some sort of speech recognition.
Those trained networks are available. The recurrent neural nets can be trained as you go.
So I think speech recognition is a big area.
Image recognition, it's really great that we finally got enough processing power to really do something with an image, really recognize the constituent parts of the image.
So those are two areas that I think of just right off the top of my head.
But I can imagine like fig trees, when the figs are ripe, you have some image recognition where...
Time to pick the figs before the squirrels get to them or whatever, but sort of prosaic solutions using this really complicated or sophisticated, not complicated, but sophisticated technology.
And there's some of that that's happening.
If you look at some of the things that are in the drone space, with agriculture, doing surveying, you know, with the mapping and the GIS work, but some of it really with specific intention in agriculture of maximizing the potential for a certain area, making sure that areas aren't over watered or under watered, that the right you know, the right environmental conditions are in place.
And it's That's pulling in this data, and that's stuff that's becoming accessible, and it's still being driven by the...
I think by the smaller community, not the bigger industrial groups yet.
There's a lot of interest and a lot of people that are looking into it.
At this point, those aspects are still things that are experimental.
There's an IoT ecosystem. component that I've seen discussed and start to be experimented with as well.
There was a a project that is now defunct but called the Thing System.
This was put together by another, make colleague Alistair Allen and partner of his, Marshall Brain.
This was something that would pull all of your IoT devices together and and turn it into an intelligence system.
And you've got some of that with Nest, and there's people that are, making their own DIY nest thermostats and using using AI components to replicate the stuff that Nest does.
But imagine something that takes your Nest, takes your Tesla, takes your Philips Hue light bulbs and everything else.
And it understands where you are and what you want and it's processing all of your daily motion and learning who you are and getting to the point where it knows where you left your keys because it's you know because it becomes that smart and those are things that again you know these are these are These are community projects.
This is not something that's happening, I mean, I wouldn't be surprised if it's happening at Google as well, but the stuff that I know of is the stuff that individuals working on and i think they've got they've got big expectations and hopes but it's it's um It's something we can do ourselves and it is happening by ourselves.
Just as a result of my project showing up on the internet, I've had a number of people get back to me.
There are all kinds of things that people have thought up.
The critter camera in general seems like something that's interesting.
In fact, there was a guy in Australia, who had all kinds of different kinds of critters raiding his chicken coops, And so he was talking about training it for I don't know what, dingoes or whatever they have in Australia.
Right. And I think that's entirely reasonable.
I think all he really needs is a pile of pictures of dingoes and there you go.
There was another guy who talked to me about, well, what I'd like to do is put a camera by my cat's door, find out when the cat's got a mouse in his mouth. in his mouth and then not open the cat door so the cat doesn't bring the mouse in the house.
So there are lots of those that have shown up just as a result of feedback I've seen from people.
I can see that with kids, too. They're muddy, and you don't want to let them in the front door, and so you keep them outside. until they get hosed down.
All sorts of applications. Bob, I want to ask, how did your neighbor's Did they notice that this was a deep neural net at work or were they just like, hey, those sprinklers seem to go on. in this kind of oddly and coincidentally timed way.
Well, I actually haven't. talked very much with the neighbors and cats about this.
My next door neighbor though, we did have a conversation about the fact that his yard now seems to be you know, sort of the local litter box.
And I said, well, you know, I'm sorry about that, but I...
So that's the feedback I've had. I had some other neighbors that have been interested in the project and have been following it along as well.
Do you ever think of a dog? One quick answer to the problem, and that might have been a dog.
Yeah, I think. Maybe not as much fun. We're not really animal fans to have them for ourselves.
So Bob built this non-lethal kind of cat sprayer, and in fact, probably some of the cats just watch it and have fun with it.
I mean, is this rolling out like so many other technologies do?
Have you seen this happen again and again before?
If you think of some of the ways that Technology has become more accessible.
A couple that we really watch closely through the make lens is electronics prototyping.
And you think of what Arduino has done for electrical engineering in that capacity.
And you think about fabrication, digital fabrication and 3D printing, and now there's access to CNC tools.
If you think of... AI as another developing, growing pillar.
I think that the first things that happen is you get access to the technology, but then you have to have something that keeps people around.
And this is stuff where Bob's projects are perfect.
These are perfect examples for that. Cat sprayers and ant laser beam devices.
Yeah. Which I'm really... Pretty captivated by the sound of.
But that's the, you know, that's the fun stuff.
And people go, oh, I get it. Okay. And then I, you know, and they start to iterate.
And they start to think of their own problems and how the technology can fit into that.
And you grow the community that helps support just the concept in general.
And I think that's what's happening. I think that some of it, we're really right at cusp of this.
I mean, it's just starting, but it's something that there's a lot of potential and And it's neat because it's something that I'm not a trained engineer in electrical engineering, but I can access this stuff now and I can do these things.
Bob doesn't work specifically in the the neural network division, but he's able to do this just by reading a manual.
And that's incredible. But I think part of that excitement is... making sure people have access to stuff they can do, And at Make, we're always trying to focus on, here's the project, here's the learning path, and here's the project, and make it playful and fun so that people... want to do it and then they understand it.
Once you've built it, then you get it and then you can make it your own.
Bob, I have to ask, do you have something else on the drawing board?
What's next? You've tackled ants, cats? Yeah, I actually am. trying to get back to actually I'm building a bookcase in the garage at the moment.
But I'm trying to get back to this idea of driving an RC car just with a camera, nothing on the RC car. that isn't stock, and then controlling the transmitter to the RC car with a computer, and then just doing all of the localization for the car via a camera.
So I do intend to get back to that. And I think that's going to be interesting and a lot of fun.
Mostly what I'm going to do is chase my grandkids around with the car.
There you go. Well, that sounds like a great next project.
Bob, thank you so much. Mike, you too. Likewise.
Yeah, thank you. Postscript here. If you are listening to this podcast on iTunes, Google Play Music, or whatever your favorite podcast player is, do us a favor and leave us a review.
Thanks for listening.
