Hello, and welcome to the NVIDIA AI Podcast.
I'm your host, Noah Kravitz. My guests today are co-authors of a paper that was just published on peerj.com, So you can queue it up and follow along or better yet, listen to the podcast first to get a great background on the importance.
And really, it's just fascinating stuff.
And then go check it out on peerj.com. The paper is called...
Protecting Endangered Megafauna Through AI Analysis of Drone Images in a Low Connectivity Setting, a case study from Namibia.
In other words, as they're going to tell you, our guests are using high-tech technology technology in a low internet connectivity setting, which means AI on the edge, for those of you in the know, the ability for the device in this case, a drone to do a lot of the AI computations and all the things that go into using machine learning to, in this case, track black rhinos without persistent internet connectivity.
It's a great use case. It's a fascinating technical problem to solve.
And we have the two best people in the world possible to talk about it right now.
Zoe Jewell is co-founder and president of WildTrack.org, a global network of biologists and conservationists dedicated to non-invasive wildlife monitoring techniques.
And Alice Hua is a recent graduate of the School of Information at UC Berkeley in California. and an ML platform engineer at CrowdStrike.
Zoe and Alice are two of the authors on the paper And they are here to tell us some background about how the paper came to be and then, of course, about their work. tracking endangered megafauna in Namibia.
Alice, Zoe, thank you so much for joining the NVIDIA AI podcast.
Welcome. It's our pleasure, Noah. Thank you.
So maybe, Zoe, you can start with a little bit of the background. of your work, and we can get in more later to what WildTrack does or has been doing.
I'm sure you have some great stories to tell, but how this particular project came to be.
Sure. Well, we've been working with WildTrack monitoring endangered species for a long time now.
And we started back in the day in Zimbabwe working with black rhinos.
So this is a topic very close to our hearts.
And I would say over the last 30 years, we've seen such a dramatic decline in biodiversity generally.
Scientists now think that Species are up to 10,000 times more likely to become extinct than background rates.
So we're really in a major biodiversity crisis.
And another statistic that always gets me is that humans and our domestic animals account for 95%. of all vertebrates biomass on Earth.
So it's like humans are everywhere and we're sneezing out wildlife.
Right, right. So we started thinking, OK, how are we going to protect these Poor rhino, we're out there, we're on the ground, we're walking around, we're trying to find them, make sure they're okay.
But they range over very wide areas and it's very difficult for even the best anti-voting patrols to protect them.
Poachers will come in often at night with guns and they'll literally just shoot shoot these animals and hack off the horns, which retail for around $60,000 a kilogram.
So there's a strong economic incentive to kill them.
And we have to have equally strong incentives to protect them. and so there's a lot of effort money goes into trying to protect these animals and this is kind of where we started thinking about okay you know drones can protect a lot of ground, so it can cover a lot of ground, how do we marry that with the kind of protection angle so that they can inform people on the ground where these firemen are.
Just for a bit of perspective or sort of mental image drawing, if you will, for listeners, How wide of an area, how much ground are we talking about covering?
How much ground do black rhinos and some of the other species typically cover?
Well, black rhinos, in many countries in Africa, they're only in Africa, white rhinos.
But they can range for hundreds of square kilometers.
Big areas. Some of the big national parks are enormous.
And there's no way you can all be patrolled on foot, sadly.
And just to add to that, there are five types of rhinos, right?
And the northern white rhinos is like extinct in wild and getting artificially bred.
And the rest of them, including the Namibia black rhino, there's only 6,000 of them left in the world That's right.
They're tiny numbers, really. Yeah. No, it took me a second to wrap my head around that.
But 6,000 is not Not many at all. So how did this particular project come to be?
Well, so we were working with a student team at UC Berkeley on actually developing the initial AI pipeline And Alice and her team came along and said, what can we do to help?
And at that point we were doing a little bit of drone work and I said to Alice, well, maybe, you know, we could start looking at the possibility actually detecting rhinos on the ground. which would be a fantastic start.
And so Alice, being brave and fearless, jumped on that and said, that's it, we're going to do that.
And that was kind of where we kicked off.
And so Alice, is that a computer vision problem?
It is a computer vision problem because at the time I was taking a course called Deep Learning on the Cloud and at the Edge. at the School of Information.
And we have a final project that I wanted to do something that could bring practical and real-world impact.
And so our instructor connected us with Zoe, because Wildtrap is constantly looking for for collaboration with students on these wildlife conservation projects.
So we implemented a CNN which is called a convolution neural network, which is a type of deep learning. for this problem in particular and several other techniques like GAN, which is general generative Cephasario network to solve the kind of problems that we were having with drone footage being, you know, high above ground at different heights and especially of like tiny species from the footage we needed to basically look into how to do computer vision on these tiny objects from drone footages.
So how high up were the drones? So images that we collected ranging from 30 meters to 70 meters.
So that translates to about 100 feet to 230 feet.
Okay. So it was one of the issues then that from that height and with the resolution of the cameras on the drones. that the images of even these large species like black rhinos, they just weren't big enough, weren't clear enough to be easily identifiable as what they were?
Yeah, it's a little different when you have images of that height.
Definitely, there's a lot of like for Springbok. which is one of the species we try to identify.
So a lot smaller than rhino, right? So now we have even smaller species in the same model, So we needed to figure out for several of the drone images are actually 4K.
So they're very large. So we needed to implement something called tiling. which helps break down the images and allows the animal to be kept at full fidelity when fed into the model.
So yeah, that's definitely a challenge as well-known computer vision space is called small object detection problem.
Okay. And so how'd you solve it? Yeah, so we implemented a technique with tiling So tiling breaks down the images, let's say a 4K images would break them down by 1280, 1280, um, just the, the, the size of the box.
Um, and so we kept and we scale the animal according to that.
That tile so that if there's an image or if there's a tile image that doesn't have the animal will discard up and we keep everything. that has rhinos or other species that we're trying to detect in the model.
So when it gets fed, we also intentionally selected a a YOLOv5 model that would ingest a large enough image.
So that will basically keep all the animals in place without being squeezed down into a smaller space.
Gotcha. And you were doing this, the eventual use case was to do this running on drones. that didn't always have internet connectivity, if my notes are right.
And so I'd imagine that brings up, you've got dual problems to deal with in terms of energy consumption and balancing compute with power needs, and then also not always being able to go back to the cloud for compute resources?
Yeah. How far off am I? No, you're right on the dot.
And I would add to the footage is another problem, right?
Because we weren't able to get, like, a lot of wildlife footage and it involves lots of time and effort to collect this data in Namibia.
And so we needed to augment our data set to make it even larger so that the model can learn better and perform better. on top of having a large image with small animals from the drone's height and lack of internet connectivity we also have not enough footage problem.
Right, right. And so what we did is for the networking part, we used MQTT which is a lightweight messaging protocol.
And it does sporadic internet connections networking.
So I were able to say if you were in a parking lot and you're trying to send a message to the publisher, And the subscriber is not getting it because you're in a parking lot.
You're not getting internet connectivity.
The moment you exit the parking lot, you will be getting that message.
So it guarantees that you will receive it in very little connectivity.
And we're only sending a very small amount of data So we're not streaming a whole video footage over.
We're only sending a image with TPS coordinates and that is sent to a map. a park ranger can view and deploy the antiprotein units.
Got it. So if one of the species that you're tracking is identified...
By the drone system. Instead, it doesn't send footage, doesn't send a still.
It sends just a small amount of data that lights up a location on the park ranger's map.
And the image. So the image. It does send the image too.
Okay. Yeah. So image and the bounding box of the animal and the confidence interval, which this is. confident it's a rhino or other species.
And were you just using stock hardware, the drone and the, or did you have to do modifications?
So when you say stock hardware, I use NVIDIA Jetson NX.
And that is, I didn't modify it. It comes as a stock and we basically deploy our model in a containerized Docker image and it ran on the NX and does alert sending straight from there.
And just an off-the-shelf drone. Yeah, I use Parrot Anafi.
Uh-huh, okay. It's one of the things over the past couple of years of doing this show, it's just a number of folks who... are doing these really cool things that they're able to do now because the, you know, the tools, hardware and software have just become more and more accessible.
And, um, Yeah, it's wild. It's just, for me, it's one of the coolest things about hearing all these stories.
So Zoe, what happens then on the park ranger side and the anti-poaching unit side?
Or maybe a better question to ask is, With the technology part up and running, what has the impact been or where does it go from here?
Well, we're kind of excited to get this child actually further.
At the moment, it's been a theoretical model. based on practical experience that we've had so we're fairly confident that it's going to work but I think that we need to get out there, trial it, see how it performs in the wild.
There are many other elements obviously in the conservation biology arena that have to be linked up with this technology. one of the obvious things being well you know we don't want to disturb the animals so we must make sure the drones from flies and altitude that It's not going to disturb them.
So, you know, is it between 30 and 70 or maybe it depends on the way the wind is blowing that day or any other number of variables.
So we have to fly it in practical life at a height that's not going to disturb them.
That just made me think, well, do you have drone pilots on the ground or is it remote?
And then so that, you know, a step broader from that is how big of a team do you have in When you're running a research study like this, It can vary an awful lot.
I mean, a lot of people who can fly drones can do the whole thing on their own.
They can actually take the drone up and it's just one person.
When we're doing the testing, we'll obviously have a ground team for ground validation.
So they might go on the ground, find the rhino and validate what the drain is with what's on the ground.
We might have other users, we might have local indigenous trackers, so we work very closely with traditional people, communities with traditional ecological knowledge, which is kind of central to our mission.
I was going to say that it must be really important to have that connection with the Indigenous people there.
Super important. And it's something we're really trying to develop much more because, you know, when you think about it, our human origins or the origins of our evolution were in tracking.
Tracking is like the origin of science. You observe, you deduce, you test your observation by seeing whether the animal we just found is what we thought it would be.
And so it goes on. And so, you know, going back to that, I think, What excites me about this in many ways is that we have this incredible godlike technology, the AI.
But it's derived very, very deeply from that. origin of humanity from that tracking the traditional ecological knowledge.
And even now, there are many elements of that TEK that are that AI can't even begin to deal with.
We're kind of just touching the pinnacle here.
So one of the things we'd like to do is kind of engage that more, have a platform that indigenous trackers can come onto and say, well, you know, I can see this and maybe the drone could do this and, you know, to just kind of bring those two skills together.
Absolutely. Are the folks that you've talked to open to that and interested?
Yes, they very much are. Interestingly, we work with indigenous trackers nearly all over the world, but we haven't yet really made those contacts in the US.
And we would love to do that. There are lots of fantastic tribes in the US and we would love them to help us kind of work in conservation biology I mean, they're used to having had their environment wrecked, right?
We're just experiencing this. Perhaps they can help us deal with what we're doing best But yeah, we would love to do that.
How accurate has your system been in terms of tracking?
Yeah, so we use a metric called average precision.
And for Rhino, we're actually successful. very successful at detecting them were at 81%.
That's great. But for smaller and for giraffes, it's 82%.
Right. But for smaller species like spring brock or ostrich, They're on the lower end, 33 and 24.6 respectively.
And then for humans, it's 41%. 0.8, but I'm thinking this is due to the fact that we have a model that's trying to learn so much, right?
It's trying to learn from different heights, learn from the terrain there's like in Namibia is mostly dry terrain and then like we got some data from San Diego Safari Park that's totally wet and green.
So they're very different terrain and different angles.
So that's probably why the model doesn't perform well on the smaller species.
But if we were to break them down and just have each model doing just one type of terrain or one type of height, I think that will be more specialized and perform better.
And so if the AI podcast could grant you one thing to help get through a bottleneck, is it more and better data?
Is it better, I don't know, an increased ability to capture high-res video footage?
What would be the thing to sort of get you over the next hump?
I would say from our perspective with drones, drone technology is changing so quickly. all the time, but I think to have higher resolution cameras, time in the field is something you can always get more of and we can get more data. always great to have more data, more the better.
But I think that to have really high-resolution cameras with the value to fly high, that still capture the necessary resolution on the ground.
So Alice, I heard that there's an interesting story behind the name Is it the name of the project or the name of one of the rhinos you encountered out in the field?
Yeah. Yeah, so when we initially started the project, we were looking at footages with Zoe and we saw this one rhino that Zoe introduced She's like, oh, yeah, there you go.
There's Hermes. And we're like, Hermes? We were particularly moved by seeing Hermes just gracefully walking across the African plains.
And we're like, that's it. We're going to name our project Saving the Rhino Hermes.
That didn't end up on the paper, but we were very happy that we did it.
That actually makes me think to ask, Zoe, you've been doing this for how long?
And by this, I mean, working with wildlife, I'll ask it.
I'll put it that way, being sort of naive on purpose.
But how long have you been doing this kind of work?
Oh, boy. I'm not sure if I really want to tell you.
Probably 30 years. When you first started or as far back as you care to share, how did you go about tracking?
There were no drones with AI capable cameras on them flying around.
So what were the tools of the trade? How did it work?
That's right. The tools of the trade were basically either just walking out into the bush and following the tracks and looking for rhino with the scouts or...
The kind of high point of technology at that point was putting a collar around the rhino's neck with a radio transmitter.
You would go out with receivers and go and follow the bleeps. and try and find the rhinos that way, which was actually quite dangerous because, you know, you're not really following the signs of the animal.
You're just following this bleep. You get to a point where the bleep is all around you.
So it's not directional anymore. And so, you know, you could be very close but not have a clue where the animal is.
And that was... Quite dangerous. And too, I think, you know, on the ML side of things, when we say poachers came out at night, to poachy animals, we also think about how can we do also draw with infrared images.
Right, right. What's the capability like now for drones at night for what you're using them for?
Or is there any? Our current drones, the Skydio and the Sense B, do not have infrared images, nor the Parrot.
But, you know, it just, again... With a little more funding, we could probably get a drone with infrared camera.
The technology's out there. You just haven't been able to.
Technology is definitely there. There are good infrared sensors on drones.
And the nice thing is that, you know, the heat of the animal shows up as a very clear, bright, long night.
Even easier to see than it is during the day, actually.
So that's another of the things we'd like to start trying. trying.
Absolutely. So I wish we had more time to delve deeper into some of these things.
But The paper is there for folks to read, and I believe there's a NVIDIA blog post in the works as well.
But for the purposes of kind of looking ahead, which is how we like to end these conversations, Zoe, what's next for either for more work from the study that this paper is based on or more broadly for what WildTrack is doing?
Sure. Well, we've got an AI pipeline, which this paper has kind of really inspired the edge device detection into. with drones.
And what we'd love to do now is really drill down a little bit from seeing whole rhinos to seeing animal trails, which is really wild facts. interest, we identify animals from their footprints.
And now the challenge is really getting a drone that's got a good enough camera, with high enough resolution that we can pick up animal trails and identify species from their trails.
What this would enable us to do is kind of like map a rhino city.
It wouldn't be necessary to see the rhinos at all.
We could just find out where their footprints are going. where they're hanging out, who they're hanging out with, and using a kind of layer of morphometrics and analysis on top of the AI will allow us to really drill into that, you know, individual identification. not even just is it a rhino, but which rhino is it going along here?
And, you know, kind of working with our partners at JUMP, statistical software, we can kind of build the layer of morphometrics on top of the AI.
And we can also bring the traditional ecological knowledge into that. kind of like empowering the AI with these extra layers of intelligence. which is something I think that many people are interested in now is kind of like a human boosted AI.
AI is really only as intelligent as we'll allow it to be.
I mean, it's all derived from our knowledge on the baseline.
Right, ultimately it's made by humans. Right, the more that we can boost that, the more accurate, more robust animals are going to be.
Absolutely. We've had a lot of folks on the show who've talked about the importance of human in the loop AI, as people call it.
It's not replacing, it's extending, augmenting, helping.
Alice, are you continuing to work with WildTrack?
Are you on to the next phase of your journey with CrowdStrike?
What's next for you? And then also, what do you see as being next for...
Applications of AI for wildlife tracking for the kind of work that you've done with Zoe and your other colleagues.
What's next? Yeah, sure. To attack that question in multiple angles, I just want to say why it matters to work on autonomous wildlife monitoring versus what we have currently in the wildlife monitoring.
I think the, you know, the current situation is that, you know, your flight and quadcopter drones are fixed wing aircraft drones and then you manually retrieve the SD cards and then upload images for your AI model to process.
So this whole pipeline is time-consuming and it doesn't send real-time alerts and it's not scalable.
Right. You know, especially for application in anti-poaching.
So what we see moving forward And as much as we're proud of what we're able to accomplish with this theoretical model, we're still, I think, far from the complete autonomous wildlife monitoring.
I think we like to achieve like a fleet autonomous drone that can do wildlife monitoring.
It needs, you know, basically computational power and additional drones and hardware And it also needs surveyors, park rangers, wildlife biologists, robotics, machine learning enthusiasts, and like come together and collaborate and build more scalable pipeline, which can patrol for more endangered species, you know, not just right now and over a larger area of land. is what I have in my, I imagine.
For folks who would like to get involved, would like to learn more, mention at the top that the paper was published.
And then also, obviously... WildTrack as a website and other resources online.
Where can people go to learn more? Sure.
We have a website, wildtrack.org. evolving machine learning website wildtrack.ml and um alice you want to talk for uc berkeley i think If they were to be interested, definitely enroll in the Master of Information Native Science program, but also- Get the plug in for the alma mater.
It's okay. I'm just next door in Oakland.
I'm all for East Bay. Let's put it out there.
But there's different meetups as well that we have that you can definitely join and discuss what sort of projects you're interested in. you know, work with Zoe to develop that project.
I know Zoe currently has like other schools partner with Wildtrap.
I'd say also know that, you know, we always welcome volunteers.
We have volunteers doing everything from social media to software engineers helping us with our Oh, very cool.
And that you can do from anywhere. It's the internet.
You can do it from anywhere. So we definitely welcome volunteers who are committed to conservation.
Excellent. Alice, Zoe, thank you so much.
This has been a lovely conversation, but fascinating.
For a lot of reasons, it rings bells from different guests we've had on the show and just things that I've learned about uses of AI at the edge and remote areas and there's surveillance and there's industry things, but there's a lot of really interesting and vital work like this going on and different aspects of learning about and protecting the environment.
So it's always great to hear what folks like you are doing.
Best of luck with everything going forward.
Thank you very much. Thank you. Thank you.
