Hello, and welcome to the NVIDIA AI Podcast.
I'm your host, Noah Kravitz. If I asked you to imagine the greatest danger to surfers, swimmers, and other ocean goers, what would you think of?
Sharks? No, it's rip currents. Rip currents are narrow segments of water which travel away from the shore at speeds of up to 2.5 meters per second. outpacing even the quickest of Olympic swimmers.
They're the most hazardous safety risk to beachgoers around the world, In part because a recent study showed that less than one in five people can correctly identify what a RIP parent looks like.
Today's guest has taken a big step towards keeping beachgoers safe from deadly rips.
Dr. Christou Rottenbach is a coastal and estuarine physical processes scientist at NIWA, New Zealand's National Institute of Water and Atmospheric Research.
NIWA and Surf Life Saving New Zealand have developed a real-time rip current identification tool using deep learning.
The tool detected rip currents at a rate of roughly 90% in trials, and the research behind the technology was published in a November 22 edition of the journal remote sensing.
Christou is here to tell us about using AI to identify rip currents, so let's get right to it.
Prishtu, welcome and thanks so much for taking the time to join the NVIDIA AI podcast.
Thank you for having me. So first things first, I'm in California as always.
You are calling in from New Zealand? Yes, I'm in a town called Hamilton, so it's actually ironic.
It's one of the furthest points away from the ocean in New Zealand, but here we are. well you know what we're on the radio so to speak so no one will be the wiser uh so So tell us, maybe you can start by telling the audience a little bit about what a rip current is, and then we can get into what led you to the idea of using deep learning to help spot them.
Yes, anytime. So rip currents are actually a famous coastal oceanographic problem.
And when I talk about problem, I talk from a physics point of view is for us to really be able to predict recurrence is actually quite complex.
There's a number of factors that can dictate where and when a rip current occurs, you can imagine winds, waves, tidal level.
Is there a river flowing into this coastal area?
What type of sediment are you dealing with you is it payables?
Is it sand? Is it magnetic sand? The amount of coastal settings that you can get around the world is so diverse.
And morphology is the study of the change of the ocean floor, like for example, beach shapes and beach expanses, and that is very difficult is difficult to predict, rip currents is difficult to predict.
And the reason I say that is one of the biggest drivers of where rip current occurs is driven by the morphology, in other words, the shape of the coastal ocean floor, if that makes sense.
Right. No, absolutely. Do the currents develop closer to the surface of the ocean or lower to the floor, and are they a coastal-only phenomenon? heaps of different types.
I'm not sure if you've heard of undertoes before.
Yes. So undertows, also a type of recurrent more difficult to identify.
But basically what happens is you get waves.
This is at the simplest kind of way. Waves bring in water into the coastline.
You've got morphological features that then make these waves break. and basically it's conservation of mass we call it in physics and this massive ocean water now it's like on the beach but it has to go somewhere right And then it has to leave the beach.
And then sometimes if all the cards play right, especially current surface current that you can see from from an aerial picture or on the dome Different dangers, different risks, but that's in a nutshell what it is.
Got it. But they are dangerous as we were talking before. we started recording.
There's been a surge recently, or at least last year, I guess, there was a record broken over in Australia for number of drownings and a surge in incidents, I guess you would say, involving rip currents.
Yes. So when it comes to looking at beach safety, so to speak, with social science.
Because you can imagine immediately we work different exposure levels to the ocean, and usually you can think of risk in general. as exposure times vulnerability times hazard.
So hazard, you can think of it, environmental conditions, winds, wave, rip current, all those things.
Vulnerability could be who is this person?
Is this a person that's literally been surfing since their eyes opened, knows the ocean well, very much less vulnerable than someone coming from another part of the world, never experienced it, for example, like the New Zealand or southern hemisphere, coastlines and don't even really know what a wave is you know very vulnerable the listeners can't see it but i'm raising my hand i know what a wave is but i'm no survivor so Exactly.
So surfers know rip currents very well. They use them to get behind the breaker line, I think.
And then exposure. So exposure is amount of people.
So if your beaches are packed, like Bondi Beach, risk goes up.
So when we talk about, when we see these figures of record breaking drownings and things like that, There's a lot of social aspects we have to take into account per beach to know why this is happening.
But as an overarching question, Whatever the answer is for why this is happening, it means that we need to understand and warn people or understand why and when they happen and then warn people adequately to prevent these type of incidents.
So let's talk about how you came upon the research and the tool that you've been developing.
I should say congratulations on the paper being published a couple of months ago as we record this, but new enough.
Thank you very much. Yeah. So tell us about how this got started.
Yes. So just a little bit of a background to myself.
I'm a massive math geek, basically, and that's what it comes down to.
And my expertise is in a field that we call numerical modeling.
So for those who are not familiar with it, it's basically in mathematics, we've got a bunch of different languages. you can call it, I got obsessed with the language expressing the movement of fluids.
Also kind of, you can call it continuum dynamics.
And then in that language, there's a tool that we use how to express these equations in a way that a computer can help us solve them, if that makes sense.
And that process is called numerical modeling.
So numerical modeling that everyone actually sees almost daily and don't realize it is if you watch the news and the weather forecast that comes up and he or she shows like, oh, we've got a cold front moving in here and you've got this fancy animation and you can see the clouds and the rain.
That's a numerical model. So we're solving the atmospheric physics coupled with ocean physics to make these predictions.
So that's my thing. I'm a physical oceanographer, so I build these mathematical models to try and predict wave dynamics, currents, sediment dynamics, morphology that we spoke of earlier.
That's my interest and my career basically.
So in that career, I then knew that almost impossible to solve with numerical models.
It's too computationally expensive, which we'll circle back a bit later And ultimately we want quick predictions and identifications to warn people.
And that is what triggered me to start thinking, OK, I have to start because so Just back up a little bit.
Surf Life Saving New Zealand contacted us at NIWA saying, listen, guys, we need to get a science partner on board. we've got a whole bunch of different social and technical and physical problems.
Would you guys like to just get into this call with us and we brainstorm you know work these things out we take this journey together and um luckily i was the responsible person i was just like i'm so kind of when we started putting on the thinking hats and things.
And that's kind of in a nutshell where I started with artificial intelligence.
I knew... Artificial intelligence have got restrictions and so does numerical models.
And in this specific technology, I think we managed to to get a really cool methodology together.
So taking artificial intelligence, deep learning, all the stuff you've been working out on out of the equation for a second, How did people used to go about trying to predict rip currents?
I don't know much about it. I'm imagining that they spin up very quickly.
From what I was warned about with Undertow as a kid, growing up in the East Coast of the US, go into the Atlantic Ocean sometimes, watch out for undertows.
So is that the case? Did they come up in a flash?
There's a few what I would call main schools of thought of how people approach it.
Pragmatically, practically surf life-saving lifeguards. know the ocean so there's certain settings of waves and especially tidal phase like whether the tide is pushing in or out that's a really good indicator of your risk, like really good actually.
So when they close beaches, typically it's based on those very simple measures I would say.
But because the answer is simple doesn't mean it's ineffective.
It's very effective, I would say. I think it's still a really good way of doing that.
Another school of thought is recognizing fully that this is a very unpredictable ocean phenomena. than deterministically.
So we're looking at odds of occurrence instead of it's going to occur.
And that you can maybe broadly classify as a beach hazard tool.
So together with Surf Life Saving, we are working on a similar approach.
The reason we're doing that is that last word. is forecasting. is identifying but we are in the process of trying to say what will the conditions be in an hour or tomorrow because you want to plan your your holiday right or your like me in Hamilton and I need to drive to Raglan to get to the surf.
So I want to know, okay, am I going to the park?
Am I going to the city or am I going to the beach tomorrow, you know?
And those forecasting tools will use heaps of information.
So we look at offshore conditions, waves, winds, currents, all of those things.
And we look at what probabilistic combination of those tend to lead to recurrent occurrence.
But now we're in that catch-22 again, because to make that connection, which can also be an AI, by the way, Sure.
We need to know when recurrence occurred.
So bringing us back to the original problem of needing identification.
So even this protocol, this AI tool, can now actually help with the development of the forecasting tool as well.
Very cool. So how did you go about building the model?
Very good question. So like I mentioned, I'm not claiming to be a data scientist pick up anything with regards to mathematics or if I'm going to get nailed for that statement.
So I reached out to a colleague of mine working in Auckland, He is, beyond just being a fantastic human, he's very competent and so easy to work with.
And the two of us, I took this idea to him and said, Nilesh, What are we going to do?
How are we going to approach this? And he came up with a lot of the AI novelty related to this bigger project. um basically what we're doing there's a few steps to the ai there's something called transfer learning so transfer learning means that we're not building this brain from scratch.
So we took approximately 1.3 million images categories already from ImageNet and it's a full fledged brain.
So this brain can identify cats, dogs, cars, trees, all of those things, you know.
But what we found then, and these ocean aspects to it as well, but when we deployed this and asked it, show us where rip current is, it got confused very quickly. it said oh what's is this a beach is this a rip current wait is that a person on a beach you know so we realized we're just gonna have to make this thing specifically more smart for the coast.
And that's, I guess, where the one aspect of our novelty came in.
We then trained specifically on coastal phenomena What makes it more novel is we didn't tell it, look, this is exactly in this little bounding box, this is a rip.
So we didn't use bounding boxes to tell it specifically, this is where rip current is. and over-specified it, we used a process called interpretable artificial intelligence.
In other words, we trained it and then it will have a loop where it comes back to us to show its results.
It's almost like taking the black box a bit out of it so we could see where it's missing the plot.
And then with Missing the Plot, it empowered us to say, oh, it's almost like we've got this baby AI.
And we're like, okay, we just need to show it some TLC and teach it a bit more about a beach.
And we taught it more about the beach, and then it got better and it got smarter and it got smarter.
And think what it does. That's exactly. And at the end of it, but artificial image augmentation.
So we saw that in the cameras that we use here in New Zealand, some days it was raining, There were fog, foggy days.
Sometimes the camera just drifted and you were looking at the beach at a tilt. random shadows.
So we artificially graded a lot of these aspects. and then also through it to the AI.
So let's start building a robustness to not get confused give you some of the statistics.
So when we just have a convolutional neural network, which is a very fancy word for this AI of use, we are just underneath 60% accuracy in our trial data set.
The reason that is not good enough is it's almost flicking a coin.
So I mean, it means nothing. When we added these augmentations I spoke of, percent accuracy when we just add a transfer limit. like just under 70%, when we added all of them, we jumped to that 90, 91%.
Right. Highlighting that we really did need all those aspects.
How long ago or how long have you been working on the project and training the model?
Actually not such a long time, right? Yeah, that's right, yeah.
So we kind of kicked off this project almost a year to the day to when the paper was published in remote sensing.
So most of it was actually just engagement with our main cycle, the surf lifesaving, like really walking this road, understanding their needs, you know, because So I'm very passionate about science communication and I'm very passionate about having bespoke products and tools and science for shoot into the sky because this is the gun you've got.
Be specific. You know, it's just, you know, it's just like, I've got a hammer and you just start smashing anything.
No. Right tool, right problem. Right. So is the use of artificial intelligence, deep learning, other other related tools.
Is that something that NIWA has, I was gonna say historically, but recently.
Is that something that NIWA has been engaged in or is this kind of a first time maiden voyage kind of thing?
So to be honest, NIWA, and I mean, I'm probably talking about my pay grade here, but we've heavily invested in data science. in general recently we've specifically employed employed eight data scientists that literally is sent out amongst the crowd of scientists and NIWA covers a range of scientists.
They went from freshwater ecologists to atmospheric scientists, meteorologists, yeah ecologists the whole show and they are basically there to do exactly this so like go and talk to these domain experts and let's find cool solutions, like bringing this interdisciplinary research together.
Was this your first experience hands-on working on an AI model?
I've touched on it before. So in a previous life, I worked for the South African Weather Service and there I led the development of the first operational forecast, marine operational forecasting platform.
What that means is The first platform we predicted high resolution waves in the near shore and storm surge.
And in that process, myself and my team and some of my team, also my PhD
PhD students, we started touching on the potential of using artificial intelligence for prediction So waves and storm surge, but also for signal correction.
So numerical modeling and AI are not always head to head in competition. they work beautifully together.
And if I can maybe just elaborate a bit more about that, I think the audience might think it's interesting.
Please. The strength of numerical modeling is physics.
So in other words, it is not threatened by climate change, if I can put it that way.
If climate change presents changes in the environment, those changes are captured by physics. so the forecasts actually capture it, which makes it very powerful.
AI can't necessarily do that because AI is based, it can only predict what it has seen.
It learns patterns from what it has seen.
So if you bring those two together and numerical modeling can tell the AI brain, listen, you haven't seen this yet, but let me give you a glimpse into the future.
Then AI can say, okay, thanks, numerical modeling.
What you can't see is that there's a pattern coming up here, you know?
So that's why... pattern statistics, pattern recognition, complex nonlinear pattern recognition together with solving actual Earth system physics, such a team.
As I'm listening, you paint such a great picture and you explain these things so well.
As I'm listening to you describe this, I'm thinking Either comparing these two things or even just sticking to the current developing tool, what about the process surprised you? whether down to the technical aspects, the hardware, any aspect of it.
It could be more on sort of the mathematical and prediction side, but was there something about this process that you know, either you're expecting it to break differently or just you haven't thought of and turned out to be important.
Yeah. I think what surprised me and it's still, to be honest, part of the project that's very much in development is the potential complexity of social science associated with it.
And I'll paint a bit of a clearer picture on that.
You would think let's you know what we've got this algorithm let's go wild guys deploy drones on every beach go crazy, we develop an app, people get to the beach, scan it, boom, Bob's your uncle. you know solve the world's problems right but the thing is there's so many it's not how it works It's not how it works.
The nuances are things like, for example, you're lying on the beach and every hour you hear, of a drone flying by.
So that's irritation number one. Number two, you could be like, Who's looking at me?
I'm literally my skimpy speedo lying here. trying to just get some private sun rays and meanwhile someone's photographing me or even more complex is you know what there might be some cultural or religious construct that people are like, I'm honestly not comfortable with being on a public beach where I'm formed without my consent.
So Those things actually makes it very difficult, which also implies that drones might not be the solution for your beach.
Doesn't mean it's not a solution. for maybe popular city beaches it is, but other beaches it might be a fixed camera because of staffing issues within surf lifesaving, or it might be as simple as a rover with a high angle camera on a pole, but when the surf lifesaving service to their patrol they actually get a camera or aerial image of the ocean you know that canal So I think that surprised me, plus made it even more exciting.
And I'm really looking forward, so if this research goes forward and we get more funding, we're definitely planning on having, say, for example, five case study sites in New Zealand and we'll really go all out with community engagement and that's kind of where that word came in transdisciplinary research so what that basically cross over just science disciplines and you actually engage with the public.
So my research is steered and co-developed by society, you know.
So without any of this basically made up silos that a lot of science tends to work in.
Right, right. And so I may have misspoken earlier.
And I think you did a good job of correcting it for the audience. but talked about this as being a forecasting tool when actually currently it's an identification tool.
Is that right? Okay. So how would it be used now or looking forward if you're able to get funding, anybody out there listening, and continue the research what would this look like in a real-world, real-time application?
Yeah, so I'm one of those scientists that before I do anything public, I want to test the living poop out of it.
I'm not sure if I can say that. Like, I want to be sure.
You can say poop. We'll allow it. OK, I can say poop.
That's a crazy question. So hence the paper as well.
So the academic side of it was I wanted scientific colleagues, peers to review our technical and come back and say, you know what, we agree with this, go for it.
So that was step one. That will also show the funding agencies like, okay, these guys have actually done their homework and their background.
They're not just shooting from the hip. But the next phase would be this very close relationship with surpassing.
So we'll deploy, Based on those feedbacks, we'll deploy options, whether it's cameras, drones, whatever.
So I'll develop, together with social scientists, methodologies we call it then we'll test it with surf life settings so Let's assume a drone.
So when a drone identifies a recurrent, it will notify through what is called a patrol app.
So Surf Lifesaving has got this app where they're getting data feeds from currents, winds, waves, all these things, and they can make their decisions based on that.
So this is just going to be an extra data feed which is very useful and then they can say okay drone 3 identified a current like just now And then part of this validation protocol will be they actually have to go out for me and just click a button to say there was a recurrence.
Right. Because that's kind of building in even more intelligence.
I want to check, you know, are we doing this thing correctly?
And if we're successful with that and all of the social evaluations and things, the final step would then be to deploy it.
Deployment might mean having a QR code at any beach to rock up with your app.
Your Surfly Saving app will definitely work through our partners. where like science support in the background and you scan it and it will tell you, OK, there was a recurrent and where it is and even more so we've got some partners within our research team that's working on virtual reality displaying of this information. the drone will give us the Earth coordinate where the current was seen, but you actually you don't think an earth coordinates right and you don't think from aerial views you want to actually see where is this thing So you can hold up your phone, and where the rip current is, it will color it red.
Right, right. Very cool. How long do rip currents last?
It depends. So it's usually bound by the tidal cycle.
So depending on the level of the tide, because if the tide turns around and starts pushing in, it can change the morphology.
So the tide is a big indicator in most of New Zealand.
I tie this what we call semi-journal, just a fancy word for meaning twice a day.
You've got a high tie and a low tie. Right, right.
But I know there's places in the world that's journal, one high tide, one low tide. so i would say the tide is a big restricting factor so you know in that six to twelve hour but probably even shorter than that.
So I would say four to six hours. But the duration of a single rip current could be long enough that it would be useful to be wandering onto the beach in New Zealand for the first time to get an alert on my phone saying, hey, there's a rip current 500 yards to your left.
Stay away from it. You can hold up your phone to get an AR view of it if you want to see what it looks like, but stay clear of that area for a while.
Yes, exactly. Because if you think about it, a typical beach girl won't be on the beach more than four hours.
And if you hit it wrong, there's a rip current that holds for hours.
So four hours doesn't seem like a lot, but it could, from a perspective of a beach go, that could literally be the whole time you are there.
Right, right. So Christy, we've got a couple of minutes left here.
And you mentioned being a math geek, a numbers geek.
But we were talking a little bit before we started recording.
And it sounds like maybe you've got a little bit of hardware geek in you.
So do you want to talk a little bit about the role of GPUs and the role of... One of the things we've talked about over the years with guests is just how... the acceleration of hardware development and the power of GPUs and the available compute has just taken off in the past five, 10 years and really just helped accelerate the human side of being creative and spinning up all these different uses for the tech Talk to us a little bit about your experiences and what this notion of available GPU power has meant to you.
Yeah, no, definitely. Thanks, Noah. Well, maybe just one thing to motivate the random kid out there who might be listening to this.
Absolutely. I literally grew up in the middle of nowhere in the desert in South Africa.
When I went to university, I did not know what a computer was.
I remember in my first year, I had computer science as a subject.
And I remember the lecturing talking about, okay, we're going to code how to print out Hello World.
My brain was broken i was like wow why you know it makes no sense anyway and here i am talking about like super computing gpus and stuff so Life comes at you fast.
Exactly. So, yeah, with that little intro.
That's fantastic, though. Yeah. Yeah, so a lot of these numerical models that I've been talking about before is classically designed to run on CPU computing power.
As you can imagine, we're solving Earth system physics and everything influences everything. the computing power literally you need supercomputers and with supercomputer i just mean almost like thousands of computational cores classic cpus a lot of consulting engineering firms actually do not have access to supercomputers and a part of part of my experience was actually how do we use limited computational ability to still answer real world questions.
And that kind of led me on this path of thinking, okay, what if I did not work for a or some science institute.
I had my own little business and I wanted to actually do these things.
And that's where GPUs come in. So again, it's not just me.
There's science teams with me. And I have to give full recognition to my colleague, Dr. Cyprien Bosserel.
He's based in Christchurch in New Zealand, and he was the first one to translate a numerical code called XSpeech that's GPU compliant.
That can run on GPUs. and when i moved to new zealand and started chatting to him i then got the idea telling him like okay let's just benchmark this you know let's just see how much faster is these gpus compared to CPU computations.
And luckily we have a whole bunch of GPUs.
So we've got almost laptop, desktop, based typical GPUs all the way to the Nvidia B100, A100, like the beasts.
Yep, the beasts. And the results were mind-blowing. a laptop grade GPU outperformed.
And I mean, you can look at my papers for anyone who's interested in the actual numbers, but it competed with 160 cost of productivity.
The A100 was so fast and efficient that it couldn't compare.
I actually had to plot the results on a log scale just to get everything on the same.
You know? using GPU power. So part of the message in one of those papers, especially with regards to that speech model is trying to motivate the rest of the coastal engineering and coastal science community to say, hey, guys, girls, let's make an effort to translate more codes to be GPU compliant or compatible.
Because for us, it's changing everything. because what it meant was I can now build forecast models at a five meter resolution for all coastlines.
Yeah, that's fantastic. All right. We could talk about this forever, but we're going to bring this conversation to a close, but I hope we can. get a chance to catch up down the line.
But leave the listeners, if you would, Christy, with a couple of things and in no particular order.
One, definitely let everyone know where they can go online, your faculty page, the research paper, social media, anything. to learn more about this project, about the work that you're doing with NIWA and with Surf Life Saving New Zealand.
And then as much as you can, tell us what are you working on now?
What's the state of the project? We mentioned and kind of kidded about looking for funders or I kidded, it's serious business. but where are you at with it and what's on top for this new year for you?
Definitely. If people are interested in specifically what I do, I'm one of those crazy people that honestly work on like 15 projects at once.
So if you want to be entertained, you're welcome to follow me on. on LinkedIn.
But then Niwa specifically, I would invite you to just Google my name Niwa or just go to the Niwa page.
A lot of our resources sources are disseminated there you can also follow me on social media all of it from Twitter to LinkedIn.
Not just my research, but all of the groundbreaking research that we do produce is broadcasted there.
You'll stay up to date with that. And yes, alternatively, if someone wants to contact me with specific details, my contact details is on the NIWA website as well.
Great. As for this specific project, the next step is waiting for funding.
We've shown we've done our due diligence and proved a minimum viable product.
So now we're just, if we get funding, we'll roll out the things as we described earlier.
Yeah, and that's kind of the plan going forward.
Thinking a bit more broadly, which I know is in your wheelhouse, where do you see this technology taking, you know, rip current identification other things having to do with coastal formations and the physics of ocean and ocean life, all of these things. over the next five years or so as available data sets increase, as the ability to create synthetic data increases as GPU power is more available.
You know, when you sort of let yourself run free, what do you see in five years? on the five-year horizon i think it will enable us just more for monitoring general coastal monitoring so deploying any sort of instrument in the ocean and the coast is no different it's very aggressive on instruments you know so We generally have a problem of actually knowing, oh, what was the actual inundation extent of that storm?
What was the beach stability like during that storm? storm, how much has the beach eroded?
How many people were there on the beach?
What was the actual exposure if you want to calculate the risk?
So all of these question marks are missing and a lot of it is actually just image data that we could process to answer some So I think we've started with rip currents because it's the number one threat to beach health and safety, but I think it's opening the door to answer a lot of Other questions which aren't less important, but yeah, that's the hope that I think.
Fantastic. Well, again, Chris, this has been just really enjoyable for me and I'm sure for the listeners as well thank you for taking the time to come on the show and talk to us about rip currents and um you know here stateside we've got a few months until beach season so um Hopefully, by then, some of the work that you've been doing will help keep each other safe in other parts of the world as well, and we look forward to tracking your continued progress.
Fantastic. Thank you so much for having me.
Thank you.
