Thank you. Hello and welcome to the NVIDIA AI Podcast.
I'm your host, Noah Kravitz. Airport control towers are icons of the aviation industry, but a Canadian startup wants to use artificial intelligence to make them a relic of the past.
Sea Ridge Technologies believes AI-powered video systems can do a better job at watching runways, taxiways, and gate areas than today's system can.
By seeing airport operations through as many as 200 cameras, air traffic controllers will have a better view and better tools than ever. even if they're working miles away from the runway itself.
Joining us today is Chris Thurow, Head of Research and Development at Sea Ridge Technologies.
Chris, thanks so much for joining the NVIDIA AI Podcast.
Thanks for having me, Noah. So first off, where are you calling in from today?
I'm currently calling from my home office in Berlin, Germany.
So Sea Ridge is a Canadian-based company in Ontario, I believe, but you work out of Berlin.
Yeah, that's right. They are based in Ottawa, but the same as we do for our customers.
We can work remotely, no problem. I think some people call that dogfooding. or it's a variation on dogfooding anyway, but I like it at the remote workplaces.
It is the wave of the future in many ways.
But we like the international flavor. So again, thanks for joining us from far, far away.
So let's talk about Searidge and your role as head of research and development there.
What are you guys doing and why are you doing it?
What's wrong with the current system of airport control towers and how are you making it better?
We are trying to help air traffic controllers to do their job better and We are focusing at the moment on those air traffic controllers that sit in the tower.
So they have a very good elevated view on the airport and They can do their job very well.
However, those towers are very expensive to build, to maintain.
And for contingency reasons, they cannot do their work while the tower is under construction or under renovation or something like that.
So that poses a problem. we deliver systems that give the air traffic controllers eyes where they usually don't have any.
And we do that by supplying very high quality specialized cameras all around the airfield To A, give them the same view they have from the tower today, however, in any place they would like. plus additional views that they currently don't have from, for example, blind spots that they have right now, or any areas of interest that they normally wouldn't be able to see.
Now, you mentioned before going further and getting into your technology, you mentioned renovations and construction and work on the control towers and everything. that would prevent air traffic controllers from doing their job while the construction is going on.
Is that something that's kind of a widespread thing or happening now that a lot of these towers are older and need to be renovated?
How does that play into how Sea Ridge developed its technologies?
Sea Ridge didn't really focus on that aspect at first.
However, we noted that Towers are made for a very long life cycle, and it just happens to be so that these life cycles end right about now.
Right. So there is a market for that. And those airports or air traffic control agency needs to find a way of dealing with that.
And other than just shutting down the airport or running at a very reduced capacity, there's not really another way other than using technology to solve the problem.
Is this a global phenomenon that's happening?
It makes perfect sense that with the advent of commercial aviation and airports, consumer airports, for lack of a better term. springing up that these towers might have been built kind of all around the same time period.
And so now they're starting to see a maybe an unforeseen end of life cycle happening.
But is this a global thing or mostly in the US or Canada?
And not just specific to the towers, but where Sea Ridge is doing its work.
Yeah, so I would say it's a global thing.
This aspect, the renovation aspect, would only affect very large airports.
The majority of airports around the world that are not international are quite small and it wouldn't affect so much them.
However, our technology would still benefit those airports greatly in the sense that, for example, if you have very remote airport with only a few movements a day, it is very costly to have one or two air traffic controllers being on site, even convincing air traffic controllers to go someplace where there's not much civilization after all.
So what we could do is we could provide those sensor data and then images from the airport and remote that to some place where there is civilisation. and where there is maybe an air traffic control center, we call it.
So the facility where multiple airports Converge and air traffic controllers may be able in the future to even handle multiple airports So you could have one center and the air traffic controllers would come in to work there.
And on Monday, I might be running air traffic control for an airport in Montreal.
And on Tuesday, I could switch to New York City or what have you.
Something like that. Or perhaps the one person wouldn't do it, but something like that.
You could run multiple scenarios from the same And I should confess, I don't travel a whole lot, but I've traveled quite a bit in the past two weeks.
And I actually... flew home just yesterday as we're recording this.
So I have images of airports burned into my head.
Making up all kinds of wrong ideas here.
Well, let's talk a little bit about how the technology works.
And you alluded to it that The cameras that you deploy give the air traffic controllers similar views to what they are afforded by the traditional physical towers.
But they can also... give them views that right now are in their blind spots.
Beyond the camera data, what else are you providing and how does the system work?
Actually, we started our company with the idea of providing tracking just based on camera images. you know that the traditional surveillance source is radar.
However, radar also has its downsides and is very costly as well.
Much more costly than an off-the-shelf camera, of course.
So we started out by using off-the-shelf cameras, placing them on the airfield and providing tracking data for radars that had blind spots.
Once we did that, we realized we have so many great camera images from HD cameras that actually look quite nice. that we started stitching those views together and that evolved over time to providing an out the window view with additional tracking functionality for the air traffic controller.
What does the additional tracking capability look like?
So if you imagine an air traffic controller sitting in the tower, they have huge windows 360 degree all around them and they can they can see pretty much any target if they know where the target is and if they know what they're looking for.
This gets a little bit more tricky, of course, if the view is degraded by weather, or if it's nighttime.
And they also have, of course, already systems in place like a radar display where they can correlate the out-the-window view with those targets.
However, this is a very tedious job. challenging and they have to do more than just looking out the window and correlating target.
They have to constantly talking to due to pilots and ground staff and so forth.
So what our system provides is an integrated system view, so to say.
So we would provide images from the area of interest using cameras and lay on top bounding boxes, for example, for the target.
So they immediately see where a target is.
It can be highlighted. And we also combine that with the flight data plan.
So all the bounding boxes have data tags saying what aircraft it is. where it's going, what gate it is assigned to, and so forth.
So in one monitor image, for instance, they would have the information they usually can only get looking out the window and correlating that with multiple systems that they would have on their desk.
I may be dating myself here, but as I'm listening to you, I'm thinking of images from you know, TV and the movies mostly of a monochrome screen with little blips on it.
And each blip, you know, represents an aircraft and the, incredibly mentally taxed and stressed out air traffic controllers, you know, monitoring all of the blips and listening through the headset and Frankly, what you're describing just seems much more modern and much more useful to the air traffic controller.
What kind of feedback have you gotten from folks using your systems?
So since we have to put our system to very vigorous tests and showing them to a regulator.
We developed processes showing the air traffic controller how to use the system in incremental steps.
So we started out by just using the outside window view that we create by stitching cameras together.
And the feedback that we got was this is pretty much the same as looking out the window, which I think is a huge achievement.
But the real benefit came in when integrated with other sources and showed sort of an augmented reality view.
And that's where the throughput could really be increased, the security could be increased, and so forth.
The benefits are huge and the air traffic controllers love these systems.
Our guest today is Chris Theroux, Head of Research and Development at Sea Ridge Technologies.
Sea Ridge is updating the air traffic control systems, the towers... the folks that sit in the towers high above, airports all over the world who are responsible for the safety, incoming and outgoing flights,
They're getting the benefit of AI, GPU-powered AI, and all kinds of video technology from the work that Chris and his colleagues at Sea Ridge are doing.
So Chris, I want to ask you about AI and GPU tech in particular.
You've got a good picture now of sort of, and I like your term, sort of an augmented reality view of what's out the window. with data overlays and combined with the other information coming in.
And then, of course, the air traffic controllers being able to do this from remote towers.
And obviously there's a lot of data that's going in and out.
How does AI and machine learning come into play?
So many years ago when there was no CUDA, no graphics cards in sight that could do any custom programming any custom calculations we had to use we had another choice but to use traditional computer vision algorithms to do our tracking and that was Many years ago, and it was very hard to adapt it to an air traffic control environment. conservatism and safety-oriented.
So the first time we have tried... GPUs and employed an artificial neural network. we could see that this would give us huge benefits over the traditional tracking methods that we had in our repertoire.
I guess it was around 2013, when we first used a GPU and a neural network to do the task.
And we were immediately seeing that this was solving many of the problems we had before.
And forgive me this kind of naive question, but when you say tracking, is this primarily kind of tracking the flight paths of the different airplanes or what else is being tracked?
Yes, so we are interested, or actually our customers, the air traffic controllers, are interested in anything moving So this could be aircrafts, vehicles.
But sometimes if it's in the right spot, I mean, for example, on the runway, we would even be interested in seeing other objects like A deer, for instance.
I remember one time we tracked a fox all the way on the runway.
Did the fox make it? Yeah. Was she okay?
She was. Okay, good. Yeah. So let's switch gears a little bit and talk about your background.
Aviation in your background, data science.
How did you wind up at Sea Ridge? So I was always interested in computer vision and computer graphics specifically.
I started at Searidge through an internship that I got from helping out a friend and that friend just happened to be someone high up in the ATC industry.
So I saw the company grow. When I started at Sea Ridge, There were only seven or eight people.
And I worked my way up there from an intern to an engineer and eventually up to a research and development had.
And in that time, I saw many very interesting shifts in the industry all sparked by new technologies that suddenly became available, definitely the first thing that comes to mind was the remote tower technology that was mainly sparked by GPUs just getting faster and shaders becoming available so the GPUs got programmable that was a huge step and and recently of course deep learning and when deep learning came into play we were able to provide a much better product in terms of video tracking capabilities to our customers.
And how long have you been with Sirich? Yeah, around 10 years.
Not bad from intern to head of R&D in 10 years.
It's an inspirational story, I think, for a lot of the younger listeners out there.
Kind of looking back over that time, you mentioned, you know, the advances in technology and the rise of deep learning and then GPUs powering a lot of the advances.
Yeah. What, if anything, along the way has surprised you, either about the aviation industry or traffic control in particular? or even separate of that, just the technology itself?
I find it fascinating. It surprised me a little to see how mystic all this deep learning topic is on the surface at least.
But once you actually start implementing this technology and then or start programming even, you begin to see how simple it is under the hood.
Well, it is complex in the sense that there are many, many neurons. but just the singular entity that's doing the processing is very easy to understand.
It's very simple. And the power just comes from the massive numbers of neurons, which is really fascinating to me.
So I'm going to put you on the spot for a second, just following up on that.
For someone who's kind of new to deep learning, whether they're coming from a different angle within computer science or a different field entirely.
Any suggestions, any tips, any books they could read or blogs or anything that comes to mind that could help that sort of demystification process to kind of get somebody to that? moment of, oh, okay, I think I can understand that when you break it down, like you said, it is kind of a simple mechanism, even if it gets complex at scale.
Absolutely, yes. I started out reading a very good introduction book to deep learning with even some programming examples.
And that really helped me understand. And once I started programming, everything really fell into place.
The book is from Christopher McLeod, and it's called An Introduction to Practical Neural Networks and Genetic Algorithms for Engineers and Scientists.
Excellent. Whenever we can drop a very specific bit of knowledge on folks that can follow up on, that's fantastic.
So rolling back up to the present day now, a sea ridge powered tower that spans multiple runways, so a multi-runway remote tower, recently opened up in Budapest. in Hungary.
And this is the EU's first certified multi-runway tower.
Is that right? That is right. And so where else is Sea Ridge operating right now?
How many How many airports? How many countries?
What's the reach of Sea Ridge's tech right now?
So Sea Ridge systems are really deployed globally.
We have systems on almost all continents.
I think we're working on getting Australia right now, but all the other continents we have already reached.
So we have around 50 deployments In really a lot of countries, I don't know the exact number, but it must be over 20.
With most of the installations being in Canada, It's kind of natural since the company started out in Canada.
Right. And that's where really the remote aspect of the remote tower really comes into place a lot.
You must be gathering an immense amount of data on a daily basis from tracking all the flights and all the other, the ground data, everything else coming into the system.
I heard something about, it's sort of speculative, but about an AI assistant. as a possibility, using AI to help the air traffic controllers.
What's that all about? Interesting, yeah.
I would say that the AI assistant would be the future.
If we're speaking about the next five or 10 years, I would say.
So right now we are using AI as a technology for video tracking.
And I think that we can really leverage that as kind of our foot in the door now with this technology in the ATC industry.
As many people see AI as being, well, maybe dangerous, or at least see it as cautious We really showed in this industry that AI is not nothing to worry about.
If you use it correctly, it's just a very powerful tool.
And it's not conscious or anything. It's not trying to kill us all.
The AI is redirecting my flight from Hawaii.
We're not looking at that sort of a future.
Exactly. But that really was the first step for us. we would really like to use this technology for other more advanced features.
So one of my ideas is to use AI in the ATC industry for assisting air traffic controllers.
If you look what an air traffic controller does, it's really fascinatingly complicated and complex.
It's really heavy on multitasking, and I can't imagine that air traffic controllers don't miss anything.
So they're... They're human after all. Exactly.
So I believe if we train an AI... to see what an Air Traffic Controller does and eventually become maybe almost as good as the human, it could provide the human with some additional alarms. something like hey you missed that aircraft or would you like me to assign that aircraft to a gate Something like that.
Right. So that will be helpful. There are other applications I could think of, for example, optimal gate to aircraft assignments. which constantly changes and then cannot be really planned ahead, although we have a flight plan, but there are so many variables.
In it, it would be just really painful to actually program something that could figure that task out, it will just be so much easier to just give all the data to or to a neural network and let deep learning figure it out. something that I think would strike a chord with the average traveler.
The obvious safety implications aside, which are first and foremost time spent on the tarmac, taxiing, waiting for a gate to open up, waiting for a takeoff position, that kind of stuff.
My understanding is that the Sea Ridge Tech is already yielding increases in number of aircrafts that can come in and out per hour, reduced wait times, that sort of thing.
And from what you just described, I immediately imagined Not that I waited too long yesterday when I got home, but we did have to wait a little bit until the gate opened up.
And, you know, when you're in an airplane, every second is... is welcomed if it's time you get back.
Absolutely agree. You know, if you fly through London in the winter these days, you may be able to witness our system in action when your plane needs to go through de-icing and there's maybe a waiting line for the de-icing pads. our system is able to get you through those de-icing paths because our system tracks all the aircraft and that even through very thick fog or snowstorm.
Excellent. It sounds like the possibilities for utilizing the system are near endless, just with I can't even think of, obviously, all the variables that go into running an airport, but it just seems like there are more and more things that... the humans on the ground and in the towers can get a little assist from the AI tech.
Yes, I think there are so many parts of the journey through an aircraft or through the airports that can be improved.
It's really fascinating. Well, if you want to find out more about what Sea Ridge Technologies is doing and has done already, and a little bit more about some of the details, of how they're doing it, what technologies they're using, what computers they're using to train their networks, that kind of stuff.
There's a great blog post up on the NVIDIA blog right now.
So you can check that out. It's called More Power, Less Tower.
AI may make aircraft control towers obsolete.
So anybody who's interested, go check that out.
And Chris, Is there a Sea Ridge website or any other resources online where people can go to learn a little bit more about your work?
Yes. Our website is searidgetech.com. And we even have a YouTube channel with some example material that you might find interesting.
Perfect. Well, Chris Theroux, thank you so much for joining us.
The Remote Podcast, talking about remote ATC technologies just seems all kind of perfect to me.
But thanks for making the time. And obviously, best of luck.
I think I speak for anybody who's ever flown in an airplane.
We're all rooting for you guys to just keep improving the state of the art.
Well, thanks very much, Noah. Appreciate it.
Thank you.
