This is the way it feels to move through summer in Lululemon. Iconic Align softness without the front seam.
For our smoothest look and feel ever.
Summer won't know what hit it.
Stretch your limits in the non -stop flexibility of the new Lululemon Align No -Line Pant.
In select stores and at Lululemon .com.
For Scientific American Science Quickly, I'm Rachel Feltman.
This Fourth of July, some celebrants who flock to their local parks and waterfronts might not take in the iconic sights and sounds of a fireworks display.
In some cases, those traditional explosives could be replaced with swarms of colorful drones.
Drone light shows have been popping up more and more in recent years, replacing or supplementing fireworks at the Olympics and even some Super Bowl halftime shows.
They're dazzling, precise and a lot safer than explosions.
Besides the obvious risks of setting off incendiary devices, firework shows also raise environmental concerns.
Studies suggest these big displays have a marked impact on local air quality in the hours that follow.
But swapping out fireworks for drones isn't simple.
Every one of those displays takes painstaking effort from a team of engineers.
They have to plot the movement of every single drone frame by frame.
Today's guests recently published a paper that offers an AI -powered solution.
revolution. Matt Schwager is an associate professor in the aeronautics and astronautics department at Stanford University.
And Eduardo Montejano is an associate professor in the Department of Computer Science and Systems Engineering at the University of Zaragoza in Spain.
Thank you both so much for coming on to chat.
Sure. Our pleasure.
Thank you. Why don't we start with just a quick overview of this study?
You know, how did it come to be?
What got you interested in this particular aspect of drone swarms?
I've been doing research in multi -robot systems for some time.
Also, I've been collaborating with Mac for many years as well.
And with all the development of all these new AI techniques that have been successfully applied to other problems and applications, we thought in collaboration with mainly one student, although there are more people in this research, but here probably I would like like to highlight Pablo Puello mostly, but we decided or we discussed how cool it would be to try to apply all these new techniques to this problem of controlling hundreds or thousands of robots for animation displays.
So speaking of those animation displays, when compared to fireworks, what problems do they solve and what problems do they raise that maybe your paper was trying to address?
I think we consider sort of animation displays with drone swarms as being much more flexible and sort of artistically richer medium for entertainment.
So in fireworks displays, right, there's a big bang and a big flash, but the engineer has actually very little control over exactly what the fireworks do and what they look like, right?
But with drones, you can program the lights and you can program the motion of the drones to display a very clear image.
For a sporting event, you could have somebody playing the sport floating in the air, or for the Fourth of July, you could have words spelled out, you could have the American flag or whatnot.
So it's much more flexible, and, you know, there's more control by the artist and the engineer as far as what they want to convey.
There's a challenge, though, which is that drone swarms, especially large drone swarms, require a lot more engineering expertise and quite a bit more infrastructure to control and to deploy, especially to do that safely.
And so this was one of the targets of our research is to basically make the planning of these large -scale drone displays much more automatic and to kind of empower people without that kind of special knowledge to create their own drone displays.
And could you kind of paint a picture for us currently what does it look like to put on one of these displays?
What's required in the background?
Right. So these are usually managed by large engineering companies and And there's usually a team of engineers, specialist engineers, who make sure that all the drones are properly charged and have landing stations.
They would have to go out to the site where the display is going to be performed and engineer the site to plan where all the drones would fly and where they go and to make sure that the space is clear.
And really the target of our research is that before the drone display happens, there are artists and engineers that carefully chart the path of every drone.
At the time of the display, the drones are actually just following sort of points in space that have been pre -planned by the engineers, one point at a time, one drone at a time.
So you can imagine it's very much like animating an animated film.
It's very painstaking, very hands -on, and requires lots of expertise.
So the target of Genswarms was essentially to use generative AI to do that phase of planning for you.
So you can type in a high -level prompt, like the American flag, for example, or a skier skiing downhill.
And our algorithm would essentially produce these sets of waypoints, these sets of points in 3D space for the drones to fly along to then create the illusion of this artistic display.
So basically, you enter the image you want to end up with, and the AI tells the drones where to go, what colors to be, all of that stuff.
Yeah. Actually, at the moment, we enter just text.
So we enter a text description of what we want to see.
And then the method produces the colors and the arrangement and so on.
Although I think it wouldn't be too hard to extend our method so that you could upload a picture or a sketch of what you want to see.
And what are the specific challenges that arise when you're trying to control a group of drones with AI?
The way these models work, they have been popular by creating images and at the end of the day, they predict the color of each pixel when you give this prompt.
So the idea here is when you want to somehow translate these to drones.
Pixels is just a color and they don't have any motion constraints, any collision constraint.
So the idea is when you try to translate this idea of making pixels look what you like to making drones look what you like, you need to account for that drones cannot teleport from one location to another.
So they have some dynamics, some velocity, acceleration, some constraints in the motion that you cannot do any motion that you want.
You need to account for those somehow in your algorithm.
And also drones have some physical properties, some mass, some size, so they can collide with each other.
So there are the safety constraints that you also need to include in the planning algorithm that use this generative model so that the motion of the drones, it's also safe.
And how close are we to actually being able to use the model you created with drones?
So from the research perspective, I would say that our solution in some sense is mature enough to be applied.
But then there are all these technological challenges that Mac mentioned before about all the real deployment of drones that obviously as academic professors, we don't have the resources to deploy 1000 or 100 or whatever number of drones.
So for that, there's still a gap in terms of from research to application, but it's more a matter of maybe collaborating with companies that already are deploying drones in many locations.
So I think that the integration wouldn't be that difficult.
It's just a matter probably of having the right contact within a company that has the skills for real deployment.
But the algorithm, I think it's already in shape to be deployed.
Very cool. What other applications could this have?
Yeah, so certainly artistic displays are powerful and important, but we'd love for our robots to really help people in their day -to -day lives and also help people who are in danger.
So, for example, we could imagine using an algorithm like this for search and rescue.
You know, if you have hikers who are stranded somewhere in the wilderness and you need some way of deploying a team of drones to go look for the lost hiker, this could be a method that could be adapted to that.
We're also interested in things like exploration, maybe in a space application, NASA might consider developing a tool like this to explore the surfaces of asteroids or planetary bodies.
We're also really interested, currently our next step along this research journey is drone or other robot swarms for construction.
construction so currently our algorithm you type in a prompt and the drones will organize themselves into a shape right that looks like what you asked for what we're looking at now is how could you type in the prompt and have the drones actually deposit material like maybe the drones can carry little square blocks how could they deposit the material in the right order to construct something that is useful or interesting for an artistic display so you can imagine drones constructing Maybe constructing a bridge in a remote area where people maybe need to pass over some difficult terrain, or maybe
there's an emergency scenario, maybe there's a disaster scenario and a bridge has been washed out and you'd like drones to automatically construct a temporary bridge, something like that.
Even when we applied this for a drone show, because the artistic component is beautiful, I would say that there are no limitations on applying this to any kind of multi -robot system.
So in that sense, we could go for other ground robots, domestic robots, construction robots, as Matt mentioned.
So the idea here is to be able to translate these high level commands specified by text that every person can more or less give these commands and then automatically translate them into plans for teams of robots to achieve these commands.
So the ambition in that sense, I think it goes way beyond the artistic display.
And what about the environmental impacts of a drone show versus a firework show?
Well, I would say that, in my opinion, drone shows are safer in the sense that fireworks are a very explosive material, and you hear accidents, and you need to produce and store them.
And then within my knowledge that is not very deep, I would say that probably the residual impact of fireworks is bigger than probably drone shows, that at the end of the day, you you can recycle or reuse these drones in multiple shows.
Noise -wise, probably they are similar, even in the sense that drones currently are quite noisy, although it's true that when you see them from far, far away, fireworks are very annoying and drone shows are not.
But when you fly them in close space, let me tell you that now having a drone flying nearby, it's more annoying than a firework.
So there, I guess, there could be arguments in favor or against each of them.
But if I have to choose drones, I would say that this reusability and safety in terms of explosive materials are the two main big advantages.
Well, and given everything that you're presenting in the paper, how do you see the world of drone shows evolving with this new tech?
Well, so I would say that in the artistic side of the problem, the idea is that with this, they are already existing drone shows are able to develop complex and beautiful animations the idea is that this will speed up and simplify this rather tedious and complex process to maybe make a scale to larger numbers of robots in an easy way maybe also in terms of the testing phase deciding the appropriate number of drones to create specific figures well in summary speeding leading up the whole creative process and hopefully that providing more beautiful, more complex animations and displays.
I think right now one of the most exciting research frontiers is figuring out how to use, you know, powerful, modern, generative AI tools that we're all familiar with, chat GPT, image generation models and so on, how to use those in ways that benefit people, you know, and myself and Eduardo being roboticists, I think we're always looking for ways to enable robots to help people, to better serve people, to make people's lives safer.
And I think this is a really exciting frontier.
One of the grand challenges in robotics is how do you orchestrate the activities of large groups of robots?
It's hard enough to control a single robot.
And now when you've got a large group, you know, there's this persistent problem of how does one human or a small number of humans tell a large group of robots what they should do?
And I think this is an interesting model that we're sort of approaching using generative AI as kind of the bridge, the interface to allow one person or a small number of people to command the activities of a very large group of drones.
Another issue that I also like to point out when mixing robotics and AI would be with the current state of the art would be explainability.
If you want to generate an image, what you care is about the output, but why this output might not be as relevant as when you are considering about the motion of robots.
So understanding and obtaining outputs that are consistent for robots, it's a very important problem that currently I would say that we are struggling because these AI models work very well.
well, but somehow they work well until they stop working well.
And having some kind of understanding of when or why these things happen is very important from a research perspective.
Thank you both so much for coming on to chat about this.
This has been great.
Thank you, Rachel. Great, thank you, Rachel.
It's our pleasure. That's all for today's episode and we're taking Friday off for the holiday.
Next week, we'll be sharing reruns of some of our favorite segments from the past year.
We'll be back with a new episode on July 14th.
In the meantime, you can quench your thirst for fresh science news by reading Scientific American online or in print.
Science Quickly is produced by me, Rachel Feltman, along with Fonda Mwangi, Kelso Harper, Naima Marci, and Jeff Delvisio.
This episode was edited by Alex Sugiara.
Shana Poses and Aaron Shattuck fact check our show.
Our theme music was composed by Dominic Smith.
Subscribe to Scientific American for more up -to -date and in -depth science news.
For Scientific American, this is Rachel Feltman.
Have a great weekend.
