Thank you. Welcome to NVIDIA's AI Podcast, and we are recording from the floor of the 2017
GPU Technology Conference, a gathering of the AI faithful here in Silicon Valley.
Over the last few years, data-intensive machine learning solutions have supplanted rule-based software systems in many technology-driven companies.
Think about Amazon, Netflix, and Uber. But the gaming world hasn't exactly followed suit, at least not as quickly.
Most games are still a delicate mix of hardwired behavior in the form of traditional code, and somewhat more responsive behavior in the form of large collections of rules.
Our guest, Danny Lang, VP of AI and Machine Learning at Unity Technologies, is taking a different tack using deep learning to help with game creation, that subtle combination of art, story, and software.
Danny, welcome. Thank you very much, and thank you for having me.
How is it that the gaming world hasn't jumped on that machine learning treadmill or bandwagon as quickly as some other industries and other companies?
I think there are a couple of reasons. One of them is that early on, the gaming world did transform try sort of early neural networks and maybe five, ten years ago tried it and were not particularly successful.
And then there's also a great deal of art involved.
You really want to control your storyline.
You really want to make sure that you're audience, your players are having a top rated experience and you don't really want to leave that up to the computer to decide.
Unity technology is all about helping people make games better, more profitable, et cetera.
What you describe is, I think, the sort of crux of it.
Games are this magical art, you know, in many ways.
So how does AI help? What can it do for game makers?
I think that AI can help. adding a lot of more automation to games.
You mentioned Uber, you mentioned Amazon.
That's my previous employers. And I think a lot of...
Information systems have really improved over the years by getting AI and machine learning in there and implementation.
And I personally believe that we can bring that to game development as well and basically improve the productivity of the game developers by taking away some of the more traditional So tasks such as?
Today, a lot of that is either hardwired or rule-based, but we think that those behaviors of those NPCs can be learned.
And in the past when gamers or game developers, I should say, try to adopt these neural nets, what was the outcome and how did it feel and kind of play out as it were in these games?
Those nets were very shallow. So they had the ability to learn sort of what I would call a very direct behavior.
So in many ways, they became very, very good at very trivial tasks.
And the human players did not have much fun with them because it was easy to guess what their behavior would be.
I see. So they didn't have that nuance and that kind of surprise that we all like in games.
How have you then trained or developed AI machine learning systems that can allow for that and give us that thrill and surprise that we like when we're playing games?
Yeah, with the advent of deep learning, in particularly LSTM or very long-term memory networks, These models can learn complex behavior that has elements of strategy in them.
The system will basically remember 50 or 100 steps back... that it did something there that led to a surprise later on and a win.
So you will have either, you know, we have seen in poker plays where they will bluff, or they will confuse their opponent.
And that's sort of very entertaining behavior, and I think it's worth giving it a try.
One of the things that AI does, and deep learning in particular, you teach things to do things.
So drive cars, look at radiology images.
Can you kind of program or teach characters in these games to behave like these characters might if they were fully sort of formed?
Yeah, there are a couple of ways of doing that.
Either we can basically use imitation learning.
We can basically train the agents on human players.
So they can basically learn from human players and incorporate that into their models.
Or we can use reinforcement learning where we essentially set up some rules of actions that the agent can engage in and then some rewards function. and then let them play it out in automated games.
Millions of those spent hours and hours.
Against themselves or against these other agents.
So it's similar to how AlphaGo taught their system to play Go.
It played against itself, essentially, right? yes essentially and you you will see and we have seen examples of that where the npc will develop A strategy that is in many ways very human.
It will try to bluff. It will try to get you to follow it. so that you as a human player get in a tricky situation where it can maybe win over you.
Are there types of games where this has been super successful or that it lends itself more to one type of game than another?
I think that's too early to say. We are running a number of experiments and we are trying to explore and we are trying to work with game developers.
We're going to make... some of these deep learning capabilities available as a service so that game developers can start experimenting with it and bring it into their games.
What does it free up game developers and the other people involved in making games?
What does it free them up to do more of or less of for that matter?
Yeah, so what it does take away is a lot of the manual work.
There's a lot of manual work in basically coming up with initial rules. watch the games and then go in and change the rules to improve the behavior.
That's something that takes hours and hours and days of work. that time will be freed up.
Maybe you can now, with more productivity, create more games or you can create richer games.
Does the gameplay change? Have you seen it?
For example, in the AlphaGo and its Go tournament, it started making moves that no human would make.
Do you start changing games in a way that like, oh, we didn't think about that.
This is interesting. Yeah, that's definitely what I anticipate.
I anticipate that we will see that kind of behavior where an NPC will behave in a way that...
I dare not say the word, you know, highly intelligent or very strategic where we will not initially as human players understand what the plan is and what it's like. up to pardon the interruption but we have a favor to ask Leave us a review on iTunes, Google Play Music or whatever your favorite podcast platform of choice is.
It helps more people find us. As always, thanks for listening.
Now, back to the good stuff. Now, we've been talking about games, but there's a lot of the non-gaming world that's using games as far as simulation goes, to train driverless, autonomous driving systems. famously on Grand Theft Auto.
Are there other applications you see for the gaming world to kind of be that sandbox and that learning environment environment for other kinds of applications.
Absolutely. So Unity has a first-rate film quality rendering engine.
So I can put 3D objects. In a scene, I can have all kinds of lights in that scene.
So I can have a morning, I can have a midday, afternoon, I can have night.
I can move the camera around basically in every perspective in that scene. that will produce enormous amount of either video or still images that I can train my deep networks on.
And so it could be like if I'm trying to teach a system to drive, you can help them with that.
And here's a... you know, a city scenario and here's what it's like driving through the city, but can you make it, can you make the physics real and can you make the situations as real as, as you can possibly make them?
There's a physics engine in Unity, but in these cases here, it's not as much about the physics.
It's basically about... creating the scenarios where people step out in front of the car.
Those scenarios are really hard to recreate in reality.
You don't want to have people. stepping out on your vehicle.
Who wants to volunteer for that job? Do you see it outside the autonomous driving space?
What are some other applications for this kind of gaming environment that can help us learn how to do other things seen several examples of robots on robot arms and robot behavior being simulated in Unity. where you can basically train your robot in a virtual environment before you build it.
You can even... basically experiment with different dimensions and different constructions of your virtual robot There's friction and there's inertia in Unity.
So you can get a very realistic environment in all of these cases.
My guess is you can probably get 80% of the way there and then you build the physical robot and you basically finish the learning process in a physical environment.
Do you have a sense for both game developers and then what you just described, let's say we're designing and building a robot, the time saved and even the economic savings saved?
For games and also for, it's hard to say I'm sure for robots, but anyway, for games at the very least.
Yeah, I think there will be a lot of time saved.
You can set the system to train at night.
You can come back next morning and you have an NPC training.
There was no human work going into that.
Born and matured overnight. That's interesting.
Yes. Often these things take, you know, something like six to 10 hours to train.
And what are they training on? And like you said, they can play against themselves, but where does the data come from initially?
So we basically set up what we call a headless Unity.
So basically a Unity engine running on a server sending its views back to a deep learning system.
It can be TensorFlow. It can be running in Amazon Web Services and AWS.
And you basically have the system interacting with that model and improving over time.
In some ways, there's a massive pool of data that the Unity system can pull from and help train these non-playing characters, NPCs, When it sees fit?
But you basically have the game play against itself.
Right. And every time the NPC dies, every time it wins, it just starts all over.
And you keep doing that thousands or a million times, and in that sense, you generate enormous amounts of data for that game.
Right. Can you generate that data more broadly for games and for a world in general?
Yes. What we are working on is to create skills, which is basically deep learning models that can, for instance, read a map and navigate a maze, but have that as a skill that is an asset that you can basically download and use in your game with your maze that we have never seen before.
Right, that's what I'm trying to get at, where these attributes or these skills, like you say, Look, I can dial it up.
The games won't be the same, but at least I don't have to re-teach all these systems to have a particular skill.
So you can pick and choose and build your game around that.
Yeah, imagine having, you know, like a friendly NPC walking down the street.
A very basic skill. We see them in a lot of simulation games.
They don't do much. They just walk there.
They disappear. disappear. That could be a skill that you basically, you buy a city, landscape, you buy a city, you buy a character and you buy a skill.
And that skill is actually a deep learning model that has been developed once and for all.
Right. Craftiness, assassination skills, you know, ninja skills, cooking skills, whatever it is.
I see. Does it change our gamers? What do they need to be mindful game developers and like what needs to change and what can't change for games to be the things that we love?
There's one thing that cannot change, and that is the game developer is the creative part.
The game creator... is in control of the game and it needs to be entertaining.
And it needs to be amusing and create enjoyment for the player.
So the NPCs should not take over here. They should be a tool to create that storyline that is exciting.
Do you see this applying to... film at some level too to kind of narrative storytelling that's not game related but could be more linear Yeah, if we look a little out, we can see that some of these NPCs could be trained to react differently. to stimulate from the audience.
In a VR scenario, you have a digital experience. where you basically, if you pet the character, it will follow you through the game.
If you don't pet it, it will sort of wander off.
Oh, that's cool. How do you want to see this evolve and what are the signs that you're looking for that it's working and that people are taking this up?
As a technologist and as a unit is a platform provider, what we really want to do is that we really want to put the tools out there so that the game developers can take control and use the latest advancements in deep learning to basically create much better games.
Well, I'm with you there. Danny Lang, Unity Technologies, thank you so much.
Thank you very much and thank you for having me.
