Hello, and welcome to the Nvidia AI Podcast.
I'm your host, Noah Kravitz. I was watching a travel show the other day, and the presenter, a British man visiting a restaurant in Japan, pulled a gadget out of his jacket pocket.
The device was basically two handheld plastic boxes connected by some sort of cable. kind of looked like a commercialized version of the old tin cans and string telephone that were popularized in books and movies, not to date myself, but back when I was a kid.
But the gadget on this show didn't just let people talk to one another.
It translated what they said between different languages.
So in this example, the presenter told one end of the gadget that he wanted to order a beer.
And then the other part of the gadget asked the waiter for a beer.
It spoke out loud. in Japanese. So apparently technology can do this for us.
And yeah, I'm guessing AI is involved somehow. but I don't have answers, only questions.
Lucky for us, we've got an expert on AI-powered translation joining us on the show today.
Spence Green is CEO at Lilt, a Silicon Valley-based AI-powered enterprise translation software and services company.
Lilt's mission is to make the world's information accessible to everyone, regardless of where they were born or which language they speak.
And Spence is here to talk to us about how they're doing it.
Spence Green, thanks so much for taking the time to join the NVIDIA AI podcast.
Thank you, Noah. So tell us about Lil. Well, simply, we translate text.
We translate text and words for large organizations.
And you could ask, well, surely other companies do that.
What we do that's different is that we use technology to make this more efficient and affordable.
I think most people in the world now are familiar with Google Translate and automatic approaches to language translation.
And this has been really, really useful.
15 years ago, when you used to travel around, you had to hold a phrase book and, you know, try to stumble through language to order things in a restaurant or read road signs.
And now we can all do this on our phones. and we can speak into our phones and have it translate.
So that's been revolutionary. What hasn't changed is if you're an organization, which for us means companies, governments, and academic institutions, how you publish information in other languages.
And historically, this is a very labor intensive function where you send documents to a service provider and they hire people who are professional translators to type translation and they send it back.
And even today, this costs something like for a single eight and a half by 11 size sheet of paper, this costs between 60 and $80.
So it's really, really expensive and we're trying to build and provide technology that's going to drive down the cost and increase the quality of the service. so that every organization can make all of its information available to anyone.
So I have about a million questions already, and I'm also...
I'm mentally processing the scarring memory of having a Parisian woman hand me my hat when I tried to speak French to her in Paris 25 years ago.
Back in the days of those handheld books that you mentioned.
I guess my first question, and maybe this will open up into the other ones.
Why can we have apps on our phones, Google Translate, that I can speak into and it can translate it out loud for me into a different language online?
But doing the translation on the level of companies and universities and such is still such an expensive proposition.
Yeah, well, let me put it to you this way, Noah.
Let's say we took this podcast and we took every word that you were saying And we publish that into 40 different languages automatically.
Would you be comfortable with whatever the machine did without any sort of quality guarantee that what you said is being represented accurately?
You know, depending on how technical the discussion was, the machine might actually make me make more sense.
But but I see what you're getting at. Yes.
Yeah, yeah. So for businesses, businesses where your business reputation is on the line, it's your main product or your main website. they want a quality guarantee.
And these consumer technologies give you translation that has two important attributes, which are it's free and it's fast.
But it doesn't come with any kind of guarantee.
And so when businesses, when organizations and governments translate, what they want is a quality guarantee.
And the only way that you can do that is you have a human being somehow in the process.
Of course, that's going to slow everything down and make it more expensive.
And so the crux of the problem is how efficient can you make that human intervention?
And so how does Lilt approach that? So our approach is one that actually goes back to the late 1960s, which is In those days, it was called interactive machine translation, meaning you have a human that's interacting with a machine to accomplish a task.
These days, we call this a human in the loop approach.
And so the way that this works is for our customers, we provide an end-to-end workflow where they can send us Text that they want translated and this could be all kinds of text.
It could be a web page. strings that you need to translate a piece of software.
It could be a product manual. And then that is routed to professional translators who work in an environment that looks like predictive typing.
So they see a view of the content that they're working on, And the machine translation system gives them suggestions for how to translate that content.
And that makes them a lot more efficient because instead of translating every word, they can rely in some cases up to 75 percent on machine assistance. which is going to make it faster and more efficient.
And then we send that content back. And the more efficient that we can make translators, the more we can reduce the cost.
So the translator is, are they listening to an, well, I guess it would depend if they're translating from audio or text. but they're following along kind of the predictive text translation and then intervening to make corrections as necessary?
That's exactly right. We don't do speech right now.
We just do text. I'm still caught up in my Parisian nightmare.
Your Parisian example. Yeah, that's right.
But typically, if you do movie subtitling or something like that, there are two steps.
One is a transcription step where you take the audio and you transcribe it to text.
And the next is a translation step. And that's where we would step in.
And Lilt was founded 2011? My co-founder and I met working on Google Translate in 2011.
Both of us were researchers. And we worked together first as a research project, publishing papers together for the first four years.
And then a really exciting thing happened right at the end of 2014, which was we got this technology to work reasonably well and then deep learning approaches to machine translation in parallel. started to work.
The first modern paper on neural machine translation was published in December of that year, and then the one that was the sort of seminal paper was published in January of 2015.
And so these two research directions converging convinced us that The technology was going to happen and we thought that we had unique perspective on it and the skills to build it.
So we decided to try to bring it to market directly ourselves as a company.
And so that was in 2015? right at the very beginning of 2015.
So the past five and a half going on six years now.
How has it been? What's the process been like building the company?
And this is a bunch of questions at once, but kind of to end with, what are some of the interesting challenges and advances in the field that you've been a part of in those five, six years.
Well, I sort of think of the modern version of the company as only about two years old.
The first three years... Our market's kind of complicated.
The translation market is a very large, mature market.
It's the text translation market is this It's over $20 billion, and it's a market that most people haven't heard of, but it's very large.
And it's highly fragmented. The top 10 providers control less than 10% of the total market.
So it's really fragmented. And it was...
We thought we had some hard technical problems to solve.
We found that those were matched by the business problems we had to solve, which was finding how you use this technology to serve a market need.
And indeed, I think this is one of the main challenges of starting machine learning companies is that machine learning is often a feature and not a company.
We had to figure out how we could get this technology into the enterprise.
The breakthrough was Deciding that we weren't just going to sell software.
We were going to build a services business and we were going to hire and manage the translators ourselves. and then go to enterprises directly as a complete solution. and manage the whole workflow for them, which is a very different, there's a class of companies doing technology enabled services now.
It's happening in bookkeeping. It's happening in logistics.
It's happening in fraud detection. And we are one of the companies that's using machine learning to partially automate traditional business function.
And so are you competing then primarily or I'm guessing maybe it's a mix of, you know, human only or at least largely human dominated games? translation companies?
And then also, are there any that are in your market that are purely technology based?
It's exactly as you suggest. We primarily compete with the companies that primarily have sort of just a fully manual services approach to translation and may not use any type of machine translation or machine learning at all in their production workflows.
Is your advantage primarily speed and efficiency?
I think that the main advantage is that I think about why do human beings build tools You build tools to amplify human productivity.
So imagine, I don't know, harvesting a field of wheat.
You can certainly do that with 100 people with sickles.
But wouldn't it be better if you had a harvest combine and you could harvest the whole field with one person?
And that's sort of the transition that our market is going through, which is that sure, you can translate a bunch of text with a bunch of people.
It's just going to be very expensive and very slow.
And if we have this objective, which is to realize a world where every customer can have the same experience with, digital products and services that you and I have, Noah, which is, you know, we can have the full experience in the language of our choice. then we have to use technology to do that and we have to bring the cost down significantly.
We've had some folks on the show before talking about natural language processing and just the complexity of solving problems. solving problems in that field and the issue of context in teaching machines to read and write or speak language.
Can you speak a little bit about, and for the uninitiated, the concept of localization and how that plays into the work that Lilt's doing?
Sure. Companies that want to sell a product or service in another country There are a bunch of things that they have to do.
It might be setting up an office in that country, hiring salespeople in that country, setting up a marketing campaign.
And one component of that usually is providing the product or service in the language of that linguistic community.
There are subparts of that, that could be the text, it could be maybe they have training videos, so they have to do video translation.
It could be that there are cultural elements, so they may need to provide local knowledge and culture training to their employees before they start trying to sell in that market.
But certainly one part of it is ensuring that the product or service is available in the language that's dominant in that market.
Here's one where it can get complicated.
So we have a customer, Canva, which provides these design templates.
It's a really fast-growing startup in Australia.
So they have these design templates, which are like event cards.
And so they'll have a design template that'll be like, Let's get margaritas at 5 p.m.
Right. Well, if you're going to translate that into Arabic and use that design template in Saudi Arabia, that's not really the message that you want to be sending.
Right, right. our translators will take that template and they will convert it into something that's appropriate for that local culture.
So, you know, it could be we're going to have Iftar at 5 p.m. or something like that.
And so the language that's used is influenced by the what we call the pragmatic context or sort of the world context in which the language is used.
And that's the part that's very, very difficult to operationalize in a machine learning system right now.
It's just sort of common knowledge about the world that you can really only get from people right now.
Along with that concept of pragmatic knowledge, pragmatized context, What are some of the other kind of more technical leaning kind of big hurdles that you're grappling with or maybe you recently have overcome? as pertains to kind of the machine learning side of the process?
Well, I think One thing that I've found to be interesting is we had this sort of thought that machine translation systems get a lot better.
And indeed, over the last three years, they've gotten better at a faster rate than at any point, certainly that I've been working in MT, which has been about 12 years.
And one could argue probably they're getting better at a faster rate than at any point in the 70-year history of working on these systems.
But what's interesting is that customers' quality expectations for the type of brand that they wanna project into a local language those seem to be going up nearly as quickly.
And that extra linguistic information is really hard for a system to get right because Oftentimes, a neural machine translation system these days can produce a correct translation, but what the customers are paying for and what they're wanting is a preferred translation.
So does it sound like our company? Is it going to create the right impression in our buyer's mind?
And so on. And There seems to be an awful lot more of that.
I can hypothesize why that might be. The world's really competitive. people are getting more marketing departments are getting more sophisticated when they try to sell into local markets.
I mean, certainly we see this happening in English companies will spend I don't know, a month trying to get the hero text on their website right.
I was going to say, I want to blame it all on Instagram.
But that's right. Right. So like just getting that succinct message.
Right. Yeah. And that creates an additional learning challenge for these systems we're trying to build.
Yeah, that's fascinating. We're speaking with Spence Green.
Spence is the CEO at Lilt, a company based out here on the West Coast of the U.S. in San Francisco. and they are working on AI-powered translation software and services for the enterprise.
Spence, I want to change gears a little bit and talk about your background, how you got into working on machine translation.
My notes say you studied computer engineering at the University of Virginia way back when.
That's accurate. So you've had an interest in computer engineering for a while anyway, I'm guessing it predates starting at UVA.
Have you always been interested in languages also?
How did these two fields come together for you?
Yeah, well, it's true. I mean, I've been programming since I was a kid.
And so I never really had to think very hard about what I wanted to do with my life.
Right, right. But... after undergrad, I sort of had a quarter life I don't know if you want to call it crisis or, you know, a series of moments that I set moment.
Yeah. Where I set a series of goals for myself.
And one of those was to learn another language.
Okay. So I moved to the Middle East. You know, this was like after 9-11 in the mid 2000s.
And I thought this was going to be an important region in my life.
And I wanted, you know. while I was alive and I wanted to learn more about it.
So I moved to Abu Dhabi and I was, you know, I was working as an engineer, but then also learning the language.
And I met someone who I ended up spending a lot of time with who was helping me learn Arabic.
And one evening we were talking and this was somebody who was working as an immigrant like me in the UAE.
And I said, what did you do back in Egypt?
And he said, I was an accountant. And he was working as a building watchman in the building that I lived in. there in the UAE.
And I said, well, why aren't you working as an accountant here?
And he said, I don't speak English. And I said, but we're in a Arabic-speaking country.
And he said, I know, but you get paid different.
If you speak English here, you make one salary.
If you don't speak English, you make another.
And This is like a source of inequality that I don't think we talk about very much because you can learn another language, but actually adult language learning is really hard.
And it's really only accessible to people who have the means to pay for it, as I did while I was there.
I had a tutor. And it was right at that time that Google Translate came out.
And so I was amazed by this technology. And I thought this was a place where my sort of technical interests could have a positive impact in the world.
So I left my job and I went to grad school to start working on machine translation.
And, you know, within a couple of years, I met my co-founder John and was working on MT right after the Arab spring.
So it was like, you know, within just a couple of years, I was actually able to start contributing to this technology.
And both of us have had this interest in information access for most of our professional lives.
And I think what we converged on was information that organizations create and control most of the world's knowledge.
That's what's not being unlocked to people like my friend who can't work as an accountant in the UAE because he doesn't speak English.
So we started to think about how we could build these systems to solve that problem.
That's amazing that you're your series of goals you set for yourself are uh quite prescient already it's it's pretty incredible I'm curious, when you went to the UAE, what year was that?
2005. Okay. And were you able to get by speaking English as you started to learn Arabic or what was that like for you then?
It was. English is the lingua franca of that country, so most people speak English.
But what I noticed was the people that I reported to and that I was working with were native Arabic speakers.
And it was my intention to learn the language.
And I think that it's less the case for us as English speakers that language is a part of our identity, but I think that's more true in other linguistic communities.
And I noticed that there was appreciation of interest in the language by an English speaker.
And that further reinforced this observation that I had that language isn't, you know, sort of an intensely personal part of, for most of us, for our identities.
And, If that language is not the lingua franca and we're sort of compelled to learn some other language just so that we can you know, survive, basically, this is really something that we don't talk about very much.
I'm going to put you on the spot here. I don't know if you have an answer, but what are the hardest languages for a native English speaker to learn, do you think?
And my understanding, I think that like Chinese and Arabic, Mandarin Chinese and Arabic are both pretty hard for English speakers.
Yeah. And is that because of just a totally different both written and spoken set of characters or is it is it contextual or just kind of everything?
Yeah, you have to, obviously the orthographic, the writing systems differ a phonologically, Chinese is a tonal language, so you have to learn that with Arabic there are phonemes, there are sounds that you make in the language that we don't say in English. there's not really cognates.
So like if you're learning German or French or something, English borrows vocabulary from those languages.
And so you can sort of orient yourself. When you go into a language like Arabic or Chinese, you sort of lose that.
And so you're really starting from zero.
And so is it similar then for a machine translation service as far as which are the most challenging languages to translate back and forth to?
Yeah. That's a really interesting question.
This is especially true of modern systems. their ability to translate is primarily, I would say, a function of the amount of data available in those languages.
And that is primarily a function of government support. the machine translation systems are trained on bilingual data, like the Rosetta Stone.
So you have a sentence in one language and a sentence in another, and you have a bunch of these sentences like a lot, like billions of them.
And that's what you train a system on. And there are organizations in the world like the United Nations that by law, or by charter provide all of their text in six different languages.
So if you're one of the six official UN languages, there's just by default all this training data available to train on.
Now, if you want to train a system for English to Thai or English to Swahili or something, there's not nearly as much data available.
And there's not as much data available on the internet for you to crawl because there are not as many of those people on the internet.
So you sort of see translation quality as a function of data availability.
Right, right. And that just speaks back to what you said about language inequality.
That's exactly right. It's sort of a self-perpetuating cycle.
Yeah, that folks don't think about that much.
Yeah. How many languages does Lilt work with right now?
So we support 45 languages out of the box.
And The system is language independent, so it can translate into any languages for which we have data for.
But all but... very few number of companies translate into that many languages.
So we have somewhat of a different objective than than Google does, which is to fill out all languages into all other languages.
Most businesses have no need for Korean into Arabic translation or something.
So instead, we focus in the languages that businesses are trading in and try to make those really good.
And so what's next, either for Lilt in particular or kind of for the industry, which, you know— then sort of rolls out to the world of languages and language equality and such.
Where do you see things headed over the next couple of years?
Well, I think one of the exciting things about translation is that it's not a solved problem.
I mean, both John and I, my co-founder, we like really big problems.
And we've been at this one for 12 years.
It just feels like we're starting to figure things out.
We're just still at the very beginning. And the translation of any sentence, the longer the sentence, the more possible translations there are of it.
Right. Indeed, most of the sentences that we say every day and that exist in the world are new sentences that nobody's ever said before.
And new words and phrases are being introduced into language all the time.
Language is productive. So this is a problem that we can work on for a long time.
And I think The technology is getting a lot better, but there's still a long way to go.
There were some really big breakthroughs in the architectures used about three years ago.
It's not quite clear what the next breakthrough is.
Everybody's using this transformer architecture right now with various hacks on it.
There hasn't really been an architectural breakthrough in a couple of years.
And so I think there's just an awful lot more to do on the basic technology.
And then on the business front, sort of getting that into the hands of serving customers, okay, so we've reduced that sheet of paper from $80 to something like 30 or $40, but wouldn't it be super if it was $1 or $2?
And we're a long way from that. So that has to do with technology.
It has to do with process. It has to do with operational efficiency.
It has to do with business model. It has to do with the appetite of customers for saying, yes, we're going to make the step from translating into five languages into 25 languages and we're going to operationalize that.
There's just an awful lot to do there. If you go to the website of any major public company, you'll find it's maybe available in five languages.
That just tells you we're only at the beginning of this.
Excellent. Well, the fact you can hear it in your voice that you're excited to keep banging away at the big problems, which is great to hear.
It's fascinating stuff. Spence, for folks who want to find out a bit more about what Lilt's doing, either on the business side or I don't know if you have a research blog or that type of information out there.
Where can they go online? What's the website?
Sure, on lilt.com is all the business stuff about our product and service.
You can go to lilt.ai, we have a research team, we publish in the major NLP conferences.
We had a paper this year in ACL so you can see the basic research that our team is doing.
And then we have a blog, liltlabs at labs.lilt.com. with more information about localization industry and the application of technology in the industry.
Excellent. Well, again, thank you so much for taking the time to come on the show.
This is a fascinating conversation, and I wish you all the best in all of your efforts.
Thanks so much, Noah. Thank you. Thank you.
