I'm thankful that most academics are such terrible writers.
Great. It gives you a job.
I'm Angela Duckworth.
I'm Steven Dubner. And you're listening to No Stupid Questions.
Today on the show, how do you get people to work for free without their even knowing it?
It's like better than Tom Sawyer, really.
He just painted a frickin fence.
Yeah. Also, there are millions of academic articles published each year.
But how much is this research actually benefiting us?
Not even other scientists are reading these articles.
And not even scientists' mothers.
Very likely not. Angela.
Steven. Nice to see you.
It is great to see you.
Special occasion today.
Very special, I think.
Our first ever guest on No Stupid Questions.
And I think he may be a remarkable guest.
I think he's absolutely a remarkable guest.
But we should probably lower the expectations.
I think he'll be an okay guest.
He's fine. Great, guys.
Great. So Luis Fonon is probably best known for being the creator of Duolingo, the popular app for learning foreign languages.
Hi. Luis, welcome. Could you give us a kind of 60 -second version of your entire life?
I am currently the CEO of a company called Duolingo.
We teach languages.
Was born and grew up in Guatemala.
Came to the US for college.
I was a professor of computer science at Carnegie Mellon University.
And something else that I did that became pretty popular is these distorted characters that you have to type all over the internet to prove that you're a human.
It's called a CAPTCHA.
And it stands for something clever.
It's an acronym, yes.
It's completely automated public Turing test to tell computers and humans apart.
It's a mouthful. And this was to get the other end of a computer to prove that you were a human and not another computer.
That's correct. For example, to prevent people from buying all of the tickets for a concert on Ticketmaster.
That didn't work very well.
Because they still do.
They do. But it prevents other things.
For example, it prevents things like people from writing programs to obtain millions of free email accounts.
Luis Fanon, I have a question for you now.
And it goes like this.
What do Luis Fanon and Tom Sawyer have in common?
I'm going to guess.
It probably has to do with tricking somebody to paint a fence.
I was not going to use the word trick.
A positive version of trick.
I'm going to use the word entice.
So you remember the Tom Sawyer story?
You know, I actually don't remember it super well.
Anja, do you remember the Tom Sawyer story?
I don't actually. Okay.
So in The Adventures of Tom Sawyer, Tom Sawyer's aunt Polly tells him his Saturday chore is to whitewash or paint this really tall fence.
And he's bummed because you know, he'd rather be fishing or swimming or playing.
And then he decides, well, what if I persuade people that this is not grunt work, but a novel and awesome thing to do.
And then his friends start coming by.
And pretty soon, not only have they painted the fence in turns, but he's made them pay him for the privilege of painting the fence.
So here's a short passage from when the middle of the afternoon came from being a poor poverty -stricken boy in the morning.
Tom was literally rolling in wealth.
And so in the least pejorative way possible, I thought of you when I was reading about the way you've built your companies, which is incredibly ingenious.
And it seems like it helps everybody.
But can you explain what I mean by you being Tom Sawyer's in that regard?
For a while, a lot of my research had a lot to do with how do you entice people to do things that computers cannot do.
When I was a first year graduate student, I helped invent these captures.
And can you tell the story of how you even knew that that was a function that computers needed?
I was at Carnegie Mellon University.
And the guy who at the time was the chief scientist of Yahoo, his name was Udi Mamber, came to give a talk.
And the talk was about 10 problems that they didn't know how to solve at Yahoo.
And Yahoo, we should say, was a big deal.
At the time was the single biggest Internet company.
Yahoo was like Google is today.
It was huge. And then they had these 10 problems.
I now understand that the reason he came to give this talk was to recruit potential employees.
I thought he actually needed help solving these problems.
But one of the problems he talked about was they didn't know how to stop people from writing programs to obtain millions of email accounts for free.
The people who wanted to do that were spammers.
Each account only allowed you to send like 500 messages a day.
But if you wanted to send 50 million emails, you needed millions of accounts from which to send the spam messages.
So they wanted a way to stop this.
And people were just writing bots to sign up for millions of email accounts?
That's right. And so first we came up with a more general idea.
What if we could come up with a test that can distinguish whether the thing that is getting an email account is actually a human or a computer?
See, a human's not going to get 50 million email accounts because they'll lose patience.
But a computer would get 50 million.
So if we could distinguish between humans and computers, then we could solve the problem.
And by the way, this is an old idea in computer science distinguishing between a human computer is called the Turing test.
This was proposed by Alan Turing.
Many people call him the father of computing.
His idea was a computer will be intelligent once somebody cannot really determine whether they're talking to a human or to a computer.
Now in this case, it was a human judge trying to tell whether they're talking to a computer or human.
In the case for this problem that we were trying to solve, we needed a computer to be able to determine whether it was talking to a human or a computer.
We needed a test that a computer could administer and grade but not pass.
At first, this seems paradoxical.
But then when you're your professor, you actually realize this is not that uncommon.
You can administer tests without being able to pass them.
I definitely did that.
So anyways, the idea was that the more a general idea and then at some point we came up with a specific one, which was at the time, computers were not very good at reading images of distorted text.
But humans were. So this is something that a computer can actually administer and grade.
So a computer starts by putting some letters on an image then it distorts them.
All you have to do is just match the story letter.
So that was the idea with a capture.
Were there other ideas you had that didn't work?
There were others that did work, but just not as well.
For example, we could give you a bunch of pictures of like flowers, and then just ask you what are these pictures off and then you would have to type the word flower.
Is that too open ended?
Yeah, it was too open ended.
It was harder. It required people to know how to spell.
I was going to say there's a little bit of human skill here.
Not everybody knows how to spell.
Whereas the beautiful thing about the distorted letters is that for every letter, there's actually a key on the keyboard and humans happen to be pretty good at that.
And it worked beautifully.
Yes, it worked beautifully.
I mean, pretty quickly, essentially, every website out there was using it in one way or another to stop people from writing programs that would abuse the system.
At the time, about 200 million times a day, somebody would type a capture on the internet.
So could we somehow figure out how to get them to do something useful?
And this is where it occurred to me that you could get them to help us digitize books.
Google was trying to digitize all the world's books.
And the way that works is you take a book and you take a digital photograph of every page of the book that gives you images with words in them.
The next step is that the computer needs to decipher all of these words.
But for older books, the computer could not decipher all of the words.
Or even probably some fonts.
Yeah, it was harder or the picture was not super clear or something.
The computer could not recognize about 30 % of the words.
The way we hooked up the whole process was we took the pictures from the book.
And then the words of the computer could not understand.
We would send them to people while they were typing captures on the internet.
So Google ended up buying your company, correct?
But even before that, you were licensing your technology to the New York Times, for instance, to digitize its archives.
Yeah. So the captures were not just random words.
The captures were not just random words.
They were actually coming from books that the computer could not recognize.
And we would use what people were typing to help digitize this stuff.
Angela, you didn't know that.
I didn't know that.
So as you learn that, think back.
You've probably filled in a few thousand captures in your life.
I feel like I didn't get paid for any of that.
That's what I was going to ask.
Do you feel like I didn't get paid or do you feel like, oh my gosh, that's so awesome that in the process of me doing this thing that I needed to do to get what I wanted to get, I also helped digitize an archive.
I think it's great.
I wonder, did anybody ever know?
A lot of people knew.
And we actually didn't try to hide it.
There was a big question mark in there that when you clicked on it, it would tell you what it was.
It turned out most people didn't click on it.
So there was like a little button that said, You're helping digitize things.
So we were getting these words and we were sending them to people while they were getting a Facebook account or whatever.
They were helping us to digitize the New York Times.
Can I just say, I think that's awesome.
That is genius. I mean, I just think it's so clever that you were able to repurpose labor.
It's better than Tom Sawyer, really.
It is better than Tom.
He just painted a freaking fence.
And it's going to need to be painted again, like in three years.
And once the New York Times is digitized, it won't.
Is it still true though?
Haven't computers gotten better at being able to read distorted text?
So computers have gotten way better.
What has happened over time is that the modern versions of CAPTCHAs, the secure ones, are unlikely to use distorted characters now.
And they're back to this original idea of pictures of things.
Like you get pictures of street signs and you're asked, Which of these is not a stop sign?
Why can't they do that?
I mean, gosh, that sounds like a great task for a computer.
I think in most cases they can.
This comes because something like Google Maps is trying to map all of the world and they're trying to recognize all of the stop signs in the world.
Whenever the computer is not very sure that it saw a stop sign, it says, Oh, okay, here's what I'm going to do.
I'm going to send it to somebody that's trying to get an email account somewhere and I'm going to ask them, is this a stop sign or not?
And so you're helping basically build much better maps or build better image recognition or something like that.
So is there any of this win -win logic at the heart of Duolingo?
There was at the beginning.
There no longer is.
Oh, I was hoping there still was.
So just explain what Duolingo is and how it works.
So primarily it is an app to learn languages.
It's now the most popular way to learn languages in the world.
There are more people learning languages on Duolingo in the United States than there are people learning languages in the whole US public school system.
We have 10 times as many people learning Irish on Duolingo than there are Irish native speakers.
And you have a lot of people learning Game of Thrones and Star Trek languages.
Are you kidding? Make of it what you will.
There are more people learning High Valyrian on Duolingo from Game of Thrones than there are people learning Irish.
Can we also just quickly do the business proposition?
You're still privately held, correct?
It is a private company.
Venture funded. Venture funded company.
And the last I read the valuation was in the neighborhood of one and a half billion dollars.
Yeah, about one and a half billion.
So most of your product, the vast majority is free.
Correct. So how are you worth one and a half billion?
Yeah, so you can learn a language entirely for free on Duolingo.
In fact, this was the mission of the company, in part because learning English can really change your life.
It's nice to learn French to go and order a croissant in Paris, but really learning English in non -English speaking countries can double your income potential.
There is data that shows that learning a second language if you're an English speaking American is pretty much useless.
I think it's the lowest rated skill from employers in a recent survey.
But if it were High Valyrian, maybe.
I wasn't on the survey.
Well, but there's other things.
By the way, I of course have to be here to tell you about all the benefits of learning a language.
It is also good for your brain.
It's good for executive function, prefrontal cortex.
Yeah. But the mission has really been always to be able to teach for free.
Is that related to your coming from Guatemala where English is an incredibly valuable skill?
Yes, it's entirely not only is it an incredibly valuable skill.
It's a very poor country and everybody wanted to learn English and nobody had money.
And all the ways the word to learn English were really expensive.
So if I'm writing the Louise Fanon biography, like the hagiography, I say, listen, if you, Angela, are feeling a little bit exploited that you didn't get paid for entering all those captchas, you should know that he plowed the money back into a way to teach everyone English around the world for free.
I'm good with them.
I feel like we're even now.
So the way we make money is we had to be consistent with our mission.
So the first thing that occurred to us is just put an ad at the end of a lesson.
It turns out we have enough users that that actually made us, you know, tens of millions of dollars a year.
Soon after that, we added a subscription where the main thing the subscription did was turn off the ads.
We make way more money from the subscription than from the ads.
What percentage of your users turn off the ads?
So 3 % of our users are paying subscribers.
But even being only 3 % outweighs the revenue from the ads.
Significantly. 85 % of our revenue comes from those 3 % of the users.
That's the way Spotify works as well.
There are a lot more free users, but the premium, well more than Makeup for it.
It's exactly the same thing.
So last year we made $90 million from this.
And then this year we're probably going to double it.
We're thinking about going public.
So I'm not supposed to give too many estimates.
It's just us in the room.
There's no microphones or anything.
So you said that in the beginning Duolingo used the translation for a different purpose.
Can you describe that?
In the beginning it was a very similar idea to this book digitization.
What can we get out of people learning a language?
Can we get something useful from them?
Computers cannot yet do.
And at the time, the idea was, could we get them to translate stuff for free for us?
So what we did is we actually had contracts with CNN and BuzzFeed.
They would write all their news in English.
They would give it to us.
And then we would have these users that were learning English on Duolingo.
And then after a certain point, we would say, hey, you want to practice what you just learned?
Here's a CNN article in English.
Why don't you help us translate it to your native language?
Imagine they were native Spanish speakers.
So they would, to practice their English, would help us translate the article to Spanish.
And then we would get multiple users to translate the same kind of paragraphs and stuff.
And then a computer program would kind of combine all of it.
And then we would give a translation back to CNN who would pay us for this translation.
It had one huge problem, though.
Translation is a crappy business to be in.
Because the prices are always going down, and computer translation is getting better and better.
Can I say you're the first computer scientist I've ever spoken with who afterward I understood a little bit better about how to think about problem solving, and it didn't just make me feel confused.
Oh, thank you. There are other computer scientists like this.
I'm sure there's a few.
But I've also had some wildly baffling conversations where I really try to understand.
Because, you know, computing is a marvel, especially for those who don't really understand it very well.
It seems like magic.
And I want it to not be magic.
I've often asked to understand different pieces of it, and I kind of gain a little bit of an understanding.
But the way you approach it, it's not about the bits and the bytes.
It's about humanity and how to kind of manipulate the relationship in the right way.
So I appreciate your explaining things so well.
Well, thank you. Still to come on No Stupid Questions, Louise Spon -Anh answers Angela's question about the volume of academic research being produced these days.
We could do something like if you publish too many papers that have very few citations, you get tenure taken away.
Oh, I like that. Relegation.
Relegation from the Academy.
See, you just gotta watch out.
You just can't publish dumb papers.
Welcome back to No Stupid Questions.
Today, Angela and Stephen are speaking with Louise Spon -Anh, the computer scientist who invented CAPTCHA and is now the CEO of the language app Duolingo.
I am really interested in Louise's position on scientific productivity.
I do believe we should be producing scientific insights into everything.
But I realize that there's something like one or even two million, depending on the estimate that you read, new scientific articles produced every year.
And the typical article will get, you know, zero, one, maybe two citations, meaning not even other scientists are in any obvious way reading these articles.
And not even scientists' mothers.
Maybe not. Most likely not.
Very likely not. So Louise says somebody who began as a professor in computer science, or at least was at one point in your career and then moved outside of academia into the real world, what do you think about whether there is utility in these many, many, many articles that are being produced?
Well, that's not an easy question.
But what I never really liked this about being a professor is I think a lot of people, including me, I was guilty of this.
They think the end goal is the paper, not the result.
And some of the people that I admire the most did not have that many papers throughout their career.
Each one of them really changed the world.
I think that we should strive to be a lot more like that.
In computer science in particular, you hear of people who have 14 papers at the same conference.
I can tell you this.
I kind of don't care how smart you are.
You cannot have 14 world changing, amazing ideas in one year.
You just can't. But to some degree, you can't blame the perpetrators so much themselves because the incentive structure is set up to encourage that, right?
100%. The problem is you need a way to measure it.
The measurement kind of got in the way, right?
So measurement was always number of citations, number of papers.
These are supposed to be proxies for the impact of your scientific discoveries.
But now I feel like we're chasing the metric and that has led to kind of this like citation arms race 20 or 30 years ago, to become a tenured professor.
There was an expectation of some small number of citations and papers.
And now it's just exploded where you can't even get interviewed for a job unless you have a reasonably long VITA with lots of articles.
It's crazy. And also things like Google Scholar really helped to do this because now you can count very easily how many citations somebody has.
And then your worth as a scientist is you have 2 ,000 citations or 10 ,000 citations or 30 ,000 citations.
You mentioned people who are great in their field, which I guess you mean your field, computer science, right?
Yeah. Who've published only maybe a handful of papers.
Can you give an example?
A good example was my PhD advisor, Manuel Blum, very prominent in the field.
I mean, he got the Turing Award, which is kind of the Nobel Prize for computer science.
He's a cryptographer.
He helped invent modern cryptography.
Has more than a handful of papers.
But he really didn't seem to publish more than a couple of papers per year.
And there's a few of them that are really the beginnings of some amazing things.
What motivated him, and if he wasn't motivated by citation count, how would you characterize what woke him up in the morning, kept him working so hard?
I think in his case he truly, truly wanted to understand things.
When I started working with him, I would try to explain something to him and he would say, I'm sorry, I don't understand.
And he did that many, many times.
At some point I started kind of wondering, was this guy smart?
I've seemed to be saying very simple stuff.
But he just wants to have a really deep understanding of things.
And at some point I started realizing what a deep understanding meant.
For example, if you have a really deep understanding of something, you can explain it really well.
These people who can't explain things super well, they probably don't have that great of an understanding of things.
So did you learn at that point that you didn't really fully understand even your own idea?
Correct. And he helped you do that without telling you that's what he was doing.
Exactly. And I got much better at explaining certain things.
I mean, it'd be like the first line and he'd say, I'm sorry, I don't understand.
And it really kind of pissed me off.
He still doesn't understand one thing and he's still at it.
He wants to understand consciousness.
This is the thing he has wanted to understand his entire life.
What does that mean, to understand consciousness?
I'm not even sure. Maybe you should turn the table and say, I'm sorry, Manuel, I don't understand what you mean by this idea.
Can you explain it to me?
It's a pretty complex thing, right?
I mean, consciousness, I'm sure you guys know way more about this, particularly Angela being an academic in this field.
I mean, there's something about this, the thing that's controlling us.
Right. You know, if you're sitting in the park, you think you have consciousness and you see a squirrel chasing an acorn and you presume that the squirrel does not have the kind of consciousness we have.
And then you go into a tailspin of like, but how would I know?
Do computer scientists actually think about questions like this?
Yes, yes, yes. Particularly early artificial intelligence researchers really wondered, what will it take for a computer to be able to do everything that a human can?
And then you start wondering, well, OK, do they need to be conscious, you know, from the outside?
If all they're doing is the same input output behavior, who cares whether they're conscious or not?
So you say early AI or maybe even early computer people.
It reminds me a little bit of like the earliest economists going back a few hundred years ago.
They were moral philosophers and economics was just kind of like the nitty gritty way of measuring how people were changing as society and industry was changing.
Do you see a similar kind of generational shift among computer scientists that there was more, whether it's philosophy or that kind of higher order thinking than now?
Because I think a lot of people now when they hear about someone going to school now for computer science, they almost think of it as like a very, very, very high end trade school, because there's an industry.
It's a great vocation.
And so I'm curious whether the field has continued to produce and encourage that kind of philosophical thinking.
I think it has definitely shifted from the early days.
I think these people were a lot more philosophical and they really were trying to answer very fundamental questions about what it means to be a human.
I mean, now you go to school for computer science and you can get a very high paying job at Google or at Facebook.
So it's a different type of person that started going into the field.
That doesn't mean that there's nobody out there thinking about all the deep questions, but overwhelmingly, I would say it's become a lot more pragmatic.
Okay, so getting back to Angela's original question about all these papers as a layperson, I would think, wow, two million new papers every year.
That's fantastic. We're broadening scientific knowledge so much.
But if the preponderance of it is some form of job auditions, slash academic status climbing, slash virtue signaling, slash whatever, then it's not so good, right?
It's very hard to say whether this is good or not.
I mean, there's the other argument, which is like, if you're trying to be creative, and 99 % of the stuff you come up with is going to be crap.
There's some research that says, look, you just need volume of ideas.
There is this theory of creativity from Dean Keith Simonton.
It's called the equal odds rule.
And it says exactly that.
You want a good idea?
We'll start with 10 ideas.
Maybe one of them would be good.
Okay, you want two good ideas, generate 20 ideas.
I mean, it's really just a linear function and more is better.
How should a non -academic researcher feel about all the research produced by academics, the vast majority of which is written for a tiny cohort of your peers in a language that is literally indecipherable to the layperson, maybe not psychology papers, but comp sci and a lot of economics.
Economics, yeah. Should we feel that it's a kind of, this is going to sound too fresh, like some elite, masturbatory festival where you guys are all in your lovely ivory towers, doing research that very rarely trickles down or affects us and yet is often funded by some version of taxpayer.
I'm not saying that academic researchers should focus on news you can use kind of research, but it does seem extraordinarily cordoned off, including the fact that academic journals will cost $20 ,000 a year.
Very few people will actually access them.
So how do you just feel about that from the production end?
That's an excellent question.
I think there are some academics whose role a lot of times is to translate research, and I think they get very good at it.
Angela being one of these people, not your entire role, but you've done a really good job just translating a lot of research.
I try. I'm thankful that most academics are such terrible writers.
Right, it gives you a job.
Yeah, I would be useless.
I think there should be more people that try to translate things to the general audience.
Should universities spend more time doing that themselves, or should the journals, I mean some journals actually to their credit now do, like NBER, the National Bureau of Economic Research, they send out all the working papers every week, which is usually 25 or 30 papers.
And then once a week, they do a digest.
It's kind of written in journalism form.
And those are the papers that get written about in the New York Times and Wall Street Journal much more often, because reporters might be bright people, but they may not read all 30 economic papers every week.
And so I figure that's a good function, but I still feel just that there's a kind of loss to society.
That there's all this brain power and good ideas.
Because every time I meet an academic, it can be in a very obscure field.
And I always ask them, you know, what do you think about?
What are you working on?
They always have something amazing.
Always. And I'm like, I've never heard a thing about that.
Where would I read that?
And it's like, well, in the Journal of Left -Handed Cardiological, you know.
Okay, so here's a theory, as probably Manuel will tell you, it's not easy to write clearly, even for your peers.
Also, if you're not a very clear thinker, or an especially good writer, there's a kind of retreat into the...
Dazzle them. Yeah, with syllables.
With formulas. With formulas and passive tense sentences that are really long.
I have mixed feelings.
On the one hand, more scientific discovery is better.
But it's absolutely true.
There's this strange incentive system at work where you are personally awarded on volume, and it would be better if we had an incentive system that rewarded quality and not quantity.
Right now, I think it's rewarding both, but the metrics are so much more visible when it comes to quantity than quality.
I don't have any brilliant ideas of how to make the system better.
We could do something like if you publish too many papers that have very few citations, you get tenure taken away.
Oh, I like that. Religation.
Religation from the Academy.
So you just got to watch out.
You just can't publish dumb papers.
Okay, I want you to start that movement.
And I'm going to sign on to be your enforcer.
That would be so much fun.
We will not be loved by the academic world.
It's okay. It's not my world.
No Stupid Questions is part of the Freakonomics radio network.
This episode was produced by me, Rebecca Lee Douglas.
And now here's a fact check of today's questions.
In the first half of the episode, Luis explains that the original purpose of CAPTCHA was to help prevent bots from signing up for spam accounts.
He says that each account only allows you to send 500 emails per day, which is true.
That's the sending limit for both Yahoo and Gmail.
However, he goes on to say that if a spammer wants to send 50 million emails, they would need to register for millions of accounts.
Luis may be a computer science genius, but he might need to brush up on his basic arithmetic.
If Yahoo allows you to send 500 emails per day, you only need 100 ,000 accounts to send 50 million outbound messages.
That is, within a period of 24 hours.
This would still be quite challenging for a single person.
Say it takes a human one minute to register for a new Yahoo email.
Even if you did nothing but register for accounts all day and you didn't eat or sleep or use the bathroom, it would still take approximately 69 days, 10 hours and 40 minutes to register 100 ,000 accounts.
So you need to have some extreme devotion to fake Nigerian princes or to scientifically unproven male enhancement drugs.
In the second half of the episode, Angelus says that there are between one and two million scientific articles published each year.
But according to the International Association of Scientific, Technical, and Medical Publishers, the number is actually more like three million.
Luis was concerned that some academics publish 14 articles a year, but authors have been known to publish more than 72 papers a year and equivalent to one paper every five days.
Stephen laments how challenging it can be for lay people to access academic journals, which he says can cost $20 ,000 a year.
Depending on the quality of the journals and the level of access, this number can actually be much higher.
A subscription to Elsevier, the world's biggest publisher of academic research, can cost universities a whopping $10 million a year.
However, in recent years, there's been a push for open access to scientific journals.
Government agencies like the National Institutes of Health and the National Science Foundation now require grantees to make their work open access a year after publication.
Scientists are also increasingly publishing pre -publication versions of their work online so that anyone can view it without a paywall.
Unfortunately, that means that the studies haven't yet been peer reviewed, but many believe that this is a step in the direction of open access to scientific information for all.
That's it for the fact check.
No Stupid Questions is produced by Freakonomics Radio and Stitcher.
Our staff includes Allison Craiglow, Greg Rippon, James Foster, and Corinne Wallace.
Our theme song is And She Was by Talking Heads.
Special thanks to David Byrne and Warner Chapel Music.
If you'd like to listen to our show ad free, subscribe to Stitcher Premium.
You can also follow us on Instagram and Twitter at NSQ underscore show and on Facebook at NSQShow.
And if you heard Steven or Angela refer to something that you'd like to know more about, we put together a page of relevant links and resources for each episode.
Just go to freakonomics .com slash NSQ.
Thanks for listening.
Okay, so let's just begin with a greeting that feels not too manufactured or unenthusiastic.
Hi, Steven. Yeah, very manufactured.
Salutations. No, also bad.
Stitcher.
