This is 20 Product with me, Harry Stebbings.
Now 20.
Product is the monthly show where we sit down with the best CPOs in the world to discuss how they build their product teams.
Today we've got one of the best joining us in Jem Kansu CPO at Duolingo, one of my favorites at Duolingo.
Jem leads product strategy for over 90 million monthly active learners.
Since joining Jem, has played a pivotal role in driving user engagement, revenue growth and product innovation.
He's launched Duolingo Math and the wildly successful Duolingo Music.
Under his leadership, the company is consistently ranked as the number one education app globally.
This is a masterclass in how Duolingo has created a product of true consumer delight.
But before we dive into the show today, are you struggling to beat model benchmarks or implement Gen AI in your product?
If so, you need Turing.
Turing is an AGI infrastructure company backed by incredible investors like Foundation Capital and Westbridge Capital.
And they do two things.
Number one.
They help leading companies in AI labs like Salesforce, Anthropic and Meta enhance their LLMs with advanced reasoning coding multilinguality, multimodality and more.
Number two.
They combine human and artificial intelligence expertise to deploy cutting edge AI systems for awesome companies like Rivian and Reddit.
Right now, Turing offers a free five-minute self-assessment to help you pinpoint your place in the Gen AI journey, get tailored next steps to optimize your model strategy and then finally learn how Turing can refine and implement your models for better performance.
Take the guesswork out of Gen AI.
Visit Turing.com forward slash 20VC to start your free assessment today.
And once your Gen AI strategy is on point with Turing.
Now let's switch gears to the future of mobile and how one company is turning your screen into real returns.
We spend nearly half our waking hours glued to our phones, upwards of 50 hours every week.
Recently, one company transforming this reality stood out so much I personally became a shareholder Mode Mobile.
Mode Mobile created the Earn Phone, the smartphone that pays you for daily activities.
Instead of big tech profiting billions from our attention Mode returns over 325 million directly to users through earnings and savings.
Mode's revenues surged an incredible 32481 three years, recognized by Deloitte as 2023's fastest growing software company in North America.
And here's why I'm excited.
Mode's equity offerings have raised over 30 million from 20000 plus retail investors one of 2025's standout public raises.
And you can now join me as a shareholder with as little as 1000 at investmodemobilecom.
Forward slash 20VC.
For a limited time, unlock up to 100% bonus shares and a free earned phone.
Email us for the investor brief at 20VC at modemobilecom or check out investmodemobilecom forward slash 20VC.
We've talked about making money from your phone.
Now let's talk about making your product do exactly what you want it to.
Whether it's the software you build or the software you buy, your tech stack should be creating results, not creating roadblocks.
Well, Pendo's no-code software experience management platform makes your software better with tools to see where users get stuck, guide them with in-app messaging and constantly improving your UI.
It's so easy that over 14000 businesses use Pendo to increase revenue, lower costs and reduce risks.
Businesses love the control.
Engineers love the freedom.
Everyone wins.
Star for free today at pando.io forward slash 20 product.
You have now arrived at your destination.
Gem, dude, I am so excited for this.
I just mentioned it, but I freaking love the Duolingo product.
I told you.
I love that you love it.
It's so nice for me because I interviewed lots of CPOs and I don't love their product.
And you kind of have to pretend, you know, it's like you're on a date where you're like, oh, really?
That's what you do?
Oh, fascinating.
And here it's like, I love it.
And you know what I love is the attention to detail.
Yes.
So special.
Yes.
And you're learning Spanish.
I'm learning Spanish.
Yes.
Bruno S Americano.
Attention to detail for you.
Thank you so much.
No, we take pride a lot in the pixel, perfect details, like i think you'll see that in the product right, it's like every screen will have a very specific design language, characters will be very specific, etc.
So, like i'm glad you noticed it totally, let's have some chronology, because otherwise i'll just go off on a tangent.
Can you take me to the moment?
You actually joined your lingo?
Yes, how did that come to be Yes?
So I joined in 2016.
It was kind of a dark, rainy day in September.
I'll skip the funny part of the story.
Anyway.
No, I like that.
That's good for the story.
Yes.
September 2016 in Pittsburgh, Pennsylvania.
That's where our headquarters is.
And I was the first PM that was handled the baton for we were pre-revenue.
We would like to have revenue.
And can you help us figure out how to monetize?
Obviously, I was an early career PM, and I knew ads.
I had worked at Google before.
My first experiments, my claim to fame at Duolingo was putting ads on Duolingo.
As you can imagine, this is good fame.
Yeah, exactly.
Good fame and bad fame at the same time.
And then I worked on monetization for about six years.
So I built all of our revenue features ads subscriptions, in-app purchases and everything that's under our subscription umbrella.
Subscriptions obviously became the lion's share of Duolingo's revenue.
And then I kept on building our teams that work on revenue and building our features that work on revenue.
And in the past three years now, I've moved on to being our head of product.
So I lead the product management team and now I dabble with all of our product efforts.
I mean, my word, how exciting.
It's been a great journey.
So you're the guy who puts ads in my Duolingo?
I did that.
Yeah.
Brilliant.
You're welcome.
You know, I think it's well done.
I am just jumping around to be fair, but my favorite dude on product is Gustav Soderstrom from Spotify.
And he always told me that the details are not the details.
They are the product.
And I'm just intrigued when you think about product building, how do you think about the details being the product versus?
For Christ's sake, we actually need to build product and move on.
I think I have a very similar saying I repeat a lot.
Consumer products live and die in the pixels.
Meaning if you are working on a consumer product, you have to figure out how to make your change pixel perfect for the user.
So, for example, product managers are sometimes viewed as these like I'm the mini CEO, I decide strategy and then people execute.
That is like the worst definition of a product manager that I've ever heard.
Why?
Because everyone says that on the show.
They love saying that.
First of all, the CEO is the CEO of the product, not the product manager.
The product manager has to understand what the user will respond to when you kind of give them what they're going to see in the app.
So they have to understand user design really well.
They also have to be able to stretch into business strategy, et cetera.
That said, this is why consumer products live and die in the pixels and attention to detail matters.
If a button that's like the dark shade of green versus the light shade of green will make a difference in user behavior, a product manager or a product designer, for that record, like people working on the product, should understand what that means.
And I think this is also how we build product teams at Duolingo, where everyone should understand pixel perfect design and why certain specifically what the user is going to see, how that's going to change user behavior.
Can we start at the start of the process, which is the design process?
Yeah.
What does the design process look like for you today at Duolingo?
And has that changed over time?
It's evolved.
It's evolved quite a bit.
I joined Duolingo when it was 50 employees.
Now we're about 850.
So things have changed and we've gotten a little bit more process.
But at its core, the way it starts is you start with all of us use the app heavily.
So most of the ideas that we work on come from our own usage of the app.
Not all of them, but a lot of them.
So when we were testing GPT-3, we're like, okay, this is a good way.
We've tried many different things to teach speaking.
And then GPT-3.
Was this like light bulb moment where like, we can actually teach speaking better if we use LLMs?
So the design process got started with like, you have the technology and you have Duolingo.
How do you bring them together?
Through various iterations back and forth, we prototyped a thing where Lily talks to you.
We really liked it.
We made a feature out of it.
Now it's called Video Call with Lily.
Now a lot of our users have access to it and are using it.
So I was walking through Belgravia, which is a very smart part of London, the other day.
And I was on Duolingo.
And I was going, voy a la playa.
And this guy comes up to me and he's like I'm sorry to interrupt your Spanish call, but I love your show.
And I'm like you've literally been standing there listening to me go.
Bruno S Americano and Voyager La Playa thinking this guy is a fucking idiot.
So, yes, I agree.
And it's fantastic integrating it.
But that design process then, is it fast?
And who needs the approvals?
Don't laugh.
Do you do Figma?
Do you just go straight to prototyping now?
How does that work?
Yeah, great question.
So we use Figma for a lot of our designs.
That said, for something that is an experience you've got to experience, Figma is good for us to say this isn't headed in the right direction, let's go to prototype.
We have a product review process where our CEO Luis, is involved, I'm involved, our head of design is involved and our product leaders are involved.
Where teams come, they show their change and we give feedback.
They refine it and they either go to prototype or they go straight to an A-B experiment.
So depending on the size of the change, we might do extensive prototyping.
In the case of video call, we prototyped for like maybe two, three months.
We did one change, we tested it, we didn't like it, we iterated and then it was ready to go out to our users.
What does extensive prototyping entail?
Extensive prototyping is I think you let's say we have an idea.
We think calling Lily is going to be a good way to practice our Spanish.
The way that the first prototype is going to look is like it's not going to be well animated.
The illustrations are going to be janky.
Maybe there's going to be like error codes in like some part of the UI.
But we'll test the core interaction of can she give us smart answers?
We'll try that.
And then we'll be like, okay, if she can, let's move to problem number two.
It needs to look engaging.
We solve that.
So I think extensive prototyping is you prototype, test, and refine what the spec is.
So it's like a step one utility, step two design, step three perfection.
Pretty much, exactly.
And I think for smaller stuff, like if it's not an entire new feature, you can do it all in one go.
So you validate if it works and then you kind of do the other steps.
Is there subjectivity to if it works?
And a user behavior like Lily, you could be like, well, I thought that was really engaging.
And another person would be like, I did not.
Yes, 100%.
What do you do then?
Well, I think we bring a lot of smart people in the company together.
We have them all tested and see if we can align for the most part.
We don't all have to align.
But generally, if enough high judgment, people are like yeah, this kind of works, like I can actually practice speaking better than everything else I could do on Duolingo, which was the case.
We're like, we have something good on our hands.
Let's make it better.
You mentioned Lily there.
I obviously spoke to Severin the other day.
Wonderful, elegant man.
And he told me about, like two of the I think it was PMs who came up with the idea for chess and brought that forward.
And they weren't really sure.
How do you think about the ideas you choose to pursue versus not choose to pursue it?
This is a great question.
You know, I would love to say we have this like super ironed out process.
You know, you follow ABC and you have the perfect answer to the question of what to invest in.
The reality is it's a lot messier than that.
But Chess is a really good example of bottoms up innovation in the company, where it's not an executive that had a great idea and said yeah, we should go in this direction.
So as a product manager and a product designer, they didn't have an engineer on the team.
They went and prototyped.
Basically, they're like we think chess will be a good subject for Duolingo to do.
We're like, kind of weird, doesn't exactly fit.
But we're like, yeah, cool idea.
They went and prototyped and almost like a full chess lesson experience with no engineers.
So like our PM vibe coded basically the whole thing.
Our designer created excellent looking designs.
We tested it and we're like, this is a pretty good Duolingo lesson.
So like they kept iterating actually like on this vibe coded prototype.
And then until a point where we're like, this is very much headed in the right direction.
We really enjoy using this prototype.
And then we staffed a team with engineers and now it launched this week.
So in a span of eight months it went from like I don't know, this is kind of an interesting idea to like a full launch of a new Duolingo course.
How does that differ in terms of timeline from traditional idea to launch?
It's way shorter and way less people.
Our experience with doing this was with math and music before.
It was triple the amount of people and double the amount of time it takes to get a course out.
The big unlock there is, I would say, maybe two things.
One of them.
AI coding certainly speeds up because you can prototype quickly, get feedback iterate, rather than, If you couldn't do that, you have to have an engineering team.
That's like building prototypes.
And that's a much slower process.
The second one is honestly, I think having great people.
I mean, we had a great PM and a great designer, and they know how Duolingo works, what we think is good.
So they can independently make really quick decisions.
So I think staffing a great team with great tools was the big unlock.
When you look at that process and you look at the team that went into it, what does the structure of the product team look like today?
And how will that change over time?
This is a great question.
I think today the traditional product team is you're going to have somewhere between five to 15 people.
That's the traditional product team that has engineers, designers, and product managers at its core.
And it might have other functions like data science, marketing, depending on what they're doing.
What I expect is going to happen which is maybe the chess example is a good example is these teams are going to be smaller because you might not need as many people to do the task at hand, which might be just like.
We just need to create a prototype of this thing.
We just need to figure out if this is the right design.
We're probably not going to need as many people.
So we'll have smaller number of people on each team.
But honestly, I'm hoping by doing that, we'll be working on much more innovative bets quicker.
Because again, like this test example is a good one.
If we can do that, like 10x more, well, Duolingo will hopefully be 10x better over time.
Will it be 10x better or will it just be 10x broader?
Because there's the question of like, you can go for breadth of subject or depth of engagement.
And that is a challenge.
I love a Matthew McConaughey talk and this is why his show is so successful because I bring in very wise leaders.
But he says, you know, options make an enemy of us all. and actually constraints are sometimes good.
How do you think about ensuring feature simplicity without creep?
I think focus matters a lot in product.
I think you can really distract yourself by going too broad.
So our goal is not to 10x the number of subjects Duolingo teaches today.
Making Duolingo 10x better in everything we teach is the main goal.
That said, I think if we find a great subject that would be additive to Duolingo and we believe we can do it really well, I think you'll see us doing that.
Right now we teach four subjects like language, math, music, chess.
The goal is not to get to 40 subjects and do them all equally the same.
I think make what we have 10x better is the primary goal.
Can I ask when you think about design and when you think about product and eng, does what it takes to be 10x as a product person?
Change now.
Like you said that they vibe coded.
Do we have just hybrid people?
I think so.
I think what is likely to happen is there is clear work definitions of what an engineer does today, what a product manager does and what a designer does.
I think these are going to converge way more than what's happening today in technology organizations.
So I don't know if they'll converge fully.
I think that's maybe a little far, but they will converge in the sense that they'll be a lot more.
All of them will be capable of taking over each other's jobs in multiple ways.
Because AI tools will enable them to design faster, or PMs to start designing more, or PMs to vibe code more, or designers to do the same, or engineers to stretch into design or PM.
So I think these roles will converge more.
That's already happening, honestly.
If you were advising like a young product person today, 20, I'm your little bro.
I'm 23.
I know we both have the good looks to carry it off.
But what would you advise me as a product person entering this very different world?
Learn AI tools in and out today, and they will look different almost in like four weeks.
So I think you should be experimenting with how AI tools build product. like very heavily.
This is what I advise my own team.
Like we have basically told and aligned with our team, we will be AI first as a product team.
This is very similar to when companies went mobile first in like 2012.
It felt really artificial and weird because, like tech companies, looked at their user base and said oh, 1 of our users are on mobile, 99 is on web.
Going mobile first is kind of stupid.
Like we're investing in 1, but like quickly, in like three years, they realized that was the right bet to take.
But it was quite artificial.
Anyone that's going into products should bet on the same thing, which is you should build AI first, even though, when you use these AI tools and say, all right, create me a great app, you end up with like a dinky web app.
In my opinion, that will get solved in, I don't know, maybe 12 months.
But I think if you know how to use these tools to create product, build product and you can kind of harness their power to build great things, you'll come out ahead.
There's a difference between idea and action.
And I would like to say that with my team here.
Obviously, we have a big media company and I would like them to be bluntly trying tools.
And I'm like, should I implement an hour a day testing time?
Like jokes aside.
I think you 100% should.
What do you do like deliberately to make sure people actually are?
Because people are lazy, bored and tired.
There's going to be inertia, right?
I think it's like just saying, yeah, we should all use AI.
I think that doesn't actually get groups to be like, yeah, like I'll go do this tomorrow.
I mean, some people will because they're excited.
I think what tends to work is, I think, in one incentives you should incentivize people to automate repetitive tasks.
I think you should award and highlight people doing that.
That's what we do.
I think that helps.
How do you reward people?
One is recognition.
I think that's what we do right now a lot.
So if one of our product managers automates the task, we encourage them to share it with others and we clap.
I think that is one way to recognize someone that has done something great.
I'd rather have 25K.
You give me a clap?
I mean, money is another incentive.
So I think you can reward people with compensation.
That's another obvious one.
And I think the third is in an organization.
I think it's also in a tech company, or like Duolingo.
We bake it into how we think about performance.
So if someone is actually creating more output with AI, well, they'll probably get promoted faster.
What do we do in product today that we definitely won't do in five years?
Oh, there's a lot.
I think specs.
Companies write product specs that go from someone's idea, that turns into a product spec, and then that goes into design, that goes into engineering.
This process is like dead.
Like, why would you write specs if you can go from your idea to an exact prototype?
So this whole middle assembly line of creating product is a dead process.
Do you lose the design process?
Does Figma become obsolete in the world of rap, let's say?
This is a really good question.
I don't think it becomes obsolete.
But the way we use these tools and maybe it will be Figma, maybe it'll be a different tool is going to change significantly.
I think we'll go from basically text of an idea to a prototype of an idea, rather than everything in between.
Maybe Figma is the tool that we use to do that.
Maybe it's something else.
But then now there's like three, four steps to get there, or actually even more.
Like there's basically text to design and then design to code is kind of how things go.
Actually, there's text to spec, spec to design, design to code.
This is going to all merge, is my guess.
So this process is changing already.
Does this favor you as a big company?
Because traditionally... I like that you call us a big company.
Well, you're a public company worth like $20 billion.
We don't see ourselves as such, as you can see, but yeah.
Sure.
But it's $20 billion.
$20 billion.
Well done.
And so my point being there, traditionally, that would have been a bad disadvantage that you have.
That process, respectfully, is cumbersome.
And my team's seed don't need to do that.
Now you don't need to do that.
It's to our advantage.
I mean, we can cut process.
We're not doing process because we love process.
I think if we can speed up all of our product teams to us, that's an advantage.
Like we would love to disrupt our own process as soon as we can so that like, every individual can ship product faster.
Will you have more or less engineers in five years' time?
Oh, this is a great question.
Honestly, I don't know.
Well, that's not a question.
That's not a good answer.
Give me a yes or a no.
Yeah, I'll give you my take.
We will have more engineers.
However, the definition of an engineer might look different in five years.
Meaning, this hybrid I'm going to call it the prototyper role, where it's like a hybrid of engineer designer, PM will be way more common.
And I think that will be a lot of the headcount growth that organizations experience rather than I think right now.
When you say engineer, it's like software.
Engineering is like a very defined discipline.
I think we'll end up with these like undefined builders that can use AI tools really well.
And I think companies will employ a large number of those.
When we look at your cursors or your windsurfs or any of these tools that our engineering teams are using today, do they make 10x engineers 100x, or do they make average engineers 10x?
I think my read right now is more of the latter.
And also the magnitude of multiplication is way lower than like 10x-ing things.
I think it's maybe in the realm of like 10 to 20 more productivity of an average engineering capacity.
That's not that exciting.
No, I don't.
So my read is like, it's not that exciting.
And I think there's a lot of hype generated around.
Like you know, we're now replacing, like I think some companies are putting out statements like okay, like we're not going to hire engineers anymore.
I've seen really hot takes.
HubSpot said the other day that they now produce too much code to ship.
That sounds insane.
I hadn't heard that.
Yeah, yeah, that's amazing.
And then Salesforce, 20% of their code is AI generated.
Microsoft is 30%.
I think these stats are very much of the autocomplete variety.
Like what Curse is a lot of times good at is.
You start writing code and then you can autocomplete with AI.
That's great.
That's awesome.
I mean, it's not a 10x unlock if you can do that.
Like meaning if you're writing an article, if you can complete 20 of it with autocomplete, it's the same.
It's like, yeah, it's good, but it's not a massive life-changing thing.
What is a massive life-changing accomplishment in that product paradigm then or productivity?
The two things that really become 10x unlocks is when we figure out text to excellent looking pixel, perfect design with just prompting.
I think that is not there, but I think it will get there.
So for example, we think that Bruno as Americano is kind of the experience we should build.
So we build a character like Bruno that you can talk to and he's American.
He teaches you American English.
Let's say that's what we want to build.
Without touching a single piece of code, can we create the best animated Bruno character that talks to you in American English?
If we can do that, that is a massive unlock.
And I think this is not super far.
So that's like text to design and then design to code.
We have the design that we can build a prototype of that thing that is good enough to go into Duolingo and go out to real users.
That is a 10x unlock.
Hablo mucho espanol.
Muy bien.
Gracias.
I feel this is going well.
The joys of also saying end of day is you're just like, ah, fuck it.
Why not?
Why not?
It's not 10 a.m.
I can't be asked to be formal.
When you have that unlock, the amazing thing is it means that bluntly, everyone can create product to an extent.
I think we're about to enter a wild time with this.
Yes.
Yes.
Does that excite you?
I'm so excited.
Or worry you?
Because it now means that you have infinite supply of product.
Yes.
And that means that discovery becomes even more challenging.
Yes.
On a personal level, I'm super excited.
And for the discipline of product management, I'm also super excited.
Maybe you've had multiple folks on this show, and I've heard this come up, actually.
It's like PMs do have imposter syndrome.
Because they're like, when they're interacting with engineers, they're not technical enough.
When they're working with designers, they're not creative enough.
These tools level up everybody to have much, maybe not at the same level, but to a similar leveling field where we're all operating with like strong tools.
So it helps PMs be a lot more productive and independent for themselves.
So I think you uplevel product managers, which is good.
Like for my function, I love that.
But you uplevel everybody to be way more productive.
This unlocks much easier product building.
And I think this is very similar to just like a 10x productivity gain, where the best ideas will eventually win.
So creating 10x more prototypes doesn't?
I mean it creates maybe chaos at first, but then the users eventually get the best of hundred ideas rather than best of five ideas we can ship today.
So I think, just like creation goes up and when creation goes up, the best things that go out to users are just going to be better.
I agree completely.
When you look at the tools again, who wins, Cursor or Winsurf?
I'm going to say Cursor.
Why?
Because internally at Duolingo, I'm seeing way more usage and way more creation come out of Cursor.
That said, maybe we haven't experimented enough with Windsurf.
Can I ask you?
Everyone says to me I look at these tools and you know that neither of them have lock-in or defensibility.
And as we discovered from my apt due diligence on some companies we mentioned before, I naturally go.
It sounds very interesting.
Tell me more.
But I don't actually understand why.
Why did they not have defensibility?
And is that wrong? i feel like these statements come out of is there a data lock-in so like if you're feeding it more data is the tool getting better or is there like a social lock-in where i don't know you're collaborating with folks and you know switching is painful for example a lot of enterprise tools have heavy data lock-in like we use jira for for example task management in the company it's very hard to switch out of because all of our history has been now fed in there and switching out means we kind of have to leave that data and go elsewhere we use jira for A la poubelle.
I'm going to introduce you to Carrie at Linear after this.
Okay.
That sounds good.
Yeah.
And he is amazing.
And this is called Venture Value Ad.
Here all week.
And honestly, they're brilliant.
They're much better than Jira.
I mean, I've seen their designs.
It looks awesome.
I'll say the hardest thing is to switch out of the thing we have.
So, even if we love the product, switching out is going to be a big debate because there's a huge cost.
And I think people bring this up because you can switch from cursor to windsurf.
You don't lose anything, right?
If the windsurf is better tomorrow morning, well, we'll start using that.
Maybe these will change like meaning the more code you build with one will just make it so much better.
For example, now ChatGPT has memory.
I think that is an amazing lock-in for ChatGPT, because now, since it remembers our chats, it can now give me smarter answers.
That is a big lock-in for me to not use a different product.
I do have to ask, because you mentioned at the beginning, like being central to the monetization.
And I spoke to Severin and he said that, bluntly, it was your fault that they waited so long.
Jokes aside, why did it take so long, do you think?
And how do you reflect on that decision?
The hindsight bias is easy to apply to our past and say, oh, we waited too long.
But I think at the time, a few things were, I would say, maybe headwinds to starting to monetize.
One of them is Duolingo obviously has a very strong mission today, and that certainly was the case then.
When it first came out, you can find Luis's talk.
This was a lot easier to find on YouTube, and I think it was on our corporate pages.
I think we removed it now because we moved away from that.
But the monetization plan for Duolingo was going to be, everything was going to be free.
So as users went through did language exercises, they would do translation and then we would crowdsource those translations as a service to anyone who wanted to buy it.
So if CNN wanted to translate an article from English to Spanish, they would feed it into Duolingo's system.
That article would be broken up into chunks, fed into users as an exercise.
And as crowds translated that as an exercise, it would populate back a translation.
That was the original plan.
And, as Luis announced, Duolingo.
He also said we will never have ads, never in-app purchases and never subscriptions.
And funnily enough, those are the exact three things I did as part of my job at Duolingo.
And your lesson from that is never say never.
That's lesson one, that things change and your view on these change.
But I think two.
The reason there was a lot of resistance in the company for monetization is we launched with the view that if you did these things you would screw up your amazing, great consumer product.
I think what honestly, my job eventually turned into is obviously building these things, but also changing the organization's mind that there is a way to do these without going against our mission of providing access to free education and and doing them well.
The big unlock there was at the time and honestly, mostly today, all education apps would lock all of their content behind the paywall because that's so straightforward.
Like, why did you come to this app?
To learn?
Well, what are we going to charge for?
Let's charge for the learning content.
We did the exact opposite.
We're like learning content is completely free, but we are going to monetize the bells and whistles.
We're going to do freemium.
So most of the stuff will be free, but the premium features could be part of the subscription.
Even our investors at the time was like, that's not a good idea.
I've never seen that work.
If I was one of your investors, I'd say, hey, growth is everything.
Engagement is everything.
Retention is the fucking holy grail.
Don't stick in ads when we don't have a massive company yet.
I think that was also the common wisdom at the time.
It was the Facebook model.
Just grow.
It doesn't matter.
Ignore monetization.
And honestly, it was good advice.
But I think what we ignored is you can still have high retention.
You can keep growing your product, but also monetize at the same time.
I think it felt like it would kill things and it didn't.
It was fine.
Like, I mean, putting ads on Duolingo, if you do it tastefully, it doesn't kill your retention.
So you can still have very high retention and monetize the bells and whistles, but have high growth.
Can you just take me to the moment where you were like okay, we're going to move from a zero revenue company to trying to make money.
Yeah.
Was there a meeting?
Can you take me to that moment?
It probably wasn't one meeting, but there was a few key moments where I remember being like ah, how am I going to figure this out?
One of them was sitting with Luis in honestly, probably one of my first weeks.
And I'm a very early product manager trying to figure stuff out.
And I'm like, all right, Luis, like here's our ads roadmap.
I calculated how much money this is going to make.
Here are the experiments I'm going to run.
And he's like, all right, what is it going to look like?
And I show him like the prototype or like we look at it.
He's been looking at it.
He's like, these ass look like shit.
Shit is his first response.
He's like, I'm not sure if we should do it.
And I'm like, okay, where did we?
I hear you.
I understand that this doesn't look like the pristine every other page of the app.
It's different.
It's another advertiser's property.
But we can improve the look, we can change the design.
Anyway, it was an iterative process from there to be like he's like let's test, let's see what happens.
And the other worry was like, oh, this is going to kill retention.
We tested, it's fine.
It's like neutral on retention on the first experiment.
And we're like, okay, our users are not running away.
We're improving how the ads look and when they're surfaced.
And then I think over time, it started snowballing to be like, okay, we can do this.
We can do ads.
Once we did ads, the second phase was well, No ads was like a request.
People started sending us being like can I pay you guys to remove ads?
We're like, well, let's try that.
That's a good start.
Before we move then into the paid, then do you accept that you cannot do ads without hindering user experience a little bit?
There's certainly a cost.
I had it the other day and you have the timer in the top left.
Yeah.
And I'm like, listen, it's annoying.
I stay, of course, and I love dueling are just the same.
I can't say I love ads as a user either.
So I think you sacrifice because it is not a property you control.
Meaning if I bring an advertiser, they have to show their asset.
And the asset comes with a five second timer on, I don't know, a video for selling you shorts, whatever it might be.
It's a cost, I think, that you take.
That said, there are many different ways to do ads.
So I think treating ads as this, like black box carrier of like it's only bad and it can only hurt, is a bit too simplistic.
What's the lessons on what it takes then to do ads well if it's too simplistic to just black box?
I think ad load is the key thing there.
We stick to one.
You will only see one ad at the end of a lesson.
And that's not mid-lesson.
That's not start of the lesson.
You finish your lesson and you take a breather.
That's the only time we show you an ad.
Is what we aligned on versus you can put a banner ad on top of you know there's many tasteless ways to do it.
Which optimize for cash though.
Correct.
Because you can charge more for in-play versus post.
Correct.
But I think that is short-term thinking at the end, because you maximize a lot of maybe revenue now because you're putting a lot of ad clicks across the app.
But the moment you ruin your free user experience, you kill your growth engine.
Can I ask you, do you get paid on a views or on a clicks?
We get paid by impressions.
So views, I guess.
So the rate that these ad networks pay you depend on how clickable these impressions are.
It kind of both touch eventually the ad revenue you make if you're running in programmatic ads on your products.
Okay.
And so I then say, you know what?
Fuck this.
I want no ads.
And you're like, hmm, what happens then?
A lot of users obviously were like, I love Duolingo, but you know, I'll pay you guys, whatever.
Can I remove the ads?
That no surprise because that's a common model.
That was the idea for our subscription.
So it's like we had ads, we had a no ad subscription.
That was the start.
And what we saw also at the time was there was a lot of user love for Duolingo.
It sounds like you also love Duolingo.
I just wanted to contribute something back.
So I bought the subscription, even though I didn't know what was really in it.
No, I'm not that Christian.
Definitely not.
The $20 billion market cap made me feel.
I was like, no.
But basically the no ads was a starting point and it was feature light, honestly, at the time.
The subscription was kind of a nice to have more than anything else.
If you were advising me, as a founder, on doing that premium subscription, what would you say, having been through that?
It really depends on what business you're building.
But if you are building a consumer app business, I would very, very highly advise you to build a freemium model, meaning you should have a very great free user experience without asking anybody to pay.
The reason for that is, if you have a premium product, let's say it's all paid you can only use the app if you subscribe, or it's very limited.
What you have to build is also a paid marketing engine.
So you have to spend a lot of performance marketing to acquire subscribers, generate revenue from them and spend it back on performance marketing.
That's how paid products kind of have to grow.
Free products, if you build an amazing free product, it takes way longer.
But what you end up building is an organic growth flywheel.
Meaning you tell your friends about Duolingo.
I don't know if you do.
I tell my friends about Duolingo.
Let's go off of me.
They can go experience the full thing for free.
Whether they pay or not doesn't matter.
If they like it, they will also tell their friends.
It just expands the entire pie of people who can use your product without having to pay, and we have a lot of users that use the app for free for five years.
Eventually, one january they're like oh, i'm gonna go ham on my french this year i'll subscribe, it doesn't matter.
But they use duolingo so to build basically a large user base.
And if you want to build a large business, it's way more defensible to build a great free user experience and do it freemium rather than full premium.
The other problem with premium is competition.
So like the reason Duolingo has grown quite a bit in the past 10 years, or even gained initial success, was because it had an amazing free user experience.
If you created the exact equivalent of Duolingo today, what you would probably do is okay, like you wanna make some money, apps have tried this.
Actually you kind of copy Duolingo exactly as is.
You make it fully paid.
You're like, I'm gonna go fully paid and spend on paid performance marketing.
Well, what happens is the users who you can acquire will land on your app and say okay, you asked me to pay 80 100, whatever it is, or 10, or I can just go to Duolingo and do it for free.
So your paid costs go really high.
So it makes it really hard to compete with a free, amazing product.
So this is a very roundabout way to say if you're building your consumer app business, I think you wanna make sure you have an amazing free growth flywheel makes you defensible and it kind of spreads your product without you having to spend marketing money.
How do you think about the right time to hit them with a paywall?
You want to build the trust, the love, the loyalty without being like, okay, level one paywall.
Exactly.
So I think it's twofold.
One of them is honestly like hitting with a paywall is kind of the wrong approach, in the sense that I think you want people to discover the paywall by experiencing features.
That is the best way to honestly like convince someone to buy something.
Meaning.
Let's say, i would like to convince our users to buy duolingo max and its key feature is video call.
The best way to do it is have them try video call.
Actually, so a lot of our ways to do that is maybe show them a video of how video call works and then say if you're interested, here is duolingo max.
Or have them go into the experience, have a call themselves and then at the end say this is part of Duolingo Max.
That is kind of the best way when it's like contextual and users discover that there is something to be paid to like experience.
I think just like having people showing kind of what I would call like generic ads works way less better.
And it looks spammy in the app.
So we try to do everything contextual.
For example, if we show you an ad, we will say right after there's an ad-free version if you'd like to upgrade.
And that is in a way like you've discovered an ad.
Okay, maybe you didn't love it.
Okay.
But you can also find an ad-free upgrade.
I just wasn't expecting Pornhub Premium to be an advertiser by that.
Honestly, for a second I was like, oh shit, did that happen?
Oh no, the ad network surprised us.
It was the first useful ad I've had on Duolingo.
If the only ad you clicked turns out.
If the only ad I'm like, sign up, sign up, sign up.
The joys of doing the show is also like you hear the show and you're like, God, it's so serious.
And then behind the scenes, you're like, what a ride.
Behind the scenes, it's born up in you.
Yeah, it's much more fun that way.
Can I ask you when you look at the different paywall pages that you've experimented with, are there lessons on how to bucket different offerings?
Oh, yeah.
What are those lessons?
There's a lot.
The biggest one I will say is what you say in your copy in strings makes a massive difference.
And obviously you don't want to kind of sell something that you don't have.
But how you talk about entering that experience makes a massive difference.
I'll give you an example.
We have tested many ways to on the button that is going to start your subscription.
We tested probably endless number of things.
Turns out, if you say try for 000, which is not a normal human like I wouldn't talk to you and like would you like to try something for 000 pounds?
Like I wouldn't say that.
But in the string, if you write multiple zeros and you hint at the fact that there's a free trial with multiple zeros, conversion goes up a significant amount.
Overstart your free trial over anything else.
Start my free week, start my free trial.
Start your free trial or come up with any other combination.
We've tested probably many different things turns out if you put multiple zeros.
I think it like lights something up in the brain that you're like oh yeah, I get it, it's very free.
It's because it's like 000, so Copy matters.
That's one.
Because with subscription there's natural aversion to entering a thing that might be hard to get out.
We also have seen talking about how easy it is to cancel is like an unlock in conversion.
Do you guys send reminders?
We also send reminders.
Because I signed up to Max and I signed up because of the reminder.
There you go.
So that actually makes a big difference.
And honestly, this is one of those changes I love because it's user friendly.
If you don't like the experience, I want to remind you to cancel.
We don't want people who forget and pay because of it.
But that relieves the anxiety of entering a subscription free trial.
Anything about the different packages, like give people just one option paid versus not paid, max versus premium versus free, anything on like choice.
I'm always fascinated by the paradox of choice.
I think comparison is super important.
What we've also seen is if you give one, it's for example, we have multiple packages now.
There's super, there's family plan, there's max.
We think you are a great fit for super individual.
If we only show you one package and you have no anchor for different packages, it converts way less because it feels like if that's not for you, you're just gonna decline.
If we give you more than three packages on one page, it's too much like, too much choice.
You also convert way less.
I think two to three is we've seen as a sweet spot.
So you want some anchors to compare to.
And most of the time, if you give a choice of three, you want the middle option to be the option that you really recommend, because that makes choice very easy.
Humans love.
I mean on a menu too, if you think about it, if you have the lowest price, middle price and highest price, middle price feels like the safe choice.
If you don't know what to choose, very similar to if you're kind of offering someone a subscription choice.
I think you wanna make the choice easy by recommending one.
And recommended one being the middle one feels safe to choose.
What percentage of people take the recommended?
Very high.
I don't remember the exact percentage, but certainly more than, it's the majority, more than 50%.
Wow.
Yeah.
Any other big lessons on copy matters?
I'm sure you've been through many different iterations.
Oh my God, we've tested so many things.
Yeah, copy matters.
You know how every company eventually puts like a checklist in front of you where like oh, if you do this package, you get like two checkmarks.
But if you do this other package, you get like ten checkmarks.
This has become like a very common way for subscription purchase flows.
Weirdly, that also works because it's the psychological difference you can see visually.
I'm like, oh, what I have now has one check mark.
What I don't have has like 10 check marks.
Well, 10 check marks feels better.
It feels like you're getting a lot.
And I think that feeling of getting a lot feels better.
I think the other thing that we've learned is there's a lot of emotion on purchasing anything.
Certainly something that is an upgrade to a consumer experience, like Duolingo, going from free Duolingo to super Duolingo or Duolingo Max.
And I think you want to convey that emotion in your design.
Meaning Duolingo takes pride in like a fun, being fun and animated.
While our flow is being animated on Duo doing funny stuff through the flow.
It works well because it like conveys that this is not just a boring subscription as a service purchase.
It's like part of your experience and it's an upgrade.
So anyway, you can use illustrations, animation and things like that that make it a fun flow that moves metrics at the end and increases conversion.
What percent of revenue is subscription today?
Today, it's more than 80%.
What revenue line does Duolingo not have today that will be meaningful in five years?
We do have this revenue line, but it's incredibly small today.
But I think it will be way bigger is merchandising.
We have a store.
You can buy Duolingo merch.
Our most selling item is the Duo plushie.
It's like a very funny plushie of Duo.
I think we are one of those brands where our characters are quite Duo is quite recognizable at this point, and we're trying to build more IP around our characters.
There's a lot we can do on creating better merch and expanding our merch offering.
Surely content is it.
You want to become the Disney of the next generation.
Why not?
Exactly.
We would love to become the Disney of the next generation.
I think that is hard for sure, but that would be awesome.
But I'm actually the founder of this idea and now company to That was my idea.
So thank you, Lewis.
I will take over.
Would you say that you're an education company, an entertainment company, a content company?
I know it's all semantics.
Honestly, neither is the reality.
We have maybe somewhat of an identity crisis is also the reality.
We don't identify as an education company, but we also don't identify as a game or entertainment.
So We're kind of in our weird own league and own market that we've created.
The reality is what we compete for, if you're like, OK, but what are you?
Who do you compete with?
We eventually compete for screen time.
We would like to make your screen time very useful for you to learn something and get better.
The reason we don't fit into education very much is if you look at the education space honestly, this is a bit of a hot take, but it's all boring ass products.
So I think we would like to not be a boring-ass product.
As you can see, we try to do everything super fun.
And I think if you say, well, you guys do a lot of fun, then are you a game?
Well, we're not a game either.
We also want to teach you something.
So we're kind of in this zone, league of our own is how we see it.
But what we eventually compete for is your screen time.
So everything that takes up your screen time that might be social media, that might be gaming, that might be streaming, whatever it is at the end is what Duolingo competes for.
It's also a hard one on multiples.
Gaming multiples are shocking.
Education, tough.
And also that's not the product we're building.
We're not building just for entertainment, but we're also not building just to educate and ignoring the fun element of it.
So that's why it's like right in the middle.
Okay.
The final element of this talk.
We've got the ads, we've got the premium, and then we have in-app purchases.
How do we think about what worked there and the biggest lessons to doing that so successfully?
So in-app purchases for us was very similar to how games have designed in-app purchase mechanics.
So we have a virtual currency.
It's called Gems.
It's not named after me.
It's the actual diamond Gems.
Sure, sure.
Yeah.
But yeah.
But you were the one who came up with monetization.
So yeah, yes.
Gems was one of the first features I built, actually.
So there's some tie in there.
In 20 VC, we hand out Harry stars.
Yeah.
I didn't come up with it actually.
No connection whatsoever?
No connection at all, but it just happens to be that.
You do know if you're going to listen back to this and go, this guy was a fucking joker.
I am curious what you cut and what you keep.
So it's going to end up really being interesting.
It's going to be fun.
It's a surprise for you and me.
But so you give gems.
We give gems and you can either buy gems or earn gems, and you can spend those gems in virtual features.
So you can, for example, one of the most popular features is a streak freeze.
So streak, as you might know, is a very popular feature.
People take pride in their streak.
They don't want to lose it.
But if for one reason or another, you can't do Duolingo that day, you can use a streak freeze.
This is a very valuable item.
People love it.
And if you earn gems, you can spend your earned gems on it.
If you're out of gems, you can buy gems to purchase a streak freeze.
Items like that exist and that's part of our gem economy.
Have you ever had someone sorry to interrupt you?
Have you ever had someone lose a streak and then contact you and be like listen, i was tracking up the like everest and you know i've dropped my phone and and so i couldn't do my streak all the time.
What's your favorite story?
High volume request.
I think that most of the stories are time zone switches.
And honestly, this has happened to me.
So I actually have full empathy for this where I actually did my lesson.
But the reason is I flew to Australia.
I was on the plane for 14 hours.
And when I landed, the day had already moved forward.
So I lost my streak.
So you had to call up the head of product.
Completely get it.
And we fixed those.
So like if you actually reach out to customer support and they were like, I have this legitimate reason.
We're like lenient.
We're like, of course, like we'll fix your streak.
Don't laugh.
Do you have to prove it?
Honestly, we really don't ask that many questions and we're lenient.
But yeah, so there's like time switches and there's like, I was on a mountain with no connection.
What was I supposed to do?
And of course, we understand.
You're like, get off the fucking mountain.
We understand.
But also, yeah, like use a streak freeze maybe and then get off the mountain.
Buy the freeze.
That's why they're there.
But okay, so lessons on how to do that effectively.
Is it just create addictive apps that people don't want to?
I think the core experience, of course, matters a lot.
Addictive sounds like the bad version of this, but habit building like you want to build good habits.
And the reason we use that term is like if you want to learn the language, you have to practice, if you want to be fluent, thousands of hours.
And to get to thousands of hours, you need a daily habit.
You can't like binge your way to fluency.
So that's why everything at Duolingo is built around a daily habit.
Once you've done an experience that people want to use daily, you have good daily retention.
Then you kind of layer on these benefits that are kind of extra, like streak and a streak freeze.
You don't need it, but it is really core to your daily habit.
And then freeze is this extra benefit that you can monetize.
That's how we've thought about it.
Totally get that.
That's fascinating.
What do you think people do wrong with in-app purchases?
You see so many.
You're a great product mind.
What do people do wrong?
I think people do really aggressive things and that ends up hurting the experience.
So for example, streak freeze is so extra, right?
If you don't, nothing really changes.
It's all up to you.
Like your French is going to be the same either way.
I think what sometimes apps do is they make really core functionality and in-app purchase.
And I think then you start killing the fun, the retention producing elements of your app.
So imagine if Duolingo was every new unit of a lesson was an in-app purchase.
It was like a dollar per unit.
You would get pretty sick of it pretty quickly.
It's like buying a chapter of a book for a dollar every time you could do it.
That's a monetization model, but it's a pretty stupid and bad one.
So I think you want to find things that people find joy in buying that doesn't hurt the freezer experience.
I think it actually goes to trust, which is when you do that I actually lose trust in you and it makes it a transactional relationship that I don't even feel.
Though that I'm a premium, like the one hit pain of.
Like I'm a premium fine, you pay the toll and now you enjoy the holiday and you feel good.
But if it's like consistent pain inflicted, fuck that.
No, it's very annoying.
Like we talked about with ads the same way, tasteful monetization really matters.
It builds user trust.
And I think user trust goes a long way for the business because you tell your friends they download the app.
What are the metrics that matter most to you?
There's two metrics that drive the business.
So if you really squint at a business like Duolingo, you have to look at two metrics.
One of them is daily active users.
And the second one is revenue.
But if you just want to spreadsheet a business, you have to look at DAUs and revenue.
But I think for me, what matters is retention.
Well, because that's like company metrics.
Correct.
But if you think about product metrics, the inputs that drive the outputs.
Retention.
Retention is everything when you think about building product, because it is the best metric to distill down if people are finding value or liking a feature or a new app, whatever it is.
What is your daily retention?
I mean, if it's a daily app, daily retention.
If it's a weekly app, you look at weekly retention.
For us, everything we build is around daily usage.
So it is daily retention.
So if we say, for example, chess, we just launched chess.
The thing we're looking at day in, day out is what is the daily retention?
Because if it's really high, well, it's going to compound to high DAUs.
We know it's going to.
For example, you could look at DAUs, but DAU has many inputs that are muddying the waters of.
Is this a sticky habit building product.
So daily retention is the king of all metrics for consumer products.
And just so I understand, daily retention is do I come back tomorrow?
Correct.
High bar.
High bar.
Exactly.
With mobile products, that's why it's so important.
Because you have many different things that can steal your time on your mobile.
You don't, maybe.
I didn't have anything.
You're like, I don't know what apps are.
I don't know what apps are.
Okay.
So we have daily active users are like cool.
What is good?
I think it's going to really depend on your product.
There's no like easy benchmark you can put.
And also it really depends on what your user base looks like.
If it's all new users, you have a different metric than if you have a bunch of users that are already settled users.
I'm going to call them like Duolingo where they've already been using for like three years or above.
When they're settled users do those cohorts, in terms of usage, look quite like Candy Crush users, like the very, very settled gaming users.
I think so.
I'm not knowing Candy Crush's metrics myself at all, but what we intend to see is, if you've reached a 30-day streak so you've built up a 30-day streak your retention is very close to 100.
It's not 100%, but it's very high because you've built a habit.
It's not crazy because you've done it for 30 days without breaking your streak.
You have built a habit, so you're very likely to continue with that habit.
But if you look at retention of users who are streak less than seven days, it's way lower because they actually haven't built a habit.
Or maybe they're unsure of doing it or they're just playing with it.
One way or another, they haven't built a habit.
Do you see wildly different retention numbers in paid versus free?
We do.
However, the difference is a little hard to parse on its causation versus correlation.
Do they retain better because they are a subscriber, or they were likely to retain because they were more engaged and that's why they decided to subscribe?
It's hard to parse in the data.
But if you look at just pure users who choose to subscribe their retention and users who choose not to subscribe retention, is much higher.
What caused the biggest change in retention from a product perspective?
Three things.
Wow, that was very... This is something I've thought about a lot.
You had that answer ready to go, didn't you?
I was like, that's a good question.
You're like, I got that one.
So one of them is the streak.
No surprise. having the streak and making the streak exciting.
So when you finish your first lesson, there's this flame animation that lights up.
There's a haptic, there's a sound.
We make the streak like one of the most important things on Duolingo.
The details being the details, it's exceptional, really.
Details matter a lot.
So every tiny thing that might not even make sense to talk out loud, but when you see it, that made the streak more exciting, has helped improve retention actually.
So the streak is this incredibly important mechanic that instills daily habit, helps you learn a language or learn a subject.
That is one of the biggest things for retention.
I would say second as a feature, it was leaderboards.
Turns out no surprise in hindsight, but if you see your name ranked on a leaderboard, there's this human tendency to not be at the bottom of that leaderboard but instead move towards the top.
So a lot of our learners, even though when they came to Duolingo they didn't come to compete in French.
We have a leaderboard feature where you see your name among a cohort of similar learners.
You want to rise to the top.
A lot of users want to, at least.
So that drives a lot of usage, a lot of retention.
Who's like, no, I'm happy languishing at the bottom.
I think some people are like, I don't want to compete.
But a lot of users.
When you see your name, you're like I would rather move to the top, not to the bottom.
I'm one of them, for sure.
For sure.
Do not understand the people I'm happy to be there.
Well, you're not getting hired, are you?
It's like the worst hiring trait ever.
Thoroughly uncompetitive, happy losing.
Exactly.
Exactly.
Okay.
Love that.
So that's the second one.
Third is push notifications, actually.
For every app, it is an incredible mechanic that you have to figure out how to make amazing.
Still.
Still.
Still.
Maybe 10 years ago, email was a heavy channel.
Email is a dead channel for most products at this point.
Including Duo.
Dead is maybe an overstatement.
We obviously invest a good amount of time in making our emails great.
For the email team, we still love you.
Don't worry.
Yes, we do love the email team.
I hope you enjoy the dark basement with the leak in the corner.
It's fine.
Lewis will let you out next year.
Yeah, where the push notification team is like popping champagne.
They're at the Belmont in Portofino, celebrating 20B.
So yes, what does it take to do push well?
You can get incredibly complicated and this was kind of news to me over my career incredibly complicated with limited real estate and copy you right in a tiny push notification.
For example, Timing is a key factor.
Content is a key factor.
Personalization of that content is a key factor.
Emoji usage is a key factor that moves metrics.
So if you get into the full details of really optimizing push notifications, which we have, you can do a lot with it.
And every year we're like, okay, we invested a lot.
Are you gonna keep improving?
There's a lot to do on improving push notifications.
I'll give you one insight.
That was kind of like tested in and out and then this turns out to be the winner we schedule.
So let's say, you did your duolingo lesson today at 1 pm.
We schedule a notification almost exactly at 23 and a half hours later from that time and we've tested many different timing and the notification will remind you to practice your spanish because we you just did a spanish lesson.
We've tested many different times but the 23 and a half hour works way better for retention because, turns out, humans do lessons.
If you're a morning person, you do in the morning.
And if we schedule it almost 30 minutes before you're ready to do your morning lesson, you're very likely to do a Duolingo lesson at the same time.
So based on your behavior, we time the next notification that has done really well for retention.
So when you get into like this type of experimentation of like, how do we actually figure out how to make it better for each of our users?
There's a lot you can do.
Wow.
That's fascinating.
Okay.
So that's like timing.
What emoji usage you said?
Yeah.
Well, emoji breaks up pattern is why I think it works.
So a lot of notifications suffer from, they fall into a pattern and now a sea of everyone sends so many notifications that, like it all blends in.
So you want to do something that is more interesting than the 85 other apps sending you notifications a day.
Emoji is one way to break pattern.
The other thing we do is kind of go full unhinged on notifications.
So we write them from Duo's voice and we make it sound like Duo is basically screaming at you or to do your not always screaming, sometimes sending you a subtle message, but it's like a personalized message rather than Duolingo saying do your Spanish lesson or practice now.
That kind of doesn't go very far.
But if Duo says, yo, what's up?
Are you doing Spanish today?
And it looks like almost a message from Duo.
That triggers you to be like okay yeah, I'll respond to Duo rather than this app that is sending me sanitized notifications.
It's like a replacement girlfriend, isn't it?
You don't need someone to shout at you.
You've just got Duo being like, yo, bitch, get your Spanish done.
I mean, you would respond to that more.
You respond because it's different.
It's different.
Yeah, it's Seth Godin's The Purple Cow.
Yeah, it's different.
And it doesn't feel like a company is sending you a thing.
It's a human would send you a thing that way.
We're going to do a growth session, me and you.
OK, so we've got a couple of fun programs we can do.
We can do a Duolingo school in New York, physical school, great for community, great for like vibe.
Did this come from Severn?
This is his like passion project of like, yeah, Duolingo should have a physical school.
I agree with him.
Don't.
I'm one of the founders.
You wouldn't know, but we were talking about this about 15 years ago.
It's founder talk.
It's founder talk.
You can do a duo movie.
Go in the Disney direction.
Okay.
I do like that idea, for the record.
Really, really cool.
I think if we want to build characters, we have to expand more beyond the app.
Yeah.
Or you can do duo date.
Duo date.
Yeah, which is like dating, connecting people, learning the same language.
You've mentioned before people getting married before who maybe met on leaderboards.
Which one do you love?
Which one do you hate?
I hate the school one, if I'm being honest, because building a physical business seems like that's not our strength.
I like the movie idea.
I love the date idea.
That's how I would pick.
Wow, you love the date idea.
I do love the date idea.
I think it is the riskiest one because what could go wrong by just building a dating platform?
However, there's already signs that people date on Duolingo.
So the best way to build product is capture latent demand that you just can't do on your product today.
We have couples that got married because they met competing on a leaderboard and then they found each other on Facebook and got married.
This behavior is already happening.
So it's not out of the question for us to figure out, well, how can we do matching better?
Wow, that's pretty cool.
I mean, it is out of the question for the record.
We're not doing that, but I love the idea.
There's so many interesting lessons to be had from hindsight.
When you look at the product decisions that you didn't do.
What didn't you do?
That with the benefit of hindsight you wish you'd done.
We talked about the obvious one, so I'll start with that, which is we didn't monetize in 2014.
We started monetizing kind of two years later than we could.
So I think that's one.
We talked about it.
We don't need to unpack it.
I also think we, this is in hindsight easier to say, we waited too long to apply the Duolingo magic to new subjects. meaning we first launched a subject that is not language, that was math, and then music followed fairly quickly, but math, we first started working on math three years ago.
We didn't have to wait that long.
We kind of had the Duolingo mechanics in place.
We could have honestly started our journey in figuring out other subjects earlier.
In hindsight this is very easy to say, but this is a long journey to figure out product market fit for new subjects.
However, we have the competency.
We kind of knew eventually we would do something like this, but instead we were investing in various projects that now, in hindsight, we shouldn't have.
Like, for example, we were trying to build a school's dashboard for teachers to use, and that is a complete different product problem.
That is a very hard problem to solve.
Instead of doing that, we should have probably done math and music and chess way earlier.
I was going to ask, what did you do that you wish you hadn't done?
A school's dashboard.
Yeah, there's another one.
I think that at the VC level, this makes sense.
But when you get into the product level, it doesn't.
Which is, why is Duolingo not catering to language teaching in the classroom?
Like, why does Duolingo not have a product that fits into the classroom?
So we wanted to solve that.
So we tried to come up with a version of our product that teachers could assign, teach certain parts of a curriculum and assign homework through the Duolingo interface.
And then they could track if assignments are being done, et cetera.
The problem with this is every language teacher wants to teach a different way, and the Duolingo curriculum does not fit any of it.
So you end up having to customize how people should go through Duolingo.
And that's not the product we have.
So, even if teachers could assign assignments, that doesn't match the textbooks they're working on in the French classroom.
Anyway, it didn't work.
How long did you assign to it?
Oh, we worked on schools.
By the way, this product still exists.
It's just like not a product.
It doesn't have product market fit.
So you can still use it?
You can still use it.
Like the school's dashboard exists.
Like if you'd like to try it out, it exists.
It's there.
It just hasn't found the adoption we wanted to because it doesn't fit the classroom very well.
But we worked on it for, I don't know, three to five years, maybe.
Why still have it live?
Why not pull it?
Because there's still usage.
It's not zero usage.
Some people find use out of it, but it's like thinking about the market of classrooms teaching languages.
It's a very small percentage.
And you have no one on it, like no maintenance.
Right now, there's no investment.
That doesn't mean we won't invest again in the future, but today we've chosen not to.
It's a complicated problem.
Anything else that we did that we should know?
There's a big one now in hindsight.
It feels like, oh, my God, why did we do that?
So right now we're talking about how video call and Lily as our speaking feature.
There was a short lived moment where we had a high tier subscription, which was basically an Uber for a Spanish tutor.
So you could press a button to speak Spanish to a Spanish speaker immediately.
So rather than scheduling, you could press.
So we figured out how to connect you immediately with someone online.
And it would be like a two, three minute condensed conversation.
Honestly, at the idea, I was like, oh, this is great.
Sounds pretty great.
Yeah.
Authentic, natural, cool.
Exactly.
We have solved the Turing problem, which is the biggest problem is scheduling and it's like expensive et cetera.
We made it part of a subscription so you could do unlimited calls on demand.
Very cool idea.
And also users have been asking us to do this for years.
Like we're like, okay, finally we figured out how to do it.
So there was two massive problems in hindsight now we know.
One is it is super scary to actually press a button and talk to a human immediately in the language you're not proficient in.
Even though you think you want it.
What we've seen when we observe users doing it, they press a button.
They're like, OK, I'm ready.
I'm going to do it.
And then the person would dial in, start speaking Spanish, and then they're like, I
Hang up, go away, is what would happen to most users that ever tried it.
So it was a scary experience to be thrown in.
And that's why AI is way better for this, because you can stumble, you can talk shitty Spanish, you're not embarrassed.
A lot of users found it embarrassing to stumble when there's a human looking at them on video.
That was problem number one.
Problem number two is you have no quality control.
So you can't run an A-B test and say, hey, let's do the opener in English.
I mean you do, but then you can't control if the human-delivered experience is what you thought it would be.
So we found it kind of impossible to operate a human-based product experience.
It's not impossible.
It's very hard.
And around that time, GPT-3 came out and we're like, oh, hold on.
We can solve this.
So we pivoted away from that.
What would you most like to change in the product?
But for some external reason, shareholders being public execs not wanting to do it, you can't do it.
I would like to make Duolingo way more social.
And I don't think there's any block or anyone against it.
I think we need to figure out a way forward and the best way to do that, and then staff teams accordingly to solve that problem.
So right now, Duolingo has social features.
So I can add you as a friend.
If you reach a certain streak, I can congratulate you.
You can do it back.
So we have this social layer, which is very congratulatory and nice.
We can do a friend quest together.
We can do friend streak.
So it's a lot about collaboration and congratulations. to achievements.
That said, I mean, I think this is kind of why I loved your dating idea.
It builds on the social potential of Duolingo.
Duolingo today taps into this reality of you're doing something good for yourself and there are others doing the same.
But if you go through the experience, you feel like it's a single player experience.
You're doing it solo.
And language learning is inherently a social thing.
Most people are learning language to do something social with others.
Maybe order their coffee in Italian when they go to Italy or date someone or whatever it might be.
There's a social need.
And I think social just fits into language quite well.
So we have to figure out how to make Duolingo more social.
We can go in many different ways, but I think that's one thing that we have to invest more.
In which way would you most like to go?
Dating is actually not the direction I would go first, but I would love to go in a way where we build better features that you can do with others.
So, for example, gifting is one area people know that I'm passionate about.
So, for example, if I can send you more virtual items when you do something well, you can send something back.
We build this reciprocity.
Imagine I send you a streak freeze.
You send me a thank you back.
When you're in need, I send you another one.
We can kind of create more of a positive gifting environment.
That would be awesome.
But it's social is one of those hard things where right now, not a lot of our users have friends.
So when you build these features that are not easy to get right, they don't like kind of hit massive adoption because not everyone has friends, but it's the chicken and egg.
We don't have a lot of features.
That's why people don't add friends as much.
So we have to invest in both.
When you look at chess, music, maths, and languages, is 85% still languages?
Yes.
Majority of our users are doing languages.
Chess is very new, like counted as kind of zero still.
It just rolled out like a few days ago.
But math and music are a small percent of our business.
Which subject?
You can do one more.
You're capped at five forever.
Which one would you add?
This will be the next business insider headline.
I think the one I would do is physics, because you know, as surprising as that might sound, it's very similar to math, meaning we can kind of apply a lot of the stuff we're doing on math and we're doing a lot of product development for teaching math.
We can apply it to physics.
I'm not sure if it's the largest market or it's certainly not the sexiest market.
It's probably the nerdiest thing we can teach.
But it's certainly we have a competency or we're building the competence for math that we can apply to physics.
God, fascinating.
That's an audience grabber, isn't it?
Speed equals distance over time.
Where do I sign up?
You sound excited.
I'm so excited.
I'm so excited.
Okay.
Are you ready to do a quick fire?
Let's do it.
Okay.
So which other product do you look at and you're like, yeah, that's smart.
I like what you did there.
The products I love are the ones I use a lot.
So I'll mention two and they are everyone uses these products a lot.
One of them is I'm an Apple fanboy.
IOS itself is a beautiful piece of software that is so well designed, so sleek, amazing animations, attention to detail, next level.
Another product I love is Spotify.
Amazingly built product.
Obviously very successful business on top.
But I think they also take care and attention to detail is high.
I'll mention a third one, which is, we actually were chatting about this, is Bump from Amo.
It's just a high craft, incredibly creative app.
So those are the apps when I use them, I get ideas for me to think about for Duolingo.
What was the biggest change PMs will have to encounter in the next five years?
The biggest change will be how they do product is going to be significantly different than today.
And the tools they use will be obviously a lot more AI based.
They will have to learn basically how to be a lot more independent rather than reliant on other functions to build the thing they have a vision for.
Meaning they will be independently coming up with prototypes themselves, rather than today they rely on machines that involve a lot of other humans to even like, visualize an idea and then turn into prototype.
AI tools will be the main thing they use to do that.
What have you changed your mind on in the last 12 months?
An example for me.
I was like you know chat GPT whatever, fuck it, we'll go to grok perplexity, you name it.
I don't think there's any lock in there.
I'm like unwaveringly chat GPT focused now.
I have memory.
I love the brand.
Similar to you, I've actually changed my mind about the same thing, but I'll pick a different one.
I thought this change in how we build product was maybe five to 10 years out.
12 months ago, if you had asked me like oh, like you know, how do you think the PM job will change?
I'm like, well, maybe in 10 years, we'll be using more AI tools.
I think it will get there slowly would be my answer.
Honestly, now, probably in 12 to 24 months, the job of a PM will look entirely different.
So I think we're talking about five years because that feels safe.
But honestly, my guess now, or what I've changed my mind on, is it's going to be in two years, is my guess.
Where do you most need to improve as a product leader?
What I'm trying to improve on is quicker decision making.
One of my fallbacks is I can debate something for a very long time because I would like to try to, like in my head, reach super 100 clarity or perfection, but that is a massive mistake because you're slowing your decision making down.
I think you want 80% clarity and faster decision making.
And you know, weirdly everything comes back to AI, but it does.
ChatGPD is actually really good at helping me do that, because it can be my sounding partner to like bounce back and get to clarity quickly, or enough clarity.
So I'm actually doing that.
Totally agree with you.
I
What actually makes me think of the stat, which is you can learn a language to, I think, 95 proficiency in like six months if really devoted.
But it takes five years to get to 98%.
Oh, yeah.
The last mile is high investment.
Yeah.
Totally.
Yeah.
Okay.
Final one.
In 20 years, where's Duolingo?
20 years is so impossible to think about.
But here's what I imagine it will be.
I don't know what platform it will be on.
Assuming we're all still using mobile phones, it will be on mobile.
But if the different consumer platform merges, it will be on that.
If we're all wearing VR headsets and living in the metaverse, which I don't think it will happen.
It'll be on that, but duolingo itself is going to be incredibly personalized in 20 years meaning, rather than, i think, following standard patterns.
I mean, it's already headed there heavily, but this will compound that's one.
It will teach many more subjects.
I think that is just like that is just going to happen.
It will teach you hopefully, everything that's good for you.
Maybe dating by then will be part of the thing.
We teach you how to date better, or maybe you'll be dating ai agents, who knows?
But It will teach you way more things.
And since it's personalized, it will teach all of those things with much higher efficacy.
So an hour of time spent will give you much more yield than it does today.
That's what I assume we'll do.
I think the other thing that will kind of emerge is these AI-first experiences.
And what I mean by that is now when people think of AI-first, they think of, oh, it's chat GPT.
It's a text box or a voice input.
What hasn't emerged much is AI.
UI creation that is also AI first.
So I think you'll see a lot of on the fly UI generation off of what you're doing instead of hard coded UI that comes into play.
So I assume in 20 years, this will be a thing that we experience where let me paint a picture.
You enter a Duolingo lesson.
Duolingo has not built a world where you want to engage in hiking up a mountain.
However, that's where your conversation leads.
That's where your interest is.
UI can generate you hiking up a mountain and then kind of put you in that position.
Maybe teach you the word for mountain while kind of you immersively experience that.
I think that type of experience will happen more, which will make it makes the whole thing way more entertaining.
I look forward to Duo Date.
At that point, I'll take one stand.
It's fair enough.
You're like, I want one future.
This is why I did the show.
This is why I did it.
Listen, dude, honestly, I do a lot of shows.
I don't have this much fun in shows.
You hear them.
There's not this much fun.
So I really appreciate you making it so fun.
And this has been such a joy.
Love it.
It's been a pleasure.
That was one of the most fun shows to do I've done in a long time.
If you want to see that episode, you can find it on Spotify by searching for 20VC on Spotify.
It makes such a difference if you leave a review or a five star, I would so appreciate that.
But before we leave you today, Are you struggling to beat model benchmarks or implement gen AI in your product?
If so, you need Turing.
Turing is an AGI infrastructure company backed by incredible investors like Foundation Capital and Westbridge Capital.
And they do two things.
Number one.
They help leading companies in AI labs like Salesforce, Anthropic and Meta enhance their LLMs with advanced reasoning coding multilinguality, multimodality and more.
Number two.
They combine human and artificial intelligence expertise to deploy cutting-edge AI systems for awesome companies like Rivian and Reddit.
Right now, Turing offers a free five-minute self-assessment to help you pinpoint your place in the Gen AI journey, get tailored next steps to optimize your model strategy and then finally learn how Turing can refine and implement your models for better performance.
Take the guesswork out of Gen AI.
Visit Turing.com forward slash 20VC to start your free assessment today.
And once your Gen AI strategy is on point with Turing.
Now let's switch gears to the future of mobile and how one company is turning your screen into real returns.
We spend nearly half our waking hours glued to our phones, upwards of 50 hours every week.
Recently, one company transforming this reality stood out so much I personally became a shareholder Mode Mobile.
Mode Mobile created the Earn Phone, the smartphone that pays you for daily activities.
Instead of big tech profiting billions from our attention Mode returns over 325 million directly to users through earnings and savings.
Mode's revenues surged an incredible 32481 three years, recognized by Deloitte as 2023's fastest growing software company in North America.
And here's why I'm excited.
Mode's equity offerings have raised over 30 million from 20000 plus retail investors one of 2025's standout public raises.
And you can now join me as a shareholder with as little as 1000 at investmodemobilecom.
Forward slash 20VC.
For a limited time, unlock up to 100% bonus shares and a free earned phone.
Email us for the investor brief at 20VC at modemobilecom or check out investmodemobilecom forward slash 20VC.
We've talked about making money from your phone.
Now let's talk about making your product do exactly what you want it to.
Whether it's the software you build or the software you buy, your tech stack should be creating results, not creating roadblocks.
Well, Pendo's no-code software experience management platform makes your software better with tools to see where users get stuck, guide them with in-app messaging and constantly improving your UI.
It's so easy that over 14000 businesses use Pendo to increase revenue, lower costs and reduce risks.
Businesses love the control.
Engineers love the freedom.
Everyone wins.
Start for free today at pendo.io forward slash 20 product.
As always, I so appreciate all your support and stay tuned for an incredible episode coming on Monday with Larry Asherbrook at G Squared.
