Thank you.
Matthew, thanks so much for being here.
Thanks for having me.
So I want to get like right into the juicy topics but make sure our listeners understand like the scale and like her role of Cloudflare first.
So correct me if I'm wrong in any of this.
66 billion market cap company today, about 18 billion in trailing revenue, and then like the biggest CDN by far with a bunch of different products now in security in particular.
What else should our audience understand about the role Cloudflare plays?
Not to nitpick on one thing, but we've never really thought of ourselves as a CDN.
We start out very much as a security company.
The whole thesis was, could you put a firewall in the cloud?
We saw that servers were going to cloud.
We saw that software was going to cloud.
It seemed inevitable to us that the networking equipment would go to the cloud.
And the big objection that everyone had was you were going to slow things down.
And so we worked very hard to figure out how could we not slow anything down.
And the goal was just to get back to parity.
Turned out we were a little too good at our jobs and everything got a lot faster.
Yes, we've ended up competing in the CDN space, but really what Cloudflare is is what the network should have been, what the internet should have been, had we known in the 60s 70s, 80s how important it was going to be.
How can we make it faster, more reliable, more secure, more efficient, more private?
That fundamentally is what we're working on every day at Cloudflare.
How long has it been since you guys started the company?
We launched in September of 2010, so we'll be coming up on our 15th year in September of this year.
Amazing.
And I don't think there's a way to ask this question without somewhat trivializing the journey.
But how did you become so dominant?
I don't know.
I mean, I think we just focused on... how did we do the right thing for our customers?
How do we solve the problems that were there?
And you know at some level.
The story of Cloudflare is that we have been customer zero along the entire journey.
So every, you know, thing that started from, could we take a firewall, put it in the cloud?
How would we get the data to populate that firewall?
We had to have a free service.
Once we had a free service, all of a sudden we had to be able to figure out how to scale enormously across millions of customers in an efficient way.
That meant that we had a whole bunch of weird stuff that was using us.
We got attacked by every which direction.
We had to build a public policy team in order to deal with those issues.
We had to build our own security.
Someone almost hacked or stole our domain at some point as a way of hacking into us.
So the next thing you know, we built our own registrar.
And so to some extent, Cloudflare has been about start with a relatively simple idea, make it as broadly available as possible, and then solve all the problems that become sort of inherent once you've done that.
Now you're in the position that I believe you've described as like the Internet's traffic cop.
I think a lot of people feel, with the sucking sound of attention toward AI assistance, that the shape of the internet is changing.
What is your, before we go to like point of view on what to do?
What is your prediction for what's happening?
So, no matter what the dominant kind of value creation model of the last 30 years of the web has been search.
Search drove everything.
It drove all of what you did online.
Entire industries grew up around that.
And the real three ways that you could drive value on the web.
In the past were you either sold a thing, whether that was a subscription or a product or something else.
You sold ads against some content.
And I didn't say the business model, but I said the value creation model, because the third part is really important, Which is a lot of people just created content for the ego of knowing that other people were doing the.
You know, The old adage is only two reasons why people create content to get richer, to get famous.
A lot of people are just doing it to get famous and and that's a a lot of what drove I mean, that's what drives Wikipedia.
That's what drives a lot of content creation that is on the web.
I think that the web is shifting now to a new interface.
It's shifting away from search and it's shifting to AI.
And we can see that through the trends in terms of Google usage.
We can see that in terms of like our own usage, where more and more people are turning to these AI systems where they used to turn to Google.
Even Google itself is sort of morphing into an AI company kind of, in their interface, before our eyes.
And as we do that, the natural thing that's going to happen is we're going to consume derivatives rather than consuming the original content itself.
A study that just came out from Pew Research that says that if Google puts an AI overview on the top of search, it is much less likely that people click on the links.
And that seems sort of like a Duh.
But the data that we have also substantiates that and shows that, compared with ten years ago, it's become ten times harder for the same piece of content to get a click from Google than it was before.
And that's because the answer box, that's because of AI overviews, that's because of the search interfaces, has gone there.
And that's the good news for content creators.
In the case of someone like OpenAI, It's 750 times harder than it was with the Google of old.
With the case of Anthropic, it's 30,000 times harder than the content of old.
And so what I worry about is that, if the value creation model of the web has been all about how do I get traffic, The new interface of the web isn't going to send you traffic.
And if that's the case, if content creators can't get value from selling a thing or a subscription, selling ads or just the ego that they get, knowing someone is reading their stuff, I worry that people aren't going to create content.
And that's going to really not only starve the web, but it's actually going to starve even the AI companies.
They're using that content as effectively the fuel for their engines.
How do you think that evolves?
Because, if you look forward, the other thing that people are talking about a lot right now is agents and the fact that you're not only getting information through an AI, it'll actually go into actions on your behalf.
So the time you actually spend on the web is going to go down effectively, or at least you're going to be dealing with one interface, which is this agent that goes off and does things in the background for you.
So do you think that ultimately the AI companies will start paying for content?
Do you think there'll be other ways to monetize it?
Do you think it's a completely different model starts to emerge in terms of how the web works?
There are going to be different solutions for different pieces of the equation.
At some level.
Agent commerce is going to be probably the easiest of these to solve, where there are going to be certain companies that say listen, we'd love your agent to come and buy a widget from us.
There are going to be others that right now are aggregators of information, that that the agents can actually disintermediate or disaggregate that content.
And they'll be actually quite threatened by that.
But ultimately, I think commerce agents and AI is probably ultimately pretty good.
To separate commerce from content and content is what you're worried about.
Content is a different piece, where content the problem right now is, the default assumption has been that you get content for free.
And it's actually interesting.
A lot of the content creators are looking to the law as the solution to this.
And generally, I'm a recovering law professor.
So pardon me for going down this weird tangent.
But it's actually, I think, really interesting, which is in copyright law the more that you are a derivative as opposed to a direct copy, the safer you are, the more likely you are to fall under fair use.
And we've actually seen a number of court cases two that happened here in California that happened within a week of each other one of which basically said AI uses of content is fair use.
The other one which said it wasn't.
There's going to be a whole bunch of things around that.
But probably the more sensible one is.
The more that you're creating derivative content, the less likely it is to be a copyright violation.
But kind of opposite of that.
The more that you're creating derivative content, the more likely it is that someone isn't going to go back to that original source.
And so I actually worry that a lot of the content creators are focused very much on what the law says today and on copyright law, which may not come out in their favor because it is it is actually protecting those derivative uses.
I think what we have to figure out is probably a different business model where content creators get compensated, and i think the good news is, as you talk to the big ai companies and you know, 80 of the major ai companies are cloudflare customers, so we have good relationship with them, we talk to them about it all the time What they all say is you're absolutely right.
We should be paying for content.
The devil is in the details though, because what they all are desperately worried about is how do you make sure that it's a level playing field?
They all believe their technology is the best.
They all believe that on a level playing field, they're going to win.
But they're really worried.
Well, if Google still gets content for free but we have to pay for it, that doesn't seem fair.
If I'm paying for it but somebody else gets it for free, that doesn't seem fair.
So what we've been really working on is, how can we create that really level playing field?
And we think, if that's the case, that AI companies will actually be quite willing to pay for that content.
What are the approaches you've been taking to try and level the playing field here?
So in order to have an economy, you have to have a market.
In order to have a market, you have to have scarcity.
No markets exist without some level of scarcity.
And the problem right now with content is that content, there is no scarcity.
They're giving it away for free.
And so we spent the last year working not only across Cloudflare's existing customers but then going across the entire publisher ecosystem, writ large, not just print publishers video, audio music, you know across, you know film across the entire spectrum and saying, you know, we think there's a problem with ai, that it's starting to actually take value and not give you anything back.
And across the board, for every publisher, from the Associated Press to Ziff Davis and everything in between, we've seen just incredible resonance with that message where they're all saying you're absolutely right.
Our business is getting astronomically harder over just the last six months and we're seeing less and less of our existing business model working.
So we need to do something about that.
And so what we did on July 1st was we announced what we called the Content Independence Day, where you could actually have independence from these AI companies.
And we for free, across all of our customers, whether they paid us or they didn't pay us.
We started blocking by default any training that was being done by any AI companies that was there.
And it was really important that we focused on that, because that meant that we could treat Google the same as everyone else.
Now what we're doing is we're working with the ITF and other standards organizations to say let's define how you have to announce what your crawler is doing as it behaves online.
We're really encouraged by the early work that's there.
As that happens, we think we'll be able to set in place really fine-grained permissions for content creators or anyone else.
To say, Humans can get my content for free, but robots have to pay for it.
And then figure that out.
That first step of creating scarcity is what you have to do in order to figure out what the market is.
And then after that, I think figuring out the market, that's going to be what takes some time.
And I think we're still experimenting with different- precedent.
We had a robotstxt file and that's where you kind of specify whether a search engine could come and crawl the content.
And it sounds like you're really extending some of those concepts on through to the AI layer.
Yeah, that's right.
And I think robotstxt was a relatively simplistic and blunt tool where today it basically says you can either allow something or disallow something.
And you can basically do it either, you can do it on a directory on your site or to the entire site.
But there's not that sort of fine-grained control.
And so we think robots of Texas is sort of like the street signs that are on the road.
A lot of people don't necessarily follow the speed limit though.
And we actually see plenty of examples.
In fact, some really prominent companies that do some very very, very shady things, where they basically say absolutely we follow the rules of the road.
But when push comes to shove, if it turns out they're blocked, then all of a sudden they're doing a bunch of things that look not dissimilar to what we see Russian hackers or Iranian hackers do in order to try and get around those blocks.
At Cloudflare, we're really good at identifying that and stopping it, and we're also really good at embarrassing those companies that do that.
So watch our blog and I have a feeling that some prominent AI companies that are misbehaving are going to get called out pretty soon.
What do you think that should the idea of a marketplace for contributing to training work out?
What do you think that does to the landscape of the types of content companies that win or lose?
I can't imagine it's going to look like it does today, because there's some notion of incrementality.
I mean, this is this is going to take us down a little bit of a tangent, but i think a lot of the things that are wrong with the world today are ultimately google's fault.
They're not the worst actor.
I'm glad we started with we're all friends here, and then it's all google's.
Yeah, it's all google's fault.
Yeah, i think google has been a net force for good in the world.
I think that they actually believe in ecosystems.
I think they're trying to do the right thing, But they taught everyone to worship if they're content creators sort of a deity, which is traffic.
And that was the proxy for value.
It was how do you generate the most traffic.
And that led to Facebook, as the next iteration led to TikTok, led to folks like the Huffington Post, which would literally write a piece of content and then A-B test headlines, trying to figure out which one generated the largest cortisol response to get the most clicks.
Or if you guys remember Demand Media.
Demand Media.
I mean, BuzzFeed.
I mean, there's a whole bunch of folks that were just trying to figure out How do we actually stimulate rage and get people stirred up so they'll click on the thing, so that I can either sell them a subscription or sell ads against a piece of content.
And again, I think that that led to a lot of metooism, that led to a lot of people writing the same story with sort of a slightly different bend.
I don't think it led to a lot of us actually figuring out how to advance human knowledge.
And so what I think is interesting is, if you think about the AI companies en masse, They're a relatively good approximation for the sum of human knowledge.
Not perfect, but probably the best we've ever had, right, where they come together.
And the reality is that they are in aggregate.
They're like a giant block of Swiss cheese where yeah, there's a whole bunch of cheese there, but there are holes in the cheese as well.
And their very algorithms.
As they come across a piece of content, they prune off that content which is already kind of part of the meaty, part of the cheese.
Whereas the parts that are in holes are actually super valuable to them.
And so I actually think that if we could create a market.
Where you're rewarding content creators not for who stimulates the most cortisol, but who fills in the holes in the cheese.
And you actually pay people for that.
That is a better outcome.
And that's actually advancing human knowledge.
And that's really amazing if we can do that.
Isn't that kind of arguably, companies like Mercor or Surge or Scale, as they do data labeling and they hire human experts to basically fill out content areas for AI companies?
So do you basically view this as like a distributed model of that or sort of a web-based?
I've spent the last year talking to a lot of people.
One of the more interesting conversations that I had um was was with danielak.
I flipped to stockholm and and saw, saw daniel, and i think there's really nobody who has compensated content creators at scale like daniel has, and it's amazing.
The day before itunes launched, the music industry was about an eight to nine billion dollar industry.
Spotify on its own today pays out over $10 billion a year to the music industry.
And so done right, these can be very much pie expanding.
There's plenty of cheese to go around if we do this correctly.
And the thing I remember he was telling me a story which I thought was sort of in the same vein, which was Spotify actually looks at queries that people have run that they don't have good answers for.
Where there's somebody searches for I don't know, I want a disco song about how fun it is to dance with your dog, right?
I don't know.
And if somebody searches for that and it doesn't get it, they actually publish that list back to content creators and musicians.
And there are several musicians that are making literally tens of millions of dollars a year just writing songs for what people care about listening to.
Unmet demand.
That are unmet demand.
Yeah.
And so I think that it's not exactly the same as data labeling.
I think it's actually saying like for the first time in human history, we can actually very accurately identify where there are holes, as the very nature of the pruning algorithms that these LLM models are trained on.
And that we could then, if we sort of basically resurface that and say hey, this is –, we don't have enough articles about the wing-toed ferret that people will actually go out and do that.
And that if we can then compensate people on that, that actually is much better than yet another article about what's happening in Washington DC, yet another article about how much –.
San Francisco is on decline or on the rise.
I mean, again, that's not actually adding to human knowledge.
That's just, you know, rage bait, effectively.
How do you think that plays out?
So if you look at some of these labeling companies that also then hire experts in to sort of provide some of the at least expert content that you mentioned, if you look at some of the models like MedPalm2 from Google, which is a couple years old now, it outperformed human physicians the average human physician in terms of output.
So if you rated its output against people, at what point do you think we've run out of good content from people?
In other words, there is some limit.
I don't think that's true.
I mean, I do think that there will be some...
Like there's always going to be people running new experiments and new tests and finding new things and new discoveries.
And yeah, maybe we can imagine some distant future where it's all robots that are doing this in the labs.
But that's a long ways off.
And so in the meantime, I think we can do that.
My black mirror kind of version of the future though, is actually one where we're not going to get rid of journalists.
We're not going to get rid of scientists.
We're not going to get rid of researchers.
You're going to still need that work.
What I worry about is we don't figure out how to compensate broadly content creators who are independent, that we actually go back to almost a time in the Medici's, where the web had historically been this incredible sort of distributor of value creation and knowledge creation.
You could imagine a world in which all of a sudden you have five big AI companies.
You have the conservative one and you have the liberal one, and you have the European one and the Chinese one.
And they all actually hire and run their own team of journalists researchers academics, the experts that fill in the holes in their cheese.
And again, that's not too hard to imagine that in some not so distant future that becomes a thing.
What I hope is that we figure out a way to compensate independent content creators and share that knowledge across all of them, as opposed to creating these silos of knowledge behind each variation of an LLM.
Do you feel like the large labs agree with you on how much can be paid out to creators to fill those holes?
Because you look at the scale of Ad revenue, you know I mean even ignoring things like commerce and whatever from the open web.
But, you know, in aggregate, like what's been paid to labeling companies like $10 billion less.
We're like really far off.
If people are starting with a very large free free today base.
Well again, I'm not sure.
Labeling companies is the right.
Is labeling companies the right, the right model?
Or is it GPU spend, or is it?
Is it?
Is it employee spend?
You know I actually think you know.
First of all, the amount that's paid to labeling companies will go up.
The amount that's paid to employees and then GPUs is continuing to go up.
And so the question is, how much value is content actually giving you?
And the answer is, you know, somewhere between zero and 100%, right?
And is it more or less than another unit of GPU time?
I mean, there's a market that can figure that out.
From that first principles view, I see it.
Yeah, and so there is some value which is there.
I think the mistake that a lot of content creators did was they actually did deals that don't scale as the business models of the AI companies scale with them.
So if you do a deal that's like 20 million and you get all my content, That's an incredibly naive deal.
It might seem like a great deal to the content providers day one, but it's exactly the opposite of what you want to do.
What you really want to do is say okay, if you imagine that there were a way for all of the content that is available, you say here's how much that is creating value.
That's going to be some percentage of whatever the subscription fee is for your AI model or if you're an ad-supported AI in the future.
It's going to be some percentage of that.
And then as the AI companies grow, which will inherently then mean the ad revenue shrinks that you share in that upside as your downside gets diluted.
And I think that there's still going to be advertising out there.
There are still going to be subscriptions.
There's still going to be tentpole content that people just have to consume even if they're AIs.
But what you also want to do is allow that content to get into the AI systems, and the content creators should get compensated for that.
And again, if there's scarcity, a market's going to determine how valuable that actually is.
One of the things that Cloudflare is known for, to your point, is really speeding up web pages and the internet.
And as we shift from serving pages to models being run, you're shifting from a world of caching and serving pages to inference.
How do you think about that in the context of Cloudflare, some of the directions that you all are going?
Well, I mean, I think we leaned in heavily.
Nobody remembers this, but back in 2020, we partnered with this you know graphics chip company in order to put GPUs at the edge of our network, in order to allow people to do inference.
And by the way, it was crickets like launch this product.
No one responded.
There wasn't a single like sales inquiry to use it.
And so we apologized to the partner who happened to be Nvidia.
And it kind of went our way.
Now, four years later, the market was ready for it.
We basically just took out the same press release and issued it again.
And then, you know, it's taken off like gangbusters.
I think that we've leaned in heavily to you know.
We believe that a lot of inference is going to happen on your end device.
But there will always be some model which is too big or too resource intensive.
And in that case, the next best place to run it is going to be on at the inside, the network at the edge.
And that's what we're delivering.
More importantly, I think that if you look at whether it's MCP or whatever, the next protocol that connects agents to services and allows these things to connect inherently, because of how much of the internet we sit in front of, they have to pass through us.
And so we're investing heavily behind those protocols, making sure that they have all of the security, the underlying rails and payments infrastructure and everything else that you need.
And my hunch is that What we solve in the content space and the rails that we create for the payments there very naturally then become one of the models to do sort of agent to agent over MCP or whatever the final protocol becomes payment infrastructure to be able to handle that as well.
So Cloudflare fundamentally is a network.
And I remember when cryptocurrency and blockchains and everything were getting big, people were like aren't you worried about this?
And I'm like, they still need a network.
As AI gets big, it still needs a network.
And so I think we sit in the center of this.
And as you especially have more agent-to-agent communication, I think the network actually becomes more and more important.
Is there a bet you're making on like?
What changes in terms of models or compound systems that like drive more model traffic to the architecture you describe where it's in network versus, you know, in large data center today?
Well, I wish I could say we were that strategic.
I mean, I think we go to wherever the market demands that we go.
And so- Including build a neocloud.
I don't even know what a neocloud is, but sure.
So I think we're fundamentally, you know, always just trying to say how do we respond to whatever either our own team needs as customer zero, or what our customers need.
And the fact that Again, 80% of the AI companies are using us.
They are constantly pushing us to, can you do this?
Can you do that?
And I think our team has been uniquely good at being able to execute and innovate and stay at least up with whatever the trends are.
And that's again.
I think I'm proud of the fact that we have ended up in a lot of these conversations and that so much of the internet does flow through us that one way or another, I think that we end up being in the center of a lot of these transactions.
Does that imply any particular belief around open or closed models, as people continue to develop capabilities?
We have closed models that run on us.
We have a lot more open models that run on us.
I tend to...
We have historically been a company that believes very much in open source.
And most of the things that we build internally.
As long as we can, we try to open source all of that technology.
And so I tend to be in the pro open models.
We work very closely with the meta team and Lama and everything that they're doing.
But again, I think there's going to be different flavors of this.
And again, I'm We're happy to have customers in either end of that spectrum.
I'm a little bit skeptical.
I'm actually quite skeptical of the.
If we allow open source models, the world is going to end arguments.
That seems histrionic to me.
There are things we should worry about like it is.
I think some of the synthetic pathogens and other things that can be created.
But it seems to me like the place to regulate that and control that is in the machine that can actually print the pathogens, not in the AI model that can come up with what it is.
That seems like a pretty flimsy argument for why we shouldn't have open source.
What needs to happen for your view?
Sorry, I'm still going back to like shape of the web.
What needs to happen for your view of like a marketplace for content to emerge?
Right.
Like what are the next signs that this is actually like happening?
I think the very tactical next step is we've got to get Google to not. be a special snowflake.
Because Google has had such a dominant position in search, they almost believe that it is their right to have access to content without having to pay for it.
And so the conversation that we have with them is we get it.
The deal that you made with content creators in the past was you.
They give you their content and you send them a certain amount of traffic.
You, over time, have taken just as much of their content, but you've sent them one 10th of the traffic that they have.
And if we just plot those trends out, going forward, it's going to become, you know, a smaller and smaller part.
At some point the content creators will say we're just gonna block Google right now.
That seems on.
That was unthinkable 10 years ago.
It seems radical six months ago.
It is what people are talking about today.
Why Google is so important is, when you talk to all of the other AI companies, Google is the one company that they're the most afraid of.
And the reason that they're most afraid of them is because they think that they have privileged access to content in a way that is much more difficult for them to do.
And so what I think we have to be able to do first of all is to say to Google listen, you can still do search indexing.
But if your bot is taking content and then transforming it in some way, making it into the answer box, making it into AI overviews, turning it into Gemini, That's different action.
That's a different deal.
And you have to be in the same bucket as everyone else.
And Google is going to resist that.
Now, I think the good news is they really do believe in the ecosystem.
I think they are trying to do the right thing that's out there.
And maybe not.
This is the good news for Google, but it's the good news, I think, for the web, which is that they have a ton of both regulatory and legal and legislative pressure which is coming down on them.
So one way or another, I think we will flatten that out.
Once that happens, I think that's when we can actually start to say we're going to shut off access to content unless you pay for it.
In the beginning most of the deals that are done, the actual money being changed will be between large content producers and large AI companies.
That's happening right now, where Conde Nast or Dot Dash, Meredith or The New York Times or Reddit is doing a direct deal with a large model company.
That'll happen a bunch.
Where I think we can play a role is when you have either a large content provider trying to make a deal with all of the AI startups that are out there, which they really do want to do, and they want to do it in a way that scales, but they can't do one-off deals in those cases.
Or you have the long tail of content with all of the different AI companies.
In both of those cases, I think Cloudflare can play a role in helping set what are sort of basic rates that are there.
And how that model looks, I'm not sure.
It might be that we negotiate, basically on behalf of a number of the content providers with all of the different AI companies a basically a pool of capital, much like how spotify does and then distribute that out it might be micro payments every time you access a piece of content.
It might be that training is actually a different payment rate than search.
That's, I think, something that we'll have to figure out.
But step one is we've got to get Google to play by the same rules that every startup, every other company is playing by.
And the minute we do that, I think the rest of the marketplace will actually happen a lot faster than you think.
For any content, company or individual providers that used to be a big part of the web.
That cannot predict today.
There's no business model for them today to make money off of content going into these AI experiences.
And they can't.
It's not easy to predict what is incremental to models.
What advice would you have for them?
So I think the first thing is you've got to get back to controlling your content.
So you have to create scarcity. from the beginning.
So how do you make sure that you're not just giving your content away for free?
And again, we've made that easy.
There are other companies that are working to try and make that easy as well.
One way or another, create scarcity and then start to have conversations.
You can see which AI companies are the most likely to deal with it.
So just today there was news that Google is starting a pilot project to start to pay news providers, something they swore they would never do.
But again, I think that they can see, because they do believe in the ecosystem.
They can see that this has to happen if the incentives for creating content go away, if you can no longer sell something, if you can no longer sell ads against something, if you can no longer even get the ego hit, because if people aren't going to the original source, you don't even know if you write some incredibly influential piece that ends up in, you know, millions of ai responses, you don't actually ever even know that happened.
You're yelling into the void.
We've got to figure something out around that piece.
So I think the first step for content creators is recognize that the business model of the web is changing.
Second, recognize there is something you can do about it, right?
You can actually create this scarcity.
And then third, actually participate, start to go out and say, based on the data.
Hey, you keep trying to crawl my stuff.
Like, let's figure out a way that we can have some fair exchange of value for that.
One other thing that you mentioned sort of um A little bit as a side note when we were talking earlier was around how you felt that a lot of the models would actually be running on device and running locally.
And then obviously, there'd be things on the edge or in the cloud.
That would be the bigger models, perhaps doing more complex tasks.
When do you think that'll happen?
Do you think that's based on when the device is advanced in certain ways?
Is it model size?
Is it something else?
Well, I think a lot of it's happening today.
On your phone, there's a lot that your phone is doing locally without it having to go out.
And there are certain places, certain applications where it has to be local.
If you have a driverless car and there's a red ball bouncing through a yard with a little you know girl running after it, whether to hit the brakes or not can't be dependent on network conditions, right?
So that has to run locally.
I think that the big place where you know where I think there's going to be exciting innovation that does not feel like it will be today is really in just how do you take, especially on the inference side, making it significantly more power efficient?
That ends up being the biggest limiting factor.
Apple has shown that it's possible and that you can actually have relatively power efficient GPUs and TPUs that are out there.
When we talk with the folks at Nvidia, it feels a little bit like talking to Intel back in our case in 2010, or Apple's case in 2005, where they were like you're doing it wrong if you care about power efficiency.
I remember sitting in Intel's research lab outside of Portland in 2011 and we were a tiny little startup.
But we were doing interesting, innovative things, and we were using their chips.
And so they brought me in, and I was like, the only thing we care about is cores per watt.
And we just need as many cores per watt as you can possibly deliver.
And they just kept saying, you're doing it wrong.
You should be water cooling your systems.
And we kept trying to explain, like, we don't have that luxury.
We have to go into what are oftentimes the oldest, most legacy data centers in the world, where there is a relatively limited power envelope.
And we've got to fit within that.
I think the same thing is going to happen in the AI space.
NVIDIA has been a terrific partner to us, but it's been sometimes frustrating to see how more GPU capacity comes along with having to stand up your own mothballed nuclear power facility.
That can't be the solution, and there's no physics reason why it needs to be.
And so I think that we're actively looking around the ecosystem trying to figure out who can deliver the most tensor units per watt, or whatever the sort of GPU equivalent is.
And that, I think, is going to be the big unlock that allows you to have more running on your device, whether that's your phone or your driverless car or frankly, at the edge of the network.
Because again, we also have to live within a power envelope which is not the same as if we were standing up a 100-megawatt data center.
Yeah, I was kind of thinking of it, I guess, from tier three perspectives.
I mean, to your point, there's the actual chips.
And in the context of mobile, obviously there was ARM and then Qualcomm as other approaches to basically get to some of the things that you're mentioning for devices.
Separate from that, there's actual model size and inference time and a few other things that are kind of overlapping but different.
And so to some extent, it's how large of a model that's how performant can you actually load on a device and when does that happen and how well can it run?
And so I was just a little bit curious how you thought about all those different pieces, because there's also the model component or side of that.
That seems to matter quite a bit.
And it's all coming.
I mean, it's all inevitable.
I'm just sort of wondering about that.
I think that we are still most of the companies are still um relatively inefficient in terms of their utilization from from everything that we can see, and so when we um, It is in our interest, based on just how our business model works and everything else, to make inference to make anything we do as efficient as possible.
Because we only charge customers based on the actual work that we do.
We're different than hyperscalers.
The hyperscalers, you go out and you rent a GPU.
They don't care if you use it or you don't use it.
There's no incentive actually for them to make.
In fact, if tomorrow, someone announced that they had made inference 100 times more efficient.
That's great news for us.
It's terrible news for the hyperscalers because our business models are very different.
I think that the great lesson of DeepSeek.
I just wish that it had been a group of students out of Hungary that had designed it, not out of China, because there were some really significant innovative steps that they took to make essentially training and inference significantly more efficient.
We all got distracted in the US by the fact that it was China, and was it real or not real or anything else?
It was real.
There was really great science that was done there to be more efficient.
I think we've just barely scratched the surface on what we can do around model compression, what we can do around really just much more efficient pruning.
There's a ton in these models that are branches of the tree that just literally, the probabilities of going down it are so incredibly low that you can prune those branches off fairly efficiently and still get incredible performance.
So I think we have an enormous amount of the kind of hardcore computer science which is different than kind of the hardcore AI science, to just say how can we now take these things and make them massively more efficient?
And my hunch is that it is not particularly long before you're running something which is the equivalent of kind of the current generation of ChatGPT on your iPhone or your Android device.
Does that advancement happen at the model providers in open source at an infrastructure provider?
I mean, it's hard to predict where it is.
I mean, I know that that's like we're not investing in how we build a frontier model.
That's not that's not our job.
We are investing on how we take any models that we're running and run them significantly more efficiently.
So that's how we think about it.
I think the question is who's gonna be the VMware of AI right and who's gonna create kind of that ability to, because we haven't even done that basic work.
Like today, for the most part, if you want to spin up a GPU, you're taking an entire VM.
There's not even a container, because you've got to get that low level.
Or you're taking an entire machine that is running.
And that's extremely expensive.
And in most cases, with most of the hyperscalers, you actually have to get anything close to attractive pricing.
You have to commit to a year Of of that.
And now it's up to you as a customer to figure out all that efficiency.
Someone will come along and figure out a way to say, here's how we can slice these things up.
Here's how we can make them more efficient.
And we're just going to speed run essentially the CPU efficiency gains, including a lot of the security things like the specter and other attacks etc.
The speculative attacks that you had in CPUs will come to GPUs.
All that stuff is coming.
We're going to speed run the last 30 years of CPU efficiency gains in the next five to 10 in GPUs.
I think fundamentally there's a lot of really interesting next-gen models around physics and materials and other things that may actually be interesting.
Obviously a few really odd future-looking things on the infrastructure side that I think will be important.
You've probably been following a lot of the agentic-related infrastructure that actually is necessary for multi-step agents.
So I think there'll be a few big companies there.
And then there's all the We will probably compete in that agentic infrastructure space.
It will probably not be one provider.
It will be something where you're going to have to have a whole bunch that actually work together in some way.
And so figuring out the standards behind that.
I think is gonna be important.
I think that, whether it's MCP or Google did their own flavor of it which was sort of just like it felt, very Microsoft-y kind of embrace and extend.
But all of that space is going to be- Sort of like temporal linear.
All those things are kind of early indicators of new agentic infrastructure that's coming.
Yeah, that's right.
I think there are entire very large domains where a lot of the architecture should probably apply.
And the data collection.
Efficient data collection is the question, because we're not going to get robots from Common Crawl.
Yeah.
But we are like probably going to get them.
We just have to figure out how to pay for the data.
Yeah.
And so I think figuring out if there are interesting models to get to generalization in these other domains is like something I'm looking at.
Yeah.
The other thing that's going to be really interesting is I was actually kind of a pretty much a skeptic around the blockchain cryptocurrency.
It may be that this shift is the thing.
Because now we're looking at this and we're thinking okay, let's say, we got to a place where it was actually micropayments for every page view.
I mean, we do It's like 15 trillion requests every day.
It's an obvious use case.
But how you then scale these things to be able to work that way.
I mean, you can't do that with Bitcoin, right?
And you can't even do that with Solana or other things.
I think it's going to be interesting how.
All of these things that have developed over the last 10 years, how they kind of come back together in interesting ways to invent whatever that next future is going to look like.
That's super interesting.
Yeah, I think a lot of people also talked about agentic permissioning and identity.
That's right.
Yeah.
Part of that, too.
Like, how do you actually embed identity on the blockchain?
And also, I mean the whole question of identity is going to be really, really interesting, because there are times where I might want you, the human, to be there and be OK with that.
There might be times where I'm willing to be.
You are the agent connected with a human to be there.
And then there might be times I'm willing to let some sort of agent that is self-directed be there.
And setting up kind of the differences in those permissions is going to be really interesting.
Browsers another place.
I mean, everybody is building a browser right now.
And that's an interesting question for us.
Like how much do we lean into supporting that versus how much do we say okay, that's actually just a way of leaking data, that's back out.
And so I think getting to some sort of way of saying here's who I am as an agent and bot browser here is what I have agreed I will do with whatever it is that I'm taking from you and and you know cryptographically signed, like I have.
I agree to these things and I think that it'll probably be the same fundamental infrastructure that regulates how bots access the web that ends up regulating how browsers access the web.
And a browser that takes data and then immediately feeds it back to an LLM might have more restricted access than one that doesn't.
And that's gonna be an interesting market out there.
That's super interesting, yeah.
And I guess to the identity side, ZK is a very natural way to actually do a lot of really interesting prove.
You have the right to a credential, but not show the credential to other things that are complicated.
And so the blockchain is perfect for that.
It feels like all the building blocks for this have been coming for quite some time.
And it wasn't until fairly recently where it felt like oh, that starts to be the shape of how these blocks come together.
But it feels like both from kind of the need of the change of the business model And through the fact that technologies have matured to the point that you know they're starting to be able to handle, you know, these volumes and have the broad adoption to actually take off.
Again the sort of all the work of the last 10 years that you know if you'd asked me six months ago how is this all going to come together?
In the last few months it's felt like oh, now you can start to see the shape of what this future might look like.
I think that's what we have time for.
Thanks, Matthew.
Thank you so much for joining us.
Thanks for having me.
Find us on Twitter at NoPriorsPod.
Subscribe to our YouTube channel if you want to see our faces.
Follow the show on Apple Podcasts, Spotify, or wherever you listen.
That way you get a new episode every week.
And sign up for emails or find transcripts for every episode at no-priors.com.
