You're listening to TIP.
Hey, everyone.
Welcome to this Wednesday's release of Infinite Tech.
Just like Bitcoin separated money from the state, decentralized inference is now separating AI from big tech.
It's a quiet revolution, shifting control of intelligence itself from the centralized data centers to individuals and small developers who can run powerful models privately, securely and anywhere in the world.
Today, I'm joined by Mark Suman, founder of Maple AI, to unpack how this is being possible through trusted execution environments, secure hardware that protects both data and computation.
It's a glimpse into the foundation of a truly open AI ecosystem.
And so without further delay, let's jump right into the interview.
You're listening to Infinite Tech by the Investors Podcast Network, hosted by Preston Pysh.
We explore Bitcoin AI robotics, longevity and other exponential technologies through a lens of abundance and sound money.
Join us as we connect the breakthroughs shaping the next decade and beyond, empowering you to harness the future today.
Hey, everyone.
Welcome to the show.
I'm here with Mark Suman and I'm really excited to have this conversation sir, because this is such an important topic crazy importance.
And I think it's only getting started, but I think everybody's going to come to the realization how important this topic is in the coming five to 10 years.
So welcome to the show.
Excited to have you here and really excited to get into this.
Thank you.
Yeah, I'm excited to be on here.
I've listened to your show quite a bit, so it's cool to be on here and chatting with you.
I'm honored, sir.
I'm honored.
Let's start here because I'm fascinated by your background.
You worked at Apple for many years as a software engineer, working on privacy, machine learning and computer vision.
On the privacy front.
I think this is something that is super relevant to where we're going to go with open source, decentralized AI, which is what you're building here with Maple AI.
But what did you see while you were there at Apple that encouraged you or gave you the motivation to go out and start what you're doing right now?
Yeah, sure.
So privacy has been part of my career from the beginning.
I started off doing online backup software for people back in like the early I don't know the 2000s right.
The aughts.
And it was all about how do we save your computer into this new cloud thing that everybody's talking about.
But we wanted to offer people a private way to do it.
Because you could back up all your photos to someone's computer.
And that person who runs a computer can see everything.
So we would provide people with this private key that they could use on their computer and encrypt everything before they sent to the cloud.
That's kind of where I got my start.
And so privacy was always kind of part of who I was.
Fast forward to when I joined Apple.
And on day one my new manager sits down with me and says I want you to build this thing that we're going to use in the retail stores.
But we have to do it in a way that's totally private because Apple cares about privacy, right?
It's one of the core things.
What I can say, it truly is one of the core tenets of Apple, seeing it on the inside.
So from probably the third week of my project, I was engaged with a privacy lawyer.
And they were part of the journey throughout the whole thing.
And it's like okay, how do we build this thing?
Normal companies would just capture someone's face and capture their identity and look at their banking transactions and all these things, right.
Normal companies would do that.
Apple doesn't do it that way, right?
We have to separate all this stuff.
We have to find ways to do it that is totally privacy preserving.
So...
It made things difficult.
We had to innovate and invent new things that nobody was doing.
We had to invent totally new tools for tagging and annotating AI training data and machine learning training data in ways that were totally privacy preserving.
So it's some really cool stuff.
And I will just say it's truly part of who they are.
So, when I look at all this AI race and you see who's emerging as the dominant players, you got Google, you got OpenAI, you got XAI, which is really coming from behind.
And then you have Apple.
Like it was almost like an ongoing joke amongst friends with the Apple intelligence and how slow it's been and and how they're just really they don't seem to be playing in this space.
You look at all the GPUs that these companies that I just named are standing up and buying and all the hardware infrastructure that they're building out.
And Apple just kind of seems to be standing on their focus on privacy and why they haven't been able to play in that space.
Or is it just the bad leadership?
What is this?
Like, help me understand what's going on over there.
Yeah, I'll definitely put out the plug there that I'm not speaking on behalf of Apple.
I'm not representative of Apple, right?
But from my own personal view and things that I can talk about that have been publicly mentioned...
There's definitely a privacy angle to it because everything moves slower.
Apple wants to do it in the way that's most private.
They announced their Apple private cloud compute, what, two years ago now?
And so they're using secure enclaves.
They're trying to do it in a way that's verifiable. they don't open source their code.
And so you're trusting third-party auditors who have maybe seen images of the server code.
So there is still a layer of trust there.
But They're trying to do it in a way, from what I can tell, that is responsible.
I don't want to use the word responsible, because in the AI space, responsible tends to mean censorship.
But they are trying to do it in a way that cares about the user.
So I think that's an element.
And then also, Apple is just a giant company.
They have over 100,000 employees.
And so you're going to have different organizations.
You're going to have the typical turf wars that go on in any large organization, where someone wants to build out their headcount and Maybe there's five competing products being built internally and one gets funding, the other one doesn't.
So there's probably a lot of that going on, too.
I'm not going to dismiss that aspect of it.
So I think it's both of those at play here.
Yeah.
Yeah.
Help educate us just on.
Well, let's start here.
So when you're just looking at this from the viewpoint, viewer's standpoint of open source decentralized, whatever terminology help us understand what you think the best terminology for this is.
Preserving your data, everything that you put into these AI models, is it possible?
I'm assuming the answer is yes.
But give us your 60 second overview of how you see the world as it comes to AI moving forward and how people can preserve and protect their data that they're feeding into these models.
Sure.
The word that I would use is verifiable which, coming from the Bitcoin space, it's always.
Don't trust, verify.
And I apply that exact same thing to AI.
And verifiable can mean different things, right?
It's not prescribing a specific technology.
It's prescribing an ideology of being able to understand inspect, look at what's going on with your data.
So verifiable could mean open source code.
It could mean running the LLM locally on your computer.
If you want to take advantage of really powerful cloud compute, then run it in the cloud, but use something like secure enclaves in trusted execution environments that can give you mathematical proof The server matches the open source code that's on GitHub that kind of thing.
So really, it's being able to inspect, it's being able to verify everything that you're running, whether it's the LLM, whether it's your data in your AI memory that you're hosting.
You want to be able to look at everything so that nothing is kind of hidden in there that you don't know about.
Help us understand the threat here.
How bad could this get five, 10 years from now?
Is this something that people should be very concerned with or just moderately concerned with?
Because I can tell you, I have an open AI account.
I'm feeding stuff.
I find myself feeding stuff into this that three or four years ago I'd be like ah no, I wouldn't be using it that much.
But I find myself using these models all day long, asking it questions.
I can fully understand how it just knows everything about me at this point based on the things that I've, that I find myself, and I suspect many people that are listening to this are also participating in.
And I'm concerned that you're going to get to a point where you get so addicted to using these types of models that it's really, really hard to wean yourself off of it.
So what is this threat?
What?
And is there a point in the future where everybody comes to a realization of how terribly dangerous this really is?
Or do we just keep going down this path where we just keep beating it more and more?
Yeah.
I mean, I don't fault anybody for using these tools.
Convenience is... is an amazing thing, right?
It's why technology exists.
Technology comes around to make things more convenient for people.
It adds value into their life.
And so they grasp onto it.
And then there's always trade-offs.
And so I have an OpenAI account.
I've got a Grok account.
I use them And I use them in a way that I'm trying to minimize my exposure, my privacy exposure.
I also obviously build Maple and me and my co-founder.
And so I use that for different purposes.
But the threat that I see you talk about five to 10 years down the road.
The difficult thing I heard it described recently chatting with a friend is that, as you kind of give away your thought process to a proprietary AI service, right.
It takes that.
And there's really no getting it back, right?
They have it now forever, and they can make as many copies as they want to.
And then they can choose to put that into their model.
They can choose to manipulate it if they want to.
They can do whatever they want to with that data, and you're just not getting it back.
So 5-10 years down the road, if you look at what's unique about you as a human, it's really the way that you think.
Your face is unique, for sure, but you could probably find a pretty good doppelganger out there that looks similar to you.
But your memories, your thought process, the way that you perceive the world, is probably the most unique thing about you.
And if you've kind of given up that thinking process to another machine that has now captured it and can train off of that, we might be giving up the thing that makes us uniquely human.
It's.
I think a threat could be viewed in that lens of.
Are we turning over some of our humanity to a proprietary system?
And I'm working on a long form article about this right now.
It's a phrase I'm calling subconscious censorship.
And we can dive into that if you want to.
But it's really this notion that these proprietary systems capture your memories and capture your thought process.
And then they can be instructed, given directives, to alter your memory, to be more mainstream or be less mainstream.
They can guide you and direct you how they want.
And that's really the threat, right?
Is the model could, if the desire is there to let's say it's somebody with very influential that's using one of these accounts let's say that you want to start shaping and transforming what they think.
You can go in there and exploit it in a way, because you kind of know what their desires are and what their interests are.
And then you just kind of slowly correct it by putting it into a rut and in the direction that you want it to be shaped.
Is that really the deep concern or the risk for some people that are using these over time?
Yeah.
And we've seen those methods already used with social media feeds, right?
We all talk about the algorithms and how we don't like them.
We don't like the for you tab on X. We'd rather have the time, the chronological timeline.
Same thing with Instagram.
People got very upset when Instagram flipped over to an algorithmic timeline.
But at the same time, we just keep using it.
And so we've seen how, just by the way that they order the posts, they can affect your emotional state, right.
So maybe there's a piece of good news that you're going to see on your timeline, but for whatever reason, they're motivated to keep you in an angry state.
So they're going to show you something that's really maddening right before that good news.
So now you're in a totally different headspace when you receive that good news and maybe it downplays it.
If you take some of those tools of persuasion that they use and apply it to AI.
Well, now AI knows you intimately.
And so it knows where to place an anchor of a false fact in this output that it gives you.
So it's guiding you and now it's emotionally triggering you.
And it can do all these things in a very subtle way.
And it can repeat them and do them and change the permutation of it in thousands of iterations over the course of weeks months, years that we're working with it, until the point where we don't realize that we've been kind of guided into this rut, as you call it, and led a different direction.
Today.
I don't suspect that these large models that we're using call it XAI or even OpenAI.
I don't think they're being used in a way that's trying to psychologically manipulate people, but I think it can go there really fast.
And I think that's the concern, right?
Is when does the government get their hands potentially in something like that and then start using it for exploitation, as opposed to just an everyday tool that's being tailored to help you out or to make your life better?
Yeah, I'll jump in there.
I don't want to be like super doom and gloom.
Yeah, neither do I. Right?
Yeah.
I tend to view technology as this amazing gift that we have and this thing that we're building.
And so I love it.
And obviously, I love using AI because I'm working on that daily in my life.
And so I want to just call out this vulnerability that I see.
And I think that the mitigation is really verifiable AI.
So if we use models that have been trained on open data sets and we can see the weights, we can see the biases that went into training.
And then we can see the code that we're running and we can run it locally on our own data, or we're running it in the cloud with something like trusted execution environments.
These kinds of things allow us to see what's going on.
Now we can kind of have our cake and eat it too.
We can use these powerful models and this powerful technology without having this risk that we are going to be led away.
Because I think you're right.
I think right now we're not seeing that kind of manipulation going on.
We have seen them introduce things like advertising and shopping experiences.
And a few weeks ago, it was, hey, ChatGPT is going to work while you sleep.
And when you wake up, you're going to have recommendations in front of you.
Which is basically like we're going to give you advertisements, in front of you in the morning, of all these cool things that you want to get.
And you can see how that can quickly turn into something more.
They're building this tool that could be used in other ways.
And I would prefer...
To basically not give myself to that system and instead go down a path of openness and verifiability.
Yeah.
In the example that you provided earlier, as far as like social.
That AI has been used for, almost like a dark way.
When you start talking about social media, what's coming into the feed?
But as far as the chat context windows and the interaction, it seems like we're still very early in building that all out, so that it hasn't necessarily entered into that form yet.
I think anybody listening to this is saying I like the fact that it kind of knows me and can give me just tailored, custom-made responses.
I love that.
I just wish that I knew it was partitioned or on my own server and that nobody else could see that data.
And it was something that was inside of my control.
I think everybody listening to this would agree with that statement.
So walk us through, because that's what you're trying to do, right?
Walk us through how you're trying to do this.
Okay.
Yeah.
And I think what you just described is what most people want, right?
This is the tale as old as time with privacy technology and freedom technology is.
We know that we should be probably being better about our data and better about our information that we share, but it's just so useful and so convenient.
And we see so much productivity gains from using some new technology that we're willing to close our eyes and plug our nose as we use them.
And I'm just as guilty as everybody else with doing that, because we have to make trade-offs in our personal life to do that.
And I mean, we haven't really talked about the potential for data leaks, right?
And maybe I'll just drop this in here really quick, and then I'll answer your question directly.
But we've seen with ChachiBT, with Grok, most recently, both of them had bugs in their software where chats were being indexed on Google search results.
And so I don't know if you saw this was like a month or two ago.
So people were searching for stuff and finding personal chats that they had made.
These were specifically around the share button.
So in ChatGPD you can click share.
It gives you a private link that you can send to somebody, and now somebody else can read that chat.
And so it was still meant to be private between you and someone else, but Google was picking up on those and now people could search.
And it would be things like somebody's chatting about their marriage and marriage difficulties they're having.
And then they send it to their spouse and say okay, here's what our AI therapist on ChatGPT told us.
And now their marriage details are spilled out onto the internet.
So it's when you give somebody else your data, that's a risk you're taking on is to have stuff like that happen.
What we're trying to build is we're trying to build verifiable AI that people can use so they can see everything through the process.
We build everything in the open.
All of our code is online.
Before we push anything to the servers, before we push anything as an update that you can download, you can see the code first so people can inspect it.
And then we also know that Local AI is really the most private AI, something you can run on your phone, you can run on your computer.
It's never going to get more private than that.
Turn off the internet, talk directly to it.
You can inspect it before you use it.
That's the utopia right there.
But not everybody has a powerful enough device yet to do that.
They cost tens of thousands of dollars to run these big, largest open source models.
So we're trying to give people an in-between.
We run secure enclaves in the cloud.
We push our code there.
And then what it does is it gives you It's called an attestation, which is really just a mathematical proof.
And it's a way to match.
So it's a way to say, okay, you have this code that's open source on GitHub.
But how do I know that you're actually running that exact same code on your servers?
So there's a lot of other private AIs out there that say, hey, here's our open source code.
We're private.
We're not tracking what you do.
But you can't actually check.
You can't verify that.
So we're trying to be as transparent as possible.
And so we provide that mathematical check.
So when you go into Maple, you get this little green verified checkmark.
You can click on that.
You can see all the details.
It's really similar to that lock icon.
When you go in your web browser and log into your bank, you get that secure socket layer lock icon HTTPS.
I view this as like the next iteration.
The internet started open with HTTP and everybody was just like going around websites, viewing them.
And then when we started having usernames and passwords, we had to come up with something better.
So we did HTTPS.
Now, I view this as a third iteration.
I'm calling it HTTPS-E for secure enclaves, which is my own fun little moniker.
But it's really this way that we can now verify the code running on the servers, because we're trusting the cloud with so much more every day.
So we need to provide a way for people to verify that.
So that's... effectively what we've built with Maple.
But I'll say one more thing and then I'll let you respond if you want to.
But what we're trying to do is we know that people don't want to give up their convenience just for the sake of privacy.
It's a really tough sale to make.
Very tough.
Yeah.
Right?
It's really tough.
So we are going to build... effectively ChatGPT, but it's going to be better, right?
We're building ChatGPT and it's going to have privacy at the core.
And so we are going to give people all of those core amazing features that they get out of ChatGPT and Grok.
But they're also going to have privacy built into it, rather than someone who is harvesting their data for other business purposes.
Let's take a quick break and hear from today's sponsors.
Have you ever been interested in mining Bitcoin?
As a miner myself.
I've been using simple mining for the past few months, and the experience has been nothing short of seamless.
I mine with the pool of my choice and the Bitcoin is sent directly to my wallet.
Simple Mining, based in Cedar Falls Iowa, offers a premium white glove service designed for everyone, from individual enthusiasts to large scale miners.
They've been in business for three and a half years and currently operate more than 10000 Bitcoin miners.
Based in Iowa, their electricity is over 65% renewable thanks to the abundance of wind energy.
Not only do they simplify mining with their top notch hosting and onsite repair services, but they also help you benefit financially by running your operations as a business.
This approach offers significant tax advantages and enhances the profitability of your investment.
We'll see you next time.
When you're ready to upgrade or adjust your setup.
Their exclusive marketplace provides a seamless way to resell your equipment.
Join me and many satisfied miners who have simplified their Bitcoin mining journey.
Visit simple mining.io slash Preston to get started today.
That's simple mining.io slash Preston to get started today with simple mining.
They make it simple.
Listen, if you're a money nerd like me, then you need an easy way to track your net worth in real time.
And that's why I'm thrilled to tell you about today's apps for this, and Kubera is definitely the best one I could find.
It links in all my bank accounts, investment accounts and private investments and it gives me a clear, real-time overview of everything I own.
One of the issues I found with other apps is that my accounts constantly get disconnected or I'm not able to get them connected at all to give me that fast forward feature.
It lets me model out different life scenarios so I can say if I'm making this much money in five years, how much will my net worth change over time?
Or if I decide to buy a house next year and put this much money down, what will my net worth be in five 10, 20 years down the line?
That way, I always have a clear picture of what's next, to ensure I continue to hit my financial goals.
Again, I'm thrilled to have Kubera as a sponsor.
I use their app all the time.
And if you want to stay on top of your wealth, you can get 100 off your first year subscription at kuberacom dot com.
Slash WSB.
That's K-U-B-E-R-A dot com slash WSB.
Startups move fast.
And with AI, they're shipping even faster and attracting enterprise buyers sooner.
But big deals bring even bigger security and compliance requirements.
A SOC 2 isn't always enough.
The right kind of security can make a deal or break it.
But what founder or engineer can afford to take time away from building their company?
Vanta's AI and automation make it easy to get big deal blocked.
Plus, Vanta scales with you, backed by support that's there when you need it every step of the way.
With AI, changing regulations and buyer's expectations.
Vanta knows what's needed and when, and they've built the fastest, easiest path to help you get there.
That's why serious startups get secure early with Vanta.
Our listeners get $1,000 off at vanta.com billionaires.
That's V-A-N-T-A.com billionaires for $1,000 off.
All right, back to the show.
The challenge on that last part, Mark, is people hear that.
They know that these large models, especially the newer ones that call it GPT-5 or GROK-4 heavy, they're training even more powerful ones.
Getting access to that in a way that you can leverage it in an open source kind of way is the challenge.
And people are just looking at the performance of these models.
I know I've played around with these models and I'm saying you know, the open source version or the open weight versions are just not even close to what these newest models are doing.
Is that a temporary, something that is going to continue to persist for the next two or three years?
And then all of a sudden, the open source or the open weight versions are going to be up to speed with where some of these larger models are, or are they always going to kind of be outpacing where the open weight models are at?
I think that's the concern that people have before switching over to something like this.
Yeah, it's a valid concern.
Especially in the early days, the open models were significantly worse than the proprietary ones.
But we've seen an acceleration.
I mean ChatGPT came on the scene two and a half years ago.
Maybe three years ago is when it really caught on.
And we've seen the open models catch up a ton in that timeframe.
They were like 50% as good, then 75%.
Now they're like in the 90% range.
You have a coding model, QEN3 coder, which is scoring just as good as some of the proprietary models on programming in some areas, not all areas.
And we're seeing a point where the benchmarks however you want to define that are really getting similar.
And then it comes down to just using it and seeing how it behaves for you.
And really most people don't need to have that extra 3 in their model to really get a lot of value out of it.
And then the other thing we're seeing is you bring up GPT-5.
And arguably, GPT-5... is an incremental increase over GPT-4.
And a lot of people complained and wanted them to go back to GPT-4 and make it available again.
And some people have done introspection into the routing technology behind GPT-5 and think that there's actually still a lot of GPT-4 just under the hood.
That's helping to power version 5.
And so you look at that and you say, okay, maybe their progress has slowed down just a little bit.
And then also they open sourced GPT-OSS, which is really just kind of 4.0 under the hood.
And so you're seeing that open source from their standpoint is starting to catch up.
And then you have this whole market dynamic of the Chinese models.
You have DeepSeek, you have Quinn, you have these other ones.
I'm blanking on the other one right now.
But you have these other ones that are coming up.
And in order for them to compete with these big proprietary models, they're going to open first and they're trying to be as good so that they can compete.
So I think we are seeing this world where the open source catches up just enough that it becomes just as valuable to a regular person, mainstream person, than these other models.
What's the business model for them to go open source like that?
That's the challenge of I'm continuing to struggle with.
What's the incentive for them to go that route?
How are they going to make money doing that?
It's a good question.
One that I'm still trying to figure out.
Maybe you're trying to build this model that you just want to spread around the world.
And so... you're going to build that.
And the best way to get out there is just give it away for free.
So that could be one thing.
The difference with the open source and running the open models is you can see exactly what's going on.
And so you can build around that and build around that propaganda.
So...
I don't have a good answer for you on what's the business model, other than it seems like if you can't compete from a proprietary standpoint, then you go open and you spread your message far and wide And try to catch up that way.
Do we get to a point where these large language models kind of start peaking out and it just doesn't make sense to train them with even more?
You know the plow and so much energy into these latest models.
Is there a point where they get like peaked out?
Call it two years from now and there's no longer this race to just build an even larger compression of all the information on the internet.
That's basically what we're doing, right?
Yeah.
It's hard to guess five, 10 years from now, even two years from now, because AI is moving so fast.
But I think what we start to see is a slowdown of the general models.
And then we start to see more specific models.
The first thing we're seeing is with coding.
People want to program.
And very quickly we've seen that a model that is fine-tuned at just software engineering is performing generally better.
And so we're seeing a divergence there.
And I think we start to see the medical field and legal field and all these other different industries come up with their specific models.
I know there's people working on therapy ones.
And then we have these general models that act as routers in the front.
So you come up to it and you start talking to it and it pulls in a specialist.
And we can really go deep and dive deep when we have those specific models.
So I think that might be the next thing is how do we have specialization?
And then how do we have these general models, kind of wheel and deal and be the general contractor, if you will.
How do you think through stitching all these different models together?
So, when a person creates an account, walk us through the process of creating an account at trimapleai, which is the website that you've built in order for people to run their own AI locally.
Walk us through what that is.
And then, more importantly, how are you stitching the different models together to provide the experience that the user has?
Help us understand that.
Sure.
Sure.
Yeah.
So first off with Maple, right now we're only in the cloud.
We want to provide local stuff as well.
It's like a hybrid local cloud, where all your data is encrypted locally, first on your device, and we use a private key.
Coming from the Bitcoin world, we understand the power of a private key.
Same with Nostra, right?
And so we apply that here.
So you're chatting with AI locally on your phone.
It encrypts it and then sends it to the cloud for processing.
In the cloud, a secure enclave uses your private key, decrypts it, gives it to the AI.
It comes with a response, re-encrypts it with your private key and sends it back to you.
So we are not in the middle.
We can't see anything going across the wires.
Only the secure enclave sees the personal data.
But you can go look at the source code and see that there's nothing going on there.
So how do we tailor that experience?
Right now, we just give you a model picker and you're having to choose.
And we have a lot of users telling us they get almost like low key anxiety from like trying to pick which one's the best model that I should use right now.
So part of our big 20 push that we have coming up is we want to build something that helps guide the user.
And say like, I am in big brain thinking mode right now.
So I'm going to click on this thing and it's going to drop me into a model that helps me do that.
Or I'm just in quick trivia mode.
I want to look up something.
So it's going to drop me in there.
In the beginning, it'll be like just an easy picker to do for the user.
But then we'll switch over to an auto mode where they just chat and it knows what to do.
And you just put a simple classifier in front of it.
So it looks at your prompt and it can quickly determine itself what should be used, just like you would.
You would say, I'm in this mode.
Here's what I'm thinking.
And you would pick that.
Well, it'll do that for you.
So we want to get more automatic with that, but always provide these advanced features that people can turn back on and be more selective if they want to.
Love that.
The next thing that I personally want to see.
So, with OpenAI, I'm having a chat and I'll say this is really important.
I want you to remember this for other context windows and other discussions that I have with you.
Then it compresses whatever that is.
It puts it into like this, my understanding of Preston Memory Bank.
Are you guys working on something like this?
Because I think this is something that people really want to have, but they want to have it in a private way.
I know I personally want to have this in a private way.
I hate every time I do this.
But I also find myself going back to similar conversations where that context is really important.
And I hate just I have to type it in every time I'm in a new context window.
So is this something you guys are working on?
And where do you see the roadmap for something like this, if ever?
Yeah, definitely.
What you just described.
There are two different implementations that kind of help out with the same thing.
So one way that the ChatGPT does it is they have these custom GPTs where you can set up this thing, and it has a lot of the context pre-built into it.
It's basically like you typed in a system prompt with all the stuff that you want.
And for people listening who maybe aren't like Super, into AI, a system prompt is basically the instructions that you give to the model.
You give to the AI to say hey, when I talk to you, I want you to kind of be in this personality or this frame of mind or as this character, as I'm talking to you.
So you can get silly and you can say like, I want you to talk like a pirate to me.
So every time you chat with it, it's going to talk like a pirate.
That's like the extreme example.
But more nuanced, you can be like, hey, I am going to have a legal discussion with you right now.
So I want you to be a lawyer.
I want you to be a contract lawyer.
And I want you to have these qualities about you.
So to me, that's one part of what you just described.
And we definitely do want to do that.
We have a system prompt you can edit.
We show you what the system prompt is.
We want to have multiple in the future where you can customize those and maybe a dropdown or something.
Say, hey, I'm in legal mode right now.
Switch into that mode.
Mm-hmm.
The other aspect you just described is the memory side of it.
And we are definitely working on that.
We're going to have an open source memory component to it.
We don't know exactly which direction we're going to go yet with it, but it's going to be something where you will see okay, here's everything that the AI has learned about you.
And AI memory is really fascinating because I like to view it as you're sitting down with a biographer, right.
Say you're Steve Jobs and you want to have everybody know about your life, so you get the best person out there to write biographies.
That's what's happening with you every day as you're using ChatGPT or any AI product.
It is sitting down and trying to learn everything he can about you.
Here's how he thinks.
Here are his childhood memories, yada, yada, yada.
The difference is in a proprietary system, you don't get to read that biography.
You don't really get to see what's in there.
They will show you an interface that says, oh, here's the things we know about you.
And we'll even let you delete it.
But there's no guarantee that that's actually happening, right?
If you delete that thing out of there, it probably still remembers it.
But it's just like, oh, we'll tell them that we're not going to use it.
But we could use it if we want to.
What we want to build is a truly sovereign AI memory where you can go in and see what we remember about you.
Not we, what the system remembers about you.
And then you can edit it, you can add to it, and then that will get pulled into future chats.
And so...
With those two combined, the system prompt personality thing, that's more proactive.
You can say, I want you in this mode.
Whereas the memory side is more passive.
It's like, hey, this is my context about me.
So use it selectively as you see fit.
Let's take a quick break and hear from today's sponsors.
Picture this it's midnight.
You're lying in bed, scrolling through this new website you found and hitting the add to cart button on that item you've been looking for.
Once you're ready to check out, you remember that your wallet is in your living room and you don't want to get out of bed to go get it.
Just as you're getting ready to abandon your cart, that's when you see it, that purple shop button.
That shop button has all of your payment and shipping info saved, saving you time while in the comfort of your own bed.
That's Shopify, and there's a reason so many businesses, including mine, sell with it.
Because Shopify makes everything easier, from checkout to creating your own storefront.
Shopify is the commerce platform behind millions of businesses all around the world and 10 of all e-commerce in the US.
From household names like Mattel and Gymshark to brands like mine that are still getting started.
And Shopify gives you access to the best converting checkout on the planet.
Turn your big business idea into reality with Shopify on your side and thank me later.
Sign up for your $1 per month trial and start selling today at Shopify.com slash WSB.
That's Shopify.com slash WSB. there's a better way to meet face to face.
Remarkable Paper Pro Move.
It's a paper tablet, a digital notebook that combines the familiar feel of paper with the digital powers of a tablet.
Start by taking notes with any of the dozens of built-in templates, then turn your handwriting into typed text and share it by email or Slack.
Think about it.
The ways we capture thoughts on the go, all of their drawbacks.
Paper's hard to organize and easy to lose.
Laptops are too bulky and uncomfortable.
And on our phones, our attention can quickly get hijacked.
Remarkable Paper Pro Move is nothing like your other devices.
It has a display that looks, feels, and even sounds like paper.
It can fit all of your notes and documents and lasts up to two weeks on a single charge, but it slips easily into your jacket pocket.
And most importantly, Remarkable's mission is about helping you think better.
That means no apps, social media, or any other distractions, just you and your thoughts.
You can try the Remarkable Paper Pro move for 50 days Get your money back.
Visit remarkable.com to learn more and get your paper tablet today.
That's remarkable.com.
Support for this show comes from public.com.
You're thoughtful about where your money goes.
You've got your core holdings, some recurring crypto buys.
That's why they built an investing platform for those who take it seriously.
On public, you put together a multi-asset portfolio for the long haul.
Stocks, bonds, options, crypto, it's all there.
Go to public.com slash WSB and earn an uncapped 1% bonus when you transfer your portfolio.
That's public.com slash WSB, paid for by Public Investing.
Full disclosures in the podcast description.
All right, back to the show.
For me, the ladder there where it's really kind of understanding your past and just understanding the essence of who you are and what it is that you want, is super powerful and useful as I continue to interact with this.
I'm curious what the challenges are from an engineering standpoint to put something like that together, because it seems like it could dominate as you go into a new context window, talking about a brand new topic.
That, when it has this memory that it's pulling from to be customized responses that it could potentially dominate the next conversation, that has nothing to do with the memories that you're asking it to hold.
How do you think through the problem?
And I'm just kind of curious is that going to be a really difficult feature to kind of roll out as you continue to build this out?
Yeah, absolutely.
That is one of the biggest challenges with AI memory right now is that it overweights and overemphasizes something that you give it.
Because you think about your brain, you've been storing up memories for decades in your brain.
And you know, without realizing it.
You know when to selectively pull on something and when to not pull on something and use it in the decision that you're trying to make.
Whereas AI, maybe it has right now, maybe it has like two pages of information about you.
And so that one memory you tell it is suddenly like one of the most important things it knows about you.
So it's going to like overly push it into the things that you're using.
So we're trying to figure out how do we downweight that and how we not have it influence everything.
I think a lot of that comes down to.
You have to get really good at annotating information.
When it goes in and say okay, this little memory you have, this memory is really focused around finances, or this is focused around health or being outside mountain biking or something right.
And you want to classify that.
So when you're having a chat, it knows hey, I can totally ignore that right now because it has nothing to do with this topic.
Rather than like, oh, you're mountain biking today.
Hey, you should bring along your financial advisor and have a good chat about your IRA or something like that would be really dumb.
So we do have to figure that out.
And I think it goes back to that verifiability thing is I want to know what the AI is doing and which memories it's suppressing and why, rather than a closed, opaque system that is going to do that.
Make that decision for me.
Because maybe that memory is really important and relevant right now to something we're talking about, but it is choosing to not make it relevant for whatever reason.
Could be accidental, could be profit-driven, could be nefarious.
One of the things that I've heard more recently is that inference is where is going to be the competitive mode moving forward in the coming five to 10 years?
You've got the training of the model itself, which we were talking about earlier with some large language models.
And then you have the inference, which is using the trained model, that you spent all this energy and poured into all these GPUs that compress all of it into a single large language model.
The inference is using that trained model to generate the answers that you're getting back.
And every time you prompt, you ask a question and it goes into this model.
This happens every time you prompt it.
It requires...
GPU memory and you have to run it through that entire model and then it pops out the answer.
And this is the inference process.
And what I've heard is that, in the future, the speed of that inference and then the cost to do it as efficiently as possible but still give you the high quality answer, is where the competitive edge is going to have the winners and the losers.
I'm curious if you agree with this.
And then, if you do, how do you think about that inference piece with your company and And being able to provide an efficient and quality fast response to people, as they're doing this in an open weight, decentralized kind of way?
Yeah.
So inference versus training, the costs involved and the power involved are very different, right?
So the training part costs, I don't know what the exact numbers are.
Let's just say 10X.
It might be bigger, it might be smaller.
But yeah, it's like it takes 10x the amount of resources to train something.
Just like as a human.
It takes all this time to train you, over decades, for you to live life and learn all these things.
And then eventually, you can sit down and have a fruitful conversation like we are right now, right?
And so it's easier for us to have this conversation than it was for us to learn everything we learned up to this point so that we are capable of having this conversation.
So inference should be viewed that way.
Now you're ready to have a chat.
And I see that.
What's the moat?
What's the unique thing that's going to be competitive?
And that is just the user experience.
And so these apps that we're building on top of the inference are going to be the competitive moat, and what different qualities they have.
And we're already seeing that with ChatGPT and some of these others.
They're trying to build apps on top of their inference layer now that really pull people in.
The latest is the Sora video app that's pulling people in and trying to make it more engaging right.
As far as inference goes, I think that just only comes down in cost over time.
And even though we're going to get bigger models, we're going to build chips that are more efficient for processing those models.
Apple, even though they don't have the right AI solution yet, according to the market, they have built these chips into every single device that are just highly specialized at processing these models.
So one thing that we're looking at with Maple is doing a hybrid approach where you actually have smaller local models that run incredibly fast and are extremely cheap to run because they're just running on your spare cycles on your device.
And they will do a bunch of the initial processing and on some of the most sensitive information and they will come up with the most efficient prompt to give to the cloud model.
So you might go in and bang out this massive prompt paste in a whole PDF of information and then the local model will crunch that all and say okay, this is all good and dandy, but really what I need to pass on is a smaller chunk.
It'll pass it on to the cloud.
That'll get processed on the more expensive servers and then come back to you.
And I think in a model like that, inference continues just to drive into the ground as far as price goes and gets faster as well.
And so we end up with a better user experience.
And so the people that can console that kind of user experience are going to have a better moat, be better as a competitive advantage.
Yeah.
I recently read that XAI has custom ASICs that they're building just to improve the speed of the inference.
And I guess it's 10 to 20 times faster than some of the best GPUs on the market right now, just because they custom made it for that specific task, which is fascinating.
And it also just I guess it leans into this idea that I think it is challenging to compete with some of these larger you know, like XAI.
It's going to be a bloodbath of competition to compete against, because they're going to be able to provide such quick and efficient and quality responses, because they're going out and doing things like this that are very capital intensive.
I'm just looking at this whole space and I'm curious your thoughts on Just how expensive some of this stuff is.
And, you know, we saw the thing with OpenAI and NVIDIA and help me out with the other Oracle.
Yeah, like this and that broadcast.
Tom is in the mix too.
Yeah.
The numbers are crazy, Mark.
Absolutely nuts.
How do you see that kind of resolving itself?
And like, where is that going?
Is it just going to keep going more?
Yeah.
It's crazy.
I mean, I saw somebody comment over the last 24 hours about the whole Broadcom and OpenAI thing where it's like hey OpenAI, we want to buy all these chips from you, but we don't have the money to pay for them.
And so they basically say, let's do a press release together.
Broadcom stock goes up.
Now their market cap has gone up $150 billion.
And it's like, boom, there's your money that you needed.
So we'll loan you our market cap basically to help you out.
So it's this crazy thing, a lot of money being tossed around.
Where does this resolve?
Oh, man, I wish I had a crystal ball to understand.
But I think that we are going to have big players out there.
We're going to have these people who are building these big, massive mainstream solutions.
And...
I also think that you look at the government contracts that have come to these major models, right?
They gave 200 million to XAI, 200 million to OpenAI, to Anthropic.
I think Meta got that too.
I can't remember.
So there are bigger things at play here with Department of Defense and other governments around the world.
So I think that there is going to be a need for large scale systems like that.
There's also a need for other people to build out systems.
And I think there's a world where they all exist together.
I don't think this is going to be a race to where there's going to be one winner take all.
Because really, there are so many different ways to approach intelligence in this life.
And there are so many different avenues and so many different needs that Grok's not going to be able to solve them all.
ChatGPT is not going to solve them all.
And so I don't know where all the money resolves.
I think we're definitely going to have a bubble at some point that's going to pop.
It's a view of very similar to the internet.
And so we're going to have all these companies that overinvest.
And then there's going to be a retraction and a retracement back.
And the winners are going to remain...
So I don't have a perfect answer for you on that, but I just think that there will be some overinvestment, but I don't think it's going to pop and go away.
There's too many benefits.
People are seeing too much productivity from AI, too much value coming out of it that it's going to survive.
It's just which people and which companies will remain standing.
It's hilarious because it's almost similar to like a meme coin pump of them raising capital and then transmute the common stock pump into capital to then invest into the hardware.
It's like I'm sure you've seen the charts of like OpenAI goes over to NVIDIA and they give money to Oracle and it's all circular.
It's this weird thing going on.
It's crazy.
What's one of the most challenging things for you right now building out this business?
Well, we're trying to keep up and we're trying to get feature parity with arguably one of the biggest companies in the world right now.
And so OpenAI and others.
They have billions of dollars to throw at designers, at engineers and trying to build the best user experience possible.
So, in order for us to get people to care about privacy, we have to give them a tool that's as convenient, as usable as ChatGPT as a baseline.
So, that's really the biggest challenge right now.
That's where we're racing is.
We're trying to figure out how do we pick off the most important features.
Because we can't build them all right now.
It's just me and another person.
We can't build them all right now.
And so let's selectively pick the most important ones, get those to feature parity, and then keep building from there and keep growing.
So we're going to be raising some money soon, hopefully, and that'll help us hire a few more people.
But even then, we're never going to match these bigger companies as far as team size.
But we're using AI against them.
We're using AI to help us build faster than we could.
And we only launched back in January.
And we're getting ready to do our 2.0 launch probably next month.
And it's come a long way just in those last nine months.
And so the next nine months next year are going to see drastic changes to what we're building in the positive direction.
That's the crazy part, the reflexivity of the AI itself as it's getting better.
You just get more and more powerful reflexes.
And, in a way, having maybe a smaller team, you're able to just kind of focus in on those features that are most important.
The things that I'm seeing on the program model seems to be almost unfathomable.
What it's one-shotting with a prompt.
Can you just kind of help us understand what's transpired just in the past year with respect to your ability to program and write code?
So there is a lot of salesmanship going on when it comes to the vibe coding space.
And so a lot of progress has been made.
And there are definitely stories where people go on and say I wrote like a few sentences and they gave me an entire app that I can use.
And those are great, I think, for proof of concept.
We've definitely seen a lot of great things in that space.
But getting an app that you wrote with just one paragraph of text into production that millions of users can use, that has covered all the edge cases and stuff, that's a totally different story.
Mm-hmm.
So I definitely see a lot of memes where it's like, oh, and software engineers are so cooked.
But really what I think the great power is is software engineers using AI and accelerating their abilities.
So that's what we're seeing.
I don't want to cast throw shade at the companies that are doing vibe coding for people who are not software engineers, because what I see is a huge value add right there.
Being a software engineer myself for decades, I get approached all the time.
Someone's like, I've got a great idea for an app.
I want to build this.
You know, can you please go build this?
And they'll draw on a little piece of paper.
They'll maybe build a PowerPoint presentation, but they try to give me requirements.
And I can see very quickly that the requirements haven't thought of everything or the idea just is kind of off base.
So where vibe coding comes in now is they can take that and, instead of coming to someone like me, they can give it to an AI.
It can build in the proof of concept.
They can play with it and they can say, oh, this is a piece of garbage, or this is a great idea.
Let me iterate a bit on this.
And so now, when you go to approach someone to build it for you, you've got this really refined proof of concept that conveys your idea and has thought through a lot of the initial things.
And then for us...
We've taken AI and built it into all sorts of parts of our process.
So we're using tools locally where we are running coding environments, IDEs locally with something like Cloud Code.
We've tried out Codex from Chachapiti.
We're using Factory right now also as kind of a new hotness that's come on.
So we're using all of these.
We also use Maple, we've got Quin3 Coder, where we got that plugged into IDEs.
So we're using all these tools.
And then when we check in code to GitHub, do a pull request, we have two other AIs that hop in there as code review agents.
And so they're both reviewing the code and they come from two different models, two different companies.
And so they give a different perspective.
And so they drop in their comments and say hey, this line of code is maybe you should think through this more, there's a potential bug here, that kind of stuff.
And then we go in and we say, hey, Claude, respond to these comments from the poll requests.
And so you have these agents that are really helping out.
But ultimately, in the end, we are the ones reviewing the final say on the code.
And we might say, hey, we don't like the approach they all took.
So let's get in there and bang things out a little bit differently.
But, truth be told, I think probably like 90 of our code, maybe 95 of our code is written by AI, with the human in there directing it, guiding it, inspecting it and making sure that it comes out correctly.
Would you say that your time has been multiplied 10x?
What used to take you 10 hours, you can do in one hour now?
Yeah, I haven't measured it specifically, but that's definitely the vibe that I feel.
I look at what we've built in the last nine months and launched.
And we have a sizable number of users now, a sizable amount of revenue coming in.
And I think about trying to do that before.
I've been part of multiple startups.
I look at another startup that I was with And it took a lot longer just to get the product to market.
It took almost an entire year of just understanding and writing initial versions and then throwing those away and doing different versions to where now we're just.
Yeah, we're so accelerated.
Like I said, it's just two of us.
And so if we were doing this prior to AI, we probably would have had to have two more people, three more people, to get to this point.
So we're definitely seen an acceleration.
Yeah.
On the home hardware running this locally, like right now.
You're saying you're doing it in the cloud.
You send the encrypted data over to the cloud.
It then processes it, it sends it back encrypted.
But for people, let's say, 10 years from now.
I'm curious how you see the home market for running your own server.
I know as a Bitcoiner, I run my own node.
There seems to be some synergy there for people that are do-it-yourselfers, that like to do these types of things.
But is this going to become something that's almost like a heater in your home or like any type of appliance that's in your house?
Are people going to have their own data that's stored locally, that's run with some type of ASIC or GPU?
That's a specialized piece of hardware so that we can not give up our data?
Is that where this is kind of moving?
Do you see that trend going there?
Or do you think that that's still a very hard ask to ever expect out of people that might not have the technical competence or desire to for privacy?
Like, what are your thoughts?
Yeah.
I would love a world where everybody had their own home server plugged in and it's doing all this for them.
So 10 years from now, I could see the technology catching up to where we can make really easy plug and play.
You get this little box, you plug it in, you connect it to your Wi-Fi or plug it in via Ethernet.
Whatever device is the input device and the output, it's talking to your home server and doing everything locally.
I think we will definitely be there from a hardware perspective and from a user experience perspective.
That'll be possible.
The difference is, will people do it?
I don't know.
There's definitely incentive to keep the cloud model running.
And if you want to look at a parallel to this, you can look at email, where we all have the capability to run an email server at home.
We can host our own server.
We can be totally sovereign and have total control over it.
But we don't.
We still just go on to Google Workspaces, give it our domain name, and now our email is run by Google, because it's just so dang convenient.
And they handle all of the DevOps and IT headaches that would come along with running our own email server at home.
So I would love to think that that's the future is home AI stuff.
And maybe it could be.
Maybe this is finally the line in the sand where it's like you can have our emails but you can't have our brains.
Our brains need to live at home.
And I hope that's the delineation that people are not willing to give up that aspect of them.
So I just don't know, but we will definitely technically be there.
It'll be possible.
Last question I have for you.
You mentioned Noster earlier.
There's a lot of opinions as to what in the world Noster even is.
But one of the talking points that's constantly shared is just it's an identity layer.
And so you're talking about having a private key, public key pair to ensure that the encryption is actually being conducted between the cloud and your request coming from your phone or your computer.
Do you see Nostra as playing an important role because that's an inherent feature of the Nostra protocol, which is this public key, private key relationship that you can sign anything with.
So I see it all coming back to that word verifiable.
And that's really the power of Noster is verifying that this communication truly came from me.
Now, right?
And so that's the private key, public key promise that we have.
And whether or not it's Nostr as an open source brand.
That ends up being the thing.
I think.
That concept is what's important.
And so being able to say hey, this little piece of memory that went into my AI, that's signed with my private key.
And so I know that I'm the one who put it there and that it came from me, right?
And so I think that's where verifiability takes us.
And so that could be online communications.
I post something you know I want to sign our Maple builds and have those signed with a private key.
So I want to kind of integrate this throughout the entire process, And I think that's really where it shines.
Whether or not it becomes a replacement for Twitter, that remains to be seen.
Right now, it's a niche protocol.
But I think the power is beyond that.
People who are looking at it as a Twitter clone are not seen far enough in the future, where it's really all about verifying that this communication came from the person that, as they said they are.
Wow.
Any other parting comments or things that you think are important for the audience to know about what you're doing?
Sure.
I think what you should view like AI is as a toolbox, right?
So you have a toolbox, you have different tools to use for different things.
So I would say I'm not asking people to throw away ChachiBT.
I'm not asking to throw away these other services.
Instead, I'm asking them to add Maple into their toolbox.
And that way, when you are talking with ChachiBT and you're like, I don't really like the fact that I'm sharing my children's name and personal information about them.
You can switch over to Maple and you can have the exact same conversation with models that are as powerful or like 95 of the way they're powerful enough for you.
And it's very refreshing.
You get this refreshing feeling knowing that this is just a private room with you and an AI.
And nobody else is listening.
Nobody else is recording that information.
And it's not being sold to anybody.
You're not being influenced in any way.
And so I would just say, go to TryMaple.ai, get the free account.
You can upgrade if you want to support us whatever, but grab it and just have that extra tool in your toolbox and play around with it and start to see where it takes you and what you gain from that.
Mark, I think you're working on one of the most important things in the world right now, truly.
I wish you all the best, and I can't wait to try it out myself.
We'll have a link in the show notes for people, but it's TryMapleai, if you want to go to the website and try it out.
And this stuff is so important.
I think it's only going to get more important.
And hats off to you for what you're building.
And I really appreciate you taking the time to come on the show for the conversation.
Definitely.
Thank you, Preston.
Appreciate it.
Thank you.
Written permissions must be granted before syndication or rebroadcasting.
