Thank you. Hello and welcome to the NVIDIA AI Podcast.
I'm your host, Noah Kravitz. The AI community is abuzz with agents.
As NVIDIA CEO Jensen Huang said in his CES keynote this past January, agentic AI represents the next wave in the evolution of generative AI and a multi-trillion dollar opportunity at that.
AI agents enable applications to move beyond simple chatbot interactions to tackle complex multi-step problems through sophisticated reasoning and planning.
They're expected to become a centerpiece of enterprise AI systems going forward.
Our guest is John Heller, co-CEO and founder of FirstHand, whose brand agent platform is transforming how advertisers and publishers engage consumers online.
First Hand is also a member of Inception, NVIDIA's program for startups.
John, welcome, and thanks so much for taking the time to join the AI podcast.
Thank you. It's great to be here. I'm very excited.
So let's start with the basics, if you don't mind.
Tell us a little bit about FirstHand. sure as you said first hand is building the brand agent platform and that's because when we see ai We believe it is itself a new medium, not just a technology.
Perhaps I can explain that a little bit about how we came to be.
So I'd been working in advertising tech since the internet started in the 90s. and worked at DoubleClick for quite a long time.
More recently, it was the co-CEO and co-founder of Freewheel, which is now Comcast's ad-serving platform.
Right. And I've been doing ad tech for a long time.
And actually, when I left Comcast, I went back to school, finger quoted, in AI, so online courses.
Because I wanted to work in something that was much more broadly applicable and much earlier in its life.
It actually started by working on reinforcement learning robots to act as minions and artificial teammates in video games.
And that's how I started learning the guts of it.
I'm biting my tongue so we don't take us off course because that sounds fascinating as well.
Yeah, that could be a separate conversation.
But I've been working in the gaming world and some of the generative AI abilities for gaming assets. when language models really and something struck us. something very powerful, which is, and this is a metaphor for the math inside, but AI now understands the ideas and intense needs you may have, from what you're reading, what you're watching, what you might ask it outright, and it can go find the right response or take the right responding action.
And everything is presented to you in a very natural human way.
And if you back up a step and think of that happening all the way through a consumer's use of the digital world from when they're searching and becoming aware of things they might need, when they do some investigation and read up on products or services, when they go to browse or shop, when they buy, all of those modes change change pretty fundamentally.
They don't get replaced. We think they get enhanced because instead of the world of the did a search, got some directions and a link, went to a place, read up on something, browsed for something, went to another place to try to find the version I want.
Those are all separate hops. the internet where it's the same content everybody sees.
AI instead is going to understand and learn at each moment what it is you need.
And as with most things, AI data is the core.
The people who have the most and best data about a product or service are the brands.
They are the retailers and the people who sell it.
So they can create brand agents, which means your experience on the internet at all of those moments in the journey from first learning about it to figuring out what the right configuration is and comparing and browsing and buying. is going to adapt on the fly through these agents.
So it doesn't replace the web, but it changes Things from you looking at stuff someone wrote to something that's partially adapting to what you actually need, understanding your needs.
But the agents that are doing that for you are from the retailers and brands themselves because it's their data that is what you need.
And that sort of changes the internet into kind of your internet for both parties.
Right. And that really makes it a much more adaptive system form fitting to what you need in enhancement to what's already there.
But that from a media and marketing perspective changes everything.
Yeah, sure. It's interesting to hear you talk about that from the media marketing perspective.
We're recording this in early 2025. Happy New Year, everybody.
Last year I had, especially towards the end of the year, I had a lot of conversations with people about agents and agentic AI. a few of them on the podcast.
And as a sort of consumer and user of things, I keep thinking about...
And I like the way that you put it, changing it from the web to your web.
I keep thinking about it from the perspective of, oh, maybe in the future, I won't have to go seek out the information I want because my agents will know me and know what to go find and bring it back to me.
But it's interesting to hear you talk about the agents actually being on, I guess, the advertiser brand publisher side. because they already have the data about my habits.
Well, it's not that they have the data about your habits. about their products and services.
Absolutely correct. If you look ahead, consumers have and will have agents that understand them.
But that's half the conversation, if you will.
Those agents know what they need, perhaps, and what they're trying to do.
But no one's going to know more about the uses in... values of their sauces or beauty products or furnishings or cars. than the brands and the retailers that sell them.
We see two halves to this. there's the agents that consumers may use, which may just be enhancements to the web as is, they may grow from that into something new.
But the folks who can answer their needs and help them not just to learn more, but actually finish and convert, take the action, are the brands and publishers.
So for them to be able to equip their agents, but then send them to all the different places consumers need, that's a big change. of syndicating the knowledge and expertise out to where the people need it at their different moments on their journey, which is wildly different than having to get them all to come to you.
Right, got it. No, it makes a lot of sense.
And so First Hand is young. When was it founded and what was kind of, you got into this a little bit, but what was kind of the... the moment where it came together and you decided to launch the company?
So I was working in generative AR trying to figure out how to make gameplayable assets much more easy to produce and some commerce ideas around that.
When the language models first came out and we understood this change that the folks who have the best information about the products and services you might need are going to want to speak in their own voice through their own agents, but everywhere.
Right. That changes everything and that the folks who have additional information, the publishers who have the expertise on the trends or topics in general, also want to have a voice in that conversation.
That really changed things again from that the Internet to your Internet, then that sort of was the spark.
That's like, that's white space. That is not happening. any advertising is done today.
That's not how the technology stacks work.
That's not how the data management works.
It's not how the measurement works. From an entrepreneur's point of view, white space is fantastic because, first of all, it's just fun because you have to invent it. but you're not retrofitting or dealing with, there's less legacy to try to deal with.
At this point, having done marketing for this many decades, there's a lot of lessons learned about now that I can start with a clean slate, I'm going to do it this way. and then the power in ai is just it's astounding so that was just very exciting and it does change retail, commerce, advertising, martech, customer research.
That's a lot of playing fields. I need co-founders, that was thought number one.
But having done this for a while, I have just very good friends that I've worked with for decades, And Michael Rubenstein, my co-CEO, is someone I've known since, you know, late 90s, double click, and Wei, my CTO co-founder, someone that was, you know, and engineering at Freewheel, you get to work with your friends that are industry experts in white space on something that is this exciting.
Not just because of the change in the industry, but just a little full disclosure here.
The technology is just really fun to understand and work with.
Isn't it? Yeah. It's fantastic. And then just if I may, because advertising tech is just a massive, massive scale, the inference volumes and speeds and things, is a total different world.
Working with companies like Nvidia on figuring out how to make that actually properly for that kind of an industry i mean it's sleeves rolled up thumb right yeah so we we formed in august of 2023
So not too, too old, but we are 27 people now.
We were live in the summer. We, We haven't done case studies public yet, but we've got actual results that when you actually adapt the content and the websites and the journey, if you will, for consumers into something that's fitting the needs for them, the effects are multiples greater than traditional advertising.
I don't want to harp on the nomenclature here, but I want to ask you a little bit about the word agent. and brand agents in particular.
I read a definition of agents in the intro But for every conversation I've had with people about Asians, there's been at least one new kind of working definition about what that might mean.
Can you talk about the concept of a brand agent as firsthand uses it?
And specifically, how should companies be thinking about brand agents as different from other types of AI agents?
So I like to think of it from what are you trying, what's it supposed to accomplish?
So when I think about agents as I read about them, Most of what I've observed is here's a somewhat autonomous. execution engine that will go create productivity by making something more efficient.
That's a solid definition. I'll take it.
Yeah. That's fantastic. That's a wonderful, useful, make what you already do work better. use of AI.
But when you're talking about how to participate in a brand new medium, It's a very different set of use cases because you're talking about how do I help a prospect and a consumer through call it a connected journey from being aware of something to learning about what you might want. to finding what version you want to actually getting it.
There's many different objectives and purposes there.
For us, an agent is actually a composable collection of AI capabilities.
You might even think of an agent as containing many agents.
We use the word capability. And you would do one with a particular composition for, say, awareness raising, a different one for acquisition, maybe different. one for conversion improvement in upsell.
What that means is you do a few things. First, underneath the whole product is something called Lakebed. which is a kind of intellectual property rights data management platform so that agents in whatever form they take can cooperate on data across parties.
And just to clarify, Lakebed is a firsthand product, yeah. it always comes back to data.
In order for an agent to do whatever its job is, if it's for a retailer and it's happening out on a publisher, it needs to be able to, because it benefits both parties, safely use the information of the publisher.
If you're on a website about decorating, decor and home and furnishings, and a retailer wants to help you with that, Then they need to know not only all of their products and services and offerings, but everything about what you're looking at. right and the content of the page and then it's beneficial to what parties if it's done with full intellectual property and privacy control rights because no one exits an event with data they didn't enter the event with, if you will.
But it makes the agent capable. And then that agent might help you instead of seeing what you would traditionally see as an ad, it may say, look, are you trying to furnish a new home?
Are you trying to refresh for holiday entertaining?
Are you downsizing and looking to sort of deal with a smaller property?
And it learns, as I said, this is the new medium effect.
That agent, even though it's happening, retail is agents happening on the publisher property.
It's learning about what you need and it's presenting the information and here's where the capabilities come into play.
It could simply be content presentation.
If you want to have open Q&A, you could add a chat capability.
You compose the capabilities that make sense for what you're accomplishing.
But it learns, and let's say it's someone who's furnishing a new home, then it's going to present them with like, here's the five key starter pieces you need and here's how to turn a house into a home quickly with color and accessories.
And then this is new medium again. Once someone engages with that agent and the retailer, that's a first party relationship between the retailer and the consumer. which is an arcane element in marketing, but that means that when that consumer leaves that agent and arrives at the retailer, the retailer is allowed to understand everything that occurred in that agent out in the wild.
So that means when they arrive, Instead of arriving in the past, it would be the landing page everybody sees.
The landing page itself can be another agent. that is therefore composing the content of the page to fit the needs of what brought you there in the first case, and offering up additional help to find the right configuration that makes sense in this need, etc. which is beneficial to the consumer, but obviously helps sell more product and is beneficial to the retailer and obviously create a great value for the publisher in selling that media opportunity. everyone has to cooperate in a way that's safe for each other.
And this is why we think we're multiples and multiples better than traditional ads is because you're creating a utility for the consumer. that happens to be directly beneficial to the publisher and retailer doing it or the brand if they're the ones who are on the other side of the conversation.
Right, right. I'm going to ask you one of those questions where I think I know what the answer is.
And usually those are the ones that are good to ask because I'm often quite wrong.
Can you describe the difference in as plain terms as you like? between what you're describing and the chatbot experience that consumers may be having now?
So the key is that in different circumstances with different needs, if it's on a search engine, if it's in social, if it's on a publisher, if it's on your own retail site or the manufacturer brands home, you know, their site, what you want to do again, starts with what its intention is.
Then, We separate the brain of the agent, which is a few pieces from Lake bed.
What is it allowed to know? So a retailer may want it to know only about their house brand products. but they may want to know about every product they offer.
So that's sort of a business choice. Right, right. choose the sets of capabilities you want it to produce.
So it may be simply the appropriate content selection and presentation.
So it's essentially built building a mini website, if you will, on the fly to help them with sort of a choose your own adventure structure.
If you want to add to that chat so that they can ask a question you didn't interpret appropriately, then that's just additional capability. that's the agent that may happen out in the wild you sort of wrap that up as a Campaign and send it out there and say I wanted to happen this many times in these places, and if someone goes through that agent onto the destination site, so for example, in what we've seen so far,
Folks are when they're looking at an agent, they're going to the page about the actual thing that that agent talked to them about. multiples on multiples on multiples, higher rates than an ad does.
And they're going to many, many, many of them.
So hundreds of different ones. which means you're getting people to what they actually need more efficiently.
And that's not a chat experience. It's more interwoven. into the way people already serve because that last part of the agent is something we call a frame, and you can call that its UI.
What is the user experience you want to present?
Is it image rich or is it text centric? It's almost more of an app. in a way than an agent because how it appears and the way you interact with it is at least as important, frankly, for marketing as what it's able to say.
Right. No, that makes a lot of sense. And then, frankly, it continues its help to you through your connected journey.
Right, right. It surged from the publisher onto the retailer, perhaps onto the brand.
It has context throughout, which I think no one likes to start from scratch again.
Again, so an agent is this mix of what is it supposed to know?
What is it objective? Is it content presentation plus open dialogue?
Is it actually constructing the page you're on while it's talking to you?
So you pick the sets of capabilities. Then you layer on that UI.
But what that does is that's the changing the internet to your internet because it's taking those discrete hops.
I searched for something, I read an article, I saw an ad, I clicked on it, I went to the site, I started from the beginning, I found the product.
And it's turning that into one connected journey.
It understands and helps me with what I need at each of those steps.
And it's with me the whole way. with full context because that's the relationship between me and the retailer.
The location is less important. So you've spoken to this, but maybe we can put a point or you can put kind of multiple points on it.
What are some of the most exciting areas of potential or even specific problems that brand agents can solve for marketers and kind of ways that the retail experience, I mean, you've been talking about it with this idea of composing you know, your web on the fly to see what you need.
Are there other specific problems or big things that you're excited about when it comes to how the experience is going to be transformed by brand agents?
Well, I think the very, very fundamental change is Being able to take the expertise and the knowledge of the retailer or the market or the brand, and put it in the moment wherever the consumer is.
So taking their ability and putting it into an agent and sending that agent everywhere.
So you're helping the consumer at all those moments. is a massive change because that's why instead of trying to common denominator pick them one message, and figure out the targeting about exactly what this one thing should match against, this is what AI is designed for. it understands and learns what's the right thing in this situation, which means because your Internet is just as true for the marketer as it is for the consumer.
So it's taking of all the things you could do, this is what is useful to this person here in this circumstance.
So that means instead of having to fine tune and sort of try to hit the bullseye, you can kind of let this find all the bullseyes.
And you can see that evidencing by the fact that it's sending people to hundreds of different destinations instead of just the one or two.
But then if you can continue that connected journey when they arrive and have that fit to what they need, And that covers everything from like food and beauty to finance and auto and all forms of retail.
The odds of them getting to the thing they need and getting the version of it that they need and being able to ask any follow-on questions about how to actually pick the right set of it We don't have data on this yet, but I cannot wait to do conversion tracking because I am very excited about So I think that is a wonderful opportunity that has never existed before as opposed to a problem to solve. right something you said earlier the way you described the the current process of you know you you have something that you want to find or you need or you're decorating your home and don't know where to start, and so you do a web search,
And you get a list of results and you pick one and you go to it and then you read the page, scan it to find what you're looking for.
And the idea of that process changing, it makes me think of things that marketers, I've read and talked to marketers saying that You know, AI may fundamentally change the whole web experience, starting with breaking SEO.
Just thoughts on that. So a couple of things here.
The world has been built at this point. where there's a fairly central place to go ask your question. looking backward right and that sent traffic out to the places to go read the details where people might run ads to then get you to go to the place where you can consummate the purchase and learn more about the purchase.
But that hop, hop, hop is completely changing because the knowledge is pushing out to the consumer at all those different places, they still want to go to publications and read about the details and what's the best version and then to understand what's new and trendy.
So everything still exists. It's just more adaptive and it's more sort of distributed out.
I think it's less centralized trying to get someone at that first question and get them at all the moments of question throughout all their moments. of need on that journey.
And if you can make that smooth for someone and in context throughout, that's so much better for the consumer, but it is a It is still an enhancement to the internet.
I think the way digital great, it just makes it much more.
It's, it's the internet here and it makes it richer and easier to do stuff.
Yeah. I'm speaking with John Heller. John is co-CEO and founder of First Hand, and he's an ad tech veteran, started firsthand with friends from his ad tech days.
And they're using AI, AI agents specifically, to transform the way that publishers and advertisers Connect with consumers, engage with them, help them throughout their journeys, everything John's been talking about to this point.
John, you've been speaking about all the different ways, sort of conceptually and concretely. that AI, AI agents, the brand agent platform that firsthand is building can change the way that consumers go on their journeys and that advertisers, publishers engage with them.
Are there specific industries, specific experience types?
I mean, to me, this sounds like a transformative thing that's far-reaching and then some.
But are there certain areas that you think are really right for transformation and you're particularly excited about?
One of the reasons this has been so much fun and starting something that's so white space is so enjoyable is, you are learning things that you didn't understand.
I started thinking this is for high consideration stuff.
Financial services, automotive, telecom, consumer, things where you have lots and lots of details to go figure out.
Yeah. Nope. I mean, yes, but, but more, you know, food, like what dishes would make sense for the holidays and how can I sort of help you figure out the shot? shopping list for a great Thanksgiving?
Or did you know that these products just lead for a balanced, healthy life?
What we found is as much interest in how this can benefit your lifestyle in that path as much as how much this product is useful for its features.
So the interest has been very broad and it's not just been broad from industries.
It's been broad for sort of where in the distribution chain it fits. as much on their own properties as it is distributed out across where they market.
So it's as much customer as prospect. We build a platform.
This is the reason our agents are very composable and collections of abilities as opposed to just an agent is we thought that they would have broad application and the breadth surprised me.
Yeah. Talk a little bit about the experience of building brand agents.
And when, you know, in listening to you the past 20 minutes, I've been thinking about LLMs and the chatbot experience and and using RAG to put my company's specific product information or what have you in there to let the chatbot find the correct information.
How is the process of thinking about and building and deploying agents and brand agents specifically?
What's that like? Is it similar to building with an LLM and RAG?
Is it radically different? Talk about that, if you would.
It's much more nuanced. And the reason I say that, I'll use an example.
If it's a retailer with several hundred thousand SKUs, then the objective they're trying to solve for is different than someone who sells like three packages, an entertainment package, a sports package, and a family package.
That affects how you retrieve. Basically, the platform we built is exceptionally composable for many, many reasons, because the thing you're making the agent solve for affects the way you embed and tokenize, it affects the way you retrieve, it expects the way you evaluate what you retrieved, and ranked, it affects how you generate and the instructions you give it, and then how you evaluate what was generated to make sure it was doing what you needed it to do.
Because one of the interesting things about marketing is there's almost as many rules about what not to do as there is about to do.
There's quite a lot of highly flexible control layers around making sure it didn't do the thing it wasn't supposed to do as well as did what you wanted.
And all of that needs to be very composable to the objectives of the parties involved.
It's always a supply chain across multiple parties, so lots of rights management.
And then it's generating information that's never existed before because every agent because that's a first party interaction is not just generating metrics like how many times someone engaged with it, what did they click on, did they go somewhere, did they click on a citation or call to action or product image,
But it's recording the transcripts of what was actually presented.
You have a full survey dialogue transcript of all these interactions. which is almost as if all your marketing is also a customer research survey at the same time.
And that can be used to learn how to be better.
Yeah. That's a wealth of data. So it has feedback loops.
Again, completely IP rights managed protected feedback loops.
Mm-hmm. so that it's improving for the person whose data it's using to improve.
But those feedback loops, make it understand in this context, this configuration for this piece of the puzzle fits.
I think the key thing is high configurable composability and then also just frankly, in enterprise world, people like to use, like we made it so the foundation underlying models you want to use are choice.
Pick the one you want as a cartridge, partly because different companies have different desires, or maybe they've tuned a model especially for their world that they like.
But this is going to change so fast. We need to be able to plug cartridge A out and put improved cartridge B in and let the rest of the framework still operate.
So I say the composability and the learning and checking whether it did the right thing and can it do better is the key part.
When it comes to learning and checking to see if it did the right thing, evaluation and maybe even dealing with hallucinations.
Is there or are there, you know, agentic specific techniques that you're either using or sort of discovering? evaluating a response different when you're working with agents?
Or is it kind of the same? I think the key difference is you have to have lots of different types of evaluations and everything's goal-dependent. this type of a marketer with this type of an objective.
If it's a recirculation agent to increase use of time on-site for a publisher, That's a different case than if it's a retailer trying to wrap people to the right product or a brand trying to get people to understand that these are great recipe shopping lists for this holiday.
So that doesn't just change what you retrieve and how you generate, but it changes which sets of evaluations you invoke.
Right. That configurability matters a great deal and all these feedback loops are configuration specific and company specific because you would tune a house brand holiday effort is different than what you would do for everything I have on my shelves Just people to the site type of thing.
So I think, again, it's being able to manage all that to fit its purpose that is proving really important.
Perhaps a naive question here, so forgive me, but is sort of troubleshooting and running tech support for your customers made... infinitely more complicated because of all these nuanced specific use cases or is it just a new version of things that people in the ad tech world have been dealing with for years.
Interestingly, we have an entire experimentation module which means you can run this in laboratory mode, watch all of its behaviors before it goes in front of consumers, where you get yet another important feedback loop, which is no one is more expert in the quality of their output and the appropriateness of it than the editors and the marketing staffs of the brands.
They get to go into the experimentation module, have it run in lab mode, and say, that was great, that was good, that was great.
And that creates yet another feedback loop, so it improves to pass muster.
Once you've got it to a state where you're happy with what it does, that reduces the once it's live, issues set massive.
And again, to go back to the capabilities list, this is how it's different from chat.
If you picked what we call a guided conversation, which is sort of a choose your adventure motif.
It's only picking from the content you've already told it's allowed to know, and it's simply surfacing the right things phrased the right way in the right moment. and this gets back to how agents are built.
If you wanted to add chat to the bottom of that, then someone can ask whatever they want.
That obviously triggers a whole different set of protections and evaluations to keep that on track.
So again, that configurability is quite important.
Kind of at a really high level, how do brand agents, how does your technology change the way that your customers, the brands, the publishers, think about how they want to engage with customers.
So in the past, you would try to get a list of target identities.
So you're trying to find a list of folks because that list is a proxy for flags that mean what they care about and what they're interested in.
And then you would create a message that was appropriate to that list.
And then you got to go find where those folks might be and try to show that message to them.
And that's when they're out beyond your borders, if you will.
When they arrive at your site, you don't know how they got there with any particular detail and you're again starting from scratch.
So what we would say is you have a great set of products and services.
Figure out how you want to talk to your different clients. typical customer as opposed to specific target list, and then let the AI do what it does. put it in all the places where they might be beyond your property and on your property.
And then it will surface to them the things that matter to them.
And they're going to end up coming to you already having indicated what's important and being qualified. so you receive them so here's a big change you should have agents present when they arrive that receive them with that knowledge and start from step five, not step zero.
And that changes how you think about campaigns, how you construct them.
Interestingly, and this is a bit arcane for ad tech, it's vastly simpler operationally to set this up. than to try to write like a 100 line item insertion order trying to manually, this is a classic AI.
Do I build a massive if then else statement, an IO with hundreds of line items, or do I have an agent able to figure it out?
Yeah. So what's next? What's next for firsthand?
What's the future of AI agents like? Take it wherever you like.
How do you envision... The technology, the use of it, the implications kind of evolving over the next, I'm going to say three to five years, but you can tweak that to suit as you like.
I think the ability for the Internet to become your Internet has a lot of, the new medium, it has a lot of implications.
So I think where I see agents is There will start to be more wisdom around this is the right configuration at this point in the customer journey. productive configuration, these abilities.
It's going to be this guided conversation with presentation.
When they arrive, I want page composition, but only in these portions of the site.
So I think the very construction of how marketing campaigns and sites are put together changes and when it becomes this connected journey where I've understood the consumer all along the way and I've helped them all along the way, and I have that data, I think that data starts to drive a whole lot of additional insight and therefore evolution.
Not only, obviously it, feedback loops into making the models better.
But if you can ask your marketing data, you know, what are people asking me for?
I don't sell that. Right. Why didn't they like the new product launch?
Or why did they like their product launch?
Or you look at all of the things and say, what were the phrasings and offerings that got a lot of engagement?
What got no engagement? All of your marketing is customer research at the same time.
So you're learning in census level what to tune next.
That's just never existed before. Yeah, interesting times ahead, to say the least.
John Heller, for folks listening who would like to learn more about First Hand, Obviously, the website, firsthand.ai.
Other places they can go online, social media accounts, a blog.
Where would you direct them? Well, we have a LinkedIn presence, but the website's probably the best place to start.
Fantastic. Thank you again so much for joining the podcast. fascinating conversation obviously with implications for retail and brands and ad tech but just thinking about the future of the web and the age of ai and now agentic ai it's um Just to echo what you said, it must be a blast getting to build all this stuff from scratch.
It's a rare opportunity to be able to work in a world that's changing so much with something so powerful. and with partner companies that are just so capable and fantastic.
It's just fun. Excellent. Well, best of luck to you and look forward to keeping track of your progress and maybe catching up again down the road.
All right. Thank you so much. Thank you.
