I think AI is going to be very transformational for the industry, for society, for businesses.
I'm trying to step away from some of the hype and say, okay, which business problems really matter to us?
Which customer problems really matter to us?
And is AI a tool that can help us solve that problem better?
So we introduced an AI assistant and we've had a bit of a learning journey with that.
We have built something where you can build a basket in the chat.
So we are taking the ordering journey from minutes to seconds, which is great.
I think that's a real improvement.
However, we've had to do some learning about what is it that customers want, and what they really want is hyper -personalization.
There's so much discussion on this.
You know, I do think that it is going to be extremely transformative for the role of PM.
I think that the biggest thing in my mind is that I think that AI is really actually going to increase the expectations of the PM function.
From a product perspective I know people in the security industry have been thinking really hard about how to leverage even earlier gen AI technologies.
The biggest worry in cybersecurity is the lowest tech one which is phishing.
Now you look at deep fake technologies and all of these things, people can clone your voice from a three -second sample.
They'll write self -modifying code that will be running around the internet and it's already doing this looking for vulnerabilities.
And your security by obscurity isn't going to work anymore because the things that used to require human hackers are going to be automated and 10 ,000 times as fast. And so it's going to be a pretty rapid escalation in the cybersecurity space where the companies that do adopt more of these technologies quickly are going to be at a real competitive advantage.
Creating great products isn't just about product managers and their day -to -day interactions with developers.
It's about how an organization supports products as a whole.
the systems, the processes, and cultures in place that help companies deliver value to their customers.
With the help of some boundary -pushing guests and inspiration from your most pressing product questions, we'll dive into this system from every angle and help you think like a great product leader.
This is the Product Thinking Podcast. Here's your host, Melissa Perry.
Hello, and welcome to another episode of the Product Thinking Podcast. podcast. Today, we're doing something a little different.
Over the past few years, we've had some incredible conversations on this show, deep dives into product strategy, leadership, and innovation.
And as AI continues to shape the future of how we build and scale products, I thought it was time to revisit some of the most powerful insights shared by our guests on this very topic.
In this special compilation episode, we're going to go back to the moments that spark new thinking.
We'll understand how AI is reshaping user experiences, experiences, enabling smarter decision -making, and redefining what it means to be a product leader today.
You'll hear from a mix of voices, each bringing a unique perspective on how AI is not just a tool, but a catalyst for transformation.
Let's jump in. There's been a lot of talk lately, too, about how AI is going to put product managers out of a role or disrupt what we do.
What's your take on how things are going to change for product managers specifically with AI?
Yeah. There's so much discussion on this.
You know, I do think that it is going to be extremely transformative for the role of PM.
I think the biggest thing in my mind is that I think that AI is really actually going to increase the expectations of the PM function.
When you think about the impact on the role, there's a lot of things that PMs do do that AI is actually already quite good at, taking data and analysis from many different sources and using that to craft strategy and set goals or write PRVs. I think we're not all the way there yet, but you see that with the pace of model development, LLMs are actually becoming quite good at some of those types of activities.
On the strategy side, I do really think it's going to raise the expectations.
I think as PMs, we're expected to know the market, know the customer, know the business, all to inform the product strategy.
And historically, there have been so many inputs that it is difficult for PMs to stay on top of all of those things proactively.
What has actually happened in practice is that you probably loosely follow your market and competitive news, meet with customers, review feedback on some type of basis.
basis, now it's actually very quickly becoming possible to monitor and proactively engage in these sources in real time, not for customer feedback, for example.
What are the themes that your customers are talking about right now?
And how do you have confidence as a PM that the products that you're building or the features that you're developing are actually addressing the top volume of customer needs or the biggest revenue opportunities that you see out of your customer feedback.
Or on the market side, what has changed in your market landscape this week?
What are the kind of latest movements of your competitors?
Or even actually monitoring and analyzing things like 10k forms from your customers?
What are the commonalities across changes that are impacting your customer's business?
I think that on the strategy side, like PMs are now going to be expected to be able to speak to these types of inputs in near real And so product and product ops teams, I think, are going to play a really important role in ensuring that they have systems and toolings to be able to be able to capture and measure this data and use it to inform strategy.
And that's one of the areas that I think we will just see the biggest shift in, in that it will really just become an expectation of the role that you're using this technology across all those different verticals to keep a pulse of your business.
That's the raising expectation side.
I also think that maybe on execution, it will make a lot of things easier.
And there's so many parts of the PM role that end up being focused on non -value added work.
Actually, one of our customers recently used the term bad admin for this.
They said, our PMs are just stuck in bad admin.
in, they're spending so much time like writing weekly updates or preparing decks for executive strategy alignment conversations.
And on that side, I think that we will see AI significantly reduce that type of bad admin work that PMs end up facing the brunt of in many organizations.
And we're already seeing a lot of interesting use cases that our customers are coming up with to use AI to reduce that type of admin work.
So you can see already some ways where some of those more administrative tasks will become automated very quickly.
So one of the big strategies for your company is around AI and integrating AI into your roadmap.
Can you tell me a little bit about how that came about and what you're thinking when it comes to that?
Yeah, so I think AI is gonna be very transformational for the industry, for society, for businesses.
is. There's loads of excitement about it now and rightly because I think it's really interesting.
However, yes, we are looking into it.
But I think the real key is what is the problem that you're trying to solve, which is the fundamental product question always.
I'm trying to step away from some of the hype and say, okay, which business problems really matter to us?
Which customer problems really matter to us?
And is AI a tool that can help us solve that problem better?
So the result is at the heart of what we're doing rather than the technology itself.
And I think that's really important.
Having said that, I think whenever new technology starts to gain traction, it's our job to also understand it, to research it and to know how to apply it.
So we've been experimenting with a few things and in a few different areas.
One of the things that's quite interesting is personalization and improving the customer journey.
So I'm sure many of you that are watching or listening are always thinking about like, how do I improve conversion?
How do I reduce friction in my journey?
We all know that's important.
So we introduced an AI assistant and we've had a bit of a learning journey with that.
We have built something where you can build a basket in the chat.
So we are taking the ordering journey from minutes to seconds, which is great.
I think that's a real improvement.
However, we've had to do some learning about what is it that customers want.
And what they really want is hyper -personalization.
They want to say, hi, I'm feeling tired.
I'd like some comfort food.
Or after something healthy, what do you recommend?
And they want to get some instant recommendations, build a basket and go.
And that's where we've had to kind of experiment.
And there's been some learnings there on the speed, the accuracy, and the types of personalization.
and we've also been analyzing what people write about so that we can start to build it out so we also see not only do they want to order they want to know where their order is and they want to ask for help through this interface so we started to build in customer service flows and we started to look at how do you automate when things go wrong because unfortunately sometimes they go wrong and you need a refund or you didn't there's an item missing from your order so what can we do to make that experience also really frictionless how do we automate it so we just we say yep your refund's on the way
really sorry and they can go on with their day they don't need to send an email or phone somebody or any of that so these are areas we've been experimenting with with 750 ,000 partners on our platform you can imagine that the amount of data we have around menus and things like that is huge and setting up a new partner requires us to set up a new set of data every every time.
So other things we've been looking at, which are really impactful are we've, we've trialed an AI menu upload tool.
So you can take the printed menu from a restaurant, scan it, create all the data entities that you need, upload and get that partner online.
That was something that could sometimes take up to four hours and we've reduced that massively.
We've reduced it by over 50 % and every day as it's learning, as we're improving it, it's getting faster and faster.
So We can give our colleagues more meaningful work so they're not typing in menus, that they're working on customer problems and other things.
So there's lots that we're experimenting with, I think, to make the customer journey more efficient to remove some of the repetitive work in our business so that people can work on more interesting problems. And I think there's loads more to come with that as well.
Those are really cool examples of how you're leveraging it.
When you thinking about the new technologies that are coming out like AI and all of there's so many different things that are rapidly changing.
How are you, as the CPO, staying on top of these emerging trends and trying to figure out where do they fit into our product strategy?
It's a great question.
The most important is your strategy, right?
How do you bring along the entire team towards think about the AI strategy?
And the most difficult thing is how do you bust the tyranny of now?
Your massive backlog in an enterprise software, you have customer commitments, you have fewer releases because you can't push code because they have their own validation systems in our customers, in their endpoints, et cetera.
So how do you balance the turning up now?
So our approach is to actually traffic hit this.
One is make sure that you have embedded experiences, companion type experiences in all of your products, whether it's your learning product, your talent mobility products, your performance, writing your goals, connecting it, getting baseball cards for each person about them, their structure, their check -in, summarizing all of that.
Make sure that it is sprinkled in every element of Galaxy.
Galaxy. So we enable all of them with robust, usable, engaging, captivating tools to create that magical experience.
The second is then where do we then promote native experiences?
Where do we ensure that we are re -imagining the product relative to the assets that we have right now in a way that others would come and do it if we don't?
So in other words, disrupt your business before someone else does, bust the innovators dilemma, do the double flip, et cetera.
So that That is the second piece.
And then the third piece is for your existing experiences, how do you create that overarching usability factor that whether it is multimodal, whether it is personalized, whether it's adaptive, et cetera, how do you create that layer on top of it such that it's highly unifying into the next -gen form factor?
So we call all of this next -gen human experiences.
The one other thing that we've been lucky enough to do is we've been acquiring a couple companies, very AI native companies.
While we have a pedigree of about, you know, better part of a decade of doing like AI and generative AI, getting the founders in and having them in our staff and doing some reverse integration has helped us reimagine some of these things.
We're already on the forefront of generative AI for developing human skills.
And we have the generative AI product leadership now blend there as well.
So helping us disrupt ourselves has been hugely helpful.
We also do design partnership programs. So our head of design, she organizes with a few hundred customers, watching how they work.
This is more than just user groups.
This is more than basic design research. This is co -opting our customers to co -author the future and also bring their friction points, reduce that as much as possible through existing and the product technologies that we have here.
So it is a journey, but all of this is on the foundation of something that we extremely solidly believe in, which is responsible AI, responsible technology in general, accessible technology.
So there are five, six ethos that we are very passionate about.
One is privacy and security, goes without saying, but accountability and explainability, which means that if you are making recommendations, how are we accountable to this fact that this recommendation is the right one?
How can we make it explainable in terms of the fact that it's the a bias, linguistic mitigations like salesman versus salesperson kind of stuff, and then human in the loop.
If something needs to get kicked out, it can get kicked out.
So you can set and dial how much ever you want.
We also have a multi -LM strategy, a composite LLM strategy to minimize hallucinations and adjust temperature settings.
And lastly, transparency.
We are very open in terms of explaining how we built architecturally and how our AI works.
So that is our broader story, the platform portion, the innovation, founder mode, so to speak, and deep understanding and intimacy of our customers and their needs.
So let's jump into it.
Let's talk about Copilot, because this is the the thing that everybody is talking about these days.
GitHub, super successful with your launch of Copilot.
Tell us how it got started.
How did you start to imagine this like AI native product and what AI could do to help developers to fit seamlessly in.
Yes. And yeah, one of the things that I actually also love about Copilot is the co part.
We have this thing where the human is at the center and then we're augmenting you to do more and to really spend time in the things that us humans are really good at, which is creativity, or we have an advantage on, which is creativity.
But yeah, it's this thing called the GitHub Next team.
And the GitHub Next team is in charge of looking for not only Horizon 1 initiatives that can make GitHub better, but really more about Horizon 2.
Think two years from now or Horizon 3, things that may never actually happen as well.
They had been in talks with OpenAI, and OpenAI had this thing called GPT -3.
Internally, I think it was called something like Codex for us, the Codex model.
And they had been collaborating with them on that technology, essentially saying, look, we have these LLMs are very special at understanding natural language and solving problems, specifically in the coding space.
And we have seen that played out.
These models have gotten significantly better in natural language understanding and significantly better in understanding code and being able to code as well.
Well, and the GitHub Next team created a paper that said, is this science or fiction, more or less, if you want to think about it that way.
And that was to start with in GitHub on, oh my God, is this something special or not?
When I first saw it, what really impressed me and why I said, wow, this is probably going to change everything.
In our space, we have been able to autocomplete code for a long time.
we have this technology called IntelliSense, and there have been a lot of ML models that helped you do those types of things.
But what I had never experienced in my life is you being able to go through a comment and describe something in natural language and then have the system or the AI model understand that and then translate that into code.
And that natural language conversion to code was, again, something that I'd never experienced.
And I'm like, oh, my God, we're going to have to go all in on this.
Hey, product people.
I have some very exciting news.
Our new Mastering Product Strategy course is now live on Product Institute.
I've been working on this course for years to help product leaders tackle one of the biggest challenges I see every day, creating product strategies that drive real business results.
If you're ready to level up your strategy skills, head over to productinstitute .com and use code LAUNCH for $200 off at checkout.
Now, the interesting part of this dilemma is the following.
What we were playing with, at least in the research lab, was just a technology.
Like it could do things like solve Python problems. Sometimes it will take 100 shots to do it, which is not something that you could ship in a product.
Just imagine if to solve anything, you have to tell it to do something a hundred times.
That's not something that anyone will buy.
So we have this amazing technology.
We have these pre -existing tools that developers use.
How do you marry those two things?
And that's easier said than done.
Sometimes you're going to be too early with the technology.
Sometimes you're actually going to go and put it in a product in the wrong way.
And then you're going to not necessarily waste the technology, but you're too early with the wrong assumptions as well.
well. And that's the craft of product in my opinion.
It's that complexity of the problems. It's no matter how good my PRD is, at the end, the product has to actually make it and be good.
So we played a lot with that tech into making it into GitHub Copilot.
The simplicity of what we shipped at the beginning was incredible.
There were only three things that make Copilot it was successful in my opinion.
Number one is we actually made it so as you're coding, you're not interrupted.
We have this thing called ghost text.
And as you were doing your normal coding, then it will come in and then tell you, do you want to take this suggestion or not?
At the very beginning, it was a pop -up.
We have many ways of doing that in the UX, and we found one that worked.
The second thing was it had to be fast because we chose that modality on showing you a suggestion, it had to be really fast. So we got it down to 100 milliseconds, which really meant that we kept you in the flow developing as well.
The third thing is we needed to bring more context in.
So the suggestion was something that was more personalized to the code base and to you and not to what the model actually knew about in the world.
A lot of people get confused, they think that the suggestions come from all of this corpus of code and just a little bit of that is true.
the context and the LLN together with that, it's what created really generous and new thing all together.
And we just needed to tune that.
And those three things is what makes Copilot successful is the fact that it was a God's text.
The fact that it was super -fast and the fact that it was contextual and it gave you pretty high quality suggestions because of that.
And then we just watched that take off.
We were the first co -pilot in the name and in the product as well.
And we're very proud of that.
When you're thinking about AI and as it relates to cybersecurity in the future, what kinds of trends do you expect to pop up?
It's something I gave a lot of thought to.
I think there are serious both offensive and defensive considerations in what this is going to do.
And it's interesting because from a product perspective, I know people in the security industry have been thinking really hard about how to leverage even earlier gen AI technologies.
And I worked on a product like that at Citrix, which we called Citrix Analytics for Security, where we could measure the keystrokes and mouse clicks that were going on in Citrix environment and find people spoofing other users and change their risk scores based on risky behavior.
And we were actually building machine learning models of how all the users act.
And it was really cool.
And so you've been seeing more and more of that in recent years where people are building pattern matching engines based on machine learning into their security products, looking for intrusion and things like that.
What's interesting with the chatbots coming along and that capability growing so dramatically is the threat vectors are now crazy.
The biggest worry in cybersecurity is the lowest tech one, which is phishing.
And it's email phishing and it's also spear phishing.
these things have suddenly dramatically in the last few months gotten so much better.
We used to joke about like the Nigerian print scam where you get like this little chain letter that was in incredibly poor English asking you to send money.
And all the phishing training you take says the first thing is, is it really written in good English?
Is it grammatically correct?
Because if it's coming from somebody in North Korea or Russia, it's probably going to show those signs.
They don't anymore.
They're all perfect.
They're all flawless.
And now you look look at deep fake technologies and all of these things.
People can clone your voice from a three second sample.
The next phishing attack you're going to get is going to be your spouse calling you on the phone, asking you for the bank account number and her voice and her cadence.
That's where this is all going.
The next one is with these advances in the bots that can now write, they'll write self -modifying code that will be running around the internet.
And it's already doing this looking for vulnerabilities and your security by obscurity isn't going to work anymore because the things that used to require human hackers are going to be automated and 10 ,000 times as fast. And so it's going to be a pretty rapid escalation in the cybersecurity space where the companies that do adopt more of these technologies quickly are going to be at a real competitive advantage.
Wow. That sounds ridiculously scary, what it's going to be.
I've already seen some of this happening and it's blowing my mind just how crazy people are getting into hacking stuff.
That's I'm glad you're working on it.
I'm glad you're trying to fix it.
But that's really interesting.
I think it's going to pose some challenges too for leaders as they're trying to think about how do I prioritize security?
That is one of the big things we talk about in with chief product officers and CTO.
How do we make sure that we're prioritizing the things that you need to get done versus just the net new features and building the stuff that may be strategic on those areas?
is. So what's your advice for like CPOs and CTOs when they sit down and try to negotiate this?
What should they be looking at for security?
How should they be staying on top of it so that they know what's coming up and what could be helping with that?
Well, I think in a lot of ways, there's a parallel to how you think about security with the way you think about quality.
And every PM has had to make this trade off.
What do you call it?
The iron triangle. I got quality, I got features, I got time to market.
You could put security in as another box there or treat it as a sort of similar super box with quality and just talk about technical debt and non -feature oriented capabilities.
And you could put performance, security, quality all up there with all those ilities that you know make a big difference in your product.
So you need to find ways to firewall off time, resource and effort to deal with those things, because otherwise they fester.
So you are chief product officer of a really interesting company called Soul Machines, which makes digital humanoids, which is a long way away from, I think, what we think about as caricatures online like Wallace and Gromit, which was years ago.
But how, you know, how did Soul Machines get started and how are you harnessing AI to bring in these very lifelike images of people?
That's a very good question.
And it's been a while since someone's mentioned Western Roman to me.
So maybe we'll get back to that later.
So I can try to remember.
Soul Machines was started by Mark Sagar.
He was a special effects guru working on blockbuster movies and his area of speciality was emotion capture, particularly around the face on large, on major movies.
At the time of working on those films, he developed some technologies to support that work that spanned out into a research project that ultimately became Soul Machines and his big single, his big story was really about the face being access to an emotional connection for users and for people using technology beyond the way that we're used to.
So he built these digital avatars that could connect to customers potentially in more empathic ways.
I joined the company five five years ago to introduce a level of kind of design thinking as well as product thinking or attempt to bring some product thinking into what was at that stage a very research focused engineering led team and so the challenges that we had at that stage were building a design and product team to try to help not just a very engineering led company evolve into more product led which is where we are today but also transition from a very vision -based company that trying to to an aspirational selling a major vision to big enterprises to actually democratize the process of building
digital people so that more smaller companies can work on and build and create digital avatars for their own unique use cases.
And what are those use cases?
Who needs digital avatars?
So that's a very good question.
And I'll be honest, we're still in the process at the moment of trying to identify those unique use cases that actually bring real value.
A lot of the work that we've done up to this stage, and you asked about the connection with AI, and I'll come back to that shortly.
A lot of the work that we've done at this stage has been around democratizing the ability to create avatars and also reducing the time to build these very realistic, customized faces and unique faces from three to six months to build to 30 minutes really for an individual to try and create them.
The original pitch, the original opportunity that was being explored at Sun Machines as I came on board was digital assistants to help with a sales drive or to replace a call center or to connect your customer to your brand in a very personalized way.
what we're starting to find, really the value that we're starting to explore as we've opened up the platform to get more users on board for our free tier, as opposed to the previous model where we charged a lot of money for kind of a long -term project.
The use cases we're starting to see are really getting to be super interesting where it's actually where an empathic kind of really personal connection is useful so rather than being an area of transactional kind of interface which is what effectively the internet is about today and digital media is today we're very transactional very get in get out what we're finding the digital avatar doesn't work very well in that space there there's limitations with the technology the the response delays too is too great so inserting something like a fake face or a fake human between an experience that people
are used to it's actually not very successful but where we are finding that there are successes is helping people make difficult choices or helping people have difficult conversations or practice difficult conversations for example this interview today I've done a couple of times with an avatar where I had the avatar playing your role and helping me stumble through words And that practice, the loop without judgment and talking to an emotional face that actually connects to you is super effective and super useful.
One of the big use cases is really language practice.
We had a Korean influencer who used one of our avatars a couple of months back to practice speaking English, just to practice responses.
So it's that engagement without judgment is an area that we're exploring.
That's a really cool use case.
I can see, I like the idea of the practicing without judgment.
Like I personally speak really awful Italian, but it's because I can't practice.
So I can understand extremely well, but I'm so afraid to go practice it because everybody just kind of looks at me weird. And I don't get a chance to speak with native speakers living in South Carolina where actually I did find two people who speak Italian here, which is very random, but when I expect it, but I don't get to talk to them as often.
So I've always, I've been like, man, I wish I could, could practice more.
And I've looked into like other courses, but I love the AI component of that too.
Like not having a real human there judging your feedback and knowing that it's okay to say whatever you want.
I think what we found is that we went into this business of trying to create a level of utility that existed and that worked in existing channels and existing markets.
But actually as the technology matured and as ChatGPT arrived and GenAI came on board, we found that we can, people were starting to use our digital people as reprompting them in different directions like taking the one that we had in Soul Machine's website and they were reworking that to be a sleep coach giving it a brief to say I would like you to be this so that kind of looking for that personal use case that making a digital person that is adaptable because the technology is really geared towards being able to connect with the viewer emotionally or respond in terms of expression that in appropriately
it works really well in those areas where you don't want judgment or you want a response or you want to see if things work and once you layer in chat gpt you're starting to have this kind of fusion of an experience that is a little bit unique there's a lot of talk out there too when it comes to ai and everybody keeps saying this about a lot of these things people might be replaced when it comes to some of these tasks and what we're doing and obviously we're talking about some efficiency plays here what would you say to people out there who are listening product managers other people who are worried
about that What would you say to concentrate on to ensure that they have a place in this new way of working, these new workplaces?
Don't be scared of AI.
Use it. Use it to make yourself better.
If you don't, somebody else will.
And then your job may become obsolete, just like with programming languages.
Is there new languages out there?
If you're an engineer and you're not learning the new languages and you're still like programming in Cobalt or Fortran, you're going to have fewer opportunities.
Opportunity. So understand the tools, use the tools, use them in a way that makes you more productive because I don't believe, I believe some jobs are going to go away, but new ones will be created.
So be one of the people who's learning and can have the opportunity to do some of those new jobs.
I think that's definitely wise words for people out there, especially if we're working in tech because we're moving very quickly.
Thank you so much for listening to this episode of the Product Thinking Podcast. We'll be back next week.
And in the meantime, send me all your questions to dearmelissa .com and I'll answer them on an upcoming episode.
We'll see you next time.
