This episode is brought to you by Sprig.
What if your product team knew exactly what to build to achieve your toughest goals, like increasing conversions or boosting engagement?
Meet Sprig, a product experience platform that generates AI -powered opportunities to continuously improve your product at scale.
First, Sprig captures your product experience in real time through heat maps, replays, surveys and feedback studies.
Then, Sprig's industry -leading AI instantly analyzes all of your product experience data to generate real -time insights.
Sprig AI goes even further, with actionable recommendations for driving revenue, retention and user satisfaction.
Join product teams at Figma and Notion by uncovering AI -powered product opportunities at scale.
Visit Sprig .com slash The Product Manager to book a demo and get a $75 gift card.
I'm just going to be blunt.
Whether or not you're making it an explicit career goal to specialize in AI product management, it is now a skill set every software PM should be developing.
I think it's safe to say that now that the technology is here, it's only going to get better, more intelligent and more ubiquitous.
So learning how to work with AI is about as essential as learning how to work with the Internet.
In fact, you could argue that basic skills in AI are already a minimum requirement when applying for PM roles.
How can you take it a step further and stand out in the AI PM job market?
My guest today is Praveen Gujar, Director of Product at LinkedIn.
As the title would suggest, Praveen has an excellent vantage point on the growing use cases for AI in the digital product industry, as well as an intimate understanding of the breadth of skills product managers need to develop to truly succeed in this space.
We discuss both the technical and theoretical aspects of leveraging AI.
But if you take away nothing else, our conversation on data hygiene, which comes up about halfway through the episode, is critical info for anyone hoping to not just land an AI PM role, but excel in it.
Let's jump in. Welcome back, listeners, to the Product Manager podcast.
I'm Hannah Clark, and we are a community of tenacious product leaders here to share the weight of scaling your product.
Our members are SAS -focused PMs who genuinely love their work and are driven to help others succeed.
If you want to hear more about that, you can head over to theproductmanager .com slash membership.
We'd love to have you on board.
And today we're speaking with Praveen Gujar.
Thank you so much for joining us.
Glad to be here. Thanks for having me, Hannah.
Yeah, pleased to have you.
Can you tell me a little bit about your background and how you arrived at where you are at LinkedIn today?
My background is a mixture of engineering and product management.
I started my career out of the engineering program as a software developer.
But I soon realized that I think working backwards from customers and building great products is my niche thing.
So I got my MBA and started with Amazon as a product manager.
TLDR, my experience hits in two key areas.
One is in digital advertising and second one is public cloud.
And all these products that are built over these two segments are AI -powered.
I have a tenure in Amazon, LinkedIn, as well as Twitter.
So I built these products in these large organizations.
Yeah, I've been on the AIPM for a long time now.
And happy to share my experience and my expertise in this space.
That's so fantastic.
And I know that we're very keen to hear a little bit more about that.
So today's topic is going to be specializing in AI product management.
And this is obviously a huge field that's taking off enormously right now.
So to kick us off, what are some of the skills that separate a great AIPM from a traditional PM?
That's a great question.
First and foremost, if you want to be a great AIPM, you need to be a good PM to begin with, or a great PM to begin with, right?
A lot of the skill sets that are necessary to be a great PM are a must -have to be a great AIPM as well that includes really identifying the problem, working backwards from customers, really having a product sense to build the right product and driving the product execution phase, as well as the go -to
-market phase as well.
All these things are key, necessarily.
In addition to that, I think being an AIPM, there are three key things I would say are very critical.
One is technical understanding of the AI space.
I think it's very essential to be a good PM there.
I think you need to understand the machine learning and the AI concepts, the models, the different types of models, including basically training of the models, as well as how you basically interact with the AI engineers.
It's very critical in that aspect.
The second piece basically becomes the data science or the data angle to it as well.
As a PM here in the space, you need to understand data really well, understand how the data flow happens, how the data is used to train the models, and what this clean and hygiene data means, and how data can introduce biases.
All these understandings are critical for you to be a great PM.
Third, but not the least here, I think programming can be really a value addition for a PM as well.
If you know how to program in Python and R and other areas, you can relate much better to the engineer who are actually working on the space, then you can be a lot more effective as a PM.
I would classify these things as one of our technical expertise in AI space.
That covers one aspect of it.
The second aspect is applying these AI concepts to your domain.
If you're working in healthcare, if you're working in financial services, or any other domain, how do you apply the AI principles that you use, AI technology that you are so familiar with in making the products more resonant with your customer?
That's a very key aspect of it, of being an AI PM.
Third pillar, I think I would say, you need to up -level your PM skills.
One of the key aspect becomes is explainability.
How do you explain this to folks around you, including execs who may or may not be AI ready or AI proficient as well?
How do you do that?
It's a very key aspect.
Understanding the entire model lifecycle management becomes very key as well, because you are working with engineers and fine -tuning the models, and even going through the whole and entire lifecycle of the models, and understanding it, and basically working more closely with them becomes very key as
well. To summarize the technical expertise, then applying the AI's technology to your domain area becomes, and then applying your PM skills for more explainability and other aspects becomes very key.
This is what makes a great AI PM.
I'm sure that everyone is going to be really clamoring to develop those skills, because this is just such a huge area of opportunity.
Speaking of which, from your vantage point, what's the outlook right now for PMs with skills or experience in AI?
AI has now become a household name.
I think most of the attribution there goes to charge GPT and its popularization, but AI is not a new thing.
People in technology industry have been using AI in various ways.
I do come from a digital advertising background, and AI has been powering various systems and capabilities for a decade plus right now, and other fields even longer, I would say.
If you look at what, in the last two years, has been is an exponential raise in both popularity and usage of AI, that basically opens up a huge amount of opportunities for PM with AI skillset through work on different domains, different problems, and different creative or innovative solutions as well.
That explosion is a very key thing and it's going to continue in the near future.
That's the outlook at least from what we see.
Second is the expansion of the role as well.
A lot more and more in the last two years, every product and service organization is going through some kind of a evolution of adding AI touch to their product as well.
This may be enhancing their experiences with AI or rebuilding their products with the AI first -order AI integrated approach.
That opens up a huge opportunity for all the PMs working in the space to really work on some cutting -edge technology and add real value to the customers.
It's strategically important these days.
I think you may even argue that it is from a PR perspective important as well.
It's important in your profile building if you're a PM.
But more and more, this is more strategically important from organization.
You are better positioned in the organization as well as a PM if you are a AI PM because you are valid more in terms of because you have skill sets and you are strategically more aligned to the company needs.
Last but not the least, I think there is a need or a demand for AI PMs because of the explosion that we have actually seen.
You get to be a bit more in demand for your skill sets.
That opens up all new opportunities from a skill set development perspective, from building awesome products and also even monetary benefits as well.
A lot of opportunities, a lot of benefits.
Let's go back to talking about the skills and putting those into practice.
What are some tips that PMs can use to leverage AI to build high -growth products?
I think that's a great question.
We are even publishing a paper on this in the Engineering Management Review, really thinking through what does it mean to build a great product in AI.
If you look at the current trend, what is happening is everyone is jumping onto the bandwagon.
You may see a lot of organizations enhancing their products and services with AI touch to it as well.
Many of them are mostly superficially done.
For example, it can be running a chatbot on existing services.
This is very similar to what a mobile era thought as well.
When mobile became so ubiquitous, then the first reaction organization had was shrink your existing product into the mobile format.
That didn't really last for long.
Similarly, I think what we are seeing is a lot of organizations, for lack of better imagination, they are implementing or sprinkling AI over their existing product and solutions.
That's not a sustainable mode for organizations to carry forward.
What we see is where there is a sustainable advantage is when you actually build AI first or AI integrated organizations or products.
What do we mean by that?
It's basically making AI the center stage of the product and building the experiences that are powered by AI that are fully automated by AI as well.
Instead of basically just leaving AI to automate a particular workflow, empowering the AI models to make optimizations is very critical.
The second part basically is decision making as well.
A lot of AI, what is basically used because of the recent trends, is how do we use AI to answer a quick question?
The core optimization and decision making is not something that is something which will truly enable the product to be a lot more proficient.
Last but not the least, I think the organization structure becomes very important as well.
If you have a more embedded team with AI engineers working very closely with product managers, what you're producing are more AI integrated products thought through from scratch that can actually enable you to compete better with your computation and also build more sustainable mode in the future.
I would say these things are very important, like how you think about reimagining your product and services with the AI centered approach and then solving the user problem.
The user problem solution, like solving that is a paramount need for any product.
There is no shortcut there but really reimagining your product with AI making the decisions, RAI basically applying AI technology to improve and enhance current experiences is what is needed.
I'm curious about some of the use cases that we've seen powered by AI currently in the space.
So obviously there's some, I think that we could all agree that there are a lot of products that have been enhanced significantly by AI and a lot that maybe not so much.
What are some of the best use cases that you've currently seen or that you think there's a huge opportunity to expand on?
Yeah, so I come from the digital advertising space.
Let's take an example there.
I'm happy to dive deep as well.
What are the key things in digital advertising?
You need to ensure that you have the right audience that you're targeting to, you have the right content for those right audience, and you serve them where they are at this point.
So I think traditionally how this approach used to happen is we started with the AI models really trying to understand what is the best audience fit for your product and services.
And depending upon your core objective, how do you target these specific audiences?
So over the course of time, we have improved those and a lot of predictive algorithms basically enables us to understand what are the pretty two audiences that you can go after as well.
So that's one use case.
With the advent of GenAI, especially in the last two years, content creation has become very key.
For example, I think if you are a brand, if you want to basically reach through different set of audiences, how can you generate 80 % of the content that you basically have in 80 % of the variations faster and quicker that basically enables your creative team to move faster, and then basically be more
productive at the space has been very key factor from the AI space as well.
And then like you have now the audience, you have the content to go after.
How do you personalize this at scale?
Anyone can personalize when it's a small set of population, but if you are a large organization trying to reach millions of users with personalized content, AI enables dynamic audience segmentation as well as content creation so that you can actually personalize your message at scale as well.
So those are three different pieces that were being developed for a long time as well.
Now, how about stitching all them together as well to make sure that it's completely automated and optimized across these different areas as well.
And this is where you are seeing a lot of the advertising platforms are providing one click fully automated into an automated solutions for their customers, be it Google, LinkedIn, or a meta, you see a lot of these automated platforms emerging up as well.
So that's just one digital use case, right?
I think we keep talking about the use of AI in healthcare.
One of the things that I know often who works in Google specializes in scanning through x -ray images to identify anomalies that can actually cause cancerous illness as well.
The models basically that has been developed over the course of years have been increasingly proving to be a lot more effective in identifying early trends in malignant tumor or anything much earlier than what a radiologist or anyone can identify as well.
So these are great examples of how AI is improving not just productivity, but the effectiveness of the work that we actually do.
So I think these are probably just a couple of examples of what there may be thousands of such use cases.
Yeah, I can only imagine every industry has probably got something to that effect.
Let's talk a little bit about data hygiene.
So what's the importance of keeping clean data when working with AI tools?
I think you hear quite a bit about how these large language models or any AI models have trained with large and large amount of data as well.
And it's kind of a garbage in garbage or kind of a mechanism as well.
If your data basically is not of higher quality and it's curated for biases, incorrect accuracy, then I think the output that you basically get out of these models will be suboptimal as well.
So that's why data hygiene plays a very critical role.
I think that in the earlier point that I actually made as an AI PM, you need to understand the data and you need to understand the data platform part of it as well.
I think that's a very critical piece here as well.
One, basically improving the accuracy of the model that you're actually working with is very important.
Minimizing biases is very key in the AI model and that's why data hygiene becomes very important as well.
I think the performance of the model, that's basically accuracy as well, but also the training of the model is a laborious work still.
And you can spend hours and months together in training of the model as well.
And any bad quality data makes the process a lot more tedious as well.
Last but not the least, I think we can expect increasing regulatory scrutiny on some of these AI models and the output they actually generate as well.
And if the source of these tends to be an unhygiena or uncurated data, then the output of the model basically may not meet the regulatory standards that these bodies may actually enforce on in the future as well.
Take a very simple example.
If you are in a financial sector industry and you are advertising for your customers and if these advertising models are not trained with high quality data, they may inevitably introduce biases.
And that biases may be towards an underprivileged community, for example, that violates your Fair Lending Act.
So this is like a cyclical effect that you actually go through.
So that's why the impact of not having hygienic data basically is a lot more than what you can realize at first.
So training the data, like first of all having systems and platforms in place to curate the data, get the data in a way that models can be trained more effectively is very, very important.
This episode is brought to you by Sprig.
What if your product team knew exactly what to build to achieve your toughest goals, like increasing conversions or boosting engagement?
Meet Sprig, a product experience platform that generates AI powered opportunities to continuously improve your product at scale.
First, Sprig captures your product experience in real time through heat maps, replays, surveys and feedback studies.
Then Sprig's industry leading AI instantly analyzes all of your product experience data to generate real time insights.
Sprig AI goes even further with actionable recommendations for driving revenue, retention and user satisfaction.
Join product teams at Figma and Notion by uncovering AI powered product opportunities at scale.
Visit Sprig .com slash the product manager to book a demo and get a $75 gift card.
So this actually is an area that overlaps with the ethics involved in using AI.
So I'm interested to hear your take on ethics in general in the AI space and how can PMs who are new to this space ensure that they're acting responsibly when implementing AI solutions?
I think it's a great question.
I think it's something that is paramount importance for so many different organizations.
In fact, like a lot of these organizations are building up teams that are called responsible AI teams.
It's particularly geared towards making sure that the ethical and moral standards are actually maintained.
I took an example of this financial sector.
This is a great example of how when not followed right can have a societal and economical impact specifically maybe on certain sector of the population.
So if you're an AI PM, it really starts with first of all education, educating yourself about some of the shortcomings that you can actually have unhygienic data.
For example, if your models are not trained the right way, educating yourself from all these things begins the first thing to begin with.
Then basically, the second part you have to focus on is when you find out the regular data has not curated or not can introduce biases, how do you basically what are the steps that you actually take to clean it up to make sure that you retrain the models and validate the accuracy of the models after you
retrain the data that actually becomes important as well.
Then the third piece also is I went back to the principle of having building the right organization structure.
So having a diverse team also becomes very important as well, even as I would even recommend having a dedicated POC that the teams who can actually play the role of diversity or upholding the ethical standards in the team can be useful concept to apply on.
So you can actually have the person monitor, be the champion for making sure that we clean the data, we train the data in an ethical manner so that the accuracy of the models are not more relevant.
So all these are the best practices you can actually apply as a PM in the space to ensure that the ethical standards are maintained.
So switching gears a little bit, what are some of the ways that AI can help us run more efficient experiments?
Yeah, that's a great question.
So think about the traditional way of running experiments, right?
So if you are on, say, DoDash, you basically may be running thousands of experiments in a day, and you have multiple business metrics and system metrics that you have to monitor as well.
If you are a PM, it's typical that you are running two or three experiments parallelly for different features that you're running.
Now you have to comprehend like what are the implications of these experiments, work very closely with the data scientists that is allocated to you to fine tune the experiment and get the readout and then basically work again to fine tune the experiment, make a decision on whether to RAM whether to
DRAM, and also like finally generate the readout and what is actually necessary to to either ramp up the product or even demonstrate the impact of the feature that you're actually developing as well.
So a lot of these things are very human based approach as well.
And now imagine you have built an AI system that can actually optimize your experimentation capabilities, that's going to basically not only make your job as a PM lot more useful, but also the process of running experiments, evaluating them and making decisions becomes a lot more easier for the PM as
well. It really starts with even say like, when you're designing an experiment, what are the optimal parameters to consider and what are the hypothesis?
AI algorithms can actually help you to come up with hypothesis and select the right parameters to run that experiment as well.
So when you are running the experiment in and then it dynamically adjust the ramp of the experiment or even between variants to find optimal balance in running the experiment as well.
You can even make decisions on whether we have mitigated the risk in this initial ramp and then ramping up for more statistical signals.
These are something that they can actually decide more autonomously as well.
Last but not the least, identifying anomalous or anomaly within your business and system metrics because of the impact of experimentation is a really a value added feature as well.
So you can identify the impact much sooner and then you make a decision, the models can make a decision to either stop the experiment or ramp down the experiment before there is a wider impact on the business metric.
So for example, because you launched a feature and that basically say broke some sessions on your app that may directly impact your revenue.
So the models can identify much faster than human beings and DRAM this experiment to make sure that the issue that was caused by this experiment is mitigated.
So a lot of these ways like AI models can definitely help you experiment better, be more productive, and also make timely decisions.
Well, before we wrap, this has been a ton of information and all very, very useful.
I'm sure that folks are going to be looking for next steps because I know that everybody listening is very excited to start to develop skill sets even slowly into the AI space.
Do you have any recommendations for folks who are interested in developing these skill sets if they're not currently actively working on an AI product, any resources or courses that you'd recommend?
Yeah, I think there are plenty of courses on LinkedIn Learning, for example, or in Coursera as well.
One of my favorite is deeplearning .ai.
If you go to that website by Android NG, these courses are available on LinkedIn Learning and Coursera as well.
I think building the foundational understanding of AI is very, very essential.
I cannot emphasize the importance of that because if you have your right foundations and you can then build upon that to understand more advanced technology and then how to apply them in your application.
So what I would recommend your listeners is to basically go understand the foundations first and then build upon that by taking more advanced courses as well.
Podcasts can be a really good way to one learn and also keep in touch with what is happening in the industry as well because this is such a fast and evolving technology right now.
So keeping up pace is really difficult and often podcast serves as a really good mechanism as well.
Third, there are multiple blog posts that get published and blog posts can be a great mechanism to deep dive into a particular topic and understand the practical applications of the topic in your field as well or in general in any field as well.
I think these three can be really good source to go.
There are courses that universities provide, for example, as well.
The effecting of that, I'm not very sure about it, to be honest, but it's something that gives you more structured approach and gives you kind of a diploma as well to break through into the space that can actually be a very good asset as well to begin with.
So yeah, I think these are some areas where you can either overhaul, like just refresh your skills or get to know AI more better to break into this field.
These are all can be very valuable things.
So I just add to more things more than anything else.
I think you need to be curious as a PM in the space so you can actually really invest the time and really understand and dig deep.
So make sure that you learn the concepts and then apply that in your field.
That's really great advice.
Well, thank you so much for joining us today, Praveen.
Where can folks follow your work online?
People can follow me on LinkedIn, and I'm actually trying to basically build my own website as well.
Hi, it's praveengooger .com.
It's still under construction, I would say, but all my publications and upcoming book will be published on that website as well.
Right. Well, thank you so much for coming.
We really appreciate it.
Thank you so much for having me.
Thanks for listening in.
For more great insights, how -to guides, and tool reviews, subscribe to our newsletter at theproductmanager .com slash subscribe.
You can hear more conversations like this by subscribing to The Product Manager, wherever you get your podcasts.
