Whether you prefer the term data -driven or data -informed or data -dazzled, it doesn't matter.
Today's tech cannot survive without high -quality data sets and the tools to use them effectively.
But we also can't afford to think about data as the responsibility of just one or two departments in the organization.
Instead, we need to be going into 2025 with a mindset of data democratization.
In other words, our business is our customers, so what our customers are saying should be everyone's business.
My guest today is Mario Tabara, founder and CEO of Quantum Metric.
Quantum is a Gen .ai powered analytics platform that allows companies to interpret data into useful insights for different stakeholders so that everyone from the first day product designer to the company founder can bake empathy for customers into every decision.
We discussed how this works in practice, considerations around data hygiene, and some predictions on how Gen .ai will drive the evolution of analytics this year and beyond.
Let's jump in. Welcome back to the Product Manager Podcast.
We are here today with Mario Chibarra.
Mario, thank you for making time in your busy schedule to talk to us today.
Hannah, it's an absolute pleasure.
Thank you for having me.
Well, let's start it off the way we always do.
Can you tell us a little bit about your background and how you ended up where you are today at QuantumMetric?
Yeah, happy to. The background, you know, as a young child, you could have seen the writings on the wall that I'd be a future entrepreneur.
Maybe we'll get into some of that in our discussion.
But how did it end up here today?
As part of that entrepreneurial journey, I was always passionate about software, computer engineering, software engineering, and really just uncovering how do we make experiences better kind of just fell right into the center of that passion.
It started off with how do I make things faster?
How do make them more performant?
And I started realizing, look, you can make them fast, But if they're not great, it doesn't matter.
How do you make great experiences for audiences from my 13 -year -old daughter to my 87 -year -old father that have different needs?
How do we make sure our experiences work for everybody?
I do want to talk to you a little bit about your approach to leadership because you've had quite a journey in your career.
And you mentioned before that your passion has evolved from product and engineering to focusing on people and empathy.
So how has that influenced your leadership approach?
If you ask me about, you know, with the company, I was thinking about January is rolling around and it will mark my 10 -year anniversary here at Quantum.
And if you asked me nine, eight years ago what I was passionate about at Quantum, it was absolutely creating product.
It's still my passion today.
This weird thing happened and this is my fourth company.
The other companies I had, they were small.
They were one to three people.
They were building product.
They were engineering product, iterating product to success.
What snuck up on me is that we're now 400 plus team members at Quantum.
And it wasn't, I had to get to hundreds.
it was really in the dozens of people.
I realized successful companies are built on the back of the team.
And so you have to have an incredible team.
And then realizing that the CEO, the founder, the leader of the organization is responsible for leading the team.
And that all doesn't work well together.
You can have all the great product that you want, it doesn't work.
And so really creating a great success is about people leadership, investing in people.
And it really comes down to why our first company objective is happy people, healthy and diverse culture.
Our second one is customers as rating fans.
Our third one is about achieving some economical success for the year, but those first two, they don't change from year to year.
Of course, our third one, kind of, we move the goalposts each year about in terms of our success for the year.
But I think some people, some organizations get that backwards.
They focus on like, hey, let's go win a financial metric.
That's our purpose this year.
I don't think you can get there without getting happy people.
I don't think you can get there without having raving fans from your customers.
So it's really about thinking about, yes, I want the economic success.
Of course I do. But let's focus on our people and let's focus on delivering for the customers that have invested their trust and financial wares with us as well.
Right. And of course, empathy is a huge part of that.
We're going to be focusing today on generative AI and how that can make analytics more accessible to product teams.
But before we kind of get into the meat of that discussion, I want to talk about empowering empathy at scale, you know, with so many users to take into account, how do you see analytics playing a role in fostering empathy across a large user base?
If you had people that you had to serve with whatever product or service that you had, and it was numbered in the single digits, I don't think analytics is helpful.
You just go help the person.
You go have a conversation with the two customers that you have.
When that two turns into 2 million, like all of a sudden light bulb moment, I can't talk to all 2 million.
I need analytics to help me understand the customer.
And so kind of like a thought process I've been having lately is less asking and more listening.
And that's what analytics is.
It's really about listening with data.
And I think about a graph or a chart on a screen, you know, when I think about traditional analytics, yeah, it's great.
I saw the graph. It doesn't elicit empathy often for most people.
I think about when I can connect that graph to real people, that's when empathy happens for me.
and I've seen it in the eyes of our customers and the people I speak with, our team.
It's really about connecting the chart to the users that it represents.
And that's really the foundation of what Quantum has become.
It's like, how do I connect the experiences with the information and elicit empathy to enact change?
Yeah, and you know, I love that quote too, that is all about the listening.
I think that's a really interesting approach there.
And I think we don't often think about empathy when we think about analytics.
Those two almost seem like they're counter to each other, but really they do kind of enable our teams to kind of develop a sense of what are the users trying to tell us here.
Tell me a little bit about generative AI and how it can make product analytics more accessible to teams to kind of allow the teams working on the products to get a better sense of what users are trying to say.
There was an early part of this journey at Quantum and I sat down with a major retail product leader, awesome individual.
I've stayed friends with him over the last seven, eight years and he's moved different organizations.
But something that struck me when he took me into a room after we had our formal meeting with his team.
And he said these words, Mario, you would not understand the guilt that product managers have for not using data enough.
And it's stuck with me for the last seven years.
How do we make the data accessible so people don't feel guilty for not using it, that it's easy to consume, that essentially, eventually, people are using that listening skill at scale for huge amounts of data to make the right decisions that both move their business, but obviously that business is just
serving the customer.
Like how do they serve their customers better with data?
And it's hard. Like that's why he's saying it's that guilty feeling.
It's like, I've got my day job and yes, I want to use data, but it seems overwhelming to find the right data that speaks to the problem I'm trying to solve.
I don't know how to interface with it.
It's in 16 different locations across all these different tools.
I don't know which tool is going to answer the right question.
And frankly, I just give up.
I'm going to go with intuition.
I'm done. And so for us, like we've been collecting all this data, we've been trying to make it easier and easier to access.
In fact, like the premise of the whole company in the beginning, we ended up making these t -shirts of we make it simple.
We make analytics simple.
And we've done that, but you know, like we're always trying to figure out how to make it more simple, how to make it more accessible for a range of audiences from marketers, product owners, merchandisers, DevOps, jobs, executives, analytics teams, all of these teams need to access that information about what's happening
from the customer's lens and how can we make it better?
And so for us, for generative AI comes into play is what are the tasks that these teams are trying to do to understand that data?
And so, for example, a lot of product leaders have spent time watching replays of a user and their journey.
Oh, wow. I didn't know they'd click that.
Oh, I didn't know they'd go there first and then do this.
There's all these amazing insights.
The problem with watching session replays is really slow.
It's really tedious.
I've always laughed when people said, oh yeah, we have movie night, we get popcorn and then we watch people and their journey for like an hour in front of a conference room.
Like it's intriguing because it's valuable, but it's not a good use of time.
What if we could get generative AI to watch the replays for us?
So that was this concept as generative AI, because we know that Gen AI is really good at summarizing things.
What if we could point Gen AI's magic to summarize a user's journey?
And it has, I mean, our businesses that has accelerated through this year of 2024, we released this product called Felix AI in the end of February, early March of 2024.
And each quarter our businesses accelerate because it's giving more teams access to understanding of what's working and what's not for the user.
And it's doing it at scale because, you know, if it takes 20 minutes to understand an individual user and you have 2 million, you can do the math.
It just doesn't work.
if we can take two seconds to understand a single user.
And with that generative AI also use it to help create a segment.
Okay, well, this user was struggling.
Does that struggle represent one user or 10 million?
Help me do that automatically.
And that's giving that accessibility to listening to a larger group of people.
So we're really excited about how Gen .AI can enable people to listen better.
To kind of your point earlier, I think there's a ton of pressure to sort of be this data expert, really be able to parse and kind of ask the right questions and go to the right sources.
Break it down for me, like how easy exactly are we talking about when we say, you know, I understand the idea of using the Gen AI to kind of parse the data.
When like a product manager, for example, is using the platform, can you kind of walk me through maybe like a scenario, like a possible scenario, how they might actually tactically utilize that data and use that platform?
One of my favorite stories that has just really made it clear the value of what we do is the ability for us to connect this information to different audiences.
And I'll pick one that we all can relate to.
So Hannah, when's the last time that you called your bank, your telco, your healthcare provider, your airline?
If you can think for me about when the last time you called one of those providers or service providers that you work with, you probably didn't call them until you try to self -service first, right?
You're like, you're on their app, you're on their website, you try to do it, it doesn't work.
And that's when you call.
You didn't say, man, I woke up today and the first thing I'm gonna do is I'm gonna call my healthcare provider, right?
It's like you wanted to self -service.
but when you kind of got to the end of trying to self -service and it didn't work and you called in, you probably ended up on an IVR pressing one through nine a couple of times to get to the right audience.
They pick up, say, hi, Hannah, how are you?
Well, let me tell you how I am, right?
I'm trying to change my healthcare insurance for the year or look up, you know, my coverage and it didn't work.
Oh, hold on, Hannah, let me get to you with an expert.
Hi, Hannah, how are you?
You're right. Wouldn't it be amazing if we could change that conversation to, we know what Hannah was doing.
We know what she's calling us about.
We're going to route her directly to the expert.
Hi, Hannah. It looks like you're looking up coverage in the app, but you got an error.
Super sorry about the error.
I can help you with that or something else.
And really what I'm trying to convey is all of these companies have been collecting data about what Hannah has been doing on their website or app.
They just haven't been able to activate that information into the call center as just one example of an audience that could be valuable with that information.
And Gen .AI has allowed us to summarize Hannah's journey in two seconds.
In the past, we've always had it.
It's just like, was it available in real time?
Quantum has been, but even with the real -time availability, hold on, Hannah, let me watch what you've been doing for the last 30 minutes.
And then I'll get back to you.
No, it's easier just to ask Hannah what she wants or tell her to press one through nine on the phone.
The ability to summarize what Hannah was doing, and we know Gen .AI is good at that summary, that's been transformational for our business now.
That use case of the call center, obviously not for our product audience as much.
But think about the product manager.
Think about, I'm trying to understand why this user didn't convert, why they didn't sign up, why they didn't convert, why they didn't book their flight or change their service correctly or change their address, whatever it was, tasks that they're trying to achieve.
How do I understand the why behind that journey?
And it takes a lot of times, you know, analytics, session replay, there's a lot of different tools that we have, log files.
How do we make that simpler?
And so how do we make that understanding of the customer's journey easy for the product owner?
And that's what quantum is doing with gen AI.
Like, let's just transform that understanding into seconds.
And then, you know, I think there's that understanding of the individual is important, but then understand, okay, does this experience, this friction point or this optimization point represent a small use case or a big one?
And I think, I was actually at a major bank here in the US this week.
And I just, you know, smiled from ear to ear when I heard a quote that one, you know, one of the team members had said, and it said, this is helping me be an operational product manager.
And I've never heard the term, but it really speaks to me a lot, which is, you know, there's product managers.
And I think we need to transform the product manager thinking to operational product management.
How do I use data to understand what's working and what's not for my customer in real time and pivot in real time on making sure that we're making our customers happy?
I think there's a lot of product ownership that says, oh, I've got my tasks.
I've got my sprints.
I'm going to go do that.
I'm going to focus.
I'm going to be done.
When I'm done with the release, I'm done and move on the next sprint.
That's crazy to me.
I've got to use data.
I got to know, am I betting on the right change, right?
Like using data to make sure that the change or experiment or whatever I'm doing is the right thing for my users, you know, in a prioritized way.
And then when I release it, whether it's an experiment or a full release, read the data correctly to understand, is it having the right impact?
Don't move on to the next sprint.
Make sure I'm using data to like reinforce I'm making good decisions or bad and I have to change the way I've been thinking because this isn't moving the needle as I thought it would or it is.
And by the way, I want to elevate that to my leadership.
I'm making good use of our resources.
I'm impacting the customer experience.
And hey, maybe I need a promotion or a raise or a bonus this year.
But data can help confirm that I'm doing good decisions or bad.
And I think that's what an operational product manager is.
There's a bit of an issue right now in which a lot of pressures on product managers to make very, very sound decisions using data, even though they're not really data scientists themselves.
It's very easy to make incorrect assumptions or kind of misinterpret data in a way that can kind of mislead our understanding and interpretation and then thus lead to incorrect assumptions.
It kind of seems to me that if you're using Gen .AI as sort of a middleman and you're kind of using it more conversationally, then you kind of mitigate some of those assumptions.
Is that correct? Or do we still need to be kind of taking precautions to make sure that we're still making sound decisions using this tool?
I don't think it's Gen .AI or not, but I will speak to the Gen .AI, part of this.
I think it's what data source are we using.
What I love about the data that we collect is undeniable truth.
And these are words our customers say.
When you see that a customer is trying to change their service or their telco or book their flight and it's not working, you can't look away.
And like, there's just so much data sources available to any team member, but it will pick on product managers.
And if you say, Mari, I need you to go show me that my decision is right.
I literally can find a data source in the organization that proves me right.
That's a terrible approach, by the way.
Don't do it. But that's part of the problem that we have is like anyone can find data that confirms confirmation bias.
Oh, I'm doing the right thing.
And this is why I was speaking earlier about the operational product manager that's using their business data to confirm that their decisions are accurate because when they do that release and it's moved the needle for the business metrics, awesome, right?
And so I think it's about which data do I use for that confirmation?
And I think about what we're focused on is what does everything look like from the customer?
I don't care. I mean, I do care, by the way, about log files or backend server performance and stuff like that.
I don't dismiss it.
I don't care. I only care about it when I need to care about it.
I don't think that we should start there, whether we're product managers or executives and business owners.
It's not starting at the, hey, look, the API is faster.
That's great. Everyone's going to get a bonus this year.
Like, who cares? And I'm not saying I don't.
I'm saying, should I?
And what I think about it is a customer doesn't call or fill out a survey or tweet or email the CEO with a complaint saying, hey, look, I'm in the B experiment.
I'm in the C campaign and this API is broken.
They just don't talk that way.
Hey, your website sucks.
Your app sucks. And I'm frustrated.
And so how do we get the product organization to think about it from the customer's lens?
I think that's the key to, you know, behind your question.
And yes, absolutely.
I want to add Gen AI to make it accessible, more easier to understand that data, because I think it's not the struggle that organizations have access to the customer's lens or not.
There's lots of different ways to do it.
It's a question of like, how do they interpret that information to make the right sound decisions and not running off and like, oh, but this log file says this and this API says that I've seen that happen in organizations.
And I've seen this toiling of hours of dozens of people trying to make the right decision just off the wrong data.
So I think it really is grounding in.
Are you looking at it from the customer's lens?
That's really the only way a business should be run.
It's focusing on the customer first.
And then, yes, you know what?
The customer is frustrated here.
Wow, I've tracked that down to an API or the log files tracking this line of code.
Let's go focus our energy and efforts there.
But sometimes it's just, it wasn't obvious that they needed to click this button or it was below the fold and they're never going to see it.
And there's like this, all these parts from the user's lens that get missed in log files and APIs and just really the data source should be essentially listened to the customer, not necessarily ask the customer, which I would, you know, my mind, I relate to, you know, get a survey.
Is the customer happy or not.
And I find value in surveys and log.
Like, I'm not dismissing any of these other techniques, but they should be used at the right moment.
And I think it's really about looking or listening from the customer's lens first.
You've championed in the past democratizing data within organizations.
I'd like to know, what does that mean to you, first of all, and what challenges do companies tend to face when they try to make data more accessible across departments?
We've seen this evolution of consumption of data, I think, to the year 2000.
And I'm sure some listeners are like, yeah, but, you know, people use data before 2000.
But I think like digital came to life for me using data and I'll pick on things like, you know, for the OGs listening, like Webtrends as an example.
Like I remember, you know, 1995 or so using Webtrends to get analytical data from log files.
And then we progressed to companies like Omniture and Urchin was the predecessor name of Google Analytics.
We started collecting like pixels and understanding journeys and experiences that way, you know, so it kind of started off in marketing analytics.
If we kind of put it somewhere like the understanding of digital happen in marketing, we then saw the advent of the world of APM and we saw these kind of DevOps analytics coming to life.
And then we've seen experience analytics with companies like TeaLeaf and ClickTail that generally don't exist in their former forms, you know, today.
But they kind of were the initial take at how do we understand the user experience, you know, from heat mapping to session replay.
And then, you know, in the last five to 10 years, we've seen product analytics come as a category.
I stand back and I listen to our customer and I really see the consolidation of all of these different kind of point solutions or siloed analytics to how do I take this data and democratize it across the organization to understand the why?
And I think that understanding of like, like it's been easy in any one of those systems to understand the what.
Like, hey, look, there's drop off at stage three of the funnel and we only have 2 % conversion rate on our retail website.
Like that, I think every system has done pretty well.
I think, okay, okay, why?
Why are people dropping off at step three?
I think that has been like the journey to understanding that why is different for a product owner, a DevOps owner, a merchandiser, a marketer.
And because of like that data is so similar across those use cases, I don't think it makes sense to have all these point solutions.
So I think that the goal for us is how do we take that understanding of data and make it consumable by all of these different personas?
And that journey is going to look different.
The tools, like I'll pick on a UX designer, for example, they typically don't spend their time in analytics tools.
But when we overlay the analytical data in the format that they're comfortable with, which is just the screen, for example, so we take like, what are people clicking on?
And does that lead to success?
And, you know, I'll pick on a retailer like a purchase.
It starts to link the connection between the content, the layout, the merchandise to the financial goals that we're trying to achieve.
And ultimately, that's really what customers came to your website for.
They didn't come to browse and read content.
Typically, they probably came to like, hey, look, if you have the product I want and the right price point, I'm going to buy.
How do we make that easier for that consumer?
And so taking that data and making it consumable for the UX designer, for us, meant overlay the data onto the website that they're on.
Or for an APM person, getting to the Y, or, you know, DevOps person, it might be, did the page loads fast enough?
Were there any API errors?
And so on. And thinking about like how, you know, for the product owner, it might be like, hey, which experiment were they in?
And, you know, which step did it break, you know, at?
And just thinking about like, what are the different kinds of questions and the right formats of presenting that information for the different personas?
I think that's what democratizing data means for me, it's like making that same base data layer consumable by the different personas.
So we don't have 16 point solutions that are being maintained by our organization.
And, you know, I think going into these organizations, I see people fighting over like the most mundane, hey, your data is different than mine.
I've got 2 .3%, you've got 2 .4, man, there's something wrong here.
What's going on? And they just spend hours and days trying to narrow that down to like, man, if we just had one system and just agreed, because I don't think the 2 .3 or 2 .4 is the key point to like argue on.
It's like, let's just accept that our definition of a session or user could vary.
Where are they struggling?
Where can we optimize the experience?
How can we get them through checkout faster with less clicks, you know, or so on, or less friction points?
How do we make the experience better?
And that's an opportunity for Gen AI to make this easier where, you know, one of the things that we did with Felix AI, once we realized we could summarize intent and a user is, wow, we can shape this for a different audience.
You know, sure, someone might want a quick summary, but I mentioned that call center persona.
They don't want to know all the detail.
They just want to be able to, like, how do I interact with the customer in a very efficient manner?
The product owner might want to know, like, what experiment were they in, and what steps did they go through, and what features did they interact with?
A DevOps person might want to know, like, which pages load slowly and fast, and we can shape the response of Gen AI for that audience.
And that has been transformational for our customer base, because it just makes it easier to access the information that they want for the right persona.
We've got a couple more topics to cover that are a little bit wildly different.
So I'll be a little bit rapid fire here.
Data hygiene. So let's talk about that.
A lot of companies struggle with data hygiene.
How does Gen AI help to mitigate some of the issues that we see commonly across organizations?
I'll tell you, Hannah, I am like smiling from ear to ear because this is a project that's been near and dear to my heart since 2015.
It was when we created our first explainer video.
We said in that video, we're going to go do this.
I haven't found a way to do it.
Here comes Gen AI. I'm so excited that at the end of this quarter, So the next couple of weeks, we are releasing a product that does this, that actually finds the data that we need to make decisions for our business.
And I've been traveling the last couple of weeks and every customer and prospective customer I've been talking to, I ask a very simple question, how often is your data right?
And how often is it wrong?
And you'd be shocked that all of them say on a weekly basis, they're having to make decisions with not having the full data.
How difficult is that to make decisions and not having the right data?
So what is the value of having the right data all the time?
And there's been some confusion in marketing and the marketplace here, auto capture, auto tag and all this stuff.
We really haven't hit the point where we can extract the right information, maybe going beyond tagging to collect the right information to run our business.
Because I keep seeing people not having the right information.
I think for product owners, what I've heard from them over and over is we're in a sprint.
Analytics is in our timeline.
We've got a bunch of bugs that creep up at the last part of the sprint.
I can either tag the right data and get my analytical information, or I can fix the bugs.
And you know what? They all choose.
And so they end up going to production, not having the analytical data to understand, are they having success or not with the new capabilities they released?
How crazy is that? What if we can automate that with Gen AI?
What if we can ask Gen AI to say, hey, I need these pieces of information, find it in experiment A, B, and C without me having to say specifically where.
Don't have to tag it with my engineers.
Just extract the right information from our interactions, from our website users, and our app users.
And the look on the face when we demo this product, I mean, I was asked, I was talking to an industry analyst.
I said, one to 10, I want you to tell me what you think about this.
And I showed, I demoed the product working.
She said, 10 plus. This is amazing.
Yeah, it's enough to make you a little misty in the eyes.
I'm excited about 2025 success, about how we empower organizations to have the right data to make decisions in real time.
That's really exciting.
And congratulations in advance.
Okay, so let's talk about, since we're talking about 2025, let's talk about the future.
So you've predicted in the next few years that product analytics will evolve into customer or experience analytics.
What trends or innovations do you see driving this transformation?
Understanding the customer, listening to the customer is more than just digital analytics.
There's a lot of information that we can get from that's already been collected in organizations.
It's just not being pieced together.
So I'll give you an example.
Like, you know, in Quantum today, we pull survey data.
We integrate with the major survey providers.
And sometimes we want to use data to listen without having to ask where are the friction points.
Sometimes it's just phenomenal.
Just ask, Hannah, are you happy or not?
Did we use data to understand Hannah's experience successfully?
And when that matches with the data we collect, awesome, when it doesn't, okay, there's something that we're missing and we can go find part of the experience that we weren't really understanding for Hannah.
But there's more than just surveys.
What about when Hannah fills out a review?
It's not like we asked her, she just said, hey, I love this product or I'm really unhappy with this product.
What about, I'll pick on a telco as an example.
What about when Hannah had three dropped phone calls this month?
Do you think she's going to turn or renew her subscription with the telco next month?
Maybe that's not the only thing that has to happen, but maybe she ordered a new iPhone and it arrives.
Like literally one of my team members ordered a new iPhone with a telco this year, never got it, you know?
Do you think that person wants to renew and stay with that telco still?
So there's all these pieces of data that we can understand a 360 -degree view, both comprising our offline experiences, our online experiences.
And I think that fundamentally what we've been trying to do the whole 10 years of Quantum is listen to customers.
Are they happier today or less happy?
Are they more loyal today or less loyal?
It's definitely more than surveys.
It's definitely more than digital experiences.
How do we combine all that information to help executives, product owners, DevOps, all these different teams understand what are the points that matter to the customer, subscriber, whatever, kind of audience that we have, and how do we make it better for them?
How do we focus on the moments that matter?
Well, this has all been extremely interesting.
It's very exciting to hear how generative AI is completely going to be changing the game for analytics and making it so much easier and more accessible for anyone, whether you're a very data -focused PM or whether you're a lot more of a soft skills PM.
So very exciting stuff to talk about.
Thank you for joining us, Mario.
Where can our listeners follow your work online?
Feel free to follow me on LinkedIn, but for my work, but the company QuantumMetric .com, we have so much content of sharing success stories from our customers, what's up and coming and being released.
So join us on QuantumMetric .com, follow QuantumMetric on LinkedIn, and listen to some future podcasts here with Hannah and I.
Awesome. Well, thank you so much for coming.
Thank you, Hannah. Thanks for listening in.
For more great insights, how -to guides, and tool reviews, subscribe to our newsletter at theproductmanager .com slash subscribe.
You can hear more conversations like this by subscribing to The Product Manager wherever you get your podcasts.
