Every product leader faces the same impossible choice.
Move fast and break things or build sustainably and risk getting left behind.
But there's a third path that most teams never consider.
What if you could move faster by being more systematic?
What if the secret to outpacing your competition wasn't about cutting corners, but about building an experimentation engine so efficient that you could test and learn at a pace that makes your current sprint velocity look like you're standing still?
And what if you could learn how to do it from a product leader at one of the most recognizable companies in the world?
My guest today is John Kastanen, Vice President of Digital Product at American Express.
John has spent over a decade at Amex working across multiple functions before focusing on digital product strategy.
You're about to hear how his team quadrupled their experimentation velocity in a single year a counterintuitive lesson about what makes experiments quote unquote successful and how they built a referral program that went from being nothing to their second largest customer acquisition channel globally.
Let's jump in.
Oh, by the way, we hold conversations like this every week.
So if this sounds interesting to you, why not subscribe?
Okay, now let's jump in.
Welcome back to the Product Manager Podcast.
John, it's such an honor.
Thank you so much for making time for us today.
Thank you for having me.
Can you tell us a little bit about your background and how you ended up at your role where you are today in Amex?
Sure.
So I joined American Express back in 2011 as an analyst in our strategic planning group in London.
And over the years I had the chance to work across many functions within the company, all the way from marketing, business development and multiple digital functions.
So I've worked in digital now for about 10 years in different capacities both, I would say, more delivery-focused, some more strategic.
And what has really driven me over the years, I think, is the ability to make a tangible and meaningful impact for both our customers and our shareholders and driving what I call purposeful innovation, which hopefully drives marketplace differentiation for the company.
Cool.
Well, today we're going to be looking a little bit into the future and focusing on what's going on with new technology and products and trying to get ahead of what's currently competitive.
So, to get us started, can you tell us a little bit about what forward thinking with product strategy looks like, specifically at an established company like American Express?
So we think maintaining a forward thinking approach is really important to ensure we're meeting our customer current needs, but also accurately gauging what they'll need in the future.
So one of the best ways that I think has worked, and I would recommend other product leaders to develop, is to develop product strategies with forward thinking approaches that are leveraging constant experimentation.
And why that is important is that you can see what customers are responding to and you remove subjectivity in that process by measuring their actual behavior.
So, as a company, you know, without the traditional bank branches that others might have, our digital presence is our storefront.
So we have to make sure that our digital experiences live up to our brand vision of providing the world's best customer experience every day.
So to do this, what we're doing right now is we're really turbocharging and expanding our experimentation efforts.
And we want to build an always-on ecosystem that's constantly optimizing itself, based on user behavior.
And this is especially important given how some of the younger generations, like Gen Zs and millennials, are interacting with us digitally, and they already make up over 60 of our new customer acquisitions.
But also they have higher expectations of what those digital experiences need to be.
So we need to constantly be upping our game to meet those needs.
I imagine that you're doing any TikTok answers lately or... Not yet.
Not yet?
That might be an over thing anyway.
I'm not sure people are still even doing that.
Okay, well, let's dive into the experimentation journey.
So you're scaling from 30 tests to over 120.
That's this year, which is crazy.
That's massive growth.
What drove this decision to ramp up the testing and experimentation?
And how do you build the organizational muscle to support that kind of growth?
Scaling that much in just one year in the US alone is a big leap, and it's being driven by a number of factors, but predominantly two.
First is the increase that we're seeing to our site in terms of traffic, which basically gives us a much larger surface area to optimize the experiences for.
And second is our collective belief that experimentation is essential to keeping pace with evolving customer needs.
So what this scaling looks like in practice is we had to structure teams, workflows and tools to make testing a core capability, not just an afterthought of something that we do from time to time.
So we reorganize around a dedicated experimentation team whose full focus is to identify, build and run high-quality tests.
And they are also backed by a strong cross-functional support structure that is aligned around a single North Star metric, which in this case is a 10 lift in conversion rate.
So that focus, combined with the support from leadership and the sponsorship, as well as the investment that it requires, has been really behind.
What's helping us to turbocharge that velocity?
But also the impact of the experimentation program.
Okay, so I'm really interested in the nuts and bolts behind aligning a whole team in order to meet this 10 conversion lift.
How do you align these multiple functions?
We have product, we've got design technology analytics marketing, and get everybody sort of on the same page working towards this one specific trajectory.
How does that work?
It's not always that easy, I'm not going to lie.
But I do think what is important is to start by defining a metric that will resonate with all of the parties.
So in this case, what we did is when we looked at from a business perspective, and marketing Conversion is a measure of growth.
So there was a meaningful metric that resonated with those functions.
For product and design.
It was a metric that resonated as well with our typical objectives, because it's about eliminating friction in the user experience and ensuring that the product works hard and it's meant to be.
And then finally, from a tech and analytics perspective, Conversion is a great metric because it reflects how well the systems availability and performance metrics are working, as well as the insights are performing.
So conversion became like this common language for progress and one metric that united the different perspectives to make sure that the ship kind of sailed in the same direction.
So we're also focusing, in addition to having that single North Star metric, on creating transparency across the team.
So we share insights, tracking performance, we establish regular ceremonies and we identify what's working and what's not.
Because in itself, that's always evolving as well.
So that clarity, I think, really allowed us to operate as one system rather than a set of disconnected teams with different competing goals.
And the result is a much more coordinated approach to optimizing the customer acquisition funnel.
I would say that this is the dream.
I think that this is a very, very common issue, even with smaller scaling organizations, to be able to kind of work in lockstep across departments without this siloing getting in the way of people's overdoing work or not completely aligning on those functions.
So I think that's really innovative.
So let's talk a little bit about team capacity, because it's one thing to align all these folks that kind of towards this one North Star metric, but then managing the workload and ensuring that folks are working sustainably at a measured pace and not getting pulled in five different directions.
That's a kind of a different task.
How do you maintain that kind of focus in a large organization where there's always these competing priorities?
So I would say the most important thing that we did here is that we established a dedicated team that was solely focused on experimentation.
So every day when they wake up, they think about this, and it's really their core focus.
So this ensures that they aren't pulling all of the other projects that keep popping up as priorities evolve.
And their entire remit is just to run, optimize, and scale experimentation efforts.
And the second thing that they do is that they also facilitate core teams to run experiments on their own by providing subject matter expertise and project management support.
On top of that, what we're doing is that we're really looking at this from three angles.
The first one is from a talent perspective.
So we brought together the right cross-functional mix, not just across product, but across design analytics, engineering and many more functions, including legal.
But we're also upskilling team members to stay ahead of how the external landscape is evolving.
Second is the process.
So we're deeply examining each step of the full lifecycle of running an experiment, all the way from having an idea through discovery, to build execution and all the feedback loops associated with this, and identifying which are the steps that have the bottlenecks and fixing them so that they don't slow us down.
And the last piece I would say just the third one, which is very important, is tooling and technology.
So we're obviously constantly assessing what our tech stack is here, but we're also partnering with external vendors that specialize in experimentation platforms, so that we can move faster and stay focused on what we do best.
I see.
Okay, I wanted to press a little bit on the results portion.
Because I think this is something that every organization, whether they're a startup, whether they're scaling, whether an enterprise we all kind of deal with this pressure to move quickly.
And especially in this climate where innovation is moving faster than it ever has.
The cycles of development are just at light speed at this point.
And so there's an enormous amount of pressure to change courses when experiments don't immediately show lifts or results in the short term, even if we're quite confident that progressing with that experiment and kind of giving it some time to mature could yield long-term results.
So how have you changed the mindset organizationally at American Express?
And what advice would you give to other leaders who are kind of facing some of that similar sort of cultural and economic pressure, to kind of yield results from experiments and keep moving?
Yeah, so I think that there's two pieces there that I'd like to touch on.
The first one is on the speed.
I think speed is really important, because the faster you learn the better, so that you don't actually spend nine months learning if it's actually not going to drive the outcome that you want.
But then the second one, which I think is often what is the biggest challenge, is this perception that if an experiment doesn't drive the conversion lift, it's been a failure.
And that's not the case, right?
And that takes a little bit of time.
And we've shifted that culture by reframing what success looks like.
And we position learning as a valuable outcome in itself, not just the conversion lift, which is still a North Star metric.
So I mentioned earlier that we have that 10% lift in conversion.
We also have a robust scorecard that looks at other experimentation outcomes, which includes velocity, win rate, the number of variants that we're testing, etc.
So my advice for other product leaders out there is is ready to invest in building a learning culture early.
It takes time.
And it's something harder to test.
So often the default is to launch and just learn on the back of it.
So make space for the teams to test ideas without fear of failure and ensure that you have strong alignment on both purpose and process.
All right.
Well, that sounds very sound as far as a process.
I do want to talk about something maybe a little bit more fun.
We'll go into a case study for the American Express referral program, which I know has grown to become now your second largest customer acquisition channel globally, which is huge, especially for a company the size of American Express.
But it scaled from almost nothing over 10 years.
So enormous success there with that program.
So what were the key product decisions that made this program successful in your view?
And how did you scale it internationally?
Believe it or not, it actually started in international, and it started in France and it then scaled globally.
But one of the key product decisions we made about our referral program was to manage it as a true product from the early days.
Meaning that we thought deeply about the end-to-end experience and optimizing for the user journey.
And to do this, we focused on a few core things.
First, I would say the core infrastructure was really key and we needed to make sure that it's flexible and scalable, underpinning everything that we do.
And then the second one is really about simplification.
So making it easy for the user and making it very intuitive and seamless.
For example, right now, what we're doing is we're testing tactics like QR codes for in-person referrals, as well as asynchronous referrals, where you might want to refer somebody but have them check it out when they get home.
So the key is really about facilitating the virality that we get from the program, while maintaining the right behaviors.
The next one I would say is personalization, which has been obviously a hot topic for many, many years.
But we use many tactics here, including AI and data analytics, to personalize the incentives that we give to our members and their friends, but also to personalize the experience that they see.
And the last really underpinning all of this is controls.
So the same way that we have virality that is good behavior we can have the opposite behavior, and facilitating that virality requires us to have the right controls so that the program attracts the best customers and who we want.
As a result of this I would say multiple-pronged approach over the years, We continue to see a lot of success with the program, especially with the younger cohorts of CAR members, where we see about 75 of our referrals are resulting in millennial and Gen Z acquisitions.
So this aligns with what we're seeing in terms of also digital behaviors.
These cohorts are very engaged digitally and they're also themselves used to referring more, which is fueling this viral loop.
Okay, I want to talk a little bit more, because you mentioned that AI is kind of one of the kind of tools in your toolkit right now as you're moving through these processes.
I'm curious, what kinds of use cases have you been piloting?
Can you share what you've been exploring and how you're kind of evaluating whether or not the results that the AI uses is yielding are worth moving forward with or iterating on those?
I think from a generative AI it's certain relatively early days, but it's definitely going to have very practical implications about our product development and experimentation processes overall.
Right now we're piloting use cases that are focused more on internal productivity, things like speeding up how we are streamlining ideas, writing them into testable hypotheses designed to code, translation and then our ability to iterate faster.
So we're able to experiment with these tools and with generative AI because we have thankfully, a technical enablement layer that provides us with the right guardrails to do that, which allows our technical teams to focus on innovation without reinventing controls.
But I would say it's still early days.
Well, I'd like to talk a little bit about your storefront as well.
So you mentioned that your storefront is kind of your digital bank branch.
You don't have physical locations for people to visit.
How do you approach the personalization experience for prospects who are new to American Express, given that you rely on very different signals than folks who do have more of a storefront situation?
You're spot on that we don't have physical bank branches like our retail bank.
So our website is effectively our digital bank.
And for new prospects.
Personalization can become tricky as we would rely on digital signals typically to personalize those experiences.
So what we've done over the years is build a digital system that allows us to personalize dynamically even as the user progresses through the acquisition funnel.
So then, as we learn more through their interactions behavior, declared inputs, we can adapt the products, the features, the offers they see, or even the experience.
So we're bringing this test and learn mindset that I was talking about earlier to this as well.
And our digital storefront is becoming a space where experimentation is really core to how we evolve the experiences, not just the tool that we use occasionally.
All right.
And we always want to talk about trends.
We said before, we'd be talking about looking two steps ahead.
So, when we look at the future and the trends of technologies that you believe now could be most disruptive to the financial services customer acquisition space, how is American Express positioning itself to stay ahead of those changes?
There are a few major trends we're watching closely, and this is not the full list, but the ones that come to mind.
One is obviously the evolution of customer expectations around immediacy and relevance.
And I think, as AI will become more embedded in everyday consumer tools, this will become even more important.
So we need to continue prioritizing that personalization that I was just talking about, but also the ease of use of our experiences and of our core acquisition channels, such as our referral program that we've talked a little bit about before.
The second trend that is also something we're watching very closely is the way that users search for information online.
And this will continue to change, and it's been changing for the past few years.
So we're evolving our strategy to stay ahead of the shift of what search will remain a key way we engage with users.
And it's shaping both how we think about this from a technology perspective but also from a content perspective strategy, so that we keep up with the changes of core consumer behaviors.
And a final trend I'll mention is the change in competitive landscape.
We're not just fighting for share of wallet and attention with the traditional issuers.
FinTechs are now prolific and they're really defining also what user expectations are from a digital experience perspective.
So I truly believe that our ability to test and learn at speed will become a differentiator.
So that's why we're investing so much in this infrastructure that we were talking about earlier in the podcast, because modernizing our digital storefront and our branch experiences, as well as upskilling our teams across all the functions that make this a reality, to become more agile, is going to be really, really key to win in this space.
So we're very deliberate about future-proofing this and what we build, so our systems and processes are durable, flexible and not reliant on one person or one team.
And I think that last piece is also very important.
Yeah, absolutely.
And it's an ongoing process to refine all of those aspects.
Well, thank you so much, John.
This has been wonderful.
I really appreciate you taking the time out of your super busy schedule to chat with us.
Where can folks continue the conversation with you online?
You can find me on LinkedIn.
And thank you very much for having me today.
I really enjoyed our chat.
Me too.
Thanks so much.
Take care.
Thanks for listening in.
For more great insights, how-to guides and tool reviews, subscribe to our newsletter at theproductmanagercom.
You can hear more conversations like this by subscribing to The Product Manager, wherever you get your podcasts.
