Thank you. Hello, and welcome to the NVIDIA AI Podcast.
I'm your host, Noah Kravitz. AI's impact on the financial services industry has already been quite significant.
From improving customer satisfaction and loyalty to streamlining operations and reducing costs, artificial intelligence is transforming an industry that's already been at the forefront of technology innovation for quite some time now.
Visa is leading the way in leveraging AI to transform payment experiences.
And with us to talk about how they're doing it is Sarah Laszlo.
Sarah is Senior Director of Visa's Machine Learning Platform, where she's architecting the infrastructure that will power Visa's future.
Before joining Visa, Sarah, well, her resume is really impressive and too long to get into right now, But I have to mention, she was on NPR Science Friday, which is pretty cool.
Sarah, welcome. Thank you so much for taking the time to join the AI podcast.
Yes, thank you so much. So before we get into all the stuff you're doing at Visa, all the stuff Visa's doing, would you tell us a little bit about your own journey, maybe how you got started in AI and how you wound up in the role you're in now?
Yeah. So one thing that I always like to remind people of these days is that Of course, everybody thinks of artificial intelligence as something maybe that computer scientists do.
But we need to remember that the back propagation paper that opened the field back up again after sort of the AI winter was, yes, Jeff Hinton was on there, but it was also co-authored by three psychologists, David Rumelhart.
And I came up through that tradition of psychology.
Oh, no kidding. Yeah. So my PhD is in psychology and I became interested in computational neuroscience. there were many steps along the way, dot, dot, dot, but now I still do it.
Very cool. Did you practice as a psychologist?
I was not ever a clinical psychologist or therapist.
I was a cognitive neuroscientist. So I worked with human research participants, but never in a therapeutic context. context.
Gotcha. And so was computational neuroscience kind of your pathway leading to where you are now?
Yeah. So my postdoctoral advisor, David Plout, was appointed in both psychology and computer science at Carnegie Mellon.
And my academic grandfather in that line is Jeff Hinton.
Most of the people who were in that line now do this because there were not that many people in 2009 that were doing deep learning.
And so if you are one of those, you're one of the only 20 people that have 15 years of deep learning experience.
Totally, totally. Incredible. I kind of hinted at it and we don't want we want to talk about what you're doing.
How did you wind up with this role? Yeah, so I was working in responsible AI at Google.
And I did that for a couple of years. After doing that for a couple of years, I decided that I wanted to get out of California.
I was 40 years old and I was making a lot of money, but I didn't own a home.
My family was in Texas. I really wanted to get to Texas.
But I still wanted to be doing hard engineering problems, doing artificial intelligence and Visa has actually a huge presence of our AIML team in Austin.
So that's how I ended up at Visa. So let's get into that then, into Visa.
What are you doing with Gen AI? What are some of the exciting things, big problems you're solving?
What's going on behind the scenes? Yeah, so when Visa generates new technology, we're always first trying to find ways that we can prevent fraud.
There's so much attempted fraud that even though we're very good at it, even marginal improvements in that actually prevent like large dollars. or amounts of fraud.
So we're always looking at, okay, there's a new computational technology.
Can we prevent fraud better? That's always number one.
Number two is we're starting to get into personalization for our cardholders.
So I tell this story, I won't name the card, but I have a card that has an app on my phone and the offers that it gives me are ridiculous.
It offered me a Lamborghini one time. I was like, we can do better.
We can do better than this. We would like for your app to be able to minimally not get offered a Lamborghini, probably, for most people.
Are we talking free Lamborghinis? No. I think it was probably like, get a free pair of driving gloves if you test a Lamborghini.
I was trying to sell me a Lamborghini, which was ludicrous.
So we want those to be better. We would like to be able to make better recommendations.
And then of course, like one of the big themes of NVIDIA GTC this year, everybody's talking about agentic AI.
And we are very invested in the concept of agentic commerce, where we can have agents that have access to our credit card credentials that can make purchases for us.
Right. The example I've been giving all day is, hey agent, I want to go to Greece in August. do it and it just books it for me is the travel example because everybody likes lots of people like to travel or is it because of the word agent and we just latched on to travel agent I don't know, because I didn't think that I, I thought I had come up with this travel example myself, but then I've heard every other exact here using the same one.
I think it might be because travel is like, if you're going to take a flight, there's only a certain number of flights you can take.
If you're gonna stay in a hotel in a city, there's like the space that the agent needs to navigate is pretty constrained.
And it can know, like, do I have Marriott points or do I have Hilton points?
It can know that. So it seems like it's something that's tractable as a first problem.
I think that's part of it. That makes sense.
So how far along is that development? When can I...
You know, have an agent book me a trip to Austin and a Lambo.
Right. So we're still closer to the recommendation part, like recommending offers, recommending products.
Right now, what we're working on is trying to figure out the right sort of user embeddings so that the agent can have access to an abstract representation of you, the cardholder, without having access to your private data.
And so balancing that is where we are right now.
Right. Got it. I know financial services is a heavily regulated industry.
So there are lots of challenges that go along with that.
Mm hmm. When thinking about and actually developing and deploying Gen AI apps in this space, What are some of the other unique challenges that folks working in other spaces might not know about when it comes to financials?
Yeah, so one thing I've been talking to folks a lot about at GTC is that because we are so heavily regulated, And because Visa in particular has such a strong commitment to privacy and security of our cardholder data, we have been slow adopters of cloud.
So a lot of other, you know, Visa-sized entities that are trying to adopt Gen AI are very likely going to be getting their GPUs through cloud.
Sure. There's a lot of reasons for that, but we don't do that.
We own and operate our own data center, and that then puts a lot of constraints on the kind of soft and hard infrastructure that we can have and therefore the kinds of models that we can have.
And that is something that other enterprises of our size probably don't face.
Right. And so do you get around that by just building things that fit your needs?
Yeah, so actually a lot of what my team is responsible for is figuring out how to squeeze what we need out of the GPUs that we have.
Yeah. That's fun. I love that, actually.
That's the funnest part of my job. Excellent.
You have any best practices to share on that?
Yeah. So one of the things that we have had a lot of success with is we've started using virtual GPUs.
So we are able to isolate memory and compute into separate virtual GPU instances on a single card.
The reason this is helpful is that very often our users request more GPU than they actually need.
And so if we can actually, instead of giving them a whole GPU, give them a tenth of the GPU, it really improves our cluster utilization with very little impact to the user experience.
Right. Is virtual GPUs or are virtual GPUs kind of a standard thing or becoming kind of common practice now, or is that?
I hear a lot of other enterprises of our size is using that because everyone has this problem.
It doesn't take a whole GPU to run a Jupyter notebook, but people whose whole GP is to run juvenile books, but it's inefficient, it's cost inefficient, and it's wasteful.
So I think it's getting to be more standard.
Yeah, no, that makes sense. You mentioned personalization, cardholders.
That's kind of, you know, the trade-off, right, when I think about it from the consumer standpoint at and not with Visa or financial services, with anything is, okay, if I give you my info, I give you my data, like how easy are you going to make it for me?
I would imagine you mentioned, you know, that working on the abstract representation so the agent doesn't actually see my card number or what have you, right?
How are you thinking about and what's Aviza doing and down the road to provide personalized experiences?
What are some of the, I don't know, some of the cool things you're working on?
Yeah, so one of the best things about Visa and one of the biggest advantages we have is that we have an incredible data set. imagine.
You can only imagine. And I think actually maybe more so than people are with the data like consumers or users are with the data that they provide to other large entities.
I think people are pretty comfortable with Visa having their data.
I think everybody sort of uses their credit card and they know that like Visa has been around for a hundred years and that we keep the data very safe and that we don't use it for lots of extraneous purposes.
And I think people feel comfortable with that.
Yeah. We have that dataset and we are using it to make large models. that can have very good abstract representations of consumers.
The reason the representations can be good is because of the magnitude of the data that we have.
Nobody else. So I come from Google research.
I worked on petascale computing at Google.
Right. Not often did I think I was going to say that I was going to work somewhere else and have access to a data resource that no, but that like Google doesn't have.
Nobody else has, but we have that. That means that the quality of the embeddings can be really high because the magnitude of the data is high, the granularity of the data is high.
The computational resources that we can bring to bear to process the data is high.
The data scientists that we have are some of the best in the world.
And when you put all that together, you can come up with a consumer representation that outmatches what anybody else has.
The challenges with personalization, I think, are we don't ever want to expose the raw cardholder data So like the raw data is probably like, you know, any situation, maybe the most powerful because nothing has been removed from it yet.
It's still exactly what it was. And so probably one of the big research challenges right now is figuring out how to re-represent the data in a way that's effective, but that obscures the private information of the user.
Is that as big of a challenge as it sounds like?
It's very hard, yes. because, you know, it's exactly the trade-off.
It's like the more obscure the data, the personal information is, like maybe the less effective the representation is.
And so this is an exact, trade-off and it's hard to figure out what to do.
We have a whole research team working on that.
Right. Since we're audio only for the audience.
Sarah just made a beautiful graph with her arms.
You couldn't see, but it explained everything perfectly.
If you could have seen it, you would understand what I was talking about.
What are some of the other things that are either happening or down the road in terms of personalization?
Yeah, so recommendations for offers, also recommendations for product.
So one of the things, again, about our data set is we don't just know what ads you clicked on or what websites you looked at.
And so that means that we can make better product recommendations probably than anyone else can because nobody else knows that.
Right. We were talking a second ago about virtual GPUs.
As you alluded to with the data center, right, you're working at incredible scale.
What are some of the things that you've had to do, innovations you came up with, challenges you got around when it comes to, you know, just scaling up and offering reliable service?
Yeah. So especially as we move into generative AI era, the models that need to be trained are larger.
That's one part of the problem, but actually, the bigger part of the problem for Visa is dealing with the data hungriness of the new models.
We have trillions of observations in the petabyte regime size of data, and we want to be able to bring as much of that to bear as possible.
But that's obviously a very high memory intensive process.
And so part of the innovation that my team has had to be doing lately is finding ways to like process data efficiently over distributed GPU resources, keeping GPU utilization nice and high and consistent, trying to stream data where we can, trying to lazy load data where we can, trying to use just-in-time techniques where we can.
So as a platform engineer, not only does my team have to figure out how to do that, but then also help our data scientists do it too.
And so there's the challenge, there's sort of twofold challenge.
One is how do we do it? And two, how do we get our folks to adopt it?
So are there any examples of like big internal wins, kind of a, you know, use case that you can share with us talking about how generative AI has really made a measurable impact?
Yes, absolutely. So, you know, Visa has been doing AI and big data work for 50 years, which means there's some code hanging around in Visa that's like pretty old.
And we recently determined that there was a big code base that was was still revenue generating, but that was no longer maintainable because it was written in a language and I won't say which one.
It was written in a language that was no longer supportable.
And so we had a big problem because this was still revenue generating.
It was no longer maintainable. and we wanted to convert it to Python, but because it was written in such an old language, we didn't have anyone in the company who could both know that language and Python.
We were like, oh no, what are we going to do?
This problem was given to my team and so we ended up using GPT-4, to convert the old code base into Python.
This saved us $5 million. And we had one engineer do it.
He converted, I think so far he's done 50 jobs.
He did like 50 jobs in a quarter, which is crazy because he was, you know, he was averaging like converting like almost one of these jobs a day at a system. rate for like at production quality at a sustained rate for a whole quarter because we were able to use these Gen AI tools.
And as I said, that saved us about $5 million.
Fantastic. Yeah. One guy with a transformer model can do a lot.
I'm speaking with Sarah Laszlo. Sarah is Senior Director of Visa's Machine Learning Platform. where she's working on infrastructure architecture to power Visa's future, which, as we've been talking about, is big, wide future that touches a lot of people every day.
There's this concept of the AI factory, and it's picked up some steam recently.
What does that mean to you? Yeah. What that means to me is a single pipeline. that goes from a data scientist with an idea about a model they want to build all the way to the model running in production.
That's what it means to me. I like that.
Is Visa in on this model? Is it a new name for something you've already been doing?
When you think about kind of where this is headed... inside of Visa or broadly for that matter?
You know, what comes to mind? Yeah. So it's interesting because I hadn't really thought much about this AI factory terminology.
Like I had so I had heard it, but I hadn't really thought it was what I was doing until I came here to GTC and I started hearing other people talking about it.
And then I realized, oh, that's what my platform is all about.
So my platform, recently we've adopted what we call the Ray Everywhere strategy.
So we use AnyScale's Ray ecosystem. to do the whole thing, the whole shebang.
So data conditioning, model training, and model serving, we do all in Ray.
And it is an... It is intentionally trying to be more of this factory concept where there's not a whole bunch of distinct parts or distinct tools that are living in different places that work differently it's just one unified consistent pipeline from start to finish right Is the notion of standardization in an AI world, I've had some guests on the pod before talking about that, and some of them have said like,
This is a big thing we're kind of grappling with now.
What's your take on that? And are we headed towards more standardization or how might this play out?
You know, it's an interesting question because I'm sort of two minds about it.
I'm of one mind about it as a platform engineer. for a Fortune 500 company.
I'm of a different mind on it as a individual who works just because I'm a nerd on AI projects in my side.
Like I go home and I do this more when I go home.
As a platform engineer, yes, it's a big deal.
Because in the Visa case, A lot of the models that we already have, so not even looking into future models, the models that we already have, that we already have, work with big customers for.
The faster we can refresh them, the better they are.
Part of the reason for that is that Visa and fraudsters are in sort of a constant cat and mouse game.
Like it's, you know, we do something, there's a countermeasure, we have to have a countermeasure for the countermeasure, a countermeasure for the countermeasure.
And it goes on forever. And so the faster we can refresh things, like the more fraud we can prevent, the better it is. standardization helps with that.
On the other side of it, as an individual practitioner that's been doing this for a long time, like I have been a person that has said many times, like in my courses, that I call it machine learning still really.
I'm old school. Machine learning is more of an art than a science.
And so there's a part of me that's the individual practitioner that likes the sort of artful and surprising and creative endeavor.
Right. I find machine learning to be a creative endeavor, and I will miss that.
I think it'll make me sad if it becomes a fully factorized kind of thing.
Right, right, right. Yeah, no, that that I like that.
That's the notion of continuing to do it when you get home from work.
I've heard a lot of that lately from the folks we talked to on this show.
So what's the adage? If you love it, it's not a job.
It's not a job. And right now is such an exciting time.
Things that I dreamed of doing years ago, I can do now easily.
I can't get enough. Like what? Is there a big one that comes to mind?
Yeah. So I worked in image generation when I was when I was in Google and just the improvement that has come in image generation from, so I worked on Imagine One, the difference between like the quality of something like imagine one or Dali one compared to like what mid journey can create now is just, it's, it's, it's, it's, it's incredible. blows my mind.
I can't get enough. Awesome. Have best practices emerged when it comes to kind of the broader picture of I'm working in financial services.
Somehow I've been tasked with thinking about gen AI.
We got to do it. Everybody's doing it. We need to get in.
Like, what do we do? What do we do internally?
How do we think about our customers? All of that.
You've touched on a lot of this here and there as we've talked.
Do you have kind of learnings to share that someone else, yeah.
Yes, people have been asking me about this all day and all week because we are sort of at the front of this, and I hope people will be able to learn from our mistakes.
We had a guest recently, sorry to interrupt you, but it came to mind who said that, what is it that like trends only happen because some people figured out, like started doing it before it was established? a trend, right?
So you have to be there. You have to be there.
And hopefully we sort of have broken the trail.
And so it'll be easier for the people that come behind us.
One thing that I've been saying to people is that don't discount open source models, like using open source for everything that you possibly can, I feel like is a good practice.
I was going to ask, is Visa building models?
Are you customizing like open weight models or...
All of the above. We build our own, we fine-tune and rag open models, and we use proprietary models.
All of these. One of the best practices is that we do use open source for everything that we possibly can.
And that's not just true for AI. That's true.
You know, we use open source Kubernetes.
We use Red Hat Linux like we use. open source that's a part of our we use we're huge actually visa is one of the you wouldn't maybe your listeners didn't know this visa is one of the largest contributors to the apache open source Is that right?
Yeah, like we're huge Apache contributors.
So open source as much as possible is a good practice.
I think it is very important for... for governance and tech to have a good relationship from the start.
When that is not true, like things can go So I think like making sure that governance and tech are talking to each other all the time from the very beginning, like as soon as your organization has an AI governance function, they should be talking to each other. to the tech people.
Don't wait. Start that right away. I think that's an important practice.
I do think that being very aware of like what models can and cannot do and what is and is not safe. for them to do is important.
There needs to be some employee education about like, you know, don't type your tax return into the AI.
Like people, they're excited to play with the tools and they do all kinds of stuff, but there needs to be some like, here's how you do it safely.
I think that's important too. Sarah, this has been a pleasure and one of these episodes, I think, where there's just so much that you've said that can be kind of taken and examined and looked at.
So this one's going to have a lot of replayability as we say.
So it's fantastic. Thank you. Before we wrap.
For listeners who want to know more, is the best place to go just to go to the Visa website?
Yeah, Visa website is good. Recently, our CTO, Rajat Taneja, has done some really illuminating interviews with the Wall Street Journal, where he talks about his vision for us in the future.
So just look up Rajat in the Wall Street Journal and you can hear about his vision too.
Fantastic. Sarah Laszlo, it was a pleasure to speak with you.
Thanks again for coming on the podcast. Thank you.
Thank you.
