Hello and welcome to the NVIDIA AI Podcast.
I'm your host, Noah Kravitz. Most of the time on this podcast, we talk about the things, incredible things, frankly, that people are doing with artificial intelligence.
For this episode, we're going to shift gears a little bit and talk about the ethical implications of AI.
Type ethical AI into a search engine, and you'll get a long list of research and news articles mixed in with statements of principle from the world's leading corporations and think tanks.
One of those corporations is Salesforce, makers of business software used by companies all over the globe.
Browse over to the Salesforce research site at the very clever URL, einstein.ai, and there's a section devoted to ethics that includes several articles written by our guests today.
Kathy Baxter is Architect of Ethical AI Practice at Salesforce.
She's a veteran user researcher, speaker, and author, and she's here today to talk about ethical AI.
Kathy, thanks so much, and welcome to the NVIDIA AI Podcast.
Thank you so very much for having me. Before we dive in, a quick disclosure, probably doesn't matter, but in the interest of transparency, I actually used to be a full-time Salesforce employee and still do some contract work for them.
So really the upshot of that is it made for kind of a fun moment between us.
When I introduced myself to you, Kathy, earlier via the Salesforce internal chat, not on NVIDIA channels.
But that said, I don't do anything with AI.
And we're here to talk about your work. So let's get into it.
We're kind of start high level since this is kind of new ground for our show.
What is ethical AI and how do you, how do we think about ethics in this field?
Yeah, that is a fantastic question. I know that some people are uncomfortable as soon as you start talking about ethics.
You start getting into the area of morals and values.
And people tend to really feel uncomfortable, especially if they come from a math or computer science, you know, one of the quote unquote hard science backgrounds.
They tend not to feel comfortable with the squishy soft subjective aspects of technology. but we have to recognize the importance of socio-technological issues.
Technology is embedded And so we have to understand that for everything that we build and bring into society, it has an impact.
It is going to have an effect on who the humans are that are using it.
And it creates its own. power differential, who gets access to it, who gets to decide how it's used.
And so when we think about the ethics of any technology, We have to think about the goodness and badness of it.
Does it help? society more because of its presence than when it was there before.
And so at Salesforce, when we're thinking about ethical AI or responsible AI or trusted AI.
And you'll hear these words used interchangeably in the industry all of the time.
We want to make sure that we are creating technology that both our customers as well as society can trust, that they know is there. in their best interest to help them. and so when we think about ai we believe that it needs to be first and foremost It needs to safeguard human rights and protect the data that we Salesforce are entrusted with.
It also has to be accountable. We regularly seek and leverage outside feedback, whether it's from our customers or from external civil society organizations, other AI experts and ethicists. for continuous improvement.
We want to make sure that the research that we are publishing is safe to publish. that we are identifying any potential unintended consequences and mitigating them.
This helps us ensure that we are not working in an echo chamber.
So can you give us a, make it concrete, for the audience with an example, either in the abstract or from Salesforce, when you talk about... responsible AI and the impact on society and sort of, you know, a good impact versus a bad impact.
What's an example? So a good example would be creating tools that help people do what they already do even before.
Sometimes we talk about automating the dirty, dangerous and drudgery.
So getting away the harm or the tasks that humans might not be as good at good at so that it gives them the space to do the things that humans are really good at.
So creating better outcomes. So one example of this kind of application and AI for good application is our SharkEye project. where we use drone footage over the ocean. to spot sharks and we use our Einstein vision, our object recognition AI, to be able to identify great white sharks and specifically adults.
We don't want to see kelp. We don't care about surfers.
We don't want to even spot the junior sharks, which aren't very harmful.
But we want to find those great white adult sharks because Those are the ones that we want to track both from an environmental system and environmental concern, but also to help humans and sharks share the ocean safely.
And so by combining technology like drone technology and our Einstein Vision AI, then we can send alerts to lifeguards on the beach to be able to raise an alert that a shark is nearby.
So that's an example of something that humans aren't able to do. and so we can create a safer environment for both sharks and humans. by doing this kind of tracking and notification.
I now have something. Super cool to tell my seven-year-old about my day at work today.
So I appreciate that on a selfish level.
What about an example of a negative impact?
And perhaps I think quite often they're unforeseen or certainly unintentional, but we, we've had guests on the show previously talking about, you know, using AI to help them, uh, Well, lots of companies and individuals using AI to help them crunch large troves of data.
And sometimes these are used to help recommend content for people to watch or listen to or consume.
Sometimes they're used to help people make hard decisions about giving access to loans or credit or healthcare or what have you.
And I know one of the things that you work on, you've spoken about before, is bias in AI.
I don't know if an example is the best way to get into this, but can you talk a little bit about how we sometimes get these negative impacts coming from AI and how ethics play into that?
Yes. It goes right back to what you said a moment ago of those big troves of data.
Bias lives in the data and AI is only going to mirror or magnify the biases that are already living in society.
And so if we want to use AI to recommend who is going to be a good a job candidate?
Who is going to be most likely to repay a loan?
Who is going to be most likely to be successful accepted into a university?
All of those things we have to keep in mind are going to be influenced by the bias that exists in society.
And if you simply ignore that bias and think that you can fix it with a quote unquote neutral technology, That's not going to happen because the AI will find those patterns in the data.
It's going to identify for example, that your company has a tendency not to hire women. or not to hire people from a specific university or program.
You may find that your bank has a history of not making loans to people from a specific race or or a certain zip code.
Zip code can be a proxy for race, given our history of redlining in society.
And so this is one of the big concerns that we have with AI is that it simply learns the bias and the bad habits that exist in our society and then will replicate So what do we do?
How do we correct that? There are a number of different ways that we can go through.
Some of them are going to be mathematical.
We go in and we're going to make edits to the training data.
Ideally, what we really want to do is change our processes. change how we go about looking for candidates for jobs. or how we determine what makes someone credit worthy.
We can also do some edits in the model itself.
So perhaps we don't use race or zip code. as factors in our models so that we look at only other kinds of factors, like what someone's salary is, how often they have repaid loans in past.
So there are a number of different things that we can do.
But one of the magic things about AI is that we actually can become aware of biases that we might not have known even existed in our business processes in the first place.
And my hope is that we will use that awareness as an opportunity to go back and change how it is that we do business in the first place.
So talking about doing business, let's shift a little tiny bit and talk about your work at Salesforce.
Salesforce makes software. that businesses and organizations use to run their own businesses.
How does AI play into this? And what's your role?
What does an architect of ethical AI practice do?
So Salesforce has a number of different AI products. we call Einstein.
Einstein is our AI platform. And so in Sales Cloud, we are able to to examine all of these different opportunities that a sales rep has that they could call up and they might be likely to buy your product. but some are probably more likely to buy it than others.
And so Einstein can help you identify those leads that are much more likely to buy and say, call these people first, spend your time on these people so you're more efficient.
In Service Cloud, we have our chatbots product that allows customer support centers to automate some of those what we like to call one and done.
So those really easy questions that you don't wanna spend 20 minutes waiting on hold, Just to find out what is the status of the shipping of your product or resetting a password type of things.
I love it when I get the phone tree that lets me hit a button and get that automated.
And I don't know if your service cloud powers that or not.
I said before I do some work with Salesforce, but I generally don't know if you do that.
But those types of things are fantastic.
Yeah. So being able to help people get their job done as efficiently and effectively as possible. is really our goal.
And so my job has three main components.
The first one is with educating our employees. helping them develop what I like to call an ethical spidey sense.
So Yoav Schlesinger on my team teaches a whole section on ethics in our tech and product new hire bootcamp program.
So as a new hire employee, you come in, you're going to get your laptop set up, you're going to learn about all of the standard things that you need to know at Salesforce and security and all these different things. and you're going to learn about ethics.
Why does this matter at our company? What are the processes that we have in place?
Where are the resources that you can go to learn more and be able to ask that question of not just can we do this, but should we do this?
And so we have TRI. and other learning content available for our employees and also for customers, which leads to the second area. where we build tools to help empower our customers to use our tools responsibly.
We are a platform. see or touch our customers' data.
And so we want to build tools to help them identify, is there bias in their training data?
Is their model going to make decisions that are not fair and equitable to all of their customers?
And then third, outreach to the world, working with groups like the World Economic Forum, and other groups that are in roles like mine at other companies, to help build these kinds of ethical processes inside their company.
We very much believe that high tide rises all boats.
And so we want to work together and collaborate with others so that we can find solutions as quickly as possible.
So I want to hone in for a second, if we can, on The point you made about educating your customers, and maybe this is clear to the audience, but for me, my brain goes to the meta, but so Salesforce's customers are these businesses and other organizations who in turn have their own customers.
So let's say that I had a company, Noah's Podcasting Company. and I'm using Salesforce's platform and I'm using it to gather and store data about my own customers uh listeners to the podcast advertisers guests you know what have you um I, as many businesses do these days, now have this trove, a small trove, but a trove of information about my customers, some of which...
You know, some of which may be highly sensitive credit card numbers, that kind of thing.
Some of which may seem pretty benign on the surface.
But like you said, something like a zip code could be a proxy for race or other socioeconomic data. that could be used in a number of ways.
So when you're talking about training and educating and working with your customers around ethical AI, What does that look like and what are some of the things that either pop up as problems or teachable moments? or even things that might surprise you when your customers start to have the power of Einstein and your other products at their disposal.
Yeah, there's so many examples that come to mind.
I'm trying to think about which one to share first.
So we do a lot of in-app training. guidance where we give some information in the moment that if they are using what we call sensitive variables like age, race, gender, they might want to reconsider that.
If they're in a regulated industry, they may not legally be allowed to use those.
And so providing that kind of information. and then providing tools that go one step further.
In Einstein Discovery, for example, An administrator can go through and highlight all the sensitive variables in their dataset that they don't want to include in the model, so they may select age, race, gender.
Then Einstein discovery goes and finds all of the other variables in that dataset that are highly correlated. that are very related to those.
And so the example I gave earlier, zip code is often highly correlated with race. because of our history of redlining.
And so Einstein discovery will flag those correlations for you as well, And then the admin can go through and say, okay, yes, protect those.
Don't include those sensitive variables either. but they may want to include them.
So if you're trying to decide who to send coupons to for your physical store-
You need to know what the person's zip code is.
But if you're online, then it probably doesn't matter.
So creating these kinds of tools that provide information and help the customers think through what they want to do is a really important piece for us.
Our guest today is Kathy Baxter. Kathy is the Architect of Ethical AI Practice at Salesforce, and she's got a long history working as a user researcher and then as a speaker and an author. on a book that was about listening to customers?
Am I in the ballpark? Yes, it's on user research methodologies to help people who might not have a background or training in that field know how they can collect information about the people who use their products or might want to use their products.
So I was kind of struck by the, you know, sort of simplicity, but seems like a no brainer that the, training and the guidance for ethical use of AI for Salesforce's customers, the first thing you said was in-app guidance.
And it was like, oh, Well, of course, but this is great because it's for someone who either isn't thinking about it or isn't trained or is just trying to, you know, get their database set up or their what have you set up, their marketing campaign set up. having these flags in the software to get them to stop and think about it, I think is a great idea.
And not surprising that your background would be in user experience.
But let's talk a little bit about how you got into the role that you're in now and where your interest in AI and ethics of tech came from.
Yep, so my bachelor's degree is in applied psychology And master's degree is in engineering psychology, human factors engineering from Georgia Tech.
Are you a Georgian originally? I am. Yes, I've I've mostly lost my accent.
But if you listen to me speak to my mother on the phone or if I get really mad.
My accent will start to come out. Excellent.
Human factors engineering is all about designing systems that work for humans.
And the program at Georgia Tech had a strong emphasis on research ethics.
How do you treat the people that are in your studies?
How do we think about the ethics of what it is that we are creating?
So from the very beginning, had a strong grounding in putting people first and thinking about the ethics of my work.
And so yeah, for 20 years or so, worked in the user experience field and wrote a couple of editions of the book, Understanding Your Users.
Excellent. It was created during the dot-com boom.
I remember that. Lots of VCs were out there and some of them were successful and some of them not so much. lots of people were trying to find that next big thing.
And so we found we needed to help people think about how do they, again, how do they do user research?
How do they think about and understand people and how do they do it not just a methodologically sound way, but also in an ethical way.
So there is a big section on that in the book as well.
And so your current role at Salesforce, you joined Salesforce doing user research?
Yes. October 2015 started as a user researcher.
Okay. And then end of 2016, Mark Benioff, our CEO, announced that we were going to become an AI company. and the first product coming out of Service Cloud, which was the particular team that I was working with, would be Chatbox.
And immediately Microsoft Tay came to mind for the, audience that doesn't remember Microsoft Tay.
That was the chatbot that once it was unleashed on Twitter, The Twitter trolls unleashed on it and in less than 24 hours became a racist, Nazi-loving chatbot.
And it was what really spurred me to start doing research into AI to find out why did that happen?
How do we make sure that we don't do that?
And it was like falling down the rabbit hole for me.
I started doing more and more research and it It was like pulling pulling a thread on a sweater where I didn't have to look too hard before I started finding more and more and more examples of.
AI causing harm. And in each one of the cases, if you dug in far enough, you would find that the people who were creating or implementing the AI systems, it wasn't with malice.
They didn't want to cause harm. In fact, they were trying to implement AI to deal with the known human biases.
They wanted to create something that was more neutral.
But they had this love or admiration of technology, this belief that it is neutral and it could simply by its complexity and magic, if you will, be better than humans. and they didn't understand fundamentally how AI works, how it learns and adapts and changes over time. started just spending more and more time doing research in this area and working with Einstein teams across the company.
Then in April of 2018. I met my now manager, our chief scientist, Richard Socher, and told him about the work that I was doing.
And he asked if it was something that I wanted to do full time.
And I hadn't really thought about that. but then pitched the role to him, and he agreed.
This was what was needed, and so he Then went to Mark Benioff and asked for a headcount and Mark agreed, yes, we need somebody dedicated to this.
And so in August of 2018, joined his team and have been doing this ever since.
$64,000 question, but how's it been? What's it like, you know, guiding AI ethics for a publicly traded company like that?
It's it's grown tremendously in the very beginning, the conversations that I would have with people. they would be very interested.
I've never had a single person say ethics, schmethics, who cares?
We have a really strong... culture in our company of giving back and social justice and so I've never really had to convince people that thinking about and building our tools in a responsible way was necessary. but it's been instead more about how.
How do we do this? How do I give the tools to so many people across the company when there aren't that many of us. so that they can do the right thing.
They're empowered to know what is the right thing.
So it's been a matter of how do we scale, really?
I think that's been one of the most interesting challenges.
And the other, I think, big challenge is creating the tools and the frameworks when nothing exists today.
This is completely green space that we are moving into.
It's very much like the security industry in the 80s.
Paula Goldman, our chief ethical humane use officer, likes to give the example of when the first viruses and malware came out.
At that time, there was no such thing as cybersecurity.
There were no standards. Today, if you were to launch a software product without having a full cybersecurity team go through and do pen testing and red teaming your product, you wouldn't stay in business for very long.
So today, the field or community of ethics and tech is very much where the security industry was. say, in the 1980s.
And so if I was running a company that was interested in planning to use AI, the big umbrella sense of that term, whatever it means, but some sort of AI technology in our products.
And we were interested in ethical AI. What advice would you give for how to get started in designing an ethical AI practice in sort of the the wild wild west or the green space as you say of this kind of new frontier Yes.
So I think first and foremost, you have to identify what your values and priorities are as as a company and build off of that.
There are lots of companies, as you mentioned in the beginning, if you do a search on ethics in AI or ethical AI, you're going to see a lot of companies that have published their principles and you don't necessarily have to create net new principles.
A lot of that work has been done. But what you do have to know is How does this dovetail with your existing company values? you'll be hard pressed to find a company out there that doesn't have some kind of stated values.
And so as long as you know how this works, how this fits in, I think it's going to be a lot easier to talk to your executives and people in the company about that because they're going to say like, oh, yeah, this does already support the way that we work today.
I would say the second thing would be looking for processes that already exist and that you can tie ethical questions or reviews into.
Rather than creating net new processes, new hoops and new reviews and new documents that people have to fill out. no one's going to be happy about that.
You're immediately going to meet resistance.
AI was supposed to reduce my workload, not add to it.
Exactly. You're going to start getting the whole pushback of, oh, it's adding too much time. oh, this is going to cost too many resources, and then you find yourself having to defend doing that. finding those ways that you can attach some ethical questions or considerations into processes that already exist.
So, At Salesforce, we are an agile company, and one of the stages at the very beginning is release readiness planning where you are examining your backlog of features and you're trying to figure out, okay, what are we going to prioritize?
What are we going to build for this release?
During that cycle is a great opportunity for us to add in what we call consequence scanning workshops.
These were created by dot everyone. And they're pretty simple.
It's just we spend an hour and a half. You can spend as little as 45 minutes. but you just brainstorm with the team thinking about what are all of the potential unintended consequences if we were to build this feature.
And for all of the negative, because there can be some positive unintended consequences, but for all of the negative ones, now how do we mitigate those?
And then you can find additional barriers that you need to build at the same time that you're building this feature so that it can be used responsibly.
Or you may actually find that the harms are so great and we don't have adequate mitigation strategies So that's the time where we say, maybe we shouldn't build this feature.
And so finding those opportunities that you can add things in to processes that already exist is going to make it a lot easier for you to be successful than just trying to create net new things. that people are gonna perceive as overhead and not necessarily see the value right away.
Sage advice. So Kathy, I feel like we could talk about this for hours, but in the interest of time, I'll leave you with 30 seconds to predict the future.
Where is this headed? And obviously that's a huge question in terms of AI and companies and businesses using AI and let alone Salesforce using AI.
But this new industry, I love the way you likened it to cybersecurity in the 80s.
As more and more the tools get democratized and more and more people and companies start using AI, we think about doing it ethically.
Where do you see this field headed over the next couple of years, five years, whatever lens makes sense to you?
In 2019, the beginning of 2019, KPMG named AI ethicist as one of the top five jobs in that AI companies would need to be successful.
And I think we've only seen an increase in the number of people in roles like mine over the last year alone.
And we're going to see a lot more policies, regulations, and laws being implemented globally.
EU and GDPR and a number of other discussions have come up that are going to have AI specific guidelines.
And so companies are not going to have a choice.
Are they going to think about what responsible creation and implementation of AI looks like? they will have to make this a priority for the company.
And so I think as more and more policies and regulations come forward, we're going to see much more work develop around the area of standards and how do we define this and make what is Right now, something that feels very squishy and fuzzy, much more concrete.
I've no doubt that there are listeners out there right now who are already working on this or working on their pitch deck or they're building their app in their garage, so to speak, and thinking about this stuff and want to get out on the forefront of it,
And so where would you suggest that they go look online when our podcast is over to find out more about the work you're doing and Salesforce is doing or the the kind of forefront of thought in ethical AI more broadly?
You can hop over to einstein.ai and check out a bunch of my blog posts.
And I also post a lot on Twitter at Baxter KB and on LinkedIn.
And And so those are a few spots that can get you started.
Well, Kathy, again, we may have to have you back to see how the field's progressed.
I was going to say a year, but we could probably do this every week.
But thank you so much for making the time and coming on the podcast today.
And all the best to you and all of your colleagues and the companies you work with for ensuring that more AI is having a positive impact than negative.
Awesome. Well, thank you so much for having me.
I appreciate it. Thank you. Thank you.
Thank you.
