Hello, and welcome to the NVIDIA AI Podcast.
I'm your host, Noah Kravitz. For almost as long as humans have been building machines, we've also wondered if machines can have souls.
Can an inanimate object feel? If a supercomputer is smart enough, can we teach it to have a soul?
Fascinating as these ideas are, an arguably more important question might be, can we teach the machines to recognize what we as humans are feeling?
What might happen if an intelligent machine becomes able to pick up on human emotions?
Do we want artificial intelligence to react to our happiness and sadness or anger and fear?
Or are we as humans better off if the machines never learn to recognize human emotions?
Or are these questions moot because whether we realize it or not, many of today's AIs already understand at least some of how we're feeling?
Dr. Alan Cowan is an emotion scientist working on the complex and frankly somewhat mind-bending problem of understanding how AI can capture human expression. to understand everything from signs of mental illness to what moves us to inspiration.
He's the founder of Hume AI, a research company whose mission is to ensure that artificial intelligence is built to serve human goals and emotional well-being.
Dr. Cowan, welcome, and thanks so much for taking the time to join the NVIDIA AI podcast.
Great. Thanks so much for having me. It's a pleasure.
So can we start with you explaining a little bit of the basics of emotion science and maybe getting into your theory that there are 20 universal emotions. which runs counter or perhaps is a breakthrough to some long accepted ideas about how many emotions there are.
And that might set the stage for us to then get into Hume AI and its companion nonprofit, the Hume Initiative, and why you started those organizations.
Totally, yeah. Totally, yeah. So emotion science, I would argue, has changed a lot just over the last two or three years and really jettisoned Some of the traditional ideas that had driven a lot of the research.
So there's this idea that there are six facial expressions, right?
Anger, happiness, sadness, disgust, surprise, and fear.
And it turns out about 90% of facial expressions don't really fall into these categories.
And there's been this explosion in our appreciation of the complexity of human emotion, subtle facial movements that express love or admiration, cringes of empathic pain, vocalizations, laughter, laughter that's tinged with awkwardness, sighs of relief, all of these things that really hadn't been studied by the fields.
And there were A few consequences of that.
One is that when you study just these six facial expressions that are really intense and stereotypes that the field had been focused on, You're also only studying really intense emotional episodes.
And a lot of the mental states that we convey with our facial expressions, with our voice are more subtle. but are recognized across cultures.
They're recognized really robustly by different people.
And they're interwoven with almost everything we do.
So the move to study more subtle expressions has also sort of broken down the barrier a little bit between the traditional study of cognition versus emotion.
And that has really vast implications for society, right?
Because of everything we do, of everything we think about, Our behaviors are influenced by subtle emotional states.
And we're querying search engines and we're talking to digital assistants all the time and You know, we're talking to each other on a video conference or through a podcast.
All of that is tinged with emotion. And all of that data is already being processed by machines, particularly language is already understood by machines.
And I would argue that emotional language is understood in a way, in a sort of an operational or instrumental way by a lot of algorithms.
And so this explosion in sort of the range of the behaviors that emotion science studies also has pretty vast implications for technology.
Sure. And so how did Hume—and so there's Hume AI, which is the company, and the Hume Foundation, right?
And you started them both— concurrently?
Yeah. So when I left Google, I started Hume AI and the Hume Initiative.
QMAI is a for-profit research lab and technology company that's developing the algorithms that have kind of been missing to be able to recognize these subtle behaviors, you know, tinges of awkwardness and laughter, cringes of empathic pain and so forth. laughter tinged with awkward emotion of the things that you said.
And I was listening to all, but that one just stuck in my head.
Like, Right. Like, how could that not have been in the first six?
That's like the most basic. Yeah, I mean...
The funny thing is the first six emotions, the basic emotions, as they were called, only include one positive emotion. happiness, right?
And yet, there's a huge difference between when I laugh and when I go, whoa.
You know, it's a completely different feeling.
Right. And they're both positive. The difference is... probably even more evident in the voice than in the face.
So that might be part of the problem. But yeah, these subtleties have just been overlooked.
So I have a million questions forming about, you know, the different mediums through which machines understand these things and then what happens.
But I don't want to derail. Tell us more about Hume.
So I had, you know, I'm an emotion scientist and a data scientist by training.
I have that rare combination of skills. And so When I started studying emotion, it was in a way that most people hadn't been familiar with, which is in the field, which is through large-scale data-driven statistical studies. of how people express their feelings.
And so pretty quickly, I was asked by tech companies like Google and Facebook to help them meet this need for algorithms and make sense of those behaviors.
So Hume AI and the Hume Initiative really come out of the work that I was doing during my PhD at UC Berkeley, where I was kind of deep in the world of deep learning.
I was able to help drive this transformation and how deep learning was being applied to understand human emotional behavior.
But the fundamental things that we're missing where robust data from around the world with all of the subtleties of emotional behavior that actually occur in everyday life which are more rare in the data that was being used, either because it was gathered in labs or because it was taken from the internet where you see a lot of posed smiles or... kind of seductive expressions, depending on where you look.
Not necessarily representative diversity of emotional behavior.
And the other issue is that there were ethical implications to this work that were being introduced to kind of public forums and debates.
And so people were a little bit reticent to start introducing this technology before there had been a public reckoning or conversation about how it should be used.
And The Hume Initiative is really oriented toward that, latter need, which is developing clear concrete guidelines for how technology that recognizes human emotional behavior should be deployed.
What are the use cases in which we want it to be deployed?
What are the recommendations for those use cases?
Where are the areas where we don't want it to be deployed?
And then as human AI is developing algorithms to recognize human emotional behaviors, we're actually asking, everybody we provide that capability to, to adhere to the human initiatives guidelines.
Got it. And so what are a few of the top line guidelines, recommendations?
So there are some fundamental principles.
There are six principles. that we adhere to.
Some of them are, you know, pretty broad, like beneficence.
Technology should be good for people. In the case of empathic AI, AI that recognizes cues of human emotional behavior, that takes on a more literal meaning in some ways, because you can actually see in the data that you're analyzing whether people are feeling good or bad to some extent.
You can see indicators of well-being that we're not even previously available.
And so the fundamental driving principle is that those indicators should be used to improve people's well-being.
Seems obvious, right? You would think, but... Right, exactly.
There's a lot of things you could imagine using those indicators for that might not be good for people.
They could be used for manipulation, deception, all these use cases that we don't support.
But they can also be used as sort of an output of almost any AI algorithm that processes data with cues to emotional behaviors, right?
Because any AI algorithm that processes that kind of data should know whether it is causing those emotional behaviors to increase improve or for signs of negative well-being to occur more often.
Right. So when you, and, you know, hone in or jump onto what makes the most sense here, but when you were, you know, doing research or or maybe it's more now kind of actively working on the algorithms What kind of data are we talking about?
Because as I'm listening to you and I'm thinking about my own My own social media use, for instance, is a lot more limited than it was say 10, 15 years ago.
And for whatever reasons, I've kind of held on to Twitter as the platform I use most often, right?
And so my interactions on Twitter are literally limited to, you know, 240 characters, whatever the count is.
And, you know, people can post images and little video clips and what, but it's mostly text-based.
So as you're talking, I'm thinking about cues from text.
And that's kind of its own, everybody's, I think, who's ever sent a text is at least tiptoed into the, oh, my tone was lost area.
But there's obviously the web is more sophisticated.
People have video chats, all that kind of stuff.
So- What are the different types of data that you're looking at?
That's a really good question. So text is a big one.
And you mentioned Twitter. Twitter is ripe with emotion.
There's people who have been studying this recently.
It turns out when you're limited in terms of characters, you want to get as much emotion in there as possible as quickly as possible, right?
You mash that all caps button and go for it.
Ironically though, you know, the things that really draw people in that the algorithm is picking up on because the algorithm is using collaborative filtering, right?
It's saying when people are engaged with something, show that to more people.
That's essentially how the algorithms work.
And there's some amount of content understanding too, where it's processing the meaning of the text and predicting how people will respond. respond to it or engage with it based on that.
But if you're just using collaborative filtering, for example, you don't know whether the reason people are engaging with this is because it's making people pissed off or it's because it's making people happier, or because people are clicking on clickbait and then regretting it, or because it's something people are genuinely interested in.
And the algorithms are getting really, really good at... being able to predict what people are engaged with.
And it's almost at the point where they're too good to just focus on the good things.
You know, the algorithms are good enough to start looking at the things that get us... drive our attention for the wrong reasons.
So emotion in text is really important. Yeah.
Right. And so my understanding is I'm sure not 100% accurate.
I don't work at a social media company, for instance. understanding is that a lot of the algorithms now on social, and I think probably on other platforms, are optimized for engagement.
And as you kind of hinted at, I don't want to put words in anybody's mouth, but, you know, engagement...
And conflict kind of go hand in hand right now in terms of, you know, what stickiness, right?
What causes somebody to stay on the site to... keep tweeting back, all of that stuff.
And so if, you know, these... The social media and other platforms are businesses and the businesses are incentivized to drive profit.
And their profit is driven by serving up ads and the ad sales go up when people are engaged longer.
And the algorithm is optimized for engagement and that conflict is the best way to keep engagement going.
It's one thing to train the algorithm to recognize emotion and optimize for well-being.
It's another thing to get the human beings to deploy it that way, right?
So I know Hume AI and Hume Initiative are both new.
But how much of your work is on the technical side and how much of it is is on the human side of talking to the people who are controlling these levers to say, hey, look, let's rethink... incentivization, let's rethink the mission of all this activity.
It's both, but I will say people introduce the argument a lot that Facebook or Google or Snapchat, whatever company you're looking at, they have this incentive to keep people engaged at all costs, right?
And so therefore they're not going to want scared about people's well-being.
I don't think that's true. That hasn't been true in my experience.
I think what's happening is that there are these really powerful things to optimize for, like engagement, that serve as these incredibly powerful heuristics for what people want to see on the platform.
And The negative consequences haven't outweighed necessarily the...
The fact that you can optimize for these things just isn't a good proxy of well-being to optimize for instead.
Once you introduce a proxy of well-being that... can be optimized for instead or looked at at least, looked at in A-B tests as a way of seeing if new updates to the algorithm are improving people's well-being or not.
I don't think it will be that hard to drive adoption.
So we're still working on the best text emotion model.
And, you know, the field is pretty new. It's a nascent People have focused mostly on neutral, positive, negative, neutral in text.
And those kind of sentiments analysis models actually miss so much of what we're expressing and mischaracterize it a lot.
If you're training on positive, negative, neutral and you see sympathy, it's going to be negative, right?
That's a negative sentiment It actually is probably a positive emotion to be occurring.
Or are you a positive surprise? If you're categorizing surprise as negative, it's going to be, you know, to...
You really need the nuance. And if you did optimize for positive sentiment, That could arguably, I mean, people haven't tried it to the extent they probably could, but that could arguably have negative implications because you might just end up seeing a bunch of cat videos or whatever the kind of lowest common denominator is of what drives positive sentiment.
So you really do need more nuanced models to be able to say, I want to drive expressions of a wide range of positive emotions and maintain the complexity and honor that nuance of human communication, the context appropriateness of negative expressions when it's in response to somebody who's expressing... sadness, sympathy in response to sadness or sadness in response to bad news or anger. anger in response to bad news.
You really need context appropriate measures of nuanced emotions to be able to solve this problem.
Right. So are you working... I have a follow-up on that, but I just wanted to ask, are you working also then with video data and... imagery and audio and other forms of media?
So I think the text stuff is complex. And so we started with facial expressions and audio in part because you sort of need natural language understanding You know, the new, the really large models that have just recently been developed as a foundation for understanding emotion in text is emotion, like I said, is everything.
And so I think that the time is actually right for that to revisit that.
And so we are building those models. But the There's a really good foundation for understanding facial expression because of the facial identity recognition work. that's been done.
And so the models are there. The data hasn't really been there.
And so we introduced new data. And that has made a huge difference.
And so we focused on gathering data from around the world with experimental approaches where we could decouple what somebody's experiencing or expressing from who they are.
And that just doesn't exist in everyday life or especially at the internet, right?
Yeah, yeah. But it's really essential for training algorithms that aren't biased.
And we're using these experimental approaches which we're applying At a large scale, we can gather much more nuanced expressions and much more balanced expressions because we can control the tasks and stimuli and choose them carefully to evoke the right. emotion.
In that area, there's a huge amount of work being done on how do you interact with digital assistants?
Can you build digital assistants? Right. that understands your nonverbal behavior. as a means of really getting at what you're saying.
And you can see it in technologies like Siri, Alexa, they just have no ability to understand nonverbal behavior.
You can scream at them in frustration, right?
And they still won't update their behavior basically in any way at all.
They lack understanding of the tune, rhythm, timbre of speech, and that's part of also what makes them what makes it difficult to make them conversationally fluid is so much of what we do in a normal conversation with somebody is we sort of acknowledge what they actually intended using the largely nonverbal prosody or, you know, the tune rhythm and timbre of speech.
And then that provides a basis to say the next thing.
The experience that we have with digital assistants today is that we don't know if they've actually understood us.
And so we're kind of waiting for them to slowly give a response.
And it's a frustrating experience. And we kind of have to process it for a second to see if they've done what we want.
So yeah, making them more conversationally fluent is really important.
And that's important not just for the series and Lexus of the world, but for the digital assistants that are now involved in customer service, telehealth, all kinds of areas.
And not just for them to respond to you appropriately, but also to update the algorithm. if you express frustration in a response, it should back-propagate that and update the algorithm so that it behaves differently in the future.
Not either for you or, you know, personally or generally speaking for users.
And... I started with facial expression to vocal expression, but facial expression is important too, because people are now building more immersive digital assistants that have, you know, virtual bodies.
And these are being deployed in areas like telehealth, augmented reality, where You want the process to be more immersive.
You want the person to be more comfortable expressing themselves and potentially explaining what their issues are so that they can be addressed.
We're really, yeah. And then taking that even one step further, you have quote-unquote digital assistants, but they're actually robots, digital assistants with physical bodies.
And that's... And those are also being deployed, for example, in therapy settings where you have social robots that people are taking home and using them for therapies or interventions for patients. for developmental disorders like autism or loneliness and so forth.
So a little later in the conversation, I want to ask you about some of the clients that Humai is working with because You're just talking about all these areas of life, these different, you know, sort of product offerings, to put it that way, where the science, you know... is vital.
It's kind of fascinating. But before that, are there particular emotions that you found to be easier or harder? to train the AIs, the algorithms to understand.
Yeah, I mean, the ones that are hard to train the algorithms to understand are the ones that are hard to find in the data, right?
And so actually... We have a whole different set of emotions that we can understand because of our data, because of the experimental manipulation that we use And we're really good at understanding people expressing response to videos because we have a huge data set of people's reactions to videos around the world.
And, you know, cringes of empathic pain, kind of subtle facial expressions of adoration, love.
And a lot of those are subtle expressions.
We're really good at expressions that people can form intentionally, can kind of pose.
And then, you know, the more difficult ones are always going to be the ones that... are difficult to pose and hard to evoke.
It may or may not be interesting. For example, Fear, like actually genuine fear.
You probably don't want to build something that really evokes that.
I mean, you can have people who like to watch horror movies, watch their scenes and they'll express fear.
But they won't generally form like a blood curdling scream or anything.
So you use pose data for that. Right, right.
I don't think of these as emotions per se, so the answer may well just be no.
But do you delve into trying to understand, you know, irony and sarcasm and things like that?
Yeah, I mean, that's a really complex area because, you know, to some extent, irony actually has a characteristic emotional prosody to it.
We kind of hint that we're joking. We say it kind of lightly.
It's a fake laugh usually or fake amusement sounds.
There's also... kind of a contemptuous laugh that you can recognize.
You see it in people's facial expressions without genuinely smiling, where they're kind of doing a fake smile.
And so there's characteristics of that, but sometimes you just can't pick it up because the person's not using their expressions at all.
It's just complete... you know, flat irony.
In that context, irony is really a language thing and it's really something you can pick up on with the language model.
Right. And, You know, what we're trying to pick up on are sort of the mental states, the emotions, the things that people express nonverbally. that you don't find in language or that you find in language, but usually paired with nonverbal behavior that is multimodal.
And there, you know, we'll find correlations between text on Twitter and actual emotional content. you know, behavior, self-reported emotion, people's wellbeing.
Then there will be correlations between what people perceive to be emotion and text that actually don't correspond to people's real emotions in everyday life.
And that could be driven by things like irony.
We're speaking with Dr. Alan Cowan. Alan is an emotion scientist who has founded a research company called Hume AI and a companion nonprofit, the Hume Initiative. that are working on teaching algorithms, machine intelligence to understand human emotions and kind of more broadly to better align what we're doing with AI and algorithms with human well-being and sort of...
These are my words maybe, but sort of incentivize the system to prioritize human well-being.
Alan, let's jump back a little bit because I'm curious how you got into these sort of – complementary, in your case, fields of emotion science and data science.
Were you interested? Were you an emotionally sensitive kid?
You know, were you, that's something that was kind of always on your mind.
I joke, right? Can you tell? Or is my laugh sincere?
No, but without asking you to reveal too much about your own – personal information.
How did you find yourself being interested in working on this stuff?
Yeah, I mean, that's a really good question.
I actually came from a neuroscience background.
And when I got into neuroscience... My interests were kind of philosophical.
I was interested in consciousness and the philosophy of mind and That's a really difficult thing to study.
I can only imagine, yeah. I got interested in emotion kind of later in college, thinking about, you know, what are the areas where you could apply data science to basically philosophy to derive data conclusions that are almost philosophically profound.
And I feel that emotion drives everything.
I mean, this goes back to, and I took a lot of philosophy in college, this goes back to David Hume, who is not the inspiration behind this, which is focused on human-centered technology, but a nice cherry on top.
It's a nice play, yeah. And David Hume argued that reason is not only to be the slave of the passions.
And what he meant by that is passions meaning emotions.
Reason meaning everything we think about, everything that drives us to make decisions.
And his point was that at the end of the day, what matters, if your kid asks you why over and over again, why you go to work, why you need to feed your family, It's like, why does it make you sad when this thing that you don't want to happen happens?
And why does it make you happy to have this thing that you're using the money on that you're going to work to make.
And it really all comes down to that, right?
And we don't have a good answer. And honestly, like we have, a lot of evolutionary explanations that are sort of they're just those stories right but when it comes to actual behavior and the actual expression of emotion and the actual feeling of emotion That's where there's a huge amount of scientific debate.
And to me, there just wasn't a lot of data And I, you know, in college, I majored in applied math as well as cognitive science.
And originally, we wanted to use sort of data-driven methods methods and machine learning in the field of neuroscience, which I did for a while.
But there were a lot of people doing that.
And then I just saw this need for data in emotion science as I sort of edged over in that direction.
And it was just huge. I mean, people were focusing on literally ratings of, you know, maybe 10 or 20 images by, you know, 20 or 30 people.
Like those were the kind of other studies.
Yeah. were decided. And there wasn't really a sense of statistical model comparison there.
And there certainly wasn't a sense of extracting expressions or what they under the structural underpinning of expression. is from the data using machine learning because nothing really on that order existed, you know, of data sets.
And on the other hand, you had people in the field of affective computing who weren't, interested in classifying emotional behavior in data, but had basically taken the model as people had used in emotion science or developed in the theoretical models developed in the 60s and 70s of the six basic emotions, surveillance and arousal,
And we're just predicting those, just basically taking those features and trying to predict them without any thought given to how can we extract what facial expressions mean from the data. just assuming a meaning in advance.
And they were usually gathering data in one culture.
It's always perceptual judgments, which Or influenced by context and demographics and so forth.
Yeah. So I just saw this huge need to... apply data science to emotion.
And I was interested in both. It just was a no-brainer.
It started with a collaboration with Dr. Teltner, who became my advisor at UC Berkeley.
He was a pioneer. He is a pioneer in moving the field beyond those traditional practices. emotion theories and had done a lot of smaller scale studies of things like embarrassment and gratitude. finding them to be extremely important in determining the trajectory of our relationships.
But that work was still not really data-driven.
And so there was this obvious and incredible, you know, complementarity between what he wanted to do and what I wanted to do and, the methods that we wanted to use.
And so just immediately just started gathering huge data sets and broadly they supported a lot of what Docker had theorized just in terms of the complexity of emotion.
We also found that things that People had assumed were discrete facial expressions of anger and disgust.
Actually, there were just these beautiful gradients between them.
And people would take an expression that is between anger and disgust and reliably label it as being both of those things like an individual could see both.
A lot of the early work that I did was taking large-scale data and just characterizing the distribution of expressions. and what people inferred from them.
And then I moved to gathering self-report data, gathering huge bigger data sets of people, recordings of people with audio and video and training machine learning models.
And it just, it blew up from there. Yeah, no, that's great.
And this is kind of a question both for, you know, thinking about listening to you describe that period and the research you were doing, but then also thinking about now with you.
Are there specific cultures or demographics that, I don't know, stand out or notable because there are harder to get data on or the findings are a little bit different than, you know, what expressions mean in other parts of the world or what have you?
Absolutely. I mean, there'd been such limited data in the fields on cultural comparison and people... which is not for a lack of trying.
People have been trying to study emotion across cultures for many years, right?
But just the limitations on the methods of how you gather the data required them to end up focusing on small samples.
And then you can only train really limited statistical models, which allow you to confirm basically theoretical predictions, but you can't derive new understandings of what expressions mean.
So that's what we do. We've gathered data now from U.S., South Africa, Venezuela, Ethiopia, China, India, and other countries.
And what we're able to do is take a deep neural network and use it to extract what people think their own expressions mean.
But to do it in a way that's completely culture-specific.
So the model is not assuming that what people...
The way that people use the word amusement in the U.S. is the same as the way that people use the most direct translation of that in China.
It's just predicting them both separately.
And then we can see whether the model is converging and extracting the same facial expression and finding that it means the same thing in the two cultures.
And when we do that, we find, well, first of all, there's definitely more cultural universality than a lot of people would predict because there's a confusion in the field of When you're looking at small samples between the signal and the noise.
Okay. Right, right. Yeah. And so, you know, the, the, the correlations or whatever measures that you're extracting are biased toward null or no factor zero. because of the noise and you know right right what people should be have done is like looked at within the culture what's the correlation between people what's the same metric if you take two groups within the culture versus two groups of cross cultures.
And it's not as different as you might think.
So about 70 to 80% overall of our emotional behavior is preserved across cultures in terms of its meaning, in terms of We also did a study and we looked at millions and millions of videos that were naturalistic and just looked at which facial expressions were associated with which contexts in these videos and different cultures.
There we found that the relationship between expressions and context was 70% preserved across cultures you saw.
Expressions of triumph in sporting events, pain with weight training, interest in classrooms and so forth.
All the things that you might expect. Then there are some culture-specific expressions.
And the cultures that are probably most difficult to study generally are the remote culture, kind of small-scale cultures.
We consider them remote. they don't consider themselves remote.
They're remote from us. Difficult to get to.
And we haven't really made more headway in studying those, but the data is so ambiguous with those cultures.
But then the data is There's a lot of data on the U.S., China, Japan that suggests a lot of cultural dissimilarity, which is not actually true. found in our data, which really is to suggest that studies have overestimated the degree of cultural variation.
There's less data in Africa in general to And that's actually where we find the most cultural variation. data from Ethiopia is the most unique in terms of... Part of that might have to do with how people... and for meaning from expressions, and part of it might be due to the differences in language.
So M-Hark is probably the most distantly...
Well, I wouldn't say it's necessarily more distantly related from English versus Chinese or Mandarin, but it's very different.
And they're just not direct translations for a lot of the emotion terms that we traditionally rely on.
But even... Even when there are sometimes, it seems that certain expressions, maybe one in ten, have a completely different meaning in Ethiopia, which is very interesting.
Yeah. So with the few minutes we've got left here, I want to shift back to the present and what you're doing with Hume AI and the Hume Initiative.
You guys have a rock star lineup of advisors and other folks lending their efforts to what you're doing, partnerships.
There was some work with some Pixar movies, a bunch of things.
So, you know, in the last couple minutes we've got here, you've got a developer platform.
That is going online, I believe, next month.
So tell us about some of that stuff. Yeah, so we have a few promising pilots at this point.
We just launched the team initiatives guidelines and now we are ready to make our technology more widely available.
You know, part of what we were waiting for is to finish up some of the algorithms we were working on, but part of it is also we want the guidelines to be public so that we can ask people to comply with the guidelines and how they use our data sets and models. we're launching a developer platform.
It's going to provide free access to our APIs to researchers and developers around the world who are We'll be using them on scientific work and on applications that are in keeping with our ethics guidelines, like digital assistants that are better optimized for people's well-being, like social robots used in therapy settings.
You know, like metaverse applications that are optimized for people to feel the emotions they actually want, you know, than entertainment applications, for example.
Right. And that will be free up to a point for developers to try out and start building with.
So we're really excited about that. We're also providing visualization tools, which will enable people to develop the kinds of visualizations that we develop, which are... you know, not trivial.
And when you're dealing with behaviors that are kind of...
We're audio only, but we've got our cameras on to see each other while we're talking.
And I think I picked up on a facial expression when you said not trivial there.
So I... I feel like I've learned something from this conversation.
I'm proud of myself. I apologize. Go ahead.
I'm glad you're paying attention. That's great.
Yes, that was probably an expression of awe at what we're doing.
Right, right, yes. When you're dealing with data that's so complex, it's 50-dimensional, where you have nuances and...
So tinges of awkwardness and laughter, but you also have to balance that against all the other things that are in the signal.
It's not easy to just look at a spreadsheet and understand what's going on.
We're providing these interactive maps that will be generated for people's data so they can actually see in as few dimensions as possible using embedding techniques, how their data varies in this space.
They could look at, for example, a telehealth session and be able to see this is a point where the patient is Maybe not crying, but expressing a little bit of sadness.
Let's see what was going on there. Right.
Or they can look at a digital assistant query and be able to say, well, these seem to be hovering around frustration.
Let's see if these are the ones that we should be paying attention to in our data. and developing ways to identify.
And you can identify them if they appear in that visualization.
So, yeah, we're providing those tools. We have a waitlist for our platform currently that anybody can sign up for.
And we will be unrolling it over the next... month or two.
Excellent. And so before we hit record, I was saying that we usually like to end these conversations on a a bit of a forward-looking note and I'll ask some variation of, you know, where do you see your industry headed or how do you see the work you're doing shaping the industry?
I have a note on my sort of pre-production, if you want to call it that, notes that says, Why it's imperative we give AI a moral compass.
And so... I want to ask you kind of to wrap up here on something along those lines.
Maybe a better way to ask it is, you know, In an ideal world, what do you see as the best impact the work that you're doing could have on On the technology, of course, but on humanity, you know, as we go forward over the next whatever the appropriate period of time is.
Yeah. Um, I mean, that is really what we're thinking about and which adds me to this work, honestly.
Um, We're most excited about developing models of wellbeing.
And we think models of wellbeing are what AI needs to have a moral compass.
Morality kind of, without assuming utilitarianism, I think generally morality is about what people are feeling. and about their wellbeing and being able to work backwards and decide this decision is good for this person or this decision is not good for this person.
And algorithms are making decisions all the time on our behalf.
And what those decisions are ultimately optimized for is not right now well-being.
And that's been okay, actually. I think AI has...
It's been tremendous. You know, I think it's obvious that, you know, search and suggestion algorithms and So many of the other kinds of AI tools that we interact with every day are improving our lives.
But they've been using heuristics. I mean, all of those tools have been based on the heuristic that people's engagement is a reflection of whether this is good for them.
They've been optimized for different kinds of engagement, whether it's clicking on the earliest result or, or spending more time watching the video and so forth.
And that's been a useful proxy, but Let's go to 100 years from now just to be as dystopian as possible.
Sure. as per engagement 100 years from now, right?
There are obvious ways that robots that are hyper-intelligent could get us to watch a video that are not good for our well-being.
Yes. like, you know, holding our eyes open and placing our, but you know, and that's not happening yet, obviously, but the, but we're seeing ways in which, um, that objective is being pursued by algorithms that may not be coherent with our well-being.
You know, the idea that It could just be surfacing more clickbait or surfacing things that keep us engaged for longer periods of time. that cause things to be kind of habit forming.
We have kids now spending, you know, In the U.S., six hours a day on social media, almost on average, I think, is the number, which is...
Kind of hard to believe. It's astounding.
But that is what the algorithm is optimized to do is to keep them engaged, right?
Right. And if it turned out that an algorithm optimized for well-being also had people spending six hours on social media, maybe we'd rethink that and be like, well, it's actually not necessarily time that's not well spent. because it actually is optimized for their wellbeing.
The fact is it's not currently optimized for their wellbeing.
And to optimize people's wellbeing, you need algorithms to understand emotional behavior because otherwise, What are you left with?
You have the ability to ask people how they're feeling, self-report, and that gets you some of the way, but you're not going to be able to ask people what they're feeling, you know, all the time.
And that data is never going to be the predominant data used to train up. rhythms because there's just not enough of it.
And even if you could, self-report is one dimensional. and it's biased, and you kind of want to pair it with other... I'm feeling fine.
Keep the videos coming. Right, exactly. Even if you had that data, it's just not clear how the algorithm would learn to to value the different emotions that we feel when we're feeling not fine and respond to them appropriately?
You know, like how do you, How do you respond to tragedy?
You don't necessarily have to respond to tragedy by laughing.
The algorithm shouldn't be there to distract us.
Pull us away from confronting the conflicts in our lives.
It's essential to have a multifaceted view of well-being if we're going to optimize algorithms. moral well-being, give them a moral compass.
Part of that is self-report. Part of that is nonverbal behavior.
Definitely. And then other objective proxies like health, are people sleeping well, whatever data you have access to. and sometimes some of those outputs are going to be very difficult to optimize for in the short term.
Some will be easier. And the algorithm should be given a range of things that it can try to optimize for, knowing that there are many ways of being well.
And there are trade-offs between different indicators of well-being.
And I think that is absolutely the key. And hopefully within a few years, we'll start to have models that actually, in a contextual context, sensitive way, understanding the context in which expressions, emotional cues are observed are able to tell you, is this a good cue or a bad cue?
And then while keeping the cues diverse, while preserving the diversity of emotion that we all prefer, being able to optimize AI models for that, being able to include that as a metric in any A-B test?
Is this release of an algorithm? making people angry or less angry?
Is it making people happier? Is it going to do a lot?
These are things that companies should probably know.
Yeah. If the data is available, it's not yet that we can make that data available.
So I think that's That's what motivates us above all.
I want to bring up the privacy implications a little bit because that goes to people's minds, right?
People... You say like, oh, I'm collecting all this data on people's emotional behaviors.
I think it's really important to note that all of the applications that we support are only analyzing data on your device, right?
Or they're doing it if there's a reason to analyze it on the cloud, They're doing it in a way that preserves privacy, that doesn't give humans access to your data who shouldn't have access.
And this should be familiar to people who use search engines who Use any photo algorithm.
You don't want, even though every time you take a photo, every time you search the internet, your data is being analyzed by algorithms.
And that's part of the point. It's being done in ways that you should be aware of, hopefully.
It's being done in ways that improve your well-being, hopefully.
Not necessarily. Not necessarily. Arguable in current reality, but yes. our applications, we actually do require that you pursue things that Andrew did as well.
And so in that sense, we actually have a further commitment to privacy that people don't normally have, which is that if your data is going to be analyzed, you should know about how it's being used and it should be something that is one of our approved use cases, which means that it's something that's improving people's well-being.
Excellent. all of that's really important.
And, you know, what people get confused about with privacy is they think, well, lower algorithms analyzing my data is less private.
And that's not, that's not how it should be.
It's, it's, it's, Who has access to your data?
Who has access to the outputs of the algorithms that are analyzing your data? where we need to be really careful and limit access to the extent possible, particularly to sensitive data like emotional behavior.
Well, I, for one, am glad you're working on making that data available.
It's important, and it's basic human stuff that— clearly become so complex when you're when you're figuring out everything that goes into it and how to interpret it and what to do with that data.
So for listeners who want to find out more, Hume.ai has a website, Hume.ai.
I heard you have a podcast as well. So where are some places online where people can learn more about the work that that you're doing, that the foundation is doing, and then, you know, more generally about the whole field.
Absolutely, yes. So if you're a developer and you're interested in using algorithms to build applications that are designs to improve people's well-being.
You can go to our website, human.ai, to sign up for our playlist for our platform, and you can find more information on our algorithms and the data that they're trained on.
If you're a researcher interested in any of that stuff, please.
Also, we... plan to make our solutions available for free to scientists forever.
So we think there's just a huge need for that.
So yes, please sign up for our waitlist if you have data on social interaction, development, health, any of those things.
And it would be useful to code people's nonverbal behaviors for that.
And you can find out more about our scientific approaches, all of our publications on our website as well at hume.ai. in the science section.
If you're interested in our ethics guidelines, and you should be if you're a developer, or a scientist, or an ethicist, any number of things, please check out thehumanitiative.org where we have an ongoing list of ethics guidelines, kind of the first concrete guidelines for apathic AI that lists all the use cases and the recommendations and the unsupported use cases.
And we're very open to feedback on that.
We'd love for more people to be involved.
It's not set in stone by any means. And we want it to be more of a discussion.
I'm going for it. So please check out the human initiative.org if you're interested in that.
Excellent. And is the podcast, The Feelings Lab, is that still going on?
Yes. If you go on hum.ai, and you go to our Discovery Hub, you will find all of our podcast episodes.
We're on season two. Season one was about emotions in everyday life and society.
Um, season two is more specifically focused on technology.
We have, um, guests, many of being founders and CEOs of, um, Companies that are pursuing empathic technology, really, really interesting.
Music, social robots, digital assistants. metaverse applications all kinds of stuff so please feel free to check that out you can also find us on iTunes and SoundCloud and the like and on YouTube The Feelings Lab.
Excellent. Well, Dr. Callan, Alan, this has been great.
I've certainly enjoyed it. Hope you have.
Hope the listeners will. But more importantly—
Best of luck with everything you're doing.
It's, you know, timely, it's important. And, you know, I just wish you all the best with it.
Thanks so much. This has been a blast. Thank you.
Thank you.
