Thank you. Hello, and welcome to the NVIDIA AI Podcast.
I'm your host, Noah Kravitz. Many of the biggest consumer-facing uses of AI to date are coming in the realm of personalizations.
From online shopping to content recommendations, AI and machine learning are helping businesses learn more about their users and deliver more personalized experiences.
My guest today heads up technology at a company who's doing just that, but in a really unique sort of way.
Brendan Cassidy is CTO and chief scientist at Super Hi-Fi, a Los Angeles-based music company that's using AI to transform the space in between the songs.
Super Hi-Fi's press materials say the company is leveraging deep learning to craft experiences with the same depth and dexterity as the world's finest DJs and producers.
And Brendan is here to talk about what that means, how his company is doing it, and maybe if we have time, tell us a little bit about his side gig creating a line of digital modular synthesizers.
Super Hi-Fi is also a member of Nvidia's Inception program, so maybe we'll talk about that too.
Brendan, we've got a lot to get to, so let's get to it.
Thanks for taking the time to come on the NVIDIA AI podcast.
Absolutely. My pleasure. Really glad to be here, Noah.
So why don't you tell us, I read from the press materials, but we've got you so you can explain it.
What does Super Hi-Fi do? We're exploring a world of AI application that we like to think of as computational music presentation.
We're focusing on the listening experience, what it's like to hear songs play, hear content interspersed with those songs, hear a newscast put together in audio. do it and you sort of described it a bit in the intro there by having trained an AI to understand music and content at a very, very deep level and to use that knowledge and understanding to create unique experiences that won't frankly borrow a lot from the techniques of great radio and great radio audio production. to present to listeners a thing that sounds like it's carefully crafted for them.
As opposed to what you would hear on many music services today, which is a song. a gap of silence and then the next song plays.
We're not trying to behave like an electronic dance DJ that will blend songs into each other and try to make 30 minutes of music sound like it's one continually morphing song.
We're really trying to artfully take the music that is either programmed for the listener or the listener has suggested that they'd like to play, and present it in the best way possible that has energy and personality and branding. to create a more meaningful and perhaps even a longer listening experience for them.
So I know that you work with a range of clients.
There's some some big names that folks would recognize.
And I bring that up because it might set the stage for you to Walk through an example of what your service does for both the client but also the end user on a particular context.
And you don't have to pick from these, but iHeartRadio jumped out.
Sonos jumped out, Peloton, the fitness company, jumped out.
So how would you work with a company like this or pick another one and then what would it be like for the end user? we're really lucky to work with such great brands and partners that really get it and understand the value of what a transformative listening experience can do for their customers.
And we'll start maybe with talk about Sonos since it's most relevant.
Just last week, Sonos Radio HD, which is a lossless interactive version of the radio service that ships with with every Sonos device.
What we allow that experience to be, is it's not just music.
And the best example I can give is they have a station that was curated by Brittany Howard. of the Alabama shakes called the Encyclopedia of Britney.
And she went and picked the songs and put them together.
She also went and recorded a whole bunch of audio talking about the songs, why they're important to her, what they mean to her, aspects of the musical composition or the production so instead of just hearing you know uh David Axelrod piece play, you're going to hear her talk about why that song means something to her and how she's connected to it and why she selected it. and then that music's going to play in context.
Not only is her personality infectious, but the listening experience that this creates is something that is deeply personal and deeply meaningful.
It's our AI that's enabling it. So our AI has a collection of all of those audio recordings that she's made.
It has all of the music. And it's bringing those moments together and overlaying her talking on tops of the fronts of the songs and making it sound like she's live. on the radio broadcasting, but she's doing it interactively just to you.
You can skip, you can make your way through that playlist.
And that's really, you know, from a content perspective, it's what our technology enables is that broadcast experience.
It's like you have DJ that's putting together this tapestry of music and content and overlapping it and overlaying it but I
It's not done in a broadcast way, so it's not a DJ in a control booth sending out a radio signal that hundreds of thousands or millions of people listen to.
It's the AI crafting it from the raw materials just for you, the listener.
And so depending on how I interact, in this case with the Sonos app, and kind of navigate through.
I haven't used the Sonos HD radio, but kind of navigate through the playlist, whether by picking songs or by thumbs up, thumbs down type stuff, whatever it is.
My experience and the sequence that I get these things in and the sequence that I get, you know, not just the music, but Britney's commentary, like you were saying.
That theoretically would be different from your experience doing it on your Sonos app at home or wherever you are. because your AI is able to kind of piece together oh, Noah is listening to this song now, and here's the Britney clip, and here's how we're going to craft it.
Our AI is absolutely capable of doing that.
The Sonos experience is one that is centrally programmed by their music curators.
Okay. and is not interactive in that thumbs up, thumbs down sense, but you still have the ability to skip forward and have things playing for you at your own time.
But the fundamental design of what we've built is for these very large scale interactive a user could be playing anything that they want to be playing, and our AI can make meaning of that experience at scale on a one-to-one basis.
If you think about it at its broadest level, that's what we're really trying to do is take all the great lessons that you We've learned from radio and radio production and make them available for our customers. music services, fitness companies, I mean, all the types of companies that we work with, make them available to do it on whatever basis they'd like.
They could do a broadcast basis if they'd like, or they could do it one-to-one for individual customers.
So without asking you to reveal any secret sauce, as it were, How do you do it?
How do you make meaning? And what sorts of things are you thinking about and working on when you're trying to extract meaning from all of this audio content? to enable your clients to craft the kinds of things you were just talking about.
The trick for us was to learn as much about the music as humanly or machinely possible.
That is a constant quest for us in refining both the digital signal processing and the machine learning that we do, to understand what's going on in that piece of music.
We strive to have our technology behave in, and we talk about this a lot internally in an artful manner, in a manner that, um, takes into consideration what's happening in the music and given a pair of songs or given a voice reading a newscast and a music bed that would go underneath it, uh, to understand exactly where to place, how to, uh, adjust volumes, what content to go along with it, to make all of those decisions.
And what we've learned is too much information is not enough.
We just want to know as much as humanly possible.
And what's amazing today, we've My business partner and I, we've been in the music tech space for over 20 years, and we've always wanted something like this to happen.
And what's really amazing about today, this moment in time, is that A few factors have all come together to enable us to explore this world and to build this set of capabilities.
It's the availability of on-demand cloud resources that you can turn up when you need them and turn down when you need them, like upon Amazon.
It's the availability of acceleration using GPUs to have your training and your processing run as quickly as humanly possible.
When we're looking at each of our partners, we look at their music catalog, we learn about their music, and we learned pretty quickly that every partner's version of a song might be slightly different based on when they received it from the record label and what format they got it in.
And so we needed to know, as much as humanly possible about that specific version.
When you think about well, there's 55 million tracks and any country's music library and maybe 85 million total around the world. that's a lot of audio data that needs to get processed by partner.
So having compute resources that we can spin up having GPUs that make all that processing run a lot faster.
Frankly, being able to leverage all that's been made available in the open source community, and especially, and this is really a nerdy thing, but to have Docker and especially the Nvidia Docker that we can use, so that all of our developers can easily get these complicated environments that use lots and lots of software to allow us to analyze and process and slice and dice these audio files, it really opened up the door to learn as much as we can, and then train our DJ model or DJ models because we have many of them to start making that experience come to life with all the sort of myriad of possibilities that our various partners would like to do with it.
There are perhaps more fruitful questions to ask in a second, but this one's stuck in my head, so I have to ask it.
Do you name your DJ models? We don't. No, it would be fun to do.
We have basically a... these parameters that you can adjust.
Some of them auto adjust based on the content that's around and some of them you can tweak and manipulate yourself.
One of them that is a part of our latest Magic Stitch 3, is an understanding of aggression and an understanding of musicality.
And by adjusting those models, the DJ will behave differently.
And frankly, partners can start crafting their own particular sound and feel for either their entire service or certain content areas within their service by adjusting these simple values and getting a different feel from how the music connects together and how content overlays on top of it or in between it in really interesting ways.
So you mentioned this earlier and then you just mentioned Magic Stitch.
You guys dropped a press release as we're recording this in mid-November.
The press release came out just last week, a few days ago. announcing Conductor.
And so I'll ask you about that in a second.
And Magic Stitch, you mentioned a few other things.
But one of the things that I wanted to come back to was I think I was focused on in prep for this conversation and even in the beginning of this conversation focused on the whole aspect of the content itself and figuring out, you know, depending on the order of my particular playlist, which of, to go back to your example, which of Brittany Howard's pre-recorded audio clips to drop in or overlay here and there.
But it's not just the content. Your system also handles engineering tasks, like volume levels and fading and all that sort of thing.
Mm-hmm. it does everything a dj would have to do to create a well-presented listening experience and What we've learned in working on this over the last three years is every little detail matters and the milliseconds even matter in terms of like when two songs are being paired next to each other and they have different rhythmic relationships, making sure that that happens at just the right moment based on the feel and the experience that you want the listener to have.
It's more precise than I ever thought it was going to be when we set out to do this.
But that also excites me because I put myself through undergraduate and graduate school.
I was a radio DJ on air. And I always believed there was an art to what great radio DJs do in terms of. not only the show and the content and the experience they provide the listener, but also the technical and when things are happening and how they happen.
And, you know, there's a, anyone who's done any college DJing or professional DJing, you know the moment where you talk over the front of a song and and you stop talking and the music hits and the singing hits in that perfect way and you're jumping up and down and fist pumping in the air because it just felt so damn good.
That's what we're trying to make happen with the AI.
It goes beyond just, is there a space for me to do this?
It does make its way into a little bit that generative world of AI.
It's trying to create a form of art. in the listening experience.
Well said. I had flashbacks to my own college DJ experience.
It was less the fist pumping because I nailed it and more the freaking out because where I expected to be there to be music, there was silence and I had to figure out what to do.
So. From the other end of the spectrum, I can appreciate it as well.
That's when the red phone next to your console starts ringing.
Yeah, exactly. It's your program director who... for some reason is always listening to the radio station when you're on the air and they would like to talk to you about your employment status.
Yeah, I call him Producer Brian these days.
So tell us a little bit, the press release that just came out and Conductor, Magic Stitch 3 and the Dynamic Content Curator.
What is Conductor? Conductor is the name that we've given for the suite of everything that happens when someone would like to have a listening moment created. and they call to Super Hi-Fi and say, you know, I want a moment like this, or I have these two songs.
Can you tell me what to do with them? And conductor orchestrates I'm using all of these musical terms, it's a little campy, but basically it then uses our dynamic content curator to decide do I need to interject any content.
Do I have Brittany Howard talking about this particular upcoming song and it's an appropriate time for this based on this user's listening history and and you know, where they are in their day.
And then it uses magic stitch to glue all of those pieces together and to produce and to set the volumes like we spoke about a moment ago. and then deliver those instructions, we call them a recipe, back down to the playback client to assemble and create that moment for the listener. but it acts like a conductor of an orchestra.
It's pointing at the sections, it's waving its arms to make sure everyone's moving in time together.
And it, uh, you know, essentially creates the presentation that the listener is given.
I'm all for the campy names personally. I find them campy.
So I'm wondering, as you were talking about this, and I'm thinking a little bit about the the new release and the different types of clients that you work with.
How long has Super Hi-Fi been around? In business?
For three years. Three years. A little over three.
So over the past year, six to nine months in particular with COVID and people's lives changing so much, Have you seen your usage stats and or kind of interest from outside clients change?
Yes. Maybe I'll step back. There's always a question when you create a new startup business, and it goes back to that Four Steps to the Epiphany Stephen Gary Blank book, where you're wondering, am I creating a new Type of thing.
Am I entering an existing market or am I segmenting an existing market?
I'm paraphrasing and probably getting it a little bit wrong, but we kind of thought we were entering an existing market or maybe even segmenting it, right, for people who want to create great listening experiences.
I think we quickly learned we were creating a new market, a market where someone cares about the listening stage, about what happens when users are listening to music.
There have been experiments in delivering higher quality audio files, but no one had really tackled what happens in between and on top of in any real meaningful way.
I mean, there's been experiments with crossfade and some people have stuck things in between songs, but not in the artful way that a DJ would.
And so we realized that It wasn't just like taking our capabilities to market and showing them and people go, yeah, I need that.
And understanding at first we had to do the tour and let people know what it is that we do and why what it is that we do is important and valuable and interesting. really fortunate.
Our first major partner iHeart Radio, they got it because they're in the world of radio.
Their bread and butter, as it were, is great radio. audio presentation and they have some of the most talented DJs and and program directors in the world working there.
And the light bulb went off for them and we had the wonderful, opportunity to work with them on our very earliest builds of integrating of doing uh you know a b testing in their products to understand the impact that the listening had.
But to answer your question, I think we started thinking that we had something that everyone was going to understand and want, and we had to realize, oh, we had to let them know why they need to do it and explain it.
And the thing that's changed over these years is now we have companies coming to us asking for what we do.
Right. And that was when the day that started happening.
I just I don't think I could start stop smiling for over a month because People finally were getting it.
They'd spent their time and energy on just getting the availability of these experiences on all the platforms, of getting their platforms to be scaling enough to handle the demand to making their apps really attractive and pretty and all of their reporting and all of those initial problems And now we're realizing, especially for music services, like, oh, hey, we play exactly the same music that the record labels gave to us. maybe slightly different audio codecs, but it's available on pretty much every platform and it all costs about the same.
What are we going to do that's different?
What are we going to do that when someone's listening to our product, they know, that's that product and And I'm having an emotional connection to it.
And I'm going to select that product because of the way it makes me feel, not just because of what it does for me.
And I think that's been the change. And I will say, I think COVID's been a part of it too in a weird way because we all are so isolated.
And I'll tell you, if you listen to the exact same songs from that Brittany Howard radio station in the exact same order, on a traditional music service versus that radio station on Sonos, the Encyclopedia of Brittany, you will have a radically different from a humanistic standpoint.
Kind of to that point, I have found, I'm a big music listener and I have, you know, been subscribing to streaming services since they were kind of viable because just the idea of being able to listen to anything I want virtually is amazing.
That being said, I've found during COVID that when I've gone out on grocery shopping or whatever other things I'm going out for.
I'm listening to one of the local college radio stations for that humanistic thing.
And it's as much about hearing the DJ in between as it is the music.
And so you're preaching to the choir. It makes all kinds of sense to me.
Our guest today is Brendan Cassidy. Brendan is the Chief Technology Officer and Chief Scientist at SuperHiFi. a Los Angeles-based music tech company that is using AI to transform the listening experience and sort of the audio consumption experience, if you will, for for listeners on all sorts of platforms all around the world.
Brennan, let's shift gears for a second.
You alluded to how you put yourself through school. as a DJ and I know that you studied music technology and you've been doing stuff in music tech for a long time.
Can you give us a little insight into your own personal journey, your background, how you got into music and working with tech and then You talked a little bit about Super Hi-Fi's origins, but maybe kind of lead up to that.
Yeah, sure. So I've been interested in music and technology for as long as I can remember.
My mom surfaced a photo the other day of me maybe 10, 11, I've got on a set of headphones and I'm holding a TRS-80 programming book in my hands.
Making a funny face for the camera. And I didn't realize how predictive that photo was for what the rest of my life is going to look like.
Right. I had a great opportunity when I was studying for my MFA at the California Institute of the Arts. up in Santa Clarita slash Valencia to really dig into the intersection of computers and sound.
I used like, you know, music notation software and sequencers and those types of products prior, but this was actually hands-on and programming.
Sorry to interrupt, not to ask you your age, but what timeframe are we talking about?
This was middle 90s. Okay. And I had a great opportunity to study under a gentleman named Tom Erb.
He was the creator at the time of sound hack, which is a, you know, a tool for, you know, creative musicians that wanted to mangle and play with sound.
And he was patient and generous and, uh, You know, even though he was programming on the Mac and at the time I was programming on a PC, I actually created some digital signal processing shareware software. learned all about reading audio files, processing audio files, and trying to create tools and music using code.
That was a really fundamental shift for how I used computers and what I used computers to do in terms of music.
After that, I did make my way into the internet, worked for a little while at Disney.com when things were exploding in the late 90s. and made my way shortly thereafter to work for Richard Branson at his division Virgin Digital, and we created some early music experiences, Radio Free Virgin, one of the early Internet radio services, and we created the Virgin Digital streaming download platform that we launched here and in the UK. and really started cutting my teeth with music experiences for consumers and not just, you know, composing or creating music or creating software that modifies sound, but thinking about what it's like to be a music listener, a music consumer, and how do you find the content that you're interested in, and how does it get presented to you, and what is the listening experience like?
I was talking earlier about we've always wanted to do this.
What we do is super hi-fi. We've had the pleasure of building music products, not only for Virgin, but After about seven years, we spun out and formed a consultancy and started doing the same thing that we did Richard Branson for other people.
So we built digital music platforms for CBS radio, AOL radio, Yahoo radio, their next generation platforms.
One of the things I'm really proud of is we worked with Cricket Wireless and built their Move Music platform, ideated and built it. which was, you know, the first all in a handset built into your rate plan music listening experience, you know, designed for a very particular customer. and was really successful about 10 years ago. brought music to, as CDs were leaving retail, there was a segment of the population that didn't have computers. and their access to music started going down, except for listening on the radio.
This is very early days in availability of music experiences on mobile devices.
The iPhone was a year or two old, and we were able to have a full download experience, search, browse, unlimited download and play. right on both a feature phone and on the earliest iPhone and Android devices.
That was really awesome to be a part of that because it brought music to people who... technology was kind of moving away from a little bit because they, they were frankly, they were ahead of the game.
They were mobile before everyone else was fully mobile.
Right, right. But if you weren't around then and didn't hold any of those feature phones in your hand, you might not understand the... the gravity of what Brendan's talking about here, because those phones were a lot different than the phones that we all have, or so many of us have today.
And to be able to pack that in, and as you said, primary devices for a lot of people.
That's where a lot of my hair went, trying to get that feature phone version, feeling interactive and downloading and playing.
We had great partners, not only in cricket, but also Samsung was really... instrumental in helping us.
And also the SanDisk company, the product used a specialized little SD card that would encrypt and hold the music for It was really a great technological partnership in order to make that happen at the price point that it needed to happen at.
And those were the salad days of crazy hardware configurations.
And now it's all AI. You hit a button, it does it all for you.
No, it's not quite that easy. Fast forwarding and thinking about the three years, give or take, that Super Hi-Fi has been around, what, if anything, has been surprising to you about what you've done with Super Hi-Fi, maybe an unexpected challenge you had to overcome, maybe a You know, a right turn you took when you were expecting to go left.
Anything leap out at you as being kind of a surprising moment?
It's a good question. I think. There's a couple of them.
The moment where we first, like we did all of our initial training, we built the first model and It was another one of those jumping up and down fist pumping moments.
The moment we first heard Magic Stitch. segue between a couple of songs kind of artfully and lay a voiceover on the next one. and doing that thing that radio DJs call hitting the post, which is stopping at that perfect musical moment before the singing starts.
Right, yep. Up until that moment, we weren't sure if it was going to work.
And then it did. And that was that was fantastic. another thing that I think that surprised me was in our initial, I mentioned, you know, we have this great partner in, in, in I heart and, they've talked about this publicly, so I can mention it.
We did some initial AB testing on their Android platform giving some customers magic stitch turned on, some customers magic stitch turned off, no content being interjected. gaps between the songs being more artfully handled.
And they saw a nine to 10% increase in total time spent listening.
Nice. And to have that type of numerical evidence that doing something like we know it's better from, from a qualitative experience, that it's more artful, that it sounds nice.
But to have some quantitative view of that impact, that no, you're actually, they termed it really well, reducing the moments of choice.
If the songs don't dead stop and there's a gap of silence and then the next song starts and maybe they're, slightly different volumes relative to each other that the consumer has a moment to go, maybe I want to do something else. but by blending them together and creating these really sort of musical artful moments that we actually improve listening and I like to believe we improve the enjoyment of if the amount I listen is indicative of how much I enjoy my experience, then I hope we're improving the enjoyment that listeners are getting from all of the musical experiences that we have.
I'm nodding my head. I'm saying yes. I think you are.
So before I let you go, I'd be remiss if I did not ask about Inception.
SuperHiFi, you guys are part of the NVIDIA Inception program.
Can you and I'm not asking you to sing anybody's praises or anything, but can you just talk a little bit about what that experience is like for you guys?
It's been great. Not only has it given us access to resources and information, but there there's been a you know as a part of the program there's amazon credits uh that that you get and those have been fantastic.
We're a small, self-funded startup. We're not looking for capital.
We're really trying to grow this as a business.
To be able to have the support and basically we were allowed to go hog wild up at Amazon, exploring GPU acceleration and really going some places that maybe were further down our roadmap because we were being cautious about how much we were willing to do up there.
That's been amazing. I mean, we're, getting ready to unleash our third generation pipeline, which is heavily GPU accelerated and is essentially allowing us just to grab more and more and more information about the music. which then allows our AI to be trained and more intelligent and do things more artfully.
And it's made a material difference in the product.
I don't think there's any downside for joining the thing, and it's been just fantastic for us.
That's great to hear. So, Brendan, I'll ask you for our last question here to, you know, gaze into the future with your proprietary product stuff filtered out.
Where do you see this headed, whether for Super Hi-Fi or the... streaming content industry or the impact of AI on Art and artfully creating experiences in general.
You can kind of, you know, give you a wide berth there to pick from.
But over the next, you know, two, three, five years.
Where do you see this intersection between AI and creativity and audio taking us?
It's exploding right now. The amount of exploration that's happening, both in terms of how we understand music, how we can create music, how we can produce music, how we curate music to make listening recommendations. and how we ultimately present that music, which is what Super Hi-Fi is focusing on, it's really accelerating at such a rate.
And if you just go and look at the Google Magenta project and all of the experiments that they're publishing, in terms of generative music and performance analysis.
It's really mind-blowing. And my hope is that artists get access to the tools and can use them to help inform their own creative processes, that people who produce content can access to the AI tools, to help make the recordings better, even though they were just recorded at home and maybe they don't have the same amount of training in audio production as someone else.
And in terms of what those listening experiences sound like, which is where we've put our focus and energy, my belief is that. consumers are not only going to prefer to listen to a more produced experience, and it could be as simple as songs are blended artfully and you hear a branding message that just lets you know what you're listening to every 20 minutes, that those types of experiences are not only something that they prefer, that they like, but something that they demand and ask for. something more human, more connected, more meaningful than, you know, a pile of songs.
It means a lot. It means a lot. Brendan, for folks who want to find out more about Super Hi-Fi, I'm going to ask you for a place on the web to go and then also...
You know, we've been talking about the new Sonos HD radio experiences, but maybe some other places where folks might be able to go actually experience Super Hi-Fi in action.
Right. So if you listen to music interactively on the iHeartRadio app, you'll be listening to Super Hi-Fi.
If you listen to Sonos Radio, as mentioned, you'll be hearing the super hi-fi tech in action.
Do check out Brittany Howard's station. I also really love their classic rock, Rock Pantheon, which has all kinds of trivia and information interlaced in about artists, albums and specific tracks.
It's really cool. They did a great job with that one.
And also, if you ride a peloton, we power the listening experiences. and the playback of audio for Peloton's classes.
And you'll be able to hear the songs connecting together and maintaining energy and really focusing on a very different kind of thing, that driving experience that accompanies their amazing instructors.
Those are three great places to look right away.
Great. And then the company website, superhighfi.com?
Yeah, superhi-fi.com. You can also ping us on LinkedIn if you'd like.
Excellent. Cassidy, thank you for taking time out from the many, many things that you're obviously thinking about all the time to chat with us and articulate it so well.
It's as a music fan doesn't begin to cover it music's always been a huge part of my life and so i'm excited to go i'm going to the classic rock station you mentioned soon as we're done here.
But this is just great. And I look forward to.
You know, hopefully having you on the on the show again down the line or maybe your co-founder, but definitely tracking your progress going forward.
Thank you so much for having us. When you listen to it, just make sure you turn it up to 11.
Thank you. Thank you.
