Hello and welcome to the NVIDIA AI Podcast.
I'm your host, Noah Kravitz. Zoom became a household name in 2020 as it rose to prominence as the go-to video conference platform during the COVID pandemic.
Since then, the company has not only been refining their video technology, but also helping us all rethink the way we approach work in the era of digital communications and AI.
At Zoomtopia this past October, Zoom took the wraps off of a number of new AI-first products and initiatives. all in service of the company's mission to deliver an AI-first work platform for human connection.
Here to discuss everything from Zoom's approach to federated AI and AI agents to the future of how we all live and work with technology. is Dr. XD Huang.
XD is Zoom's chief technology officer and has a prolific background in artificial intelligence, coming to Zoom from like herself. where he founded the Speech Technology Group in 1993 and most recently served as Azure AI CTO and Technical Fellow.
XD is an IEEE and ACM fellow and an elected member of the National Academy of Engineering and the American Academy of Arts and Sciences.
And most importantly, he's with us right now, so let's get to it.
XD Wang, welcome and thank you so much for joining the NVIDIA AI podcast.
Thank you. I'm glad to be here. So we're recording this on the Friday immediately following Zoomtopia, where Zoom basically announced to the world that you're going all in on AI.
We want to hear all about the new stuff, of course, but first maybe we can set the scene a little for the audience.
Can you tell us kind of broadly about Zoom's approach to AI and AI in the workplace?
Yes. I think this is really the most exciting time.
I started working on AI since I was a graduate student.
This has been over 40 years. Right. Now, Gen of AI really, really transformed how activity is going to be.
So Zoom is in the forefront. We are providing amazing media conferencing for the whole world.
It's a household name that everyone understands what a Zoom is about.
Right. It's a verb even at this point, right?
That's right. So we are now facing an even more exciting opportunity in front of us.
So meeting is one of the most important business functions, but we want to expand that capability for people to work happily on Zoom platform.
Right. So Zoom Workplace is going to take advantage of general AI capability.
We believe that general AI is going to really provide an exciting new opportunity I reflected on my own journey when I started writing my first master thesis in Beijing's Tsinghua University.
The first paper I was using typewriter.
I loved expensive liquid paper. In China, in Beijing's Xinhua University, that was 1983, 82.
I remember liquid paper was expensive. It's a luxury.
Right, yeah. This type, any letter, I have to really use liquid paper.
Right. I go there to write that. And when I wrote my book, spoken language processing with my colleague in Microsoft.
I was fortunate. We had the Microsoft Word.
But even at that time, we cannot accommodate 800 pages document.
Too big? Too big. So we have to... separate file for each chapter.
But Microsoft would have done it on the job.
It's hard. For me to imagine without Microsoft Word, we have to use a typewriter to drive that, how hard it's going to be.
Because we have a lot of graphs, math, and... references right time passed quickly one of my colleagues in microsoft He wrote this book with the suspense of GPT-4, challenging it.
It's just amazing. That productivity really pushed everything to the new level.
So you can see that reflection journey. To stop you for a quick second, if you can think back to when you were writing on the typewriter, Could you have imagined where we'd be now?
You've been in this field for a long time, so perhaps you could, but I'm just curious, you know, if 30 years ago, where we're sitting today, your colleague using GPT-4 to help write a book.
Is that something you thought about back then?
Yes. I selected a speech recognition as my thesis.
And at that time, I had only IBM PCS students.
And I don't know if you know what that means.
Yeah, no, I remember. A few Apple II computers.
Yeah, I had a IIe growing up. Yeah, that was all we had.
And I actually told myself, if I could let the computer understand spoken language, I couldn't be tired.
40 years passed. I'm more excited than ever.
I'm not retired. The frontier is not our spiritual condition.
In Microsoft, we were the first to reach the human priority on the most difficult speech task switchboard in 2016.
Most people didn't believe we could have done that.
Yes, we did. Now, chat activity really redefined and opened up the imagination for the whole world. right i think okay i did a great job to really redefine the new frontier share the story why zoom yes you know going to grab this opportunity to redefine productivity Every era of computing created productivity leader.
Microsoft revolutionized desktop computing.
Office unquestionably is the productivity leader for desktop computing.
That's why I shared with you when I wrote the book, Spoken Entity Processing with my colleague at Microsoft.
We love Microsoft Word. When the work came together, Google took advantage of it.
It has productivity to support multiple people working on the same docking.
As we know, all the docks and the sheet slides They all really supported the cross-team collaboration.
Right, the collaboration, yeah. That took notice from almost everyone.
Right. One viewer is incremental influence, in my opinion.
How so? Now, in general, we are all in the same leveling field, whether it's Microsoft, Google, or Zoom.
Of course, Zoom has a unique advantage in the most important business function to connect people on the meeting. we are the leader.
But just having that meeting capability would be insufficient.
If you think about the work, we have probably a few key functions, right?
One is to consume information. We need to learn, right?
This is increasingly human value, to commit, to satisfy our own curiosity.
Genentech AI can help you to read 500 or 800 page books like what my colleague and I published.
800 pages in one page. So magically, junk data can redo this and create an amazing amount of learning for almost everyone.
We just do not consume information. That's one of the important functions.
We also need to communicate. Influence, bring people along.
So generally, I can help you to compose the draft because we understand your need.
So those two most important human fundamental capabilities to read, to write, to speak, are a group to write and to speak at the same time.
Right. Consume information to communicate.
Yeah. They are going to be really, really fundamentally helped.
So imagine if we can take a dive capability, redesign productivity, not just brought those capabilities to the existing software.
That's an opportunity zone. really, really possess.
Right. The approach for us to work on productivity suite and the approach we are addressing AI, there are three key ways I want to highlight.
Now, talk about this in Zoonotopia. Let me just explain this in detail.
The first thing I want to really highlight is our federated AI stack.
So we integrate the best from leading AI companies, Anthropic, Open your eye.
Matter. Check live, et cetera. Any open source opportunities.
We often learn this with web search leaders.
They did the neural complexity. Okay. So we federated all of them together.
In addition to our own proprietary small language model, we're training, we're developing.
That's already reaching amazing capability.
We appreciate that the small language model, because of the scaling law, need to really work together with this amazing cloud-based large language models.
So we have this unique approach to combine them together seamlessly behind to support the productivity of each individual state.
What does the small language model do in the stack?
How does it differ from what you're tasking the large language models with?
We are training the smooth language model like everyone else to train large language model.
It's just a harder task. In addition to that, we're also incorporating each individual's unique context.
So we can really make that personalized.
In addition to consuming this massive amount of tokens.
Right. Always somewhere. So if I'm, if I'm using Zoom's AI features and I give permission, Zoom can basically just ingest all of my conversations, all the meetings I have, the voice conversations, the documents, the chats, all of that and use that all as context for the generative AI going forward.
This is the feature that is coming together through our customer AI companion.
Oh, got it. Okay. A company 2.0 is horizontal, generic, not personalized.
Okay. This is customly accompanying that we will introduce later this Next year.
Next year, okay. We'll actually incorporate the ability for anyone to customize and to personalize.
This is actually a very powerful opportunity for the small language model running on the devices to really augment what the national models cannot offer because they don't understand. your personal needs, your learning pattern, your writing pattern, et cetera.
I just want to really first highlight our federal AI stack is unique, very unique in the industry.
Unlike many other productivity companies, they use only one.
Right. And so for the audience who might not be familiar, federated AI, a federated stack, Does that essentially just mean that the system can choose which LLM to prompt depending on the situation?
Or what does federated mean, the way you're using it?
There are multiple ways to federate. The way to federate the larger language models and the small language model is a new frontier.
The way we are federating this is different from federated learning.
We should be trying to really combine multiple models together to form this powerful capability that can preserve the client privacy.
What we're doing is we can choose based on different workloads because They are complaining too close.
It's almost like a super agent. That is trying to understand with different modality, different memory. spam, etc.
So, we can choose. One model is the best for different tasks.
We can also combine different models together and we can reflect like a chain of thought, rethink, and we perform the same task based on what we have learned from A small language model, for example.
So if a small language model can perform the task very well, then we stop there.
It's sufficient. So it's a very sophisticated system that can actually orchestrate multiple models together This has been developed and pushed by Zoom AI talents.
So this is a very unique approach that has set us apart from almost anyone else.
Yeah. You use the word agent, and as we're recording this, AI agents are...
There's a lot of buzz. I'm hearing a lot of buzz around the word agent and the concept of agentic AI, which isn't new, but as it's come to the fore lately, Can you talk a little bit about what that means, what the idea of an AI agent is, kind of broadly, but then specifically to how Zoom is using it?
And I want to come to that later. You want to come to that later.
Okay. Nick, why we approach AI differently with 3D... ways that are different from traditional approach right so the first one If you think about traditional productivity suite, most of the companies are using one model, either OpenAI or Gemini. to augment what they do right they build it air capability to the existing software Right.
Right. So on the back end, they're mostly using one very good challenge model, either Gemini or Brunei.
Mm-hmm. So the approach is different. We massage OpenAI Anthropic, Gemini, Meta, and our own smaller model together. to offer unmatched performance.
So that's the number one I want to really highlight.
Of course, we also integrate our partner perplexity for the amazing web search result for the web search so what is that questioning or work questioning.
And in the future, personal questioning.
We can associate them together. That's what we are pushing to differentiate the Zoom AI through our federated approach.
That's number one. Okay. Number two, our user experience is AI first.
This is what I call AUI. We are all familiar with screen-optimized graphical user interface as defined by Xerox.
Many, many, many years ago. Yes. You go back.
Right, right. Populated by Mac. Uh-huh. So both Office, Google Docs are examples of taking advantage of graphic use and effects.
Mm-hmm. So that's well understood. Okay.
And Challengivity redefined conversational user interface.
They reached 100 million users Amazing fast.
Right, faster than anyone. So what Zoom is doing is developing a UI that will seamlessly combine GUI and the CUI together.
What that means is in Zoom Workplace, they are compelling to follow will be a persistent panel on the right, Okay.
And the traditional graphic user interface services, whether it's scheduling a meeting or... have a meeting with someone.
Right. Calendar, your email. It's on the left.
Okay. Information flows seamlessly between those two in the AUI.
So we are trying to take advantage of both conversationally using the face and the screen optimized using the face seamlessly.
The facial eye emission is one where The technology intuitively adapts to your own needs.
That's getting more personal. That's what we are coming with custom AI campaigns.
Sure. When you say adapts, do you mean that the user interface changes or that it can create a conversational window sort of in context when you mean it?
Or could AI potentially just redesign the UI on the fly to match what you're doing?
How do you envision that? AI has the vision.
And with that, what information you want to consume, how you want to consume it.
Okay. That's not what an AUX is going to be.
It's not just a chat interface as chat LGBT defined today.
It's not just graph use interface like zoom meeting is defined today right also combined seamlessly in the multimodal environment right we learn that based on your individual needs right now we're trying to combine those two categories into one.
GUI traditional is so massive, right? Most of the world's services and applications, they are GUI optimized.
Yeah. ChatGPT, conversational user interface, is a new category.
And we just kind of have that to be the only one.
We'll bring those two together with information flowing across those two categories seamlessly.
Right. And trying to understand the user's needs and adapt on the fly.
Okay. That is what the AUI is going to be.
Got it. I'm already calling this world AUI.
So... This is our record. And this is the first time I'm telling you in detail what the future of Eastern Francisco is. world premiere i love it yeah so that is the principal approach zoom is taking Embracing AI natively.
That's what we'll call AI for us. You joined Zoom about a year and a half ago, a little less.
Yes. When you joined... Did you... I know Zoom has had, you know, AI functionality, AI Companion version one, and, you know, you can... use third-party apps for transcription and et cetera, et cetera.
That's been around for a little while. But when you joined...
Is this sort of, you came in and thought, okay, let's rebuild this from the ground up, AI-centric?
Was that sort of already happening when you joined?
Just kind of wondering... you know, as you stepped into the role, sort of what was envisioned and how much you've shaped things since then?
So, Eric Yuan, Zoom CEO, got his great vision.
So Zoom has invested in AI before. Yes, yeah.
Since I came, I worked with Eric and the leadership team together.
We defined AI first. Okay. Before I came, it was just adding AI, both AI, like almost every other company.
Right, yeah. We have transformed that. We got the consensus. pushing AI-first to the platform.
So what does AI-first mean? Three things.
So the first thing is that they had technology back then.
Both that, small law, small language model.
Mm-hmm. and build on the shoulders of great AI companies out there, whether it's OpenAI or Anthropical or Meta. or other open source companies like Mishra, etc.
Just that there are a lot of them. It would be a mistake not to take advantage of all of them.
Of course. Right. So it's like we form a committee working to support our workloads.
That's always better than just using one single model, trying to really perform the same task.
Right. Two brains are better than one. Yeah.
So you see how inclusive we are. On the models, we're trying to combine all of them together.
And they're using the face. And like some companies said, Charting interface is the only way or graphics interface is the only way.
I'm going to add a button here and there.
We are combining those two categories using the phrase classics. into one that will adapt to your own needs with information flow between those two categories seamlessly.
That's the second important advance. I want to use AUI as the frame to summarize this design principle.
So the third thing I want to talk about is what is the work productivity suite?
That's in the general AI era. I would say it's all about creating a true system of action.
We exist, but We have tasks to do. We take action, right?
Of course, you can say you want to entertain people, but that's not productivity.
It's weird. There are a lot of random payment software.
AI will do that. So when we say we are a first work platform, This is about AI companion is designed to understand your workflow. can learn from your pattern.
Everyone got different workflow. Everyone got a different selection of services soft right and we use ai to anticipate your personal needs Emphasize that.
And they can take action on your behalf with your permission. or with your code participating to make a better decision more than what you can just do by yourself.
Those are the really the soul and the spirit of AI first productivity.
That's very different from just to replace liquid paper with word processing.
Or just support three people co-editing the same document.
Or just about formatting this document with nice fonts.
It's about Those three things. It's about learning from your own pattern, anticipating your own personal needs, and take action on your behalf.
Whether it's tracking tasks or managing action items, is always one step ahead of you, ensuring that productivity flows seamlessly, effortlessly. throughout Zoom's whole ecosystem in workplace and the third-party solution.
Right. So if the AI companion you know, understands my workflow and then can suggest to me actions to take Either now or going forward, is it a case of like imagining the AI would say to me, hey, you should do these things in this order.
Or will it actually, you know, call up an additional tool to help me help facilitate getting these things done?
Like how does that or how do you envision that? working.
Just innovation and compelling can proactively inform you.
You are not answering the question right in the meeting.
Okay. Only you can see, right? Right. Just imagine how powerful that is going to be.
Right, we're in real time and I start to give the wrong answer.
Because in real time, a company is always augmenting your ability to influence others, make others like you better.
Right. So this is just what I want to call another phrase.
So I'll talk about Frederick A. S. Mack.
Uh-huh. Unique? Yes. I talk about the user interface, that's AUI.
The AUI, yes. This is about action-oriented task flow action-oriented task flow okay this will flow to every corner for the whole life cycle of what you need to do, because it's almost like you have a very expensive executive assistant. focus on the most important task you need to pay attention to for the life cycle of the whole project.
Until you get that project done beautifully, in a time sensitive manner and in a way you delight your co-worker your family members for better human connection.
This is the goal of Zoom's AI-first work platform.
It's action-oriented information flow. If there's something you don't need to take action, we can still... accumulate those tasks to confuse you.
That's okay. And you can decide. And you do not want to actually track those actions. we learn that pattern from you.
Yeah. And we improve our ability to track.
But if you select the O, air complaining told me this action is important.
You check that online, air complaining will work harder.
Keep your eyes open. So a week later, if you receive the piece of the email, That is relevant to the task you're tracking.
Air Confident will work 24 by 7 to update what you need to do. and give you tips and advice what you have to do better to accomplish that task.
This is what I talk about action or renting information flow.
Right. And so is the AI companion an example of an AI agent working on your behalf?
Absolutely. AI companion 2.0 already brought agent-like capabilities, like in the meeting.
We're not just actually using speech recognition to understand what is being talked about.
If you presented your slide, a company entrepreneur today understand what is presented in the slide or what you wrote on the paper that you shared your story.
We have the capability, multimodal. Or if you're shooting your points in the side panel with chat, we take that into account as well.
That's amazing. It's almost like agent, really participating in the meeting as you do.
Then we present meeting recap. You're not meeting recap.
The most powerful way for us is to identify next steps. you need to pay attention to, or your colleague need to pay attention to.
Then next step is unique. We offer unmatched quality.
We worked on this so hard in the past year and a half to improve like such, to reduce to assign the right task to the right person.
We are roughly right now, probably 80% accurate.
Okay. So we're not done. It's not perfect.
But 80% is really impressive, right? I was going to say, in my experiences with LLMs and hallucinations and accuracy, 80% sounds pretty good.
Yeah. Yeah. So we do not stop here, right?
So let's say you have a meeting you discuss.
AI Companion 2.0 identified the task and that task will show up in the upcoming through this task panel.
And a week later, through the whole life cycle, if that's something you want to chat, you want to track, you receive a piece of email from Zoom and create an update that on your behalf.
And if you want to read it right away, about that action item or status report for your colleague.
That information flows into Zoom docs. We draft the status report on your behalf.
You feel pretty happy with a few changes without doing anything using liquid paper. or Microsoft Word to format everything.
And you want to say, hey, change this into the form that I can present in that next sales update meeting.
One command. Yes. Right. And it's not as beautiful as, you know, that what a PowerPoint can do with beautiful picture.
But the key point is very much like When I was at the College of Maryland, when we presented information, it was black and white, they tear off. and printed on the reading transparency paper.
And just use really projector to talk about the key points.
Was STEM actually effective? It still works, yeah.
The Malaysian is slow. with a beautiful color background animation.
So that's the point I want to take. Zoom docs alone is actually performing most of the function because of the generality AI.
With the AI compiler, you can instruct, You can summarize in the form of a status report.
We can publish this as a blog that's more ready to be consumed for the public or In the simple form of slides, then you can communicate your points in your next meeting.
You do not need the last generation productivity suite. as we know.
So that is actually a high-level three key points.
Federal AI stack, AI-first user interface, AUI, an action-oriented information flow for productivity.
That is really the landmark, how AI works. productivity suite, or differential farm, web-centric, productivity suite, or desktop-centric.
Of course, both web-centric and the desktop-centric can add a capability that's bolted.
Bolted, sure. Yeah. What Zoom is defining is natively.
Right. Backup item, information flow into every corner of the productivity stream.
I guess today. is xd wong xd is the cto of zoom uh position he's held for going on a year and a half now before that he was with microsoft for uh quite some time.
And really, it's just a continuation of an illustrious career that started Way back, as XD was talking about in the days of typewriters and liquid paper, as we both know.
XD, I want to switch gears here. We have a few minutes left to talk I want to kind of look at things from the public standpoint and specifically from business users and the types of customers who Zoom has been working with for a while now.
When Zoom is talking to business customers about AI, about adopting Zoom's products and all these wonderful things you're building, but broadly just about, you know, using generative AI and making the investment and spending the time to upskill workers and sort of figure out how are we using these things.
How do you help your customers think about both adopting AI and also how to measure return on investment?
There's a lot of conversation we've had on the podcast and just generally in the world about. being kind of, for exciting as the past couple of years have been, still being in the early days of figuring out What can gen AI do?
How do we use it? How do we, how do we rethink things like, you know, productivity suites from the ground up with AI?
So when you're talking to a customer company How do you educate them about getting started and measuring performance?
There are a few things absolutely our customers love.
First, Zoom Workplace as a whole offers ease of use.
That's just unmatched. whether it's eating or dogs or chat.
The second thing is really, generally a companion 2.0 is offered. and no additional cost.
That's just stunning for most of the customers.
Because they get used to, you have to pay $30 a month. per person, right?
So Zoom offer this capability for the high-end customers.
We are going to offer custom air company.
We will charge $12 So you can bring your own data, get your own pattern into the they are complaining, complaining we can run that on your behalf with fine-tuning the customized capabilities.
So Zoom offers this amazing horizontal Absolutely, you know, game-changing capability to make its own workplace a very viable productivity candidate.
And for the high end, we offer you unmatched custom air companion for the info.
With $12 per person per month, it still offers the best of the TCO.
So interviews, cost-effective, and match the quality.
That's what we need to hold and sell for our customers.
And is kind of the, I don't know, goal or sort of the vision that customers who use Zoom come away with is...
That the AI, the companion is just over time going to learn more and more about how you work and what your workflows are like and how you sequence tasks and the people, the colleagues you're working with.
And the companion will just be there to help you think a couple steps ahead, help you get maximize your own efficiency, whatever the word is, is that is... Because that's a different conversation than... conversations I've had or I've read about or listened into where companies are saying, okay, we need to start with wrangling all our data.
And then we need to figure out how to clean the data and how to, you know... And it's kind of this big, deep investment process where it sounds like with Zoom, it's more like... hey, you're already using it for video calls.
And now we're going to give you this groundbreaking, you know, change the way that you do everything companion.
And it's just kind of going to be there.
And there's not a lot you have to do as the user.
So we offer the choice to our customers if they have the comfort.
Yes, who have improved customized capability to suit their needs.
If they don't, they can decide how much data they want to share or whether they want to turn off for some sensitive meeting.
That complexity is in the hands of the customer.
They can control themselves. So giving them the choice.
Yeah. On top of that, I want to really emphasize Zoom never takes any custom data. in the meeting to train our AI models.
Okay. Let's end on a kind of looking ahead note, if that's all right.
As you envision the next, I'm going to say three years, you can change that.
Two years, five years, whatever you think it is. both in terms of Zoom's mission and using AI and generative AI to help people do things smarter, better, faster in the workplace.
And then more broadly as gen AI and other forms of AI and machine learning and deep learning just continue to impact the world more.
What are you most excited about in the short term?
Again, three years, however long it is. What are you really excited about and see coming down the pike that may...
I don't know if it's the next transformational moment or just kind of a trend that's going to really take fire and change the way we do things.
What are you looking ahead to? I really think this action of making the information flow to be your companion.
Yeah. This is really just a game changer. all of us will not have enough time.
So if area campaigning can really help you to get job done quickly, you can have additional time to do whatever you want that's another productivity capability some entertainment for whatever it is yeah we'll also bring this would have will be a better place We make your worker happy, leave happy, and do whatever you want.
Be happy in between. You can do whatever you want.
So delight customers is the core mission.
Excellent. XD, for people who would like to learn more about what Zoom's doing, announcements at Zoomtopia, perhaps some of the, I don't know if there's a technical blog for developers and people more technically inclined to learn more about how you're approaching everything federated AI and everything else we've discussed.
Where's a good place or some good places online for people to get started to learn more?
Yeah, you can check the Zoom blog, you know, zoom.com.
That's actually probably the best place to learn.
Best place to start. But the even better one is really start turning on AI companion in some workplace.
Without using that, you do not know how powerful it is.
You got to use it. Yeah, absolutely. Fantastic.
XD, thank you so much for taking the time, particularly at the end of this, what I'm sure was a busy week, a crazy week for you.
But congratulations on Zoomtopia, on the work you've done so far.
And I, for one, am excited to use companion too well if i could have a panel on the side of my screen that's always telling me that the next best thing i should do that that would be a game changer for me personally so I'm excited.
I can't believe we got the air companion.
High productivity software. Yes. Fantastic.
Well, thank you again. And perhaps we can catch up somewhere down the line to see what's going on at Zoomtopia next year.
Absolutely. Thank you. Nice to be here. Thank you.
