EP 124: Databricks CEO Ali Ghodsi Breaks Down the AI Hype-Cycle | The Logan Bartlett Show

Welcome to the Logan Bartlett Show.

On this episode, what you're going to hear is a conversation I had with Ali Godsi.

Ali is the co-founder and CEO of Databricks, a business valued at over $50 billion.

Databricks sits in the center of the artificial intelligence revolution that we're seeing today.

Ali and I talk about the state of AI as well as what he's seeing going on at the ground level, his view on the federation of models that he sees coming, as well as what he sees for the future of AI over the course of the next couple of years.

We also go into the founding of Databricks, his journey to becoming ultimately the CEO, as well as the different operating principles that he runs the company with today.

A really fun conversation that you'll hear with Ali now.

Ali, thanks for doing this.

Of course.

So you've been doing artificial intelligence since 2013 with Databricks?

With Databricks, yeah.

Before that, we were at Berkeley.

We were doing also AI.

When did you actually start?

I think 2009, we started participating early, 2009, 10.

Spark, when it was originally created, was to participate in the Netflix competition, which was hey, they gave you all these movies and you had to do recommendations and they would give a million dollars, a half a million dollars, to the winner.

So we got the highest score on that together with another team.

So you're 15 years in now.

Yeah.

It obviously was considered somewhat niche and maybe academic initially, and then obviously we're in a very different point today.

I'm curious, as ChatGPT happened and that moment where it seemingly went from enterprise and business use to sort of mainstream consumer adoption,

Did that change your perspective on artificial intelligence in any way?

Not really.

I think that it kind of changed.

I think from 2013, 14, 15, I just didn't understand why people don't really get into AI.

You know, in 2014, I remember having this debate with many of the people at the company that I wanted to call it AI.

They said, no, that's like not authentic.

Would you call it machine learning?

It's ML.

And I said, look once this stuff goes mainstream.

You know, mainstream it's not going to be called ml, right?

No, but that's not accurate.

Ai is robotics and uh, i just never could understand why it's not much bigger of a deal.

It was kind of a little bit esoteric uh, machine learning ai uh, and in you know 2022, november.

There it just made all the sense, but then the magnitude of how much it sort of just shook everybody up.

That was kind of a shock.

It was surprising to you.

Was there points that you look back on actually in those 15 years that were actually maybe groundbreaking to you but not to the mainstream?

Yeah, I mean like when we started, people weren't really using machine learning for other things than ad tech.

It was really ad placement.

You know click through, you know click streams for logs, to place ads and so on.

So I would say 2014 15, when we said really large enterprises started using this stuff for more interesting things, like Expedia.

How they listed the hotels.

How you ranked the listing of the hotels gave them a huge revenue uplift.

It was kind of a game changer.

Regeneron found sort of...

So they found the marker in the genetic DNA for a disease and they were developing drugs.

So you could start seeing like, wow, that has really big impact on a lot of these companies.

That was a big deal.

But then I would say 2018 19.

We started seeing the natural language processing breakthroughs coming in with GPT-2 and so on.

It was already pretty amazing.

So it was pretty clear that the Turing test would be broken soon in a couple of years or so.

So, you know, it's been many of them.

But yeah, who would have anticipated that the whole world would kind of wake up in this way and this would be like front news every day?

Yeah, hard to believe.

I've heard you compare the ChatGP moment to the iPhone moment.

Can you compare that analogy and what the implications are for apps and maybe how enterprises or users should think about artificial intelligence?

Yeah, I mean, when the iPhone came out, everybody realized kind of what the smartphone revolution would be.

And what happened immediately was that every company on the planet said we have to build lots of apps.

We have to build our own iPhone apps.

And so there was this flurry of activity, all the companies building apps.

But you know, did the apps really make sense?

A lot of the apps that they built.

You know, I kind of joke about it and I say it's kind of like building flashlight application for your iPhone.

But then a couple of years later, the iPhone just has the flashlight built in.

And I think 2023 was one of those years that everybody wanted to say, hey, I own AI at our company.

I am the AI leader.

I am the gen AI.

So there were a lot of chatbots that were built.

And, you know, there'll be just like there's an app for that in 2007 or 8.

There's going to be an AI company for that that just does that.

So, you know, for instance, people building chatbot for HR documentation.

You can ask, you know, What's your policy on paternity leave or maternity leave?

And a company is building that, but it doesn't make sense that a company builds that for themselves.

It makes more sense that there is a SaaS app for that and everybody just buys that.

And a little bit.

We're in this moment where there's a lot of like hey, we're going to build our own flashlight app or whatever chat app for everything.

And now we're seeing slowly the companies being built and the apps being built that are going to specialize and we're all going to just leverage them.

I think we're like in the first innings of that.

To take that analogy a step further, do you think that there ultimately will be disintermediation between the end customer and maybe the HRS system, where a native AI business takes in that natural language interface and no longer would Workday or ADP or whoever it is on the back end serve as the interface to the user?

Could be.

I think that this dynamic is playing out that in this AI era, data is extremely important.

So why shouldn't everybody build their own chat app?

Well, you know, you should really ask yourself, what's your competitive advantage as a company?

What is it you can do uniquely that no one else in your industry can do?

Everybody can call OpenAI as LLM.

So you just calling, you know, GPT-4 or 506, that's not going to help.

So what did you have?

Typically it comes back to the data or some customer relationship, but typically data.

You must have some data advantage that nobody else has.

So that's what it's going to come down to.

The big companies, the incumbents, they have that data advantage, right?

They have customers and they're collecting a lot of data.

So the question is can they leverage that data advantage and incorporate the AI and do the innovation that's needed?

On the other hand, we've seen in these waves before that you'll have these smart founding teams that are just completely disrupted, thinking about things differently, and they'll rise and they'll do things in a way that the big companies are just they're asleep at the wheel and they're not going to do.

I think we're going to see both.

I think we're going to see a lot of company startups that fail, just like in 99, when the internet was about to come.

But we're also going to see a few that become the next Google, Facebook, and so on.

But I think we're also going to see some incumbents that actually figure out how to leverage the data that they have and do the investments needed to crack the code on this.

We're going to see both of those.

How exactly that you know, flashes out.

I would probably bet more on the startups, to be honest, than the big ones.

Interesting.

They move too slow.

The big ones move too slow.

And you know, if you have like a killer app, you can get the data flywheel, probably collecting data on it, and grow the usage.

And then you'll have enough data advantage to be able to do that.

And if the big guys are not paying attention, it'll move too fast.

And before you know it, it's too late to react.

Where do you think we are in the hype cycle of AI right now?

I think.

I think that we're gonna see there's.

I don't think like hey, we're in trough of disillusionment.

I think it's.

You know frankly, I think it's like somewhere at the top and And we're going to see that soon.

We're going to see more and more companies not doing so well.

And I think there's going to be a narrative that, hey, this AI thing was complete.

It was overpromised.

There's nothing there.

The quality of the models aren't there.

They just don't work.

This stuff doesn't work.

It wasn't worth it, all these companies and so on.

So I think we're kind of going in towards that very, very soon.

But then I think right after that is you know when we actually are going to see those very companies start coming out with these applications that are completely game changing for humanity.

The good news is crypto is back, so people can bounce back and forth between one and the other.

I'm curious.

As we record this, I guess early November of 24, there's been a bunch of reports about scaling laws and maybe the limitations that we're reaching some of the outer ends and diminishing returns on that.

Do you have any opinion on where we are with scaling laws right now?

Yeah.

I mean, we actually put out a research paper.

Uh uh, you know my co-founder, Matej Zaria.

Uh it's, it's about the compound AI systems.

We call it, and it was actually put out probably six months ago or maybe even more.

And it kind of predicts this uh, which.

What it basically says is that to really eke out that last quality bit now and really like move up the ladder, we really need to build these agent systems where we have many parts, many components that work together.

And, in particular, it's going to be really important that we can evaluate the the quality of that whole AI system on the particular task you want to do.

Just to take that silly example that I gave, HR docs Q&A.

How well do you answer questions from employees on HR documentation?

Not how well do you pass the bar exam or medical exam?

General intelligence not that, but this specific.

Only once we have this evaluation we can start building these agentic systems where we can start doing hill climb.

So keep improving the quality for that specific task.

So we call these compound AI systems.

And these will also leverage a lot of tools, And they will also leverage classic machine learning, because classic machine learning has much better numeracy.

It's much better at dealing with numbers and so on calculators, things like nature forecasting and so on.

So that's what we call compound AI systems.

And the beauty of those is that you can, when you run them and they get something wrong, you can debug which agent got it wrong first.

Where was the reasoning step that was broken?

And you can then improve it.

You can get better and better.

And you can do this hill climbing and keep increasing the quality of the model or the AI system.

Scaling laws.

You pre-bake this gigantic models.

You wait six months and the model comes out.

And maybe it's bad at predicting something.

Maybe it's numeracy is not good and you can't even know why it's not good.

There's no way for us to know, hey, why did it get this thing wrong?

I don't know.

You do a bunch of matrix multiply, a bunch of attention layers, something pops out.

It's inaccurate.

I don't know.

Let's go train a bigger one with more data.

So it just seems much easier to do this now, more agentic workflows at inference time.

So it makes a lot of sense.

I think there's going to be a lot of innovation.

It is nothing, you know, I think everybody's now focused on this.

Do you think there's an implication for like do you think that results in a federation of model types and different capabilities that people are respectively good for?

I think you will build AI systems that are very, very good at a particular task.

We see it all the time.

Our customers, they typically want to do the same thing again and again and again.

Like they have massive data, like they have database with SEC filings, right?

Or they're an insurance company and they have all these insurance documents.

And they want to repetitively, again and again, on massive amounts of data, repetitively do the same task, again and again and again.

So then for them, it comes down to What's the highest quality I can do on this specific task?

I don't need to be good at AGI, artificial general intelligence.

How can I do this again and again really, really well?

And can you give me the lowest cost for that?

Because I'm going to do this at scale.

It's going to cost me a lot of money.

So then it makes sense to specialize a system that's really, really good at that.

Having a generic, gigantic model is just way too costly.

And actually it won't do as well, even though it's really smart.

It won't do as well at that particular task.

Repetitive tasks that you want to do.

So we see this all the time.

Like one of the surprising things on Databricks is, you know, we have models of all kinds of sizes.

You can use LAMA 405 billion parameters, but in aggregate, we get more revenue from tiny models.

And that was surprising to me because you know per model you make much more on.

A bigger model uses much more GPUs and so on.

But...

Customers want to take the big models, distill the smallest possible that can get good enough quality and just use that because that saves them money.

And actually, in aggregate, we make more money on that.

So yes, I think we're going to see that.

And you see that in life also.

Are all the tasks that humans are doing, all the jobs that are happening in our day.

Do you need Terence Tao for every task?

You know, are you getting the smartest person possible on the planet doing them?

Or many of the jobs are repetitive and some things you know it's good enough and you want to pay this particular price for it.

So it makes sense that you have a smarter model that's specialized.

What do you think about the concept of general reasoning or the path to maybe not AGI but a more holistic understanding and reasoning that sort of underpins these distinct models?

Do you think we're approaching a step function in that?

Or do you think that we've reached some certain plateau in it and now it's about the data and the different functional areas?

I mean, look, the scaling laws, it's like on a logarithmic scale.

So moving to the right, we are going to move to the right.

And yes, that means that we're going to get improved intelligence.

But the next 1,000x is not going to yield as much as the previous 1,000x.

Plus one of our researchers put this out over the last 11 years we've had 100 million times improvement in compute that we've thrown at the problem.

We're not going to be able to get the same over the next few years.

We're going to improve it, but not as much.

There's just not going to be enough energy compute to throw at it.

And we're not going to have enough data.

Like, you know, Lama was trained on 15 trillion tokens.

What's that?

10X the size of the web.

Are we going to 100x, 1,000x the amount of data that can do that?

Getting that high quality data, it seems just it gets exceedingly expensive.

It's much easier to build these compound AI systems, these agentic systems, where you can actually debug and get really good at the task.

So yeah, I think there's going to be a slowdown.

I also think that a lot of those things where people are like look, we're going to really crack the code and come up with that super AGI, that kind of would require some kind of recursive self-improvement loop.

I don't see that happening, because the next model is more expensive and requires even more people.

You have to be even more careful about it.

I would believe that if you would say hey, GPT-5 will cost, you know, 1 and it's going to take one second to create.

If we were headed in that direction, Then you could see like okay, well then it can spawn off and we can just apply a genetic algorithm, do lots of them and pick the best, and so on.

But we're not headed in that direction.

It's the opposite.

It's like, we're going to do a very specialized data center.

We're going to build a model over one year period of time.

We're going to be very careful.

Lots of humans involved.

So I don't have a crystal ball, but my guess is that for many more years we need humans in the loop.

And we need this stuff to be verifiable.

Just like we saw with cars or with autopilots in airplanes.

We want to have a human in the loop that can verify that it's not veering off in the wrong direction.

And it's just accelerating the work that they do today.

And I think for that, we're going to use more of these agentic systems.

So it sounds like a lot of lead bullets.

There's no silver bullet that you sort of see on the horizon at this point.

When you hear people talking about 50 billion models or throwing all this, it's very good for your business, I'm sure as people think about this.

Do you think that all of the stuff we were just talking about it means that that's a little pie in the sky and overly ambitious for diminishing returns, as people sort of talk about these models at that level of scale?

Well, again, I don't have a crystal ball and maybe they're right.

And if they are, it's amazing.

You've been pretty good historically in predicting some of these things.

I think that you know the reasoning that they have makes sense, which is like hey, we're freaking, creating AGI.

It's worth any cost.

It doesn't matter what it costs and it doesn't matter what dilution and your economic model is done.

I don't care about them because look at what we're creating.

Makes sense to me if you actually achieve it.

But, you know, I would probably lean more on the skeptical side.

You know spending that much money.

I guess I'm too much of an operator running a company that needs to think about.

You know you know growth rate LTV, CAC.

You know you know so on and so forth, operating margin that I think you know if you got to spend that much money to produce a model.

And then you have to do inference on that model, for it to sort of pay back the investments that you need to train that model.

And it seems right now the case is that before you've even gotten the money back for how much it costs to train the model, it's already commoditized and there's a better version by someone somewhere.

So then economically, it's just right now not feasible to do that.

It seems that for those particular tasks the cost to create these compound agentic systems That's worthwhile.

And the ROI is there.

Why not do that?

And it can drive a lot of value and continue doing that.

And then maybe we'll be surprised when one of these labs come up with that super model.

But I'm skeptical.

I think that there's going to be a barrier.

We got to overcome some big barrier to get to that complete autonomous.

That can just get the accuracy right and it's reliable.

If we froze model development right now, do you think that the existing capabilities that we have in AI is enough to catch up at some point with all the tooling and the applications?

And all that with the excitement and the promise that we're seeing?

Or do you think these incremental gains that we're sort of talking about in the more niche sort of specialized models, we need to keep making those improvements to catch up with the potential of what we could actually accomplish?

Absolutely.

I think that humans are the longest pole in the 10th year.

If no human uses any of this stuff, it's absolutely useless.

You can't even come up with that super AGI thing if Nobody's using it.

It doesn't matter.

Humans take a very long time to change all of their behaviors habits, processes in these companies, and so on.

It's just very, very hard.

So I think, with what just we have now, There's huge untapped potential, and it's not yet been applied in every workplace, in every function, in every process, in everything that we're doing.

And I think we can even improve what we have today with those kind of techniques that we discussed.

So I think even if we freeze it and we say, hey, that's it.

Actually, we didn't know how to create a single model that's even better.

This is the best we'll see in the next 20 years.

I think there's going to be massive implications on the economy and, you know, on what we can do and so on.

So I would still be extremely bullish on this.

That's really true to what I believe in.

Is there something that you see from your seat and purview that you have?

That maybe wouldn't be intuitive to the average listener that's paying attention to all this stuff, but you're able to see.

So clearly the same way that you maybe did 10 years ago you're like hey, why doesn't anyone care about this AI thing?

Is there another insight?

Maybe not at that quantum, but something else that you sort of see, that you feel like might be mainstream or well understood in a couple of years, that is underappreciated today.

Yeah, I mean, I think it's the fact that you're focusing the task.

I kind of said it.

You focus the tasks on the repetitive stuff that today is costing you a lot of money.

And if you can just get the reliability up on those specific tasks, it's massive improvement for these organizations.

And it's pretty game changing.

We haven't actually reaped that yet.

Those use cases are sort of right now being worked on.

Folks are building them.

They're not in production.

It hasn't had impact on all the companies that are using this stuff, but it's just around the corner.

It's guaranteed it's going to appear.

It will probably take two, three years before we see it, and then we'll see the impact of it.

And actually I fear or suspect that there might be before that we hit the trough of disillusionment and then actually these use cases land and we'll see the companies that are doing that.

So you can see it baking already.

It's just not in production yet.

Are there use cases that you've seen people use Databricks or other systems for that have gotten you particularly excited?

Back to the Regeneron analogy that you were saying 10 years ago.

Maybe something today that you guys see on the field that is exciting or makes you particularly looking forward to the future of what could happen.

In every vertical, in every industry, we see lots and lots of use cases.

You know, like financial services, you know, there's so many financial advisors, FAs out there.

You know they basically now we generate automatically for one of our customers you know large bank automatically in advance.

What advice they should give to you?

It's customized.

They understand everything about you, your wealth situation, and so on.

They can give you customized advice.

It affects thousands of thousands of these folks that are doing that.

In insurance, sifting through massive, massive amounts of documents to assess risk.

It's like they can do it.

And, by the way, one thing that people underestimate is that the models are very fast at reading data.

They're slow at producing text.

They're much slower.

The other insight I would say that's surprising is it's probably at least a 10 to 1 ratio between how much people are using the models to read stuff than to produce.

So they have.

When they use Databricks, they use us 10 times or 100 times, sometimes more or even more, to read stuff and then produce very little text or very little data out.

Whereas people are focused on that output that comes out.

It's usually much, much smaller.

So, you know...

Underwriting is another one that we're seeing.

There's so many going through all the SEC filings and figuring out what's happening, extracting signal.

So figuring out alpha, we see it in all the financial sector.

So there's so many use cases in healthcare, electronic medical records.

There's just so many of them.

They're all handwritten.

Being able to actually find structure in that, that's a treasure trove by itself.

So use cases are on that.

So there's hundreds of hundreds of these you see all the time in every industry.

They just haven't been productionized yet.

They're not fully being used by the workforce.

Do you think that read to write ratio, do you think that stays around that?

Are there any trends that you sort of see as read going up and comparison to write?

Or you think at some point the writing and the creation catches up with the reading and summarization.

I mean, hard to see that because how much are we going to read as humans?

I mean, there's only so much and we read slowly, right?

Whereas having the AI crunch a lot of information So it has that context, makes a lot of sense.

So if anything, that probably increases.

And if we can do that faster, so that's where the trend is headed.

We're seeing even more and more of that when we look at our data than when we started.

As you counsel customers, be it big enterprises or smaller ones, about figuring out where AI and automation can exist within their organization and how to continue to dive deeper into that journey?

Are there any takeaways or key points that you keep hitting that you feel like hey, if you could tell all customers this is how to set up a team or this is how to think about it, any frameworks or primitives or whatever that you think about?

First of all, you got to get your data strategy right.

A lot of companies now want to sort of skip that step and jump straight to the AI.

And the data teams have been at it, trying to get the data of their company organized, secured.

You know, get it in one place so they can actually get access to it, not in different silos.

And, you know, that has been a journey and it's not finished work.

That work has not been on the CEO's radar of the Fortune 500.

That's, you know, it's been sort of an esoteric topic.

Maybe IT was sort of... worrying about it.

Now the CEOs are waking up after ChatGPT and they're saying, I need AI.

It's game-changing.

I see it now.

I've used it myself.

Give me AI.

And they're sort of skipping that step.

So it's extremely important that you go back to the basics.

The data...

It's extremely important.

So you have to get that in a good place.

And if you don't have that in a good place, you're not going to actually going to be able to succeed with AI.

The second thing is focus the use cases of AI that you want to do on where you have a data advantage.

So for your company, what is your data advantage?

And, frankly speaking, oftentimes I find myself talking to customers telling them maybe don't do this project.

And I say, why not?

We're very excited about it.

We're going to do this and that.

We are ready to go.

We'll pay you a lot of money for it.

And it's like, yeah, but what's your advantage in this?

You know, there'll be an app for that by someone else.

So let's focus on places where you have a competitive advantage against your competition.

And it typically has to come back to something you have around the data and you do have it.

So let's focus on those use cases because we want our customers to be successful.

Only then long-term will we succeed as well.

So I would say that's the second thing I say.

The third is there's a lot of people problems in large organizations.

There's now many people that are fighting over owning AI in the organization.

Like I'm the person who owns AI at our company and so on.

Uh, so figuring out a way so that you can get the strategy aligned in large organizations is.

I know it's boring, but this is the hardest part of a large organization which so many different people running in different directions, having so many ideas of which vendor to use, what kind of models to use, what kind of people to hire, which project to do.

So if you can concentrate sort of the decision making to one person in the organization and not have lots of departments have political fights over it,

It streamlines and moves things faster.

Typically you actually have the head of that department being a change agent person.

It's someone that doesn't do things that everyone will love.

They'll come in and make the difficult decisions, streamline the data processing, the security around, the privacy around it, because everybody's worried about that and then focus on those use cases that actually matter for that organization.

This is as important as a technology.

I care a lot about technology and geek out on that, but this is just as important.

Projects will get not funded, or they get stuck in security or they go nowhere because different departments are fighting each other.

So we need to overcome these things as well.

This is why I'm saying it's going to take a few years before we see the impact of these things.

As you think about the landscape and the term that you used earlier to sort of talk about, was it compound?

Yeah, compound.

Compound data and all that.

Agent systems.

Yeah, compound agent systems.

How do you think about what areas to potentially pursue?

There's a bunch of venture companies, companies in spaces like inference or evals or whatever it is like.

As you think about different areas to potentially go into for Databricks as a business, how do you evaluate no pun intended, I guess those markets to potentially pursue?

Yeah, it makes sense.

Look, it's just for us, it makes a lot of sense if you already have all your data with Databricks.

You already trust us that it's secure.

You know that we can handle the security, the privacy.

So the question is, what's really adjacent to that?

And for me as a CEO where is it we can actually bring extra value to the customer that they otherwise couldn't get elsewhere?

Typically comes back to the data that they have with us already.

So typically comes with governance around that data. with guardrails around that data.

You know, you don't want to build, this actually happened with a customer.

They had a chatbot and if you asked it, what's the best, you know, car I could buy?

And if this was a car manufacturer, it recommended a competitor's car, you know?

So how do you put guardrails around that?

How do you deal with PII data?

How do you deal with AI regulation that's coming?

A lot of this we can standardize in Databricks.

So it makes sense.

We can automatically produce documentation So you're compliant with the regulation that's coming.

We can put guardrails so that it doesn't do those kind of things.

And since we have the data, we can create this data loop so that you get this flywheel of when they use the model.

We create the evals and automatically feed it back to making this model smarter and smarter, and doing that.

It seems we can do that because they already have the data with us.

So it's just advantageous for us to do that.

I think in this space, like for startups, they should focus on the applications.

I think applications are going to be where...

A lot of value in the AI space is going to accrue in the future.

We'd love to be their infrastructure.

So they use us for their data processing, for their governance, and for their AI.

The applications, I mean, there's a lot of creativity.

There's a lot of things that you're seeing happening.

I can't predict what's going to be the apps that are actually going to win in the future.

It's kind of like in 2000 trying to predict Facebook or Twitter.

No one could have done that.

No one...

Ever would have said oh you know, there's going to be a company that does you know?

140 character tweets.

It's going to be the same thing now.

So we just want to be that infrastructure for them, so that they can build their AI systems on top of us.

And we help them with that data firewall.

If we back up a little bit to the original early days of Spark.

So we talked about the Netflix competition and that.

I had heard that maybe you tried to actually sell the technology in the early days and there weren't really any takers.

Can you just go back to that early story?

What did you see and what was your path to getting it off the ground into some commercial entity?

Yeah, I mean, what was really happening is that we were seeing that the Silicon Valley tech companies were leveraging AI.

This is 2009 or 10.

We were seeing what Facebook was doing.

We were seeing what Google was doing.

And it was very clear that doing AI on the data is going to be the most important thing.

And at the time everybody was stuck on doing basic analytics, backwards looking analytics, call it BI, business intelligence, which is backwards looking.

Like, how was my revenue last week?

And they were excited that they could do that really, really fast.

But we were asking, hey, what about, what's your revenue going to be next week?

AI.

And there was just not much appetite for that at the time.

And we went around and we tried to push the technology out because we just wanted to do research.

But it was very hard to actually get anyone to actually see this information.

You might remember the phrase, it was the Hadoop wars, Hadoop days.

It was impossible to get anyone's attention for that stuff.

So it was kind of a little bit out of despair that we started the company and said look, if we don't do it ourselves, no one is going to do it.

I remember actually one of our first customers said, We said, hey, this is open source.

Just take it because we don't want to make money on this.

And I said, no, you got to charge us for it because we want an enterprise that's behind this.

And that's kind of when we realized that, OK, there needs to be like an enterprise business.

No one else is going to do that.

We probably have to do it ourselves.

Was it an issue of just pure focus for these vendors?

Or was there something from a data structure standpoint that needed to actually happen to have the market evolve to the point that they actually were able to do a lot of the things that you wanted them to do?

I think there's these hype cycles that come and everybody's just focused on something.

So that era was that Hadoop hype cycle and they were obsessed with it.

And we couldn't for the life of us tell us AI is more interesting.

It's just, you know, you open the newspaper and everything is just about that.

And it's happening now too, right?

It's like, I think the market works like that.

And to create a successful company, you have to kind of bet on something that's non-consensus at the time and you have to be right about it.

So for us, it was AI.

It was the cloud.

It was open source.

Those were the things that we were excited about.

There wasn't much excitement around that.

And actually, we were told that this is not going to work.

All the data is going to be on premises, open source.

You cannot monetize it, and so on and so forth.

So I think it's similar now, right?

Were you intellectually?

I had heard a story that you, that a customer was going to offer you I don't know if it was seven figures or eight figures or something to build an on-prem version of the product, and you rejected it at a point in time that maybe you could have used.

Yes.

Now, less so.

But at that point in time you definitely could have used the seven or eight figures or whatever it was.

Did you just see this as an inevitable path you were building towards and therefore anything but that was a distraction along the way?

Or what was that philosophical decision?

Yeah, 100%.

Like...

Our view was we have to have an asymmetric bet on something that no one else believes in, and we better be right about it.

That's the only way.

So if we're going to go on-prem and do what everyone else is doing, then there are way bigger sharks that can do that than us.

And then this is going to fizzle out and it's not going to work.

So yeah, it was a month or so in to my CEO job 2016, that one of the largest banks told me you need to bring this stuff on premise.

And I said, no, we're not going to do that.

And they said, well, how much?

I said, how much what?

They said, how much would we pay you for it?

And I said... we're not going to do it.

And he said, I have 5 million.

I said, we're not going to do it.

And he said, 10.

I said, we're not going to do it.

He said, 20.

I'll give you 20 right now.

And at the time, 2016, you know, I guess we ended the year with 10 million revenue that year.

So, you know, 20 million revenue was a lot.

But the idea was like, if you go on-prem, there are much bigger companies.

So we have to have an asymmetric bet that we believe in and we continue to double down on.

And then, once that bet becomes true, we'll be big enough that it's too hard for others to catch up with us.

Similar thing, as I said, is going to happen in this AI and LLM era right now.

I think you have to pick something that you think is different than the rest of the market and then bet on it, And then hopefully you're right.

As you reflect on the journey.

One of the stories that I had heard was giving a lot of credit to Ben Horowitz and actually turning this into a business.

Can you tell that story?

Typically I don't allow other VC plugs, but I thought it was an interesting one that he actually was pushing you guys to think big and turn this into a commercial product.

Yeah, I mean, look, we came out of the research lab.

We really wanted to raise only two, three, $400,000 seed and just code away for a year or so.

And he's the one that pushed hard.

No, let's do the Series A and I want to be exclusive investor.

And you know we were actually not very excited about it.

But then he gave us a price we couldn't refuse.

And he said, no, go big.

And we gave us... you know, 14 million at the time, which is a lot of money back then.

I know it's not much these days.

And he said, go, go, go, go, go.

So he really pushed us, go aggressive, disrupt the market, think big.

You know he always was a couple of orders of magnitude bigger sort of vision for Databricks than we ever had.

Like, whatever was our biggest grandiose vision of Databricks, he always was a couple of orders of magnitude ahead and was pushing aggressively, saying no no no, you're thinking too small, like way too small.

And I think he really truly believed it as well.

So that kind of set the tone from the early days for us.

Because the early founders.

I don't know if you remember these numbers, but you guys, you remember the actual numbers.

So what were the bets for the earliest?

Well, I mean, we were around the table when we started the company and said hey, what's like a great outcome for the company.

And I think when we were around the table the numbers were something along the lines of like 150 million exit for us, or 200 and someone said 250 and so on.

Uh, so that's how we were thinking.

And Ben was immediately like, no, no, no billions for sure.

We're going to do.

And then when we got to that, you know it was like this is easily going to be a hundred billion dollar company.

And you know now where we are.

Now he's like definitely a trillion is what you need to think about.

Uh, and you know, I would also give credit for Mark Andreessen in one of the meetings.

I think we, early days we said, um, He said, what's your biggest bottleneck?

I think this was 2017.

I said, hey, we can't hire engineers.

We can't get the engineers from Facebook and Google.

They're just too expensive.

And he said, huh, so Fang is the problem?

And I said, yeah, Fang is the problem.

And he said, okay, so you just need to add yourself to that list.

It needs to be FangDB.

And that's it.

And then he moved on.

But after that meeting, we're like, yeah, okay, he kind of is right.

We should actually just offer up and, you know, hire those people and become like one of those tech magnets.

And it changed our mindset.

Like we went back, we changed our compensation.

We changed how we hire just based on that one comment there.

I'm curious, at a tactical level, I mean, that's very aspirational.

It sounds great.

But in changing the compensation or thinking bigger, what did you then go do?

We literally went back and moved the compensation for engineering to 90th percentile of the market.

We did the math and we saw how much is it going to cost us.

Are we going to be able to do this?

Is it something that we can afford as a company?

What's the dilution going to be?

This was a long time ago.

And actually when we ran the calcs we were surprised that actually we can do this and it's sustainable.

And it was surprising to us.

We're data companies.

We crunch the data and all the numbers.

What we found is that it's really your market cap divided by number of employees.

That's how equity rich you are when you're paying someone equity.

And these companies have a lot of employees.

So then it turned out, actually we are actually richer than we think if you take our market cap and divide it by the number of employees.

Because that's not how you're doing financially.

That's not about your revenue growth, but that's about what could you offer someone in equity.

And it turned out that actually it's absolutely within our reach to do it.

So let's explore it a little bit.

And it started working.

And then we were able to hire the people that we couldn't hire before.

And they came in and they completely revamped what we were doing with the product.

We just could increase the feature velocity. and the innovation cycle at Databricks massively.

And that actually helped us kind of slowly start separating from the pack.

Because in 2017, if you had talked to us, we would have been one of the startups among hundreds of other ones out there.

So these small things kind of that they did to push us, think really, really big, go way bigger.

And we did that, but we always kind of did it judiciously.

We didn't want to kind of... go crazy and, you know, burn down the bank account.

We wanted to be careful about it.

Databricks was founded in 2013.

And for the first couple of years, you weren't the CEO.

What were you doing through that the first couple of years in the journey?

And then what was the impetus for you to step in and become the CEO?

Yeah, I was running product and engineering, which I loved.

And actually in that phase it made a lot of sense, because what do you do the first two, three years of a company?

When you start a startup, you want to get product market fit, build something that just works.

So that was the focus for us.

Just how do we get... one customer that wants to use this.

And then how do we get a customer that wants to repeatedly use it?

And then how do we get a customer that repeatedly uses and pays us for it?

And so that was the first two, three years of the focus.

The technology that we had innovated with, with Spark, exploded in 2015.

And at that point, we felt like we had succeeded, but we just didn't have any monetization.

So we went through a difficult time in 15.

We had to figure out, okay, if we can't monetize, maybe there is nothing here.

And the company kind of went through a little bit of sort of introspection.

And then we said okay, let's kind of do a little bit of reset of some of the things and rethink some of the stuff and start anew in 2016.

So I became CEO in 2016.

And we made a bunch of changes that actually started helping us with monetization.

Initially, were you the interim CEO, or were you the CEO and they were still talking to other potential outside candidates?

Well, I thought I was Well.

First of all, I heard through the grapevine that they're interviewing people.

So I found out, okay, they're actually interviewing other external folks.

Were you the CEO at the time?

No.

But they were like, hey, do you want to be the CEO?

I was like, sure.

And then I found out, okay, they're actually interviewing other people.

All right, they didn't tell me about that.

But then eventually I was like, okay, I think I am the CEO.

They've announced it to the company.

I am the CEO.

I'm running it.

But then I kind of... you know, my compensation didn't change.

And I learned kind of soon that, Hey, I'm actually interim CEO.

I didn't know that.

You didn't have the title until.

Yeah.

So then it turned out, no, no, no, actually it was just a test period.

And then eventually I would say after one year um, I kind of I think they really kind of like okay no, we're going to go with him.

So the first, when you came in in 2016, there were three things that you kind of focused on.

And it sounds like monetization was one of them as well.

But can you talk through what those three were?

I think it was up leveling the management team, sort of changing the focus of the customer type and then monetization.

Yeah.

Enterprise sales.

Total pivot into enterprise B2B.

We were PLG.

I know it's still hot.

Everybody wants to build PLG companies.

But we learned the hard way after two, three years that PLG doesn't really work.

Might work for...

Very few companies.

It's a great thing.

We still do PLG.

But guess what?

If you want to sell a data platform.

We want to manage the most sensitive data of a large organization.

You need to get the buy-in of that organization.

Nobody's gonna just swipe a credit card and start giving their data to you and you're gonna succeed.

So PLG just won't work in that model.

In B2B in general, it's hard.

If you're gonna have big contracts with large enterprises, PLG probably don't work, does not work.

So that was number one, going all in on enterprise sales.

We had tried earlier to just be this PLG, product-led growth.

The second thing is up-leveling the management.

So we up-leveled all the exec team.

Like every person on the exec team was new.

So I basically got a new exec team that had much more experience.

They knew what they were doing.

And a little bit went more with experience over.

Just hey, we're just going to be smart PhDs that can figure this out on our own.

We went instead with...

Someone who has done sales for 20, 30 years, or someone who has done marketing at scale at Oracle knows how marketing is done and so on.

So that really helped also with execution because I would say we were really weak on go-to-market.

We were great at product innovation research.

So those really helped.

And then a change to how we actually innovate in open source.

Just open sourcing everything and giving it away is not going to work.

So we need proprietary secret sauce that we're not just giving away to the hyperscalers.

So that was critical also in changing.

And that's difficult to do.

If you're a pure open source company you come from academia it's difficult to get all your employees to be all in on that.

But it was the only way to succeed.

We wouldn't have been around if we hadn't done that pivot.

Did you have those views as you were running product and you sort of came in prepared to do that?

Or was there some point of introspection that ultimately led to those conclusions?

Yeah, I think 15, we were not doing super well.

And in 15, it was sort of like, okay, we are trying PLG.

It's clearly not working, right?

Yeah, we don't want to do enterprise B2B because we're not excellent at it.

We don't have experience in it.

But, you know, we ended 2015 with 1.6 million revenue.

2015.

So if we only have 15 16 million revenue from what we could glean from blogs and whatnot, an enterprise salesperson can sell a million dollars, a million and a half of software.

That's just one AE.

Why don't we hire three of them and try it out?

What do we have to lose?

So that kind of emerged in 2015.

The other thing was it was very clear that, no matter what we did in the innovation cycle side, the hyperscalers immediately had the same thing.

They would just pick that.

And in fact, they would add a little bit of stuff on top of it.

And then they had the whole platform, the whole cloud.

So they were always the good enough vendor.

So the customers were like, well, you're great.

You have great innovation, but I can also get it from the hyperscaler.

I'm going to try that first because I already have a contract with these folks.

I got security approvals.

I've already paid them a bunch of money.

Try that first.

And if it's not good enough, I'll come back to you.

So it became pretty clear that we have to be able to articulate then yeah, you can go over there, but you won't get XYZ if you go over there.

Like we have XYZ that they don't have.

And so if you couldn't articulate that, then there was nothing you could give your sellers to sell.

And then, you know, it was the experience that we were lacking.

So I think throughout the hardships of 2015 kind of made it pretty clear that we have to try something new.

And these three directions seem to be pretty viable.

I mean the one that stands out the most as like potentially a one-way door, is the licensing or monetization shift.

And it's very different than the culture, I would say, of the people that are very open source purists in some way.

Was that...

Was that a debated decision in a very meaningful way or was it just kind of declarative?

Hey guys, we have to do this to be able to build a business.

I think when I took over, I was just a little bit more forceful in what needs to be done.

But it was big debate internally.

And I made it pretty clear.

I think I used the analogy and I said, hey, we have a gun and it has one bullet left in it.

We're not going to do a thousand things.

We have like one bullet left.

We better aim it really clearly and we better hit the target.

If we don't, we're done here.

And so I made it pretty clear that this is existential, the things that we're going to do.

And I think when I kind of got people to wake up because you know, in the startup days you're excited, you're doing stuff

But if nobody makes it clear to you that hey, we're actually going to go out of business in a couple of years, it's not all great just because we have a big valuation.

I think people got in line and started thinking about okay, what are the ways we can actually do this?

Did they quickly start showing up in the business or did it take a few quarters?

No, I think pretty soon the engineers started thinking and I made it very clear to them is that this is great that you built this.

It's literally available now also on the hyperscalers cloud.

So what's the differentiation?

Literally.

And I made it very viscerally clear to them and I shared it.

We had an extreme transparency culture to make it really clear.

So get them to buy and let them understand themselves.

That we can't win.

We're guaranteed going to lose if we continue this way.

And, you know, they kind of soon got it.

They want to be successful here.

So they started coming up with innovations that we're going to this.

Maybe we don't need to open source.

Now, as you think about expanding into new products, I've heard you actually will set up different teams for each product as it gets going.

What have you learned about product management and organization as you continue to scale and iterate on these different product areas?

Yeah, I think that when you build a product zero to one, you kind of have to do the same thing that you did in the original, kind of first two, three years of the company.

So there's a lot of processes in any company once you're over 150 employees, let alone thousands.

So you got to cut that and say, look for new products, You don't need to satisfy all the security stuff.

You don't need to have all the processes.

Have a small core team that iterates really, really quickly.

Figure out, first of all, who is the sort of innovator type that you have in those early startups.

Put them on the project.

These are people that are not necessarily good at when the company scales and they're leading mature products.

You have to have one such anchor person that you put everything around and you put them in charge.

They start building that product and then you let them iterate really quickly.

So get them off the iteration cycle that the big company goes on, but get them on this much quicker, smaller one.

And, you know, encourage them just iterate inner dev loop, outer dev loop.

That's super, super fast.

And just focus on getting users and getting product market fit.

Once that happens, you can kind of move it back into the mothership.

So, you know, ideally even do it outside of the whole product if you can.

So we have a product right now called Genie, which lets you just you ask an English questions and it can answer immediately.

It produces, it understands all the semantics of your data.

We call it data intelligence.

So it understands your data and it then programs and ask the questions and give you just straight up the answer.

But to know that it's not hallucinating, you can actually see how it coded it up and behind the scenes, and so on.

That product, we actually just built it completely outside of Databricks code base.

And it didn't satisfy any of this stuff.

We didn't build on anything.

We just iterated quickly.

Just do that outside of the norm, like a startup almost.

And then only when we saw that the results are really good and people like it, we went through the kind of painful process of okay, let's rewrite it.

Let's move it into the mothership.

So it's almost like an acquisition at that point happens.

And you have to integrate it into the whole system and make it secure and so on.

So you'll give people autonomy to make the decisions on not the shared primitives necessarily.

Or if you guys are using some type of I don't know single sign on or whatever cloud structure underneath, you'll get people full autonomy to do it.

And then you'll come back and reconcile it once it actually finds product market fit.

Literally do it outside in a separate repository.

Do it on a separate GitHub repository outside of the big system and iterate very quickly is the most important thing, because that's what you're doing as a startup.

As a startup, you operate three to four times.

I've seen it.

We've done acquisitions.

And if you measure it, the startups move three to four times faster. than a company like Databricks.

And I think we move very fast and we're very innovative, but still it takes us three, four times longer.

Why is it?

It's the security processes.

It's the legal sign off.

It's making sure that it works on all the clouds in all of the environments.

And the biggest thing that slows you down is that you have thousands, of thousands of customers that you can't regress.

So you got to unshackle the new products.

Otherwise, you're just going to move too slowly.

And that's what we're seeing.

Honestly, they're really, really big companies.

They operate that way.

It takes them too long.

That's why you have startups in the ecosystem.

So you have to kind of go back and reproduce the kind of environment that you had when you had a small startup.

I'm curious.

So now you're coming up on, I guess, nine years ish at the CEO job.

Is that right?

As you reflect on, like your leadership style today, you sort of seem like the canonical leader of Databricks and it's like the job that you've always done.

I assume leadership style has kind of evolved in some ways over the course of this period of time.

Are there anything that you wish you could?

Anything you could wish you could go back and tell your earlier self taking the CEO job for the first time, or maybe your earliest founder self, when you're sort of stepping into the seat?

Yeah.

I mean, I think that you would want to grow leaders earlier on, like build trust with few leaders that you groom and continue doing that.

Because what you find nine, 10 years later and I have a lot of awesome co-founders that are still active and they run the company where it's almost a partnership, where we run it together

Uh, But it's not that many people like you know, even though we're almost 8000 employees, how many people are there that are really can really help you build the business together.

It's hard to find those leaders that build your trust with them and, you know, grow the company with.

So I would just invest more in that, much more actively, and always be on the lookout for those kind of people in the company.

You know, we were just busy executing and we didn't pay attention too much to that maybe.

So that's one thing I would do differently.

Um.

The second thing I will do differently is um, when these trade-off decisions come up and it's really hard to decide, do we do, do we go on prem or not, or whatever it is.

Um, Make a bet that is much more for the future.

Don't worry so much about the implications that it has on our finances or on the revenue.

Don't let short-termism creep in.

Easy to say now, but in the early days, when you're, it's far from clear that you're going to be successful as a company.

And, you know, the VCs are asking you, how's it going?

You have your board meetings.

They're looking for your revenue targets.

They are super obsessed.

How's that?

Is it hockey sticking?

Is it going up?

What's it looking like?

How do you compare with others in the industry?

Continue making the bets on the future and don't get carried away on those numbers.

That's another advice I would give because we took many shortcuts that...

They were not worth doing.

We kind of knew it, that this was a shortcut, but it was like, it was worth it.

Because, you know, this is going to bring so much revenue if we just take this shortcut.

I would have avoided many of those shortcuts now.

You know, err on the side of the future than the present.

Now, of course...

Maybe it's path dependent and I'm saying it just because we're where we are.

If we had done the things that I'm saying we should do, we wouldn't have succeeded.

But I still kind of, on balance, feel like we could have been more futuristic in our bets than we were.

I get the feeling you focus on empowering individual leaders within your organization and having them deputized to make the best decisions for the organization, rather than you serving as a central hub for each department.

Is that a fair character?

I think it's both.

I think I also micromanage them.

Maybe they would disagree.

How do you think about the tension between a central figure that's able to make definitive decisions that sort of sit across an organization, versus more decentralization and empowerment at an individual level?

You get more clarity if it's one person and it's more consistent and You know, that's great.

The disadvantage is that it slows things down and it bottlenecks things.

And it's maybe less empowering.

So you have to strike a balance.

You have to do it in places.

Like you know, you build trust.

Over time, you figure out which are the parts which you can completely delegate and they're going to do an excellent job and you know they're probably better than you at it, so don't pretend that you're better than them.

And what are the area areas where you really need to deep dive?

Um, and then there's some areas that there's things that only you can do as a CEO.

Like you know, it's literally tie-breaking.

It might sound silly, but over time, overlapping product areas appear and you get leaders that are kind of building similar-ish things and the Venn diagram intersects and If you leave it to them, no matter how many times you tell them you know, figure out how to unify this.

Do one.

We're not going to have two similar offerings.

They're not going to be able to do it on their own.

So you've got to step in and push.

So you've got to use the CEO card judiciously.

I think that matters a lot.

And you can build a much simpler product.

So I think, if you want a really simple unified product that makes sense, one person makes much, much more sense.

You just have to watch out for bottlenecking things and slowing things down.

Why is a willingness to say the same thing over and over again an important part of the CEO's job in your mind?

Because I think people are not paying attention to what you're saying.

Like, you know, we all have busy lives.

We're paying attention to our own life.

You know, I didn't even notice this thing that you said.

I wasn't even there.

Or maybe you said it and I didn't really rock it.

I was on my phone and I had actually a fire that day and I was having other issues that I was dealing with.

So I didn't really internalize it.

I didn't really get it.

So many, many times you tell someone something for a third time and then only then you notice that they are like you know, and then they repeat it back to you as if it's like they came up with it.

That's great.

I think it's just you got to repeat it.

But what I try to do is figure out a way to repeat it in different ways.

Don't repeat it the same way all the time.

It gets boring and old.

Challenge yourself and come up with a new angle.

Find a customer that says the same thing that you want to say.

Or use case that's a little bit different.

Use different slides.

Make new slides.

And you'll be surprised because, you know, people didn't really get it the first time.

They'll say, oh, this was completely awesome what you just said now.

Even though you're like well, it's literally what I said like a month ago, but I just refurbished it a little bit.

So I think it's super important.

Being truth-seeking, I think, is one of your values.

Is that right?

How do you implement that in practice?

Or how does that manifest itself as a cultural practice for you guys?

Make sure in the hiring process that you test folks on truth-seekingness.

It's not that hard, actually, to ask someone, tell me, what's your biggest screw-up?

You know, tell me when you really made a mistake.

Who did you really not hire well?

Why did you make that mistake?

How was it?

And so on.

Push them, push them, push them to see, you know, until they get uncomfortable.

Do they start to come up with lots of excuses?

Well, I really actually didn't want to hire this person, but my boss made me hire them, and that was a bad hire.

I knew it, but yeah, that was a mistake I made, but technically my boss's mistake.

So, like you know, keep pushing them and you can in the interview process actually suss out if someone is truth seeking and honest or whether they're trying to a little bit, you know, make it look much nicer.

You know, so that's, That's number one.

Number two, don't promote the people in the company that are not truth-seeking.

Like at least, if you made the mistake of hiring them in, then at least make sure you don't promote them.

So in the promotions of the company, make sure that you're really evaluating the cultural values.

Like really do it.

It's easy to have the cultural values, but are you evaluating the promotion?

I have a rock star.

Rock star is doing great.

They moved the needle on the business.

Should they get the promotion or not?

You don't want to be the person saying, well, wait a minute.

Can we like look at if they were truth seeking?

Oh, they're not.

Let's not promote them.

It's like, are you out of your mind?

This person had huge impact on the company.

Yeah, but they're not truth seeking.

But if you mean it, if you want that to be the culture, you have to actually do that in the promotion.

And then, of course, if it's extreme case, we might have to part ways.

You have to do that as well.

You can't have people around that don't fit the culture.

So that's what I would say.

You spend 30-40% of your time on hiring, is that right?

One of the things you said is that when you're hiring executives, you'll just find reasons to keep in touch and find a bunch of different touch points.

Can you maybe talk through that as a philosophy or how you go about doing it?

Yeah, I think actually the bigger philosophy is that I don't believe in interviews.

Because I think some people interview well and they might not be good at all.

And some people interview really poorly, but it might be excellent.

So I actually just think it's a very noisy channel.

You can't, there's not that much value in it.

So then what can you do?

One of the things this is the one that you mentioned, but there are other ones that we use as well is I try to actually have them do the job.

And I think we think, oh, you know, in programming, we can do that.

Ask them to program this thing and then if they do it well, but we can't do it for any other job.

And I disagree with that.

Just if you're hiring someone for head of marketing, work with them to fix something that's broken in marketing right now, whatever it is.

If it's a personnel problem, or if it's the webpage, or if it's the product marketing positioning.

Work with them, call them.

Call them you know in the evening, call them you know in the morning.

Ask them.

Go through it like work with them.

Basically make them do the job. as if they were already an employee in the company.

And you will have a very good sense of culturally how they are.

You know, are you in the same wavelength?

Can they actually do a good job at this?

The kind of stuff, especially pick the things that are not working right now in your company, because those are the things you would give them to fix when they come in.

And if they could fix those things, they're clearly a good hire.

So I'd rather do that, then coming in in an interview and just asking them the gotcha questions that they might have practiced.

Tell me your own weakness.

What's your biggest weakness?

And so on.

So that's one of the big things that we do.

The other is backdoors.

And I believe you should do the backdoors upfront.

So don't go through a lot of firms, especially recruiting firms.

They will run you through a cycle.

They have you meet all these people.

And then at the end, they want to just check the box.

By the way, they'll even do the references for you, which is a big mistake.

I think you should do the references up front.

I already know who you're hiring.

So that the first time I meet a person, I already know a lot about them.

I've already asked.

And I've also found this thing that if you do five to 10 backdoors on people their bosses, previous bosses you actually kind of know the person better in some regards not in their full entirety better than they know themselves.

Because you know all the things they went through.

And you know the perspective of three, four people that saw that happen to them in the previous job.

They might not actually fully comprehend what everybody thought about that situation.

Of course, they have a unique perspective.

So I think the backdoors are extremely important.

And then the other is just look at their trajectory of work.

Where were they?

Have they seen great?

Have they done something stellar?

Are they hopping around a lot?

You know, those are the three things that we kind of emphasize in hiring.

What are your techniques on the backdoor references?

Are there specific questions you like to ask or ways that no one likes to talk poorly about someone they know?

So how do you sort of drill in to make sure you're getting the actual high quality information?

Yeah, it really depends on who you're talking to and they're all different.

You can tell immediately.

Some people are not just going to tell you the truth.

This is going to be.

You know they're going to just say good things or they don't want to say anything.

So you know, that's that.

So you just move on and end the call.

Uh, but uh, and i guess i'm revealing my uh, you know, uh recipe here, so it will never work again.

But um, i think that If you can find the form of person that you know opens up, I think one thing that I like to grill them on is really, really ranking of how good the person is.

And don't ask, you know, score from one to 10 and they say 10.

Really ask them, like, does it rank number one, number two, how many have you?

And then I want to go really deep.

Who did you, who's number one, who's number two in your list?

By the way, you go after those as well, of course.

And then I really asked them, you know, go through scenarios.

Like it's just a follow-up questions.

Like what, what did they do?

Well, tell me something that went wrong.

Give me a scenario.

What was that scenario?

Really push the boundary.

You got to kind of almost to the point that they become uncomfortable.

So many times I've done it so far that you know they call back and say like you know, this is, you know what's going on here.

Uh, But yeah, you'll find when you're doing the backdoors that okay, that guy and that gal, they're like super forthcoming.

That's a straight shooter.

They're going to tell you the whole thing.

And others are not.

Front door references.

The references that the candidate gives you are similar but kind of inverted.

There, you know, the vast majority are actually people that the person is going to hire next.

If it's a manager, they're just literally giving you there.

And that's useful too, because you can interview those people and you know who they're going to hire next.

So you can gauge the quality of the people that they're going to hire.

But occasionally you find the front door that also tells you the real truth and tells you like really, what's going on.

And you learn a lot there too.

And when I do these references, I'm not looking for.

Hey, what's your one fault that I'm going to cancel?

You know that I'm not going to hire you for.

It's just to get a complete picture.

You're going to have faults.

You're going to have strengths.

We want to understand that full picture before we go ahead and hire you.

So that's kind of what we're going for.

But yeah, for some execs, I've done 15 of these backdoors before I hired them.

As a leader, you've had to ramp up in different functional areas, one of which was enterprise sales.

I don't think you had experience to it.

How did you go about learning in that area?

I think you copied maybe best practices or sort of internalized stitched together to build your own model from a bunch of different organizations.

Maybe you can talk through learning a new area or discipline and how you went about it.

Yeah, but first of all I would just say, you know we hired Ron Grabisco who you know took us from essentially zero to now over 2 billion ARR right.

Basically 3 billion ARR.

And so he's been phenomenal.

And what did you see in him?

Yeah.

I thought that he was extremely smart.

So I thought he's actually different from many of the salespeople.

He is the classical enterprise B2B sales guy.

You know.

So he's not like the engineer type, that's.

You know, he is the classic archetype of a sales leader, you know, with all the rah-rah and motivational speech and get everybody riled up.

So it's like he has all that.

So that's great.

That's the stuff we didn't understand or have.

He had the big enterprise experience because there's all sellers that sell to mid-market SMB but he had actually sold to really big, gigantic enterprises.

So that's also something we don't understand anything about.

He was thinking very big.

Like when he came in, You know it's like oh, we're going to do a 10 million contract or 100 million.

We're like, no one's going to ever pay $10 million for Databricks.

It was our mindset.

So that was also useful.

But he also was, you know, he was cerebral and he was extremely smart and went by first principle.

So there was a compatibility.

He had an engineering degree.

So he was sort of the unicorn of kind of someone who like, spoke our language and couldn't understand us and was extremely smart but also had this whole other thing that we couldn't understand.

So that hire was important.

We took us a year to hire.

We went through a lot of candidates and I think we kind of nailed that one.

It was luck.

Um, so that's important.

Um, the other thing I would say is in the, in the interview process, when you're searching like we took us a year to hire a salesperson um, take the I've seen people make this mistake take the exact hiring process super seriously.

Um, It's a learning process.

You're basically going to school to learn what a head of sales looks like.

So in that, talk to all the top head of sales and grill them and pay attention and use them also in the interview.

Since I don't believe in interviews, as I said, use the interviews a little bit of a tutorial for yourself to learn how does excellent sales look like in your organization, for your company, for your company, for your company.

And you get the after you know, talking to 20 people, you get a pretty good picture of okay, this is kind of it.

This is what they're doing.

They all have that.

All the stellar ones seem to have this in common.

This is what it's about.

And you can go deeper.

And then in the backdoor process, you figure out who's who.

You figure out who are the top people in this industry that you can't hire.

Try to get some time with them as well, to just learn from them and build a relationship with them, or get them to be advisors.

So really put yourself in the, hey, you're going to university to learn.

That's what that process is.

You know, you're not an expert and trust the people that are the best in the industry at that.

Try to find the number one person and go with them.

In hiring for Databricks, are there certain things that you feel you uniquely prioritize or the types of people that succeed within the organization that is maybe unique to Databricks?

Yeah, well, so our culture principles have been reverse engineered to exactly answer your question.

So you didn't start with the principals and went the other way.

You started with the people that succeeded and came back.

Succeeded and failed.

Okay.

We also looked at people that had perfect CVs that we hired and they bombed out.

Why was that?

And we went back and said, okay, we couldn't agree on the hiring bar.

They were like, I've worked with these 10 people, they're stellar.

And we're like, no, that doesn't look like our bar.

That was big issue.

So raise the bar, don't settle.

We'd rather have more false negatives than have false positives.

So we'd rather reject someone that's really great by mistake then hire someone by mistake.

That's not great.

And that's painful.

We learned that that way.

Truth seeking.

A lot of the execs from the big companies had this attitude of, you know, Everything's going great.

I got this.

My team's got that.

We're working on this.

Here's a nice slide deck.

Manage up.

Nothing to look for here.

But that didn't work for us.

It's like that's where the truth-seeking principle came in.

We really need to know what's broken.

Who in your team is not working out well?

What are we not doing well?

Where are we not satisfying the customer?

So that was the second principle.

Third principle, I've done this before.

I know how it's done.

This is how a much bigger, better company is doing it.

We should do it this way.

So that was not first principled.

We needed to know exactly how and why and go from first principles on every decision.

So being first principled became another culture principle.

So our culture principles are the recipe for succeeding at Databricks.

Like execs ask me all the time at the end, you know, before just we shake hands and I hire them.

They say, hey, so do you think I'm going to be successful here?

What do you think is going to be needed?

I mean, it's like, it's super simple.

I swear to God, read the culture principles.

If you really do those, you will be.

I mean, of course, the results have to, the output has to be great.

But you will work out here.

If any of this stuff doesn't look good to you, let's argue about it now, because that's going to be an issue.

Those are really the principles of the company.

So our culture principles aren't like the aspirational cool stuff that you'd like to have on the wall.

It's literally what worked and didn't work in the past empirically.

When looking for people within or to come in Databricks.

I think people that have a chip on their shoulder a little bit.

Do you, is that something you can, do you try to tease out in the conversations?

What are the types of things that you'll ask to try to figure that out?

Yeah, I mean the people that you know really really, really went against all odds, really pushed.

They have grit.

One of the things that Ron said in the sales you know, interview that kind of I remember now from 2015, I think November was he said hey, if there's a deal lost, you know, I want to know, like because there's no way anyone else on the planet could have won that deal.

Like.

I just know it's humanly impossible, because I will have done everything humanly possible to win that deal.

So I just want to, and it was a side comment he was doing with something.

And it was like clear to him that this, this guy is going to go over and beyond.

And then of course the negotiation with him to hire him was a real pain.

So that was also reaffirming that, okay, he's probably going to be great at this job.

You don't want a head of sales that's terrible at negotiating their own comp for themselves.

So you want to suss out.

Is there something people who have something to prove?

Are they trying to prove to you?

If they're super content and happy with things, you know, then it's probably not going to work.

It's probably for people that are a little bit broken like all of us are.

Like we're all, we have something to prove and we're going to work our ass off.

We're going to do everything in our power.

And you can suss that out in interviews.

It's not hard.

You mentioned work your ass off there.

That's something that I think has went out of vogue working hard for a while.

It's sort of come back in vogue.

I'm curious, you had said something along the lines I had here.

You never met someone successful who didn't work hard.

Can you elaborate on that point?

In research, in academia, like genius professors who had won amazing awards, And that's like you can be really, really smart in research and maybe you don't need to work hard.

But all of them that I met, they were working harder than us.

And we're like shocked.

They're there 24-7.

And then in industry, the same thing.

So I just never, ever met someone who's just phoning it in and they're super successful.

You find talented people, but they're not successful.

It has to go with extremely hard work.

So we had that from day one at Databricks.

And we never deviated, whether it was in vogue or not.

We were a super hardworking culture from day one.

We continue to be.

And we said, look, we have lofty goals.

To achieve them, we're going to work extremely hard.

Totally understandable if you don't want to do it.

Because maybe you have family, maybe other things you want to do in life.

That's totally understandable.

But then probably Databricks is not for you.

So it's been always true here at Airbox.

In fact, we had, you know, our work-life balance scores weren't great in surveys.

And, you know, I tried to be upfront.

We don't want to actually have terrible work-life balance.

We actually focus on teams that had terrible scores and try to up... you know, change things.

And we got actually improvements in those teams.

But I also was upfront.

I don't want to be best in class and have the highest work-life balance score in the industry.

That's not, we're not going to win that way.

Um, so it's as simple as that.

Now I tried to myself also match that.

So I don't want anyone to work harder than me.

Like, so I got to kind of match that myself.

Um, And it has to be sustainable.

What I mean by that is, I've been doing this for a decade more, and I want it to be the same way for the employees.

So if they're like really burning themselves out and they can't do it for many more years, that's not.

I don't want them to do that.

I want them to be able to do it and it's sustainable.

And then hopefully we can achieve great things together.

And if they don't, that's okay too.

No hard feelings, you know.

Or maybe in a period of their life they want to do it, but they don't want to do it another.

That's okay too.

One of the hardest things our companies deal with in scaling is when the people that got you there they're starting to question if they're the ones that can get you there to the next step along the journey.

It sounds like when you came in, you made that decision as CEO to up-level a lot of leadership.

And I'm sure along the way you've...

You've kept Ron and he's continued to scale in his role, but I'm sure there's other functional areas that you've decided to upgrade.

How have you gone about making the decision that hey, it might be time to part with someone who's done a great job up to that point, but you might need someone new to take you to the next level?

I think you have to just be truth-seeking with yourself.

So number one is like when do you know yourself that you know?

I think we kind of know there's the signals, but we just don't want to see the truth.

We want to ignore it because it's so hard because this person helped build the company with you.

It's...

So you don't want to kind of admit it to yourself, but deep inside you know deep deep deep, deep inside.

So just immediately when that is there, just recognize it and face the truth.

Be truth seeking with yourself.

Don't like yourself at least as an exec.

Okay.

Once you know it yourself, now you have to take action.

I found it easiest if you can explain to the person and get them on board that it's not really working.

Don't make it about like, just make it like, is it working or not?

Are you winning or not here?

Are you winning or not?

Just focus on that.

And if they realize that, hey, okay, I am not winning.

Well, let's discuss why.

Then I think everything else becomes much simpler.

It's like, okay, but then what do we do about, what about you then?

So after, you know, like, okay, I'm not winning.

It's not quite working.

There are many reasons.

Okay, let's figure out a way to make this work.

Then you can change, steer the conversation towards okay, what's the best thing for you, your career?

I want to help you.

I want you to be successful.

I have found that there are new roles for people where they can continue to crush it.

I have had people that you know they in that role.

It wasn't working anymore and we found a new person that leads it, or you know you do the layering or whatever.

But you find a new role for this person and they crush it and they're awesome at it and they'll be happier.

Actually, you know, after a while goes on, they'll be like hey, Ali's no longer bugging me, you know about it not working.

Second, I'm actually happier when I'm succeeding.

When I'm winning I'm happier than when I'm constantly it's not working and everything is going.

You know, I'm having a bad day every day.

So I've actually found that many of those folks, you can keep them in the company.

But you should also part ways if they're like just phoning it in and they're not interested anymore and they're just there for some memory of what happened in the early days, when everything was great.

You're better off also parting ways, too.

Let's not waste our time and our time.

Maybe zooming back out as we wrap, if someone's listening to this in a couple of years' time, and we're sort of looking forward with your crystal ball that you claim not to have, but I think you're pretty good on it.

Are there elements of the world around, Databricks or what you guys are operating that you think are on the cusp of being possible in the next few years, five years?

Just things that get you really excited to get out of bed every day and do this job, and just the potential, maybe at an industry level not even at a company level that you think we're on the cusp of seeing.

Yeah, I think what's going to happen is that we're going to see many of the things that we see today are going to be disrupted.

And there's going to be a sort of AI app for those things in the future.

We don't even know what they're going to look like, you know.

But what do they all need at their core?

Like what did apps need in the last 10, 20, 30, 40 years?

They needed an Oracle database under the hood.

The joke is you stuff all the complexity into a database and then you build workflows and UIs on top, or the two-tiered or three-tiered model.

I think in the future, you need the AI database under the hood.

And if you have an AI database, you have all your transactional data, you have all your analytics and AI in there with the model, and then you can do wonders on top.

So I think we're going to see an ecosystem of these applications in the AI space being built.

And they all need an AI database under the hood.

And that's what basically Databricks is.

Databricks is basically AI database.

We don't call it that.

We don't market it that way.

But essentially, it's an AI database under the hood.

And you will need that for every application.

Every company on the planet will need that.

And I hope that we can be the company that powers all of these applications in the future.

There'll be others also that do that as well.

But that's the goal.

And I think we're seeing already the seeds of this happening.

The seeds of it, I guess I'm curious just to elaborate on that.

Like the seeds of this happening?

Are you seeing just particularly exciting use cases, or just customers?

Look in the same space.

They need to do massive log processing securely and detect zero-day attacks.

OK, that's kind of what Databricks bread and butter is real-time processing of massive logs with AI built in.

In the observability space, same thing.

You need to observe your log processing, understand the metrics, detect anomalies, and so on.

In the customer 360 space, CDPs and understanding what your customers are doing.

Same thing again there.

It's like massive data that you have to collect from many, many different places and then get telemetry on it.

It's huge data processing.

You need to do it in real time.

You need to do AI to get the insights in doing those.

Each one of these spaces, I think, are going to be built on basically an AI database.

I hope we can be the AI database that powers all of them.

This disruption, I think, is going to happen in the next two, three, four years.

It's exciting.

Well, thank you for doing this.

Likewise.

Thank you for joining this episode of The Logan Bartlett Show with co-founder and CEO of Databricks, Ali Godsey.

If you enjoyed this conversation.

We'd appreciate it if you shared to anyone else that you think might find it interesting, as well as subscribe to us on whatever podcast platform you're listening to us on.

We look forward to seeing you back here next week with another great guest on The Logan Bartlett Show.

Have a good weekend, everyone.

[The Future of AI and the Rise of Compound AI Systems: A Conversation with Databricks CEO Ali Ghodsi]-[EP 124: Databricks CEO Ali Ghodsi Breaks Down the AI Hype-Cycle]

📋 Summary

The Evolution of AI: From Niche to Mainstream

The Shift Toward Compound AI Systems

The Data Advantage and Specialization

Leadership and Operational Philosophy

The Future: The AI Database

🎯Key Sentences

📝Key Phrases

📖 Transcript

[The Future of AI and the Rise of Compound AI Systems: A Conversation with Databricks CEO Ali Ghodsi]-[EP 124: Databricks CEO Ali Ghodsi Breaks Down the AI Hype-Cycle]

📋 Summary

The Evolution of AI: From Niche to Mainstream

The Shift Toward Compound AI Systems

The Data Advantage and Specialization

Leadership and Operational Philosophy

The Future: The AI Database

Deep Learning with ListenLeap

🎯Key Sentences

📝Key Phrases

📖 Transcript

ListenLeap Brings You Into Real Context Learning