#37 Annie Duke: Getting Better by Being Wrong｜The Knowledge Project

So I wanted to ask you about what your take on the morality of gambling is.

Oh, I wasn't expecting that question.

Hello and welcome. I'm Shane Parrish, and this is another episode of the Knowledge Project.

The show explores ideas, methods, mental models, and frameworks that will help you expand your mind, live deliberately, and master the best of what other people have already figured out.

You can learn more at FS .blog slash podcast.

My guest today is former professional poker player Annie Duke.

Annie and I first met in Philadelphia where we sat down to record this conversation.

Her book, Thinking in Bats, is about how we can make smarter decisions when we don't have all the facts.

In this deep dive conversation, Annie offers insight into poker, the cognitive biases that lead us astray, and she offers a masterclass in decision -making.

Enjoy. Annie, welcome to the show.

I want to start with someone who was originally on track to become a PhD in linguistics of all things.

Writing a thesis on something called syntactic bootstrapping, no less, ends up as a professional poker player.

That's a hell of a switch.

Well, first of all, I'm impressed that you dug up the syntactic bootstrapping bit.

I usually just say cognitive science or language learning.

Yeah, so I kind of had a winding path.

I went to graduate school.

I was studying syntactic bootstrapping, which for those people who are interested is the theory that children learn their first language because they have an innate grammar built in.

If you can access the grammar, that's what allows you to bootstrap the meanings of the words that you're hearing in the language stream.

That's what I was studying.

Right at the end of it, I'm talking five years in, on my way to my first job talk actually at NYU, I got sick.

I had been struggling with some stomach issues that year and I got really sick right as I was going out into the job market and actually landed in the hospital for two weeks.

Because the job market in academics is seasonal, I missed that season.

I was going to have to wait until the next year.

I went off to Montana where my husband at the time had a house and I was going to recuperate for a few months and then head back to Philadelphia to finish up at Penn and go back out on the job market.

It's during that time that I started playing poker mainly out of a need for money.

I thought, well, okay, so I'll do this in the meantime for money.

In the meantime, just happened to turn into 20 years.

This is a family business for you guys, isn't it?

It is a family business.

I think that nowadays people think, oh, I can see how someone would end up thinking they should go play poker because nowadays it's on television.

Everybody knows that poker is a thing that you can do for a living because they see it on ESPN.

But for me, at that time, this was in the mid -90s, it was not on television.

You were never going to come across somebody who was like, hey, I'm a professional poker player.

It was a very strange thing for me to end up doing because it wasn't something that was out in the public sphere as a way to make a business.

I think the reason I started getting into it was because my brother, when he was a teenager, had gotten really into chess and into chess like big books studying openings and moves and so on.

He found a grandmaster that he could study under in New York, had gotten into Columbia, decided to defer a year to go and study with this grandmaster before starting college.

In that year, he got connected into the games world in New York.

Chess and backgammon and poker, they're all very interconnected.

He started playing poker.

In that first year, he lost his college fund, which was about $6 ,300, but righted the ship pretty quickly.

By the time this was all happening with me, he had already been playing for 10 years and had actually made the final table of the main event of the World Series of Poker.

He figured out how to play the game really well and guided me into it as a suggestion for how to make money and then became one of my key mentors.

What were those early days of poker like for you?

I imagine there was a really intense kind of infatuation when you started, given you decided to make it your life, something few people would have the courage to pursue, especially in the time context.

It's not as prestigious then, as you mentioned, as it is today.

What's the opposite of prestigious?

Unglamorous? It wasn't even just that it was unglamorous.

I used to joke that the conversations that I would have with people, they would say, what do you do for a living?

I would say, oh, you're a dealer.

I would say, no, I'm not a dealer, I actually play.

Then there would be some variety of ways that the conversation would go.

They would ask, they would often ask what my husband did because they were curious what I actually did for money in order to support this horrible evil gambling habit that I had.

When I explained that I was actually the main financial support of the family, they really wouldn't believe it.

Very often, it would devolve into, have you thought about gamblers anonymous?

It wasn't just that it wasn't prestigious.

I mean, you were really on the fringes of society and people didn't even believe that it was something that you could do for a living.

If I felt that I really wanted to get them to understand, I would say, well, don't you think that people who are investing in the market that they can actually make a living at it?

Yes. Well, what I do is very similar.

It's just a different kind of market and then they would again ask me if I'd been to gamblers anonymous.

I might as well have been telling them I'm a professional back route player or something.

I don't know. It was interesting because it was an interesting mix.

The game was amazing for me.

It's this really incredible exercise in constantly revealing that you know less than you thought you did five minutes ago.

As you get better and better at the game, you realize you know less and less about the game in terms of the percentage of the things that there are to know about the game.

You're just essentially - You're just thinking about poker specifically, not the people at the table.

No, this is poker. You're essentially just pulling back the curtain on how incredibly complex this game is.

I think that's really fascinating.

It's this really fascinating exercise in how do you slog your way through such an uncertain system?

Outcomes and decisions are so loosely linked in that game.

You have the best hand, you get crushed, you have the worst hand, you win sometimes.

There's so much hidden information.

The cards are face down.

How are you supposed to figure out, kind of get into the process and figure out and sort of separate yourself from these outcomes that are happening and figure out how am I supposed to move forward and learn from the feedback that I'm getting because this feedback is so incredibly noisy.

Then within that context of all that uncertainty, trying to come, trying to figure out what is the optimal play, I mean I can just sit in that kind of problem forever.

You have like two minutes at a poker table.

Yes, well you have to do it very fast and then you have to sort of try to figure out after the fact, after memory has gotten in the way.

It's a really fascinating problem and I felt like it was kind of this coming together of all these issues that I was studying when I was in graduate school about how you learn in this really practical kind of real world and time pressured environment that was super cool.

I totally loved that and I think that I got really infatuated with that.

Sitting in a basement bar in Montana, that part of it was not so fun but the game was definitely worth it for me.

Let's talk a little bit about how your study of psychology influenced your poker playing.

I mean how did you learn, how did what you learn in that world carry over to the poker table and gambling?

Well here's something that I think is really interesting is that I actually have lots to say about that now but I think that that all revealed itself to me later as I went along.

I think that it particularly revealed itself to me when I started talking on these topics that it really eventually ended up as Thinking and Bets is the book.

I started talking on these topics in 2002 and I started realizing how much of what I had done in graduate school had informed what I was doing at the poker table.

Now one of the interesting things is that when you say that you studied psychology or went to graduate school in psychology people think that you're talking about clinical psychology and so people actually think oh you must know a lot about people's personalities in the sense of depressed or I'm going

to do psychoanalysis on you or something like that but most of what psychology is is actually in this space of experimental psychology.

Clinical psychology is a very small slice of it and I don't know very much about clinical psychology.

What I know about is cognition.

So what I realized is that all of the work that I'd done in really understanding how people make decisions, cognitive bias, how do people construct mental models of the world, how do people learn and really on certain systems, which is really what I was trying to figure out with syntactic bootstrapping

is how do you create you know sort of what's the choice architecture there that narrows down the possibilities enough for a child that they could actually conceivably you know pick a word out of the sound stream and figure out what it means which is actually a really hard problem.

So I realized that all of that thinking that I was doing in that space was actually really informing the way that I thought about poker because I really approached it from this decision making problem.

How are you constructing a model of your opponent?

How are you figuring out what the right strategy against that is?

How for yourself are you picking the signal out in this very noisy stream of stuff that's happening?

I think that that was very helpful and I think it really actually speeded up my learning that I had that background.

I have a couple of follow -up questions from that.

The first one is kind of like what did you learn at the poker table about psychology that's not covered in the textbooks that you were learning in grad school?

I think the key thing that I learned was that it's not as simple I think as they kind of make it sound when you get your psych one textbook.

So I think every psych one textbook has a sentence that's something like this.

Learning occurs when there's lots and lots of feedback tied closely in time to decisions and actions.

And pain. And pain, yes.

You have to have consequences.

Right. And I think that this is a pretty, I mean obviously that's a pretty sort of Pavlovian or Skinnerian view of the world.

You put a rat in a box and it runs a maze and at the end there's some cheese and then it figures out that it runs the maze faster.

And so it's this decisions, actions, outcomes, rewards, or punishment or pain.

I guess we could look at it through that lens as well.

And that that's when learning occurs, when you've got lots of that stuff going on.

And I remember, I mean just before I was ever thinking about it in any explicit way, I did have this puzzle immediately as I sat down at the table.

I was playing with people who were much older than I was, most of them were probably 40 years older than I was, had been playing for that much longer than I had.

And had really clearly stalled at some point in their learning about the game, probably about 39 years before that and had persisted in these strategies that, I obviously I had the help of great mentors that could reveal to somebody who was that new to the table some of the strategic errors that I was seeing,

but that seemed pretty obvious and they were getting feedback for it.

I mean, they weren't winning.

And as I continued to play poker, I noticed this over and over again, that people really persisted in their mistakes despite the fact that there was very clear feedback that was very quick, and there was a lot of it.

And it was painful because they're losing money.

And it was painful, exactly.

And yet they weren't learning.

And I think that that was the thing that really confused me because it is so instilled in every person who's studying psychology.

I mean, it's in your Psych 1 textbook right there.

Here's this thing. Actually, a place I just saw it was in Richard Thaler's Misbehaving.

And he's got it right there pretty early in the book, like this sentence about how you learn.

The theory on how we learn.

Yeah. I want to talk how you learn, but before we get to that, you mentioned mental models.

What do you mean by mental models?

So when we're talking about mental models, there's the world out there.

There's some objective, let's agree that there's some objective reality.

And all we have is our beliefs about that objective reality.

So what we're doing is constructing a model of the world.

So for example, we can think about something that's simple and very scientific is that in my head, I have this idea about gravity and how it works.

And that's a model of why things stay on the ground and two rocks fall at the speed that they do and whatnot.

And so that would be a model of how that works.

That's in physics, but we have models of the world as well.

We all walk around with mental models of the world.

So that's what we're trying to do is create a model of the world that predicts for us, predicts what's true in the world.

We're constructing what we think the objective truth is.

It helps us to predict things like people's actions or if I turn right, I'm going to see this particular thing here.

So you can think about it.

We all have a theory of gravity that's running around in our head, that's some sort of model of the way things work out in the world and what's true and what's not true.

And so that's what we're always doing is modeling the world so that we can act in it with intention.

How do we create better mental models?

Or I guess maybe this comes to learning in terms of updating our existing models so that we get the feedback like those people who are playing poker and losing, but they never seem to get the message.

What was different about you?

How is your learning process?

Well, I think that I was really lucky in the sense of, well, I mean it's kind of interesting because I wouldn't have become a professional poker player if my brother hadn't brought me into it.

And my brother had already landed in this group of incredible players that among them have many, many, many world championships.

I mean, they all went on to be top performers in particular, someone named Eric Seidel, who has a bunch of World Series of Poker bracelets.

I mean, a big poker champion.

He's earned $38 million playing poker.

He is amazing. And he's also one of the most rational and clear thinkers that I have ever come across.

And so I would say that I was very lucky to be plugged into that group.

Of course, it's weird because I wouldn't have been playing poker if it weren't for that group.

So take that for what you will.

But I think that they really speeded my learning up because they really taught me some things about how to sort through this system.

So I think that the thing that gets in the way of most learning, and I think that you can see this pretty clearly at the poker table, is something that you said, which is the pain.

So it's this double -edged sword.

It's like a catch -22.

You need the pain in order for it to matter to you so that you'll learn.

But the pain actually gets in the way of learning.

And why is that? Because when you're losing, it just lights your limbic system up.

I mean, it does not feel good to lose.

And what particularly doesn't feel good about losing, when you're playing in a game or doing anything where you could possibly attribute your losing to your own decision.

So these are decisions you've made based on your mental model of the world, based on the beliefs that you have.

That makes us, that's really, we view as an attack on our identity.

So if we think about, for example, Kahneman talks about this.

He says that we're all trying to create this positive narrative of our life story.

Dan Kahn talks about identity protective cognition, that this is really how we're processing the world, is to protect this identity that we have.

And with the pain, which is the loss, our identity gets attacked if we're going to attribute it to our poor decision making.

Right? So we just default to attributing it to luck because there's an element of skill and luck here.

And that way we can mentally get off the hook.

Well, that's basically what ends up happening.

So the pain is the double -edged sword.

We need the pain. We need the, well, I guess a very popular phrase at the moment is skin in the game.

We need the skin in the game in order to learn in this particular case.

But it's that very fact that you're losing maybe because of your own decisions that doesn't feel good to us.

That makes us feel like our identity is being attacked.

So now we have a choice, right?

We can say, well, in the long run, I have to take the pain because my long run goal is to become a better player and to learn.

By the way, whatever it is, whether it's poker or sales or business or relationships or, you know, ordering better food at the, at a restaurant, you know, going to the gym, whatever it might be.

So we have to be able to say, okay, I'm willing to take this short term pain, this, this feeling that doesn't feel very good because I know in the long run it's going to help me learn because I'm going to go in and I'm going to examine my decisions.

I'm going to see where maybe I could have improved, where I could have made a better decision that would have increased the likelihood that I had a better outcome.

So I'm going to trade that off.

I'm going to take that short term hit.

But in the long run, I'm going to, I'm going to feel better about myself and, and, and I'll obviously have a more positive narrative of my life story over the long run if I'm willing to do that or that choice too is I don't want to feel pain right now.

And it seems like people really take that discount.

They are totally willing to discount the, take a discount in the present in order to preserve that self -narrative to take a huge discount in the present.

And what they do is exactly what you just said.

Well, okay, if I don't want to take the hit and I lost, what am I supposed to do?

Oh, right. There's a lot of poker.

I lose all the time when I have the best hand.

And I know that I have the best hand, because the cards are face up.

I can see that I had the best hand and a very unlucky card happened.

So I can just, I can just start pawning it off to bad luck that it, you know, the game was really good and I was playing great and everybody else was playing really poorly.

And I just had a really bad run of cards.

Now what you've done there is you've socialized the result to the world.

You've offloaded responsibility for this bad outcome to the world.

Now in the short run, that feels good because you don't need to do any kind of like identity update.

You don't need to update your beliefs in any kind of way or say that those beliefs were wrong or that you made poor choices or that you caused these things to happen, but it's devastating to learning.

Talk to me about the role of Eric Seidel in your learning.

Yeah. So getting back to, you know, why I feel so lucky that I got plugged into Eric Seidel.

I had a conversation with Eric Seidel very early on.

So my history of Eric, because I met him when I was 16, long before I was playing poker, when my brother first started.

Because he was a friend of your brother.

He was a friend of my brother's.

And he was in New York and I met him because he was playing in these games with my brother.

And so that was just, you know, I knew him just as, you know, my brother's friend.

Ten years later now, I become a poker player.

And now I'm encountering him as, okay, I would like you to be my peer, right?

So now it's different.

It's like, oh, you're not just like my buddy where I've become your friend through my brother, but now I want you to be my peer.

I want to sit at the table with you.

Right. Mentally. Like I, obviously anybody considered a poker table with anyone and I had sat at, I want to mentally sit at the table with you.

So I remember going up to him after some tough loss and, you know, saying something to the, just something to the effect of, oh, you know, I can't believe I lost.

I was, I was playing so well and that guy got really lucky and I got unlucky.

And, you know, the kind of things that you say, I was telling him basically what you call a bad beat story.

And it usually goes something like I had aces and then you can fill in all the blanks too.

And then a guy hit a lucky card and I lost.

So that's a bad beat story.

So I was telling him a bad beat story.

And these are, if you go into a poker room, you'll hear like a gazillion of these in the space of about one second.

And his response was very different than other people's responses, who I had told such bad beat stories to.

With other people, generally they nodded and sympathized and then told me a bad beat story back.

That's kind of an issue.

Almost validate your.

Yeah. Like we're going to make an agreement, right?

I'm going to get to tell you my emotional hard luck situation and then you'll tell me yours and we'll make each other feel better about how unfair the world is to us.

And now we can both walk away feeling pretty good about ourselves.

What was interesting was Eric was going to have none of that normal social contract and he was actually really harsh with me.

And what he said was, I'm going to paraphrase, but it was something like, why are you telling me this story?

Like, do you think I really want to hear this?

Like, okay, I've lost with a lot of hands too.

I have heard so many bad beat stories.

I have no interest in yours whatsoever.

If you really lost because of bad luck, what's the point of the story?

There's nothing that you can learn from it.

Now, if you have a question, like if you want to talk about strategy, like I don't care whether you won or lost a hand, like we'll sit down and talk about strategy, but don't just come up and tell me about bad luck.

You're like literally wasting my time and it's annoying.

I mean, I think he actually told me I was being annoying.

And it was funny because I read him the section in the book where I talk about this particular conversation because I just wanted to like clear it with him.

I was like, oh, by the way, I just want you to know that I've got this section in the book.

So let me read you what I said.

I think I softened it slightly for the book, and he said, oh, I hope I said that.

That's awesome. He was very excited.

So that's kind of you getting out of your own way.

I want to talk a little bit about kind of like the peer group and the influence that that had on your learning.

I mean, there are studies that show that peer groups are super influential on your learning.

And if you're in a crowd of people who are blaming luck or refusing to update their mental models, you'll be on a vastly different trajectory than people who are constantly seeking to update them.

Tell me a little bit about the influence.

All of these, I presume mostly males at the time, all of these guys had on you.

Yeah. So I think that, uh, that Eric Seidel really exemplifies what you need in a good group.

So the conclusion that I come through to in the book, not by the way, on my own, but because this is the general consensus in the research is that on our own, we're, we're just biased and we just have this very natural tendency to process the world in a way that supports our priors.

And even if you know about it, even if I tell you that you're going to do these biased things, you'll still do them.

In fact, there's a lot of evidence that says that smart people who know about them are actually worse because they're overconfident, they're prone to taking risks that they shouldn't be taking.

Or why is that? Well, I think that there, I think that it's a few things.

I think one is that once they know about the bias, then they're overconfident.

They're overconfident in thinking that they're avoiding things like overconfidence.

Um, but two is that the smarter you are, the better you are kind of slicing and dicing the data, um, in order to fit whatever your prior is and you can slice and dice it to tell a story that doesn't sound biased.

So you'll say, say things like, you know, I was really trying to think though, if this was just me rationalizing my prior belief and I've come to the conclusion, no, and here's why.

And when you're, you know, when you're really adept at that kind of spin, which you have to be more intelligent to be able to do, you can actually cut the data in a way or present, present it in a way that really fits the story that you want.

You're just better at it.

Um, and it's not that you're lying by the way, you're just better at fooling yourself.

So, um, you know, when you talk to somebody like, uh, you know, Phil Tetlock, for example, I mean, he, you know, he's just very skeptical to John Hite.

They're all very skeptical that you can do much about this on your own, um, and that you really need to be in a good group in order to overcome this kind of bias.

And I think that the re it's, it's very actually intuitive to think about why that is when you're looking at other people, I mean, how often are you saying, wow, you're just really biased.

I can't believe you came to that conclusion.

Like you're completely missing this point, or you're discrediting that person on purpose, or you're giving too much credit to this person, or you're clearly overconfident.

It's so easy to spot in other people and so hard to spot in ourselves.

So if you take that, that, well, okay, it's pretty easy to spot in other people.

Um, now, well, what if we got together with some other people and let's make a deal, right?

Instead of the deal of, I'll tell you my bad beat story and then you tell me yours, what if our deal is I'm going to spot your bias and you'll spot mine and we'll have each other's back.

So to take kind of Tetlock's, uh, Phil Tetlock's way that he sort of divides these kind of two ways that groups can interact with each other.

One would be a confirmatory style of thought.

So that would be the, I'm so sad.

You agree with me. You tell me you're so sad.

And we both agree that everybody plays poker horribly and we're really great.

And everything we believe about the game is true.

And when we lose, it's cause of bad luck.

So that would be a confirmatory style.

You can see this in a lot of political thinking, right?

So, um, just pick your example.

They're all confirming each other.

Um, but the other kind of style and the style that Eric Seidel was demanding of me, what is an exploratory style of thought, which is not, we're trying to confirm what we already believe and sort of think about how great we are, which in a group form is the same as that identity protective cognition.

It says now we have a group of people who are trying to protect it.

Right. Um, and that will actually cause people to move even more extreme.

It's like interacting with a bunch of clones of yourself and you sort of become yourself on steroids, um, in the extreme.

Um, now what we're doing instead is in a group form saying, no, we're not going to tolerate that.

And what we're actually going to try to do is try to create a more accurate mental model of the world that we're going to agree that there's some sort of objective truth.

We're all going to agree that our beliefs are in progress and under construction in some way.

And that our goal is to help each other try to construct better models of the world and included in that is that we understand that we may disagree with each other and that's okay, that we may actually have different values.

So the right answer for you might be different than the right answer for me.

Um, the example I would give for poker is that strategies that might work for Eric Seidel, uh, might not work for me because I'm a woman and he's a man.

And so the perceptions of us were very different at the table.

So for example, his ability to bluff in a situation might be really different than my ability to bluff.

So we might come to different conclusions about, um, what the right strategy there was because we were different people.

Um, so essentially, if you can get the group, I mean, Eric really put it really well in that he, he sort of told me what the contract was that I had to agree to in order to be able to talk to him.

Um, so when he said, uh, you know, is there a point to the story?

He's telling me that my goal has to be accuracy.

So what he's saying is I don't want to hear you reason to be right.

I want you to reason to be accurate.

To be right would just be, I'm going to affirm my priors to be accurate would be to try to take these beliefs that are in progress and try to move them toward whatever the accurate, you know, an accurate representation of the objective truth was.

He told me he was going to hold me accountable to that.

That if I wanted to come and talk to him and have an exchange with him, that I was going to have to come to him with a real question.

And he was going to hold me accountable to the way that I was processing information in the world and check me on it.

And that I had to be willing to listen to him disagree with me.

And that was all, it was such a nice, concise way to put that.

And when you look at, you know, for example, like Phil Telach's work, you can see that this is what he, his research has shown you need in these groups in order to move the group away from this confirmatory style of thought into this exploratory style of thought.

That must have still been hard though.

Like, how do you, how do you go from your default?

I mean, the human default of explaining things away to being part of a group and as part of that group, going through what I would consider to be a lot of reflection or pain or mental labor in terms of updating your views and constantly being told, here's what you don't see about you or here's what you're missing

in this situation. So yeah.

I mean, first of all, when Eric said that to me, I was really sad.

Like, I think I almost cried.

It was horrible. Like I felt so rejected in that moment.

In part because you wanted to be a part of this group so badly.

Yes. I wanted him to approve of me.

I want, I wanted him to think that I was worth talking to, that I had interesting things about saying to say about the game.

I wanted him to engage with me.

And I think that right there is the key in answering your question of how you, how do you deal with the pain?

And I think it, it depends on what you decide the goal of the game is.

So I think that a social group can act as the best piece of cheese ever.

Like, if we go back to the rat going through the maze, getting you to think that I'm worthwhile to talk to, that feels really good.

So Eric, my brother, Dan Harrington, you know, these, these people, David Gray, John Hennigan, these people who I really wanted to think well of me weren't going to tolerate this type of talk for me.

So I needed to go to them with questions.

And now I would go to them and say something like, well, I won this hand, but I think I really butchered it.

And it would be like their face would light up.

I mean, it would be the opposite of what happened when I went up to Eric with the Bad Beats story.

Now it was like this huge smile on his face and total engagement with me to talk about that.

So what happens is that when you set up that kind of group agreement and you really live it, the group is actually reinforcing those kinds of behaviors.

So, you know, you can think about habit formation, right?

It's that you need to have this reward and the reward could be this short term feeling of, okay, I just don't want to feel bad about myself that I was wrong.

It's hard to have a reward that's in the long term.

It's hard to say, well, my reward is that, you know, five years from now, I'll be such a better decision maker.

And no, we want a reward now.

Like, I don't think that we should try to short circuit that need for some sort of immediate gratification.

It's just, what are you getting the gratification from?

And if you set up a really good group, the gratification is from the group.

They're giving you approval for that kind of thinking.

So now all of a sudden being a really good credit giver gets people to, you know, reinforce you.

Being a good mistake, admiter, like it gets people to reinforce you, you know, not telling Bad Beat stories.

And then, you know, what's really, really wonderful about that is that because I knew that if I wanted to have a really deep conversation about poker, which was the game I loved with someone who I really admired, when I was playing, I had to process what was happening knowing that this conversation was going

to occur later. And so it really helped me when I was away from the group to be looking at the world in a different way through this eye of like, well, I want to have interesting things to ask later.

Right. And being part of this group became psychologically more stimulating or rewarding than what you would be doing on your own.

Not only that, let me just add something to that.

I think that, you know, there's all this great work that's done, been done, you know, particularly recently on tribalism and tribalism as it relates to politics.

So there's a great paper actually that just came out recently from J.

van Bavel and Andrea Pereira out of NYU, which is talking about this.

And one of the things they say is that there's a few things that tribes give you.

But one of them is actually, it gives you the idea that you're different from other people.

This is actually an important thing that you get from your tribe.

They're obviously talking about it in politics.

And I think that that's important.

I do think it's important for us to feel like in some way we are different than the other people around us.

So one of the things that this group gave me was one of the things that defined that tribe that I was in that made me feel like I was different than the other people around me was when I was walking down the halls of the, you know, a poker tournament.

I was listening to other people say, I can't believe how unlucky I got, you know, and, you know, you're hearing this exchange of bad beat stories.

And then I'm going to my group and saying, I can't believe how poorly I played this hand.

Can we talk about it?

And that now is, it's different, you know, it's making me feel like I'm different, that somehow I'm doing, and particularly that somehow I'm doing something difficult, that I am overcoming something natural that other people do, and that that makes my, you know, this tribe, you know, special in some

way. Look, and I think that we all want to feel special.

I think that we are, that's why we are so tribal by nature is that we all do want to feel like we're different and somehow, you know, than the other people around us.

And that group gave me that, but in a way that was actually really constructive for learning because the difference was around this, this ability to kind of get past that initial reaction of like, Oh, I just got unlucky.

Um, and onto the more productive exercise of where did, where did I go wrong?

I think that's super helpful.

I mean, there's the saying that you are the sum of the five people that you hang around with the most.

And if you're hanging around with open -minded people, or you can create that circle for yourself or foster it or find it, uh, it goes a long way to you feeling comfortable admitting you were wrong, talking about your mistakes.

And, and I don't think, I think that even if the group isn't naturally that way, I think that if one person says, Hey, you know, I think that this will be a really good way for us to try to start interacting with each other.

And they actually create, you know, essentially an explicit, uh, contract around it, right?

That there, there's a charter for the group.

Like, let's have a group charter here.

Um, and here's what we're going to agree to.

We're going to agree that we want to focus on accuracy, not on being right.

And those two things are really different.

We want to hold each other accountable.

We want to make sure that as much as possible that we can disagree without being disagreeable and not be defensive about the whole thing.

And let's all agree to that.

Um, I think that you can move any group in the right direction.

And the great news is that you don't, you don't need to be anywhere close to perfect at this.

I mean, that's kind of what I was saying in the beginning about one of the things that I loved about poker was that the more that I played, the more it pulled the curtain back on how unsolvable this game really was, that there was so much that I didn't know about the game.

But what I understood was this little bit of progress that I was making made a really huge difference.

So I would never say that I didn't sit at plenty of tables and think about how unlucky I was getting and moan deeply in my head about the bad beats that I was taking.

And think about how, Oh, I can't believe that I so outplayed the other player and I still lost or whatever it was, but I caught it more.

It just never came your excuse either.

It did. Well, it didn't.

I mean, you know, there, the way that I kind of put it is, let's say that there is a hundred learning opportunities that kind of pop in front of you and left your own devices without say a good, you know, learning pod that you're plugged into.

Maybe you catch five of them.

Okay. So, uh, 95 missed opportunities.

Right. So let's say that because I'm in the learning pod, that 90 % of the time I still default to my natural mode, right.

That the mode that makes, um, people like, uh, John Heights, so skeptical that you can do it on your own, right?

So I'm still doing that.

Uh, that was bad luck.

And I'm just sort of shunting that off.

But if I'm at 90%, I'm catching 10 instead of five, think of what that will do for your learning.

So what I say is that having this group that you're accountable to later means that you're going to catch a few more and you're going to actually not only do that, but you're going to catch them more quickly.

So I think we've all, I mean, we can all think about, for example, the stupid decisions we made in college.

Yeah. Sure. Like, so you have all this time and space to now go look back on this decision that you made, but you've caught it so many years later that like, okay, whatever.

But I think that if you're, if you know that you're going to be accountable to a group later, now you can turn that into, I catch it a couple minutes later, right?

And that makes all the difference because it gets it in a window where you can actually do something about it.

So is your initial feeling going to be biased?

Is your initial processing not necessarily going to be the most rational?

Maybe. And that might be true for every single time, but how quickly do you go, Oh, wait a minute.

You know, there's another way to think about that.

And that's actually really irrational.

And maybe how fast are you getting to that answer?

You getting to it when you're 50 and you're looking back on your college days, right?

Or you getting to it, you know, five minutes later, or maybe when you describe the situation to the group, the group helps you get there.

You know, so I think that that's actually one of the really big values of the group.

It sounds like you learned a lot more than just how to play poker.

I mean, one of the things you just mentioned was that you learned how to disagree without being disagreeable.

Talk to me about that.

Yeah. So one, one of the issues that we have is that because of this identity, um, that we form, which is really based on the idea that we're smart and we're competent and our beliefs are generally correct and our mental models of the world are good.

And, um, you know, we're good decision makers and the beliefs that we have are true.

Um, when somebody disagrees with us, we can very often feel it as an attack on our identity.

And when that happens, I think that disagreeableness ensues because when you feel like you're personally being attacked, that the information that is being told to you is a threat, then you will react as if you are threatened.

It will cause you to be defensive or angry, um, dismissive.

Uh, it will cause you to discredit the other person or just decide that in general, they aren't worth listening to or they have ill intent.

Um, and we, we can definitely see this on our political discourse, right?

So, I mean, I think that generalization is really interesting that two people, uh, let's say that you have two people who have a disagreement about whether free trade is good for the country.

Um, you know, it's not, oh, well, we both think that we want the country to be more prosperous and maybe we disagree on how to get there.

That's a very agreeable way to have that conversation.

No, it's like you're a globalist pig and you don't care about, you know, the working man or you're provincial and closed -minded and you don't understand the value of economic theory.

So, and neither of the, you know, both of those are attacking the other person's identity, right?

It's not a question of whether maybe both of you have good intentions and are actually trying to reach the goal to the exact same goal actually.

So, it gets into this idea of sort of things about the other person and it causes you to be defensive and very dismissive.

It causes you to close your mind off.

Once you've made this agreement that, look, we're all agreeing that our beliefs are under construction, that they're in progress.

They aren't so entrenched in your beliefs because you aren't so, you aren't declaring that you're certain of those things in the first place.

Something that poker really teaches you, there is so much uncertainty in that game that in order to come out of it with any kind of emotional stability, you have to get really comfortable with the uncertainty.

You have to be okay with it.

And then what you learn, and I particularly learned from dealing with people like my brother and Eric Seidel, was that you also have to wrap that into the way that you think about your own beliefs and your own constructs.

That you have to be able to hear the other side because otherwise progress can't be made.

It kind of shifts your identity to a different place.

Your identity now becomes about acknowledging uncertainty, about how good are you at listening to the other side?

How good are you analyzing those things?

And it stops you from having that defensive reaction.

And I think naturally it causes you to be agreeable in disagreement because you view the disagreement as helpful now, rather than as a threat.

It completely reframes the disagreement.

So do you think of that internally in your head, like I'm 90 % confident in this or 85 % so that you move away from this?

Because most of us don't think in that sort of way.

And what we do is we think we're right.

And then in the face of being wrong, we explain it away through whatever mechanism is ego satisfying to us instead of updating our view.

But it strikes me that this idea of holding all of your beliefs as uncertain is a really good way to be like, oh, well, I'm not necessarily wrong now.

I can just update my view.

And that's almost ego protecting, but you're also getting a better view of reality.

I think that that's absolutely true.

And I, I mean, in part, because, you know, I was getting a PhD in science.

I think that that's much more the way that scientists are sort of trained to think, right?

I think that most people think that you construct an experiment to try to prove a hypothesis, but actually you're constructing it to try to disprove it, right?

So it's really approaching the world through this idea of why am I wrong, which I think forces you to understand that there's a possibility that you are, which automatically injects uncertainty into the beliefs.

And once you get into poker, obviously, there's, there's a lot of uncertainty and you can't see the other player's cards.

So you understand that you're always working within ranges of kind of guessing about what the other person might hold or about what the proper line of play is.

Now, people try to impose certainty on that system.

That's how you get to something like self -serving bias, which is, you know, every time I win, it's because I'm a genius.

And every time I lose, it's because I'm unlucky because we are certainty cravers.

We're really addicted to certainty.

My group forced me to really embrace it and, and really get my arms around it.

And I think that it's incredibly helpful now.

So this is the way that I was trained to think.

And I was so sort of entrenched in this world that when you ask me, is that the way you walk around with like some probability assigned to whatever you believe?

I'll tell a story that's actually in a chapter note in, in the book.

So it's kind of hidden.

I'm sure that most people have seen it, haven't seen it, but I was having lunch with Stuart Firestein, who's at Columbia.

And he wrote, he actually wrote an amazing book called Ignorance and actually another book titled Failure.

And both of them are about how great all of that is.

And he teaches a class at Columbia called Ignorance also, which is the idea that in order for science to proceed, we have to come from, from - Admit our own ignorance.

Admit our own ignorance.

Exactly. So anyway, we were having lunch in a Asian restaurant and I have, I'm like vegan and I'm allergic to gluten.

So like, it's a little hard for me to order.

And soy sauce has gluten in it.

So this is a problem in a Chinese restaurant.

So anyway, I was ordering and the waiter's first language was not English.

So I'm not speaking to a native speaker and I'm trying to explain like all the things that I need.

And it was a lot of sort of back and forth with the waiter.

And when he walked away, I looked at Stuart and I said, I think I'm 73%.

That was the total, some total of my sentence.

And he started cracking up.

And he said, I think you're a lot lower.

I'm going to put you at like 62.

But we both understood that what I was - Immediately what you were talking about.

What I was saying. And then he pointed out to me that he had never had someone say something like that to him about the probability that their order was going to come back right.

But this is something that poker players do all the time.

So like a very natural question for a poker player would be like, I'm having a conversation with a poker player and they'll, they'll say something like, you know, I mean, they could be saying something political.

Like, well, right now there's, you know, are they going to pass legislation for bump stocks?

So someone might say like, oh, you know, I think the bump stock legislation is going to get passed.

And I, my question would always be then, or their question to me, depending on who stated the belief would be, well, what do you think the price of that is?

So price is just a way to say probability, right?

And so then they could say, you know, I think it's three to two or 70 % or I mean, there's all sorts of ways to express that.

Let's get into decision making in poker.

But I want to start with kind of like, how do the stakes affect your decision making process or, and within that, kind of the context of being on television or not on television.

So let me start with the stakes.

The ideal in poker would be that the stakes don't matter except in as much as your opponents play differently depending on what stakes you're playing at.

So when you're playing at lower stakes, your opponents tend to be more beginners.

When you're playing at really high stakes, they tend to be more advanced.

What I used to say is that the players who weren't so good at the beginner levels or the advanced levels, they're both not good, but they're not good in different ways.

So at the lower levels, they tend to be more passive.

They tend to be doing more just sort of calling at the higher levels.

The players who weren't so good tend to be more raising.

They tend to be sort of overly aggressive.

So that's just sort of a general, it's not true of everybody.

Obviously we're talking about averages here, but on average that would be true.

So that should be the only reason that the stakes are affecting you is that when your opponents are making different choices, you obviously should be adjusting your strategy to try to come up with the optimal strategy against the choices that your opponent is making.

Now that's not what happens.

As you move up in stakes, it does change the way people play, in particular when the money matters more.

So when the money really matters to people, one of two things happens.

That's not a total amount of money.

It's an amount of money that matters to them individually.

Correct. Expressed as a percentage of the total money that they have.

One of the things you always want to be doing is we talk about playing within your bank role.

Which means that playing at a level where if you lose, it's not really affecting your overall situation very much.

People aren't very good at that because they're overconfident.

So people tend to be playing out of their bank role, outside of their bank role, way too often.

And then what happens is that they can very often become risk averse because they're just too worried about losing the money.

So one of the things that you want to try to do in poker is to separate the chips from the money.

You want to view the chips as tools to accomplish a particular goal in a hand and not view it as money at all.

Okay, that's really interesting.

They're like the tools of the trade.

So you're not pushing money across the table, now you're just pushing - I'm using these tokens in order to try to get you to behave in a particular way.

Oh, okay. Now, if you can do that in the ideal world, then you're much more likely to be making optimal choices.

If you can't because the money matters to you, then what will happen is that if you're kind of fresh to the situation, you'll often be risk averse.

If you get too emotionally lit up in the moment, then you might become too risk seeking.

In other words, you might be trying to sort of go for a double it up strategy, which is usually not a very good choice.

So it kind of depends on where you are in the arc of the game.

If things are going sort of okay for you and you're playing outside your bankroll, you're not going to be seeking risks very much.

And I'm going to be able to advantage that, that's the problem is that I can take advantage of the fact that you're refusing to take on enough risk.

But if you're actually losing and it's money, particularly money that really matters to you, now you'll very often start doing the opposite, which is taking on a lot of risk because you're just trying to sort of let it ride your way out of the, this really bad emotional state that I've gotten you into.

Talk to me a little bit about, uh, the role of emotions and how they affect your decisions, not only in poker, but I mean, in real life, but is there anything particular to poker that it, it highlights or accentuates?

So I think that, um, one of the reasons why poker is a kind of interesting place to look at the role of emotion and decision -making is because there's a lot of emotion.

And one of the reasons why is that, uh, one of the things that gets us really emotional is what is the most recent path that we've taken in terms of what our outcomes look like.

So what I talk about is that in poker, you have this advantage, which is that because we have this constant exchange of chips, right?

I make a decision, chips go either toward me or away from me that, um, you can see very clearly at every moment that every single decision that you make, even these little tiny executional ones actually have consequences that, uh, the executional decisions really matter to what your bottom line is because you're

seeing this, you know, you're not seeing sort of the end result, you're seeing the upticks and downticks, right?

So I think that that's, I think in general, that's good for making you understand this idea that every decision that you make has some risk to it and you should be treating it like that.

But it's bad from the emotional standpoint because you're getting yanked around by the ticker.

So if you, if you are holding a stock and you've decided that you're going to, it's a long -term hold, it's not good every hour to go look at how the stock is doing.

You're going to have like a really unhappy life if you do that, because when the stock goes down, you're going to be super, super sad, even though your idea is just to sort of have the trend line go upward and, you know, so I think I have a Berkshire Hathaway graph in the book of saying, look at Berkshire

Hathaway over the last 40 years and be happy, right?

But look at it on this 1130 on a Tuesday.

Now you're really sad.

Yeah. So what we're trying, you know, so, so poker really highlights this problem because you can't help but see the ticker.

It's not like you can say, well, I'm just not going to look at it and I'm going to see how I do at the end of the game.

And, you know, hopefully at the end of the game, I have more money.

You're forced to watch the ticker all the time.

And so it causes that emotional part of your brain to really light up.

I was thinking like another unique aspect to poker, but then I was thinking it's not unique is the TV thing, which is when you're on TV, I would assume that you're more likely to try to make decisions that other people can't criticize.

But I also think that exists within organizations, right?

Like we're often the, the TV becomes the organization, our peers, our colleagues, our bosses.

Talk to me about that.

So I think that what's really interesting is that when you know that you are being examined, this is when what I call in the book resulting really, really matters.

So that can be in an organization where how are we evaluating people as leaders?

Are we behaving like they're on television and their decisions are exposed to a critique, but not the decision process that we're going to be swayed by the outcome of the decision.

That's where the television really matters.

When I'm on TV and I'm thinking about the decision that I'm going to make and I understand that it's, it may have a bad outcome.

What's in my head is not, is this decision good or bad?

It's are people going to think the decision is good or bad because I happen to have lost the hand.

And this is the problem in companies as well.

So if I know as, you know, if I know that the person who's in the leadership role that I'm having to report to is going to be looking at the outcomes of the decision and act like I'm on television and all of a sudden do some critique when things turn out poorly, overly relying on the outcome in order

to derive the decision quality, it's going to make me, what you just said, make safer decisions.

Now, what do I mean by safer?

Decisions that are more bulletproof to criticism, more normal, more normal, right?

So it's an innovation killer.

So we can see this actually with, I didn't get into this issue of innovation very much in this particular book, but, um, you know, cause you only have so much space, but this opening, the way that I opened this book really shows this problem.

So tell me about how you opened the book, which we were talking about this before we started recording, but sure.

Um, there's 26 seconds left in the Superbowl.

It's 2015. The Seattle Seahawks are playing the New England Patriots, of course, cause the Patriots are in every Superbowl.

And I'm a huge Patriots fan.

Oh, well, we're sitting in Philly and I'm a Philadelphia fan.

So we finally got ours too bad for you.

You can have your one year.

Thank you. All right.

So it's second down and the Seahawks have one time out.

This is actually really important.

Um, and they're down by four last play of the game.

Everybody's expecting P Carol to call a run play for Russell Wilson to hand off to Marshawn Lynch.

This is the expected play beast mode.

Yes. Just plow through, uh, P Carol does something really unexpected and he has Russell Wilson, uh, throw a pass and the passes intercepted quite famously by Malcolm Butler in the end zone.

And it, this was on television.

This is, it's, you know, a big stage game over and the headlines the next day were just, oh my gosh, they were so brutal.

Um, and what I thought was really interesting was that it wasn't an argument like, Oh, was it a good decision or a bad decision?

It was, was it the worst decision in Superbowl history or the worst decision in football history?

That was the question.

And I think that what's really interesting about that is to your point.

Um, well, let's take two different thought experiments, thought experiment.

Number one is imagine if the ball were caught.

What are the headlines look like?

He's a genius. He outsmarted Bella check.

So that's, that's the general problem of resulting.

So that's resulting in a nutshell is that what you're doing is you're taking something where, where the decision process, whether the decision itself is, it's opaque.

You can't see into what's the decision good or bad.

You can't see into the process.

You don't know what was in his head or what the mathematics were, or you're not, you don't necessarily know what the formation was, whatever it might be.

It's opaque. And so as a heuristic, as a shortcut, what we do is we say, well, I know what the quality of the outcome was.

It was really bad. So therefore it must have been, I mean, and it was spectacularly bad.

So therefore it must've been a really horrible decision.

Um, and then we do the thought experiment where the outcome is really good and the ball is caught.

And we presume that the Patriots don't have enough time to drive it back down and the Seahawks win.

And we can see that now it's very obvious that it would have been, you know, he outsmarted Bella check.

In fact, we know that because the Philly special, I was just thinking about that.

That came to mind is like Doug Peterson is on the other end of possibly this, at least from an outcome basis.

We don't see the process.

Exactly. And that happened to work out.

Now that was also an unexpected play call.

They expected him to go for three points.

Had that not worked, it would have, particularly, particularly if the Eagles lost, they would have been looking at that one play and saying they should have just gone for the three points.

I mean, it's very clear that that's what they would have been saying, um, it happened to work.

And so, and he's, you know, lauded as an incredible genius.

So that that's the resulting problem.

And that that's a really big problem when that's the way that you're, um, evaluating people and where it's a really huge problem is that it's a real innovation killer.

Um, and that comes to the second point, which is that part about everybody expected him to in this case, hand off to Marshawn Lynch.

Um, we can do another thought experiment.

He hands off to Marshawn Lynch.

Marshawn Lynch doesn't get in.

He hands it off to him again.

Then he fails conventionally.

He fails conventionally.

And do you think anybody's saying that was a terrible decision?

No, no. So what is, if you're a leader and you're a result or which we all kind of are, then what you're doing is you're telling employees, don't ever make an unusual choice.

Don't do anything innovative or unexpected because if it fails, I will come down on your head.

Like I'm USA today, the day after the Seahawks lost that Superbowl.

And that's basically what happens.

You know, it's so strange.

How could IBM possibly ever get crushed by Apple?

That, what a weird thing.

IBM has all the money.

They already have the market.

They have the customers.

They've, you know, what is that?

How is that? And it's, I've always felt that it's because IBM is operating under a culture of resulting and Apple isn't.

Apple can't possibly be operating under a, they're doing something unconventional by definition.

They're, their whole business model is to make unconventional choices and to do things that are unexpected under the conditions where they're expecting most of them to fail.

But IBM, once they're in operational mode, certainly when, you know, they were a startup, they weren't, but once they're in operational mode, mode, they're a company that's just a company that's being run on resulting.

And when you do that, what's going to happen is that your employers are going to be making safe bets.

And by safe bets, I mean, once they're likely to have a lot of, unlikely to have a lot of variance around zero, right?

Like they're going to be either like little, you know, maybe we're going to have lots and lots and lots of tiny winners, because what I'm really trying to do is protect myself from losing.

And when you do that, what are you going to get?

Very small, incremental changes, right?

You'll get, going to get tiny little innovation.

But if the environment changes faster than that, you'll lose your relative position and - And that's it.

And then all of a sudden your IBM and there's Apple and Microsoft out there and you're like scratching your head as to what happened.

You know, I actually, so I want to do, this is a really simple, when people are like, okay, those are big companies, but what about my life?

This is the example that I love to give of this.

So you're driving with your spouse and you have to get to somewhere like a dinner party and you take the usual route.

So you go the usual way that you go to this person's house, to the dinner party, and there's like really bad traffic on the route.

And you know, the, oh man, I'm so upset.

And it's like, Oh, don't worry, honey, how could you have known?

Like we'll be fine.

They'll, they'll understand.

I'm texting them now to tell them there's traffic and no one's mad at each other.

But what happens when you shoot?

You're like, you know what?

Somebody just told me about this shortcut.

And it's like way faster to get there.

It's like a totally brand new route.

And you go on it. And it's the same thing.

There just happens to be really bad traffic or there's construction or there's an accident.

It's like, it's just - And then you're late.

You just fight in the car ever and you walk in and everybody's like really uncomfortable at the dinner party because they can see that you've passed.

Is this from personal experience?

No. It's such a great thought experiment though.

And you can see, and what's the difference?

In one case you did the thing that was conventional, that was sort of time test.

In the other case, you did something that was innovative.

And when you, whenever you make an innovative choice, you're exposing yourself to the resulters.

And God forbid it's on a really big stage, like national television at the Superbowl.

It's because there's more downside than upside.

In most scenarios, like in this case, if you get there one minute early, it's not because you're a genius than this route.

And but if you don't get there on time, then you, you know, it was Keynes who said it's better to fail conventionally than succeed on conventionally, I believe.

So I think that in any given try, there is more downside than upside.

Yes. But not in the long run because you miss the paradigm shifts.

You miss the ability to make really big changes.

And that's true in your life also, by the way, if you're afraid of trying new things of changing your career, because if it doesn't work out, you're going to be really sad.

And you would rather have it not work out by staying in the crappy job that you don't like.

Right. I mean, that that's your resulting yourself there, and you're refusing to make a change or make an innovative choice because you're, you're afraid of the downside and that can have really bad long -term consequences.

I mean, I can tell you there were lots of, there were many moments in poker where through these kinds of conversations that I was having with people, I thought, you know what, I think that this way that I've been playing in these particular situations, I think there might be a different way to do it.

And I understood going in that I was probably going to lose it first because I wasn't used to the strategy.

I was sort of trying it out.

I was trying to figure out when should I apply it?

When shouldn't I? What, what are the right moments?

Exactly how should I execute on this?

And I understood that there, that I wasn't going to do so well necessarily because I was trying something innovative, but I also thought that it was long run better for me because I was going to be doing something that was better for the way that the market was moving.

Poker markets move, like the way that people play changes, that how much knowledge your opponents have changes, particularly it gets on TV and you can run analytics on it and not all sorts of stuff.

But yeah, so I think that it's actually like, yes, in the short term, there's less downside, but it creates this really huge long -term downside.

And what ends up happening is that either in the business world, you're all of a sudden IBM and trying to scramble and rethink your business model in order to compete with people who have just crushed you because they were willing to be innovative.

And in your own life, it causes you to be stuck.

So talk to me about like the decision process and organization like IBM should use internally to come to the most probabilistic best solution to a problem, any given problem.

Well, so I think that there's two pieces to it.

The piece number one is that you have to really get people to feel like they shouldn't be afraid of the outcome on any given try.

Because on a single try, we don't have enough data to know whether it's any good.

You flipped one coin, right?

Like one time. What does that tell you?

You don't have 10 ,000 coin flips.

So you have to get people to feel like it's okay if the one tried doesn't work out.

And I think that as a leader, you just have to culturally, you really have to be good at communicating that, and then you have to live it.

You have to act it.

So how do you do that?

Well, work the decision process through with the group and actually memorialize the tree that you create.

So how do you do that?

You have decisions under consideration.

And what you want to do is try to figure out as best as you can, what are the scenarios that you think are going to result from a decision A versus decision B versus decision C, and then try to assign probabilities to those scenarios.

And don't be afraid of it.

Most people are afraid to assign probabilities because they think there's a right answer.

And it's true, there is some objectively right answer, and it's likely that the guess that you give is not going to be if we had some mirror into the objective reality, it's probably not going to be perfect.

When I said that I was whatever 73 % to get my food back correctly, I don't know if I was exactly 73%, but the fact that I was making a try at it is better than not trying at all.

Because if you don't try it all, you're either saying that, well, you're certain it's going to turn out in a certain way.

Or you're saying something like, well, it either will or it won't.

I mean, the fact is that it is probabilistic.

And you're probably more of an expert than anybody else because you have your own experience in making these kinds of choices.

You can certainly go and look at, there's all sorts of business cases and there's data that you can go look at to try to refine that guess.

And the desire to make the guess makes you very information hungry.

So it actually makes you more open -minded because now your goal is not like I have to have the right answer, it's I want to try to get as close as I possibly can to what the objective probability is.

And I can only do that by getting lots of information or listening to people who have different points of view so that I can start asking myself, why might this be wrong?

Why might I be off on this?

And going back and just constantly trying to refine and get that feedback.

So it causes you to be really information hungry.

It causes you to view other people's perspectives as incredibly helpful.

Again, it's that sort of identity shift around it.

And so if you can do that, if you can sort of get, okay, well, here's what I think the outcomes of this decision might be.

Here are the probabilities that I think that those might occur with.

And then in that process, make sure that you're really wrapping in the dissenting voices.

And this is very important.

And you can do that in a variety of ways.

One is that you can do, you know, you can create red team, blue team.

So you just create a team of people who are supposed to argue against, you know, what the prevailing opinion is.

If you have people in the room who really disagree on what the probability of a particular outcome is, which will happen, force them to change sides.

And we disagree. You're at 30 % and I'm at 70%.

Wow, we're way off.

First of all, cool.

We're probably going to learn something through this process.

That's actually much more exciting than us agreeing.

If we all agree. Right.

Now this is really exciting because you and I are pretty well informed.

We're on the same team.

We're in the same company.

We're kind of working with the same information.

And yet we really disagree about what the probability is.

That's exciting. And that's good for us.

Somebody has an opportunity to update their view.

We might just not know who that person is at this point.

Or we might all update our view because we might all come now to something that's more in between.

Yeah. Right. But now make us switch sides.

You now have to argue my side that I'm 70 % and I have to argue your side that you're 30%.

And the people in the room are going to judge who wins the debate.

In other words, not who's right, but who's the better arguer of the other side.

Notice that makes me construct the best version of your argument.

Hmm. My goal is to be able to argue it better than you could because I want to win.

Right. I want the people in the room to judge me.

And what's really wonderful then is that now I'm going to start thinking about, well, why are we so far apart?

Why does he think it's 30 %?

I'm likely to probably moderate my view.

You're likely to probably moderate your view and everybody in the room is now going to learn because they're exposed to what our process is.

And that's wonderful.

And now let's say most of the room agreed with me and you were this outlier at 30%, you get a chance to have your voice heard and your voice is now represented in that scenario plan that we've now created.

So that's a great way to do it.

Another way you can do it in order to really make sure that you're getting those, you know, why might it go wrong and those dissenting voices is to do what's called a premortem, which is examining the patient before it's dead instead of after.

So imagine, okay, we have this goal as a company.

We want to increase sales by X amount.

Let's imagine and that's say a goal that we want to have.

It's a two year goal.

Let's imagine at the end of two years that there's a newspaper that says we failed.

We did not reach our goal.

Let's all, everybody in the room now write down five reasons why we failed.

And now what happens is that very often in these, these rooms who end up with this team player problem, everybody wants to be seen as a team player.

Everybody wants to be seen as agreeable.

Everybody wants to be seen as sort of like, we're winning the game because we're all so smart and like cheerleaders.

When you do a premortem, now you've changed what it means to be a team player, right?

Like the people who are going to get pats on the back are the ones who come up with the most creative reasons for failure.

And that's going to allow you to now anticipate sort of where things might go wrong.

It's going to allow you to wrap that into the scenarios that you think might occur.

It's going to allow you to adjust the probabilities of success or failure.

Okay. So you go through those kinds of exercises you create now.

Okay. We're going to go with decision A, here are the scenarios we think might occur.

Here are our stabs at the probabilities and you memorialize it.

So now when the thing happens that's right up there on the whiteboard, you took a picture of it that didn't work out and it's got that percentage by it.

You know? Okay. Well, we thought 33 % of the time this was going to happen.

It was going to be bad.

It makes it very hard to result because how do you go back and say our process was really bad?

It's like, well, here it was.

Right. We kind of knew it was going to happen.

So I think that that really allows you to now focus much more on what's the process for getting us to understand what the decision is.

And obviously as the future unfolds, as it does, you can go back in and examine and say, you know, did we miss something in that process?

Or was it just, it was the 27%.

You know, were our probabilities, right?

Like you can ask that, but you've done it all together.

Right. I like that a lot.

One of the differences that I see between poker and maybe organizations is poker.

You're making very individual decisions.

You're the person making the decision.

You're responsible and accountable for those outcomes.

Those lines in organizations tend to blur in the sense of sometimes you're responsible for implementing somebody else's decisions.

Sometimes they're responsible for implementing yours.

And sometimes you come to decisions as a group instead of an individual.

How does that affect how you think about this?

Well, first of all, I think that the process for coming to what the right decision is can be similar regardless, because what I was doing was certainly in the moment I was making my own decisions, but it was my responsibility to go talk to the people who were my peers that I was trying to learn with and work

through and deconstruct those decisions and talk about strategically what I might be doing in the future.

So in some sense, that's a similar problem, right?

I have to go execute on my own, but I'm interacting with a group in order to try to figure out what my best strategic plan is.

And I think that that's actually in large sense, very similar.

That being said, obviously it would be really nice if I could go through a group decision -making exercise at the table, but that would be bad because it's one player to a hand and you need to play on your own.

But I actually don't think that it's that dissimilar.

And I think that you can view as a leader the same thing that, yes, the person has to go execute on their own, but it's my job as a leader to have worked through what the process is and what the strategy is with that person before they go off and execute so that we're all bought in.

And I understand what their reasoning is.

They know that I've bought into the process.

They understand that I know that there's some probability that it doesn't work out, but we've agreed that as they go and execute, that they're going to execute on this plan and then they're going to come back and we're going to discuss later to see how we might refine in the future.

But they're not going to be held responsible for the bad outcome.

They're going to be held responsible for having worked through the process together.

And that's actually very similar actually to poker.

So I'm executing on my own, but then I'm going off to my group and saying, hey, can we deconstruct this session that I played so that I can start thinking about what are different lines of play?

And if I make these kinds of decisions in particular situations, what are the different ways that I think that could turn out?

And let's start thinking about strategically what I should be doing so that when I go back and execute on my own, I'm sort of executing what I've learned and what I'm sort of bought into in terms of process.

Do you keep a decision journal?

I actually don't, I'm not a journaler and I wish that I were because I think it would be way better.

I'm much more of a - I would love to go back and read your journals as you were like, Meteor, Rise and, right?

Yeah. The highs and lows associated with that.

So I think that people learn in different ways.

And for me, the way that I really learned was through conversation with others.

The way that I found that I was best held accountable to my own ideas was to expose those ideas to other people.

And have them challenge.

And that's just my learning style.

When I imagine a decision journal, the kind of decision journal that I imagine would be, go talk to people about what you've changed your mind about today, right?

It would all involve, I think, go talk to other people, go find other people to speak to go.

And then I also, because I had that group when I was on my own and I was kind of thinking about those things on my own, I was always imagining it as if I were talking to somebody else.

So for me, that internalized conversation of the group became my own journal, as opposed to physically writing it down.

It's interesting though, because even in the writing process of writing the book, I don't write a whole lot of drafts because most of my drafting is in my head.

In yoga, and I guess this is bad because you're supposed to only be on your mat, but in yoga, I might be working through some ideas or walking around or I'm driving or whatever it might be.

I'm actually composing in my head and I'm working through the structure and the arguments.

And then I'm sort of imagining talking about those with other people and trying to refine them.

And then I do talk, well, here's what I'm thinking about.

This is what I'm thinking about writing in this section.

And I'm bouncing it off other people and trying to hear what they have to say and how it's playing with them.

And that's actually a lot of what I do a lot of keynoting.

And that's a lot of what that is, is you're putting these ideas out to the audience and then sort of seeing how they respond back and getting that feedback.

And that's allowing you to figure out your narratives and the way that you're expressing it and whether you think the ideas have fidelity.

How are you being challenged when the people come up to you afterwards?

So by the time I ever get it down on paper, it's been really well journaled.

It's just been journaled in this way that I have my mind happens to work really well.

What information would you require from somebody else to, or would you want from somebody else to judge the quality of their decision, not knowing the outcome?

Well, I think it depends on what the decision is.

What I think is really important that you bring up is that there are details that I need to have.

And I think that you have to think for yourself in whatever you're doing.

And you can work this through with a group of people who are also experts in whatever you're doing or trying to become experts in what you're doing, or what are the details that you always need in order to construct a good decision.

Because you know that for yourself better than I do.

I can tell you from poker, for example, there's all sorts of details that I need.

Would they be in poker?

Sure. I need to know how many people were at the table.

Were you in a cash game or a tournament?

If you are in a tournament, what point in the tournament are you?

How close are you to the money, for example?

Have you already made the money, these kinds of things?

In a cash game, have you been winning or losing recently?

What does your stack size look like?

What's the form of poker that you're playing?

Are you in early position, middle position, late position?

In other words, where exactly are you sitting at the table?

What was the action prior to the point that you have a question for me?

Were you the pre -flop raiser?

So this is all jargon.

Oh, keep going. This is awesome.

Were you the pre -flop raiser?

Did you call how many people were in between you when you called?

Because I can't... The problem is that if I don't know some very simple facts, like were you first to bet or second to bet?

Or were you the raiser or not the raiser?

Or how had the people been playing around you?

Was the person that you were playing against, did they tend to be looser or tighter?

Any of those questions, what did the board look like?

Then there's literally no point in me giving you any advice because I'm not going to give you advice of any fidelity.

I was actually talking to a friend of mine, Liv Boiree, who very early on in her career, I mentored her and was working with her in terms of...

She's a very, very good player now, but way back when, when she was first learning, I was working with her.

I was talking to her about it the other day.

I said, do you remember when you first started playing with me and you would come and describe a hand to me?

And I was a little bit Eric Seydelish here.

And I was like, well, there's no point.

Because you're not...

I have no idea. Where were you in the action?

You can't tell me all three cards that came down on the flop.

I don't know who raised or who didn't.

You just described a hand where you were the pre -flop raiser and somebody called and then somehow they checked in front of you, but they weren't in a blind...

It doesn't even make any sense.

So why should I give you advice?

It won't help you. It was vague and lacking information or context.

Right. So that if I gave them advice, her advice in this particular case, it wouldn't have helped her because I would have just been making things up because I didn't have the details.

So you can think, you can imagine fitting this to whatever you're doing.

Figure out what the details you are.

If you're in sales, figure out what you need to know.

I can imagine that there are certain things you'd need to know.

Like, where in the sales cycle are you?

Where in the fiscal year are you?

What was your relationship with the person in the other company?

Did you have other relationships with people in the other company?

Or was it just you just had one point person and how had things gone in the past?

During the conversations, what were they telling you about where they were?

How were they describing how they wanted to continue to have interactions with you?

Was it vague? Was it, we're going to set a date to be able to do this?

What are the strategies that you were using?

How did you figure out how to figure out when you were interacting with them and when weren't you?

I'm making this up, but I assume that these are...

Maybe sales people can write you and tell me that was the most ridiculous thing that anybody ever said.

But the point is, you should try to do that for yourself.

I think understanding the situation is great.

Knowing what information the person had at the time making a decision is super important.

Are there anything else that comes to mind for you in terms of how you might structure what you would want to know from people that may be less situation dependent?

More like, what range of outcomes did you think was possible?

When you're actually working through a decision with someone, you want context for one thing.

I would want to know how things had gone in the past for them with other decisions.

First of all, I want to know how things have been working out for them over the long run.

I'll give you a really simple example.

If someone's coming to me and complaining about a problem that they're having in a relationship, it's good if I know how things went in all other relationships.

If they're telling me, this person I've been dating is such a jerk, and I know that the last 10 people that they dated have all been jerks, that changes the way that I'm thinking about what kind of advice I might be giving them or how we might work together, more importantly, through the decision than if the last

nine people that they dated were all amazing and now they happen to be with a jerk.

You can see how that really matters, that context.

Yes, I want to know when you were working through the decision, what information was informing the decision that you were making.

What were the possibilities that you thought could happen?

Because maybe I disagree with you.

Maybe you say to me, I was 70 % that this was going to work out really well, and I'm like, ugh, that seems high.

Now I can quiz you on it.

Well, why did you think it was 70 %?

Because gosh, it really feels lower to me.

Let's explore that together and see why that is.

Particularly, I want to know the information that you were pulling mainly in order to drive the decision.

Why did you make this prediction in this particular way?

Because maybe I have information that you missed, that I can now provide to you.

Maybe I can start quizzing you.

Well, can you think of any information that you could have brought into it that maybe would have made you make a different decision?

What is it that you...

Do you think you could have found other information?

In retrospect, you wanted to go toward a certain conclusion, and so you drew this particular information that you had or this detail or whatever it might be.

Well, what about all this other stuff that existed out in the world?

Why weren't you considering that?

Did you ask yourself why it might not be true?

Did you seek out the advice of other people?

Did they tend to be people who agreed with you?

Were you asking people who disagreed with you?

Did you ask anybody why they thought you might be wrong?

These are all the kinds of questions that you're always asking yourself if you really want to be a good decision maker.

Hopefully, if we're in a decision pod, part of what I'm doing is asking you those questions because I know that because of the way that your mind works, you're just less likely to ask them of yourself.

So am I. I'm less likely to ask them of myself.

But it's easier to ask other people when you're part of that group or tribe.

Because we have an agreement to it.

I think that what's really cool about that kind of agreement is that going back to the idea of how do you disagree without being disagreeable.

Again, I think it depends on how you view the disagreement.

Because if we have an agreement that you're going to help me with my decision making, you're going to help me construct an accurate view of the world.

If you withhold information from me, now that's hurting me.

You're violating our agreement by not giving me a perspective that disagrees with me.

If you have one, you're violating our agreement by not sharing information that you have that might moderate my opinion downward.

Which is a huge problem in organizations where people withhold relevant information.

Right. They don't want to disagree with the leader.

Well, there's kind of two things that can happen.

So yeah, they don't want to disagree with the leader.

They don't want I don't know why Nancy is so negative.

Seems like, should be negative Nathan.

The negative Nathan.

They want to be seen as a team player.

They don't want to be seen as nacing the company.

They're afraid of disagreeing with the leader.

I mean, there's a lot of leaders who don't like to be disagreed with, right?

They make their pronouncement and they're not looking for people that disagree with them.

They don't like that.

That's why we have the word yes, man, in the English language.

And I'm sure every other language has some sort of yes, man.

I prefer sycophant.

Sycophant is a good one too.

So I mean, it's why people will surround themselves with people like that because it doesn't feel good for a lot of people to be disagreed with.

And if you're a leader, that's bad.

The other thing is that to your point where you said, well, how do you, how do you reason through a decision?

Like, how do you, what are the kinds of questions that you might ask somebody to try to figure out if a decision is good as if you don't know the outcome?

I think that part that you said about don't know the outcome is actually really significant, because I think part of the problem, and we know this from resulting, is that if you know the outcome, it actually distorts the analysis of the decision process so much.

It becomes this huge anchor because the decision process is opaque.

There's hidden information.

There's luck. We don't really know how much luck there is a lot of the time.

We don't know what we don't know, right?

I mean, we can sometimes know what information could be found, but very often we don't know what information, there's lots of information that might be relevant to the decision that we don't even know we're supposed to be going and looking for.

So this is all very opaque.

So without the anchor of the outcome, it's a lot easier to sort of look at what are the ways that we can make a good decision and figure out if we're having a good process.

Once you have the outcome, it just it's overshadows anything else that could happen and you end up taking this because it is so opaque.

It's very easy to now reason around that to make the outcome make sense.

There's a couple of ways that that's a problem.

If the leader expresses their belief about what's correct, in a sense that acts like an outcome, and the group without knowing it, and the leader without intending to have that happen, now causes a situation where disagreement with that belief is less likely and people are actually going to start reasoning

towards support of the belief.

And they don't even know they're doing it.

They could say, no, we're trying to be really open -minded.

Like we want to be an open -minded organization, but just the expression of them saying, this is what I think is the right path, or this is what I think is the right strategy, or this is what I believe to be correct.

The room starts reasoning toward that without even knowing that they're doing it.

That's one way to infect the process.

If you know the outcome, it can infect the process.

So you should feel really lucky if you have people that you can work through a decision with where they don't know how it turned out.

Now that's lucky if you can find those people.

I mean, you could do that skillfully as well, because you can create a group of people.

We do that in organizations all the time, right?

Where you're working with somebody, either a peer, a boss, or somebody else, where you're trying to reason through decisions without knowing the outcome.

And then if you're in charge of somebody or you're leading them, you're often tasked with assessing their judgment, which often means evaluating their quality of decisions before the outcomes are known.

Right. So that's the easiest way to do it is do it before the outcome is known.

But if the outcome is known, don't tell the outcome.

So if you get brought in...

Because it taints everything.

Right. So if you get brought in and they're like, we really want you to help us work through this decision, your best bet is to say, please...

Okay, that's fine. I'm going to agree to that, but please do not tell me how this turned out.

I don't want to know the outcome because I want to give you the highest fidelity advice.

And I know that as a human being, once you tell me the outcome, that's going to be distorted.

I can try, but I know it's going to mess it up.

So please don't tell me how it turned out and beg people not to tell.

Please quarantine me from the outcome.

I do not want to be infected with it.

Here's a pro tip, a story I'm not going to tell today, but I might tell another time, which is never follow a sycophant or a yes man who's been in their job for a long time if the boss also hasn't changed.

Oh, that's a good one.

A theme near and dear to you are motivational biases around self -image.

I mean, the one where you're smart when things go your way and unlucky when they don't.

Are there any tips that you have around what we're talking about within the organization aside from surrounding yourself with people where we can become more open -minded as an individual?

Because I'm thinking about this in the sense of often where we feel alone in an organization.

Like we want to make some change, like a grassroots kind of movement, but we don't necessarily have those people around us.

And we want to create it.

So let me, I just want to make sure I understand what you're asking.

Is there a way, I think I switched questions.

Yeah. Let's go with the second question.

I think that's actually more interesting in this context here.

How do we do that in the, in an organization where we're trying to create something, but the only person we control is us.

Right. How do I go about starting that?

So I think the answer is intentionally.

I mean, first of all, find, listen for people who are talking a little bit more like you, who clearly have an intent to try to do better.

Now that doesn't mean that they're not falling prey to things like self -serving bias or motivated reasoning or, you know, but you can, I mean, when I was walking through a poker room, I could hear the people who were sort of, you know, Oh, okay.

That person's really trying to get to the truth.

That person's just trying to complain over here and, you know, gravitate over toward those people.

And then what you can do is if you, once you see the kernel of that and those people say, Hey, I really want to try to get better at my job.

And I think, you know, I imagine you do too.

Like, can we help each other?

I think it would be great.

And then actually intentionally make this agreement.

Like literally say, okay, here's our charter.

This is what we're going to do.

And this is what we want to implement.

And here's how we're going to talk to each other.

We're going to work through process.

We're going to try to speak probabilistically.

We're going to create scenario plans.

We're going to try not to be resulters.

If you hear me resulting, you're going to call me on it.

If you hear me doing self -serving, you know, falling prey to self -serving bias, you're going to call me on it.

If you, you know, confirmation bias, you're going to call me on it.

So on and so forth, right?

Like you're, you're really saying.

Or you missed this.

You don't think about a wide enough range of outcomes or you don't sit down and think about it before you make a decision.

Yeah. You're, you're thinking too much in black and white.

I mean, I talk in the book about creating a decision swear jar, which is, you know, I mean, the swear jar is like, if you say the F word, you've got to put a dollar in the jar.

Well, you know, think about the things to you that are signaling that you're, you're not processing information in maybe the most unbiased way.

Like when I, when I'm saying things like you're wrong, I mean, I should have to put a dollar in the jar for that one when I'm saying I should have known, right?

Or that turned out so badly.

What a stupid decision.

Right? I mean, you could, you can start thinking for yourself, like what are those signals and do that when you're sort of in a rational place, when you're not in the midst of a bad outcome, or by the way, a good one?

Because also what should go in there is like, I knew it was going to work out and like, that's, I, I was so smart and I really thought through this decision so well and I knew it was going to work out well.

And, you know, I mean those that, by the way, you should put $2 in the jar for those ones, right?

Because it's a lot harder to spot your mistakes when you win.

Because boy, that it's like not basking in the glow of how great you are.

That, that takes a lot.

That's really hard.

Right? In some ways, that's the harder thing.

So put $2 in the jar for that one.

But if you can create that kind of stuff and then now you kind of have this idea of like, here are the types of categories of things that we could be saying, which are going to be signals and let's hold each other accountable to those things.

Right? So when you, you know, if you say I'm wrong, I'm allowed to say to you, well, am I wrong or do you disagree with the thing that I said?

Because if you disagree with the thing I said, that's different than me being wrong.

That means that we have a disagreement about what the truth is.

And so let's start to try to explore that.

Right? So, you know, that's where you get to speaking to each other like this.

Well, like I see your point, like, let me try to understand what your point is and then let me see what I can add to that.

Because I have, you know, I'm thinking about it this way because, you know, I'm looking at it through this frame or I'm imagining this extreme case.

And I think if I stress test your idea at the extreme, it feels to me like maybe there's a problem.

You know, that's where you start this very moderating language because your own beliefs are moderated and you move out of the extremes and think about how much more open you're going to be in that exploration process with me.

I mean, it's going to change everything about the way we communicate.

So find those people, like, listen for them and start to pull them in and you can start to create that culture.

If you have the kind of relationship with the leadership that you're interacting with, go talk to them about it and say, like, I really think this would be a good way to start doing things.

Can we start trying to implement some of these ideas that we don't have to implement all of them at once, but how about if we just start doing red team, blue team?

Like, can we try that?

Right? We're going through a strategic planning process.

What about taking, you know, Bob, Sue and, you know, what, Larry and Corda and they have to argue against us.

We send them off to argue against that.

Can we try that and see what comes from that?

Because I read somewhere that that was a good idea and you can start maybe trying to get these kernels in until you can, you can actually from the bottom up create these cultural changes, I think.

I have three more questions for you before we, um, we kind of come to a conclusion here.

One of which is, um, what are the three main takeaways from your book?

Uh, thinking and bets.

So, okay. I don't know if I'm going to do these in order.

So don't take these as number one, two and three.

I'm just going to think about what the three main takeaways are.

I think that the main takeaway is say I'm not sure more.

Because I think that we confuse confidence with certainty.

And we think that if we say I'm not sure or something like that, meaning, I mean, another way to say I'm not sure is I'm 62%.

Another way to say I'm not sure is I, I believe this thing to be true, but just so you know, I only looked at two articles on it.

So all of my information is coming from these two places.

So acknowledge the uncertainty in this situation.

Exactly. So these are all different.

There's all another way to say I'm not sure is I think that Elvis died somewhere between age 40 and 47.

Okay. So that's another way to do it, right?

So these ways that you're wrapping in, I'm not sure into the way that you're speaking.

What I mean by, I think we, I think we conflate confidence and certainty that we think that in order to be seen as confident communicators to be competent, to be believable, that we must express things with certainty.

And there's all sorts of bad things that come from that.

Uh, the first thing is that you close people off to telling you what they know because well, for one of two reasons, either they're embarrassed because they think they're wrong.

I mean, after all you've just said, I'm sure.

Um, and so maybe they, they're like, oh, I don't have my phone to Google this right in front of me.

So I don't want to say anything cause I, I think I might be wrong.

But the other thing, and you can think about this as a, the leadership problem is that it, as a leader, if you express things with certainty, people in the room might not share with you because they don't want to say that you're wrong, particularly in front of a group or maybe just alone to you.

And so they kind of keep it to themselves and they keep their opinion to themselves.

Um, particularly opinions that they themselves aren't certain of, if there's any uncertainty to their opinion, they're going to be unlikely to express it to you because they don't, because of this, either they think they're wrong or they don't want to tell you you're wrong when they're not necessarily

certain themselves.

And that's bad because now you just lost the opportunity to explore and learn and, and calibrate the beliefs that you have.

Um, but the other thing is that when you express the uncertainty, I think that you do become a much more believable communicator.

When I say to you, I, well, here's my belief, but I, I only read it in these two articles.

So I just want to let you know.

And these are my sources.

What have I told you about myself?

You do a lot of work that you've thought about it that you're uncertain about the position, but that you have an initial position that can be updated.

Right. And doesn't that make me more believable?

Because I've told you that I'm thoughtful that I thought about the process.

So I think it, I want to say it does, but I mean, there's a lot of research that speaks to like, Connor Dick, that right?

Like intuitively, I want to be that person who's like, I love that person, right?

Who does that? But we tend as a society to value something different, which is this overconfidence.

I'm not wrong. I know exactly what to do in almost every scenario.

We see that play out through politics.

We see it play out in organizations, right?

Well, we see that, we see that in the pundit tree, right?

In the pundit class.

They like to express things.

Right. Or internally in organizations, right?

Not knowing what to do or being uncertain about it can often confront your view as a knowledge worker.

Your view is a leader.

Um, you feel like you'll lose respect from the people around you and you might.

Right. I think that there's a difference between when, you know, a candidate has to express it to the general public, particularly when that, that public can't talk back and you as a leader in a room with the people that you're working with and being able to model that kind of behavior, I think it's

going to cause people to respect you more and it's going to cause them to share back with you and see that the process is better and you're going to be considered more believable because you've put thought into the process.

Um, I think you're also expressing to them that you'll value your, their opinions as well.

So yeah, I understand that when a political candidate is talking to a hundred million people that there isn't time to waffle.

You, you have to just say that, you know, things for sure, which by the way I think is a shame.

Wouldn't that be more realistic though?

Like if somebody got up, I often have this dream where I run for office and I'm never going to do it.

So don't anybody hold me to this, but I get up in a debate and somebody asks a question.

I'm like, I don't know.

I would hire the smartest person available to figure that problem out.

Right. Like I can't be an expert on healthcare and education and taxes and all of this stuff.

You need to rely on people.

Well, I would, I would say a couple of things there.

I mean, I think that we have had some candidates who did speak that way.

So number one, uh, but so it's a flawed strategy.

Well, no, I think, well, I think I can think of one candidate who won two terms, who did modulate a lot of what they said.

That being said, I think that it depends on what country you're in.

I think that, I think that it depends on the culture that you're sitting in and how much the culture is sort of this culture of confusing certainty and confidence.

Right. So I think that our general, there's a difference between sort of the initial reaction of how do we react in the moment versus like, if you ask somebody, what do you want really in your president?

I think that most people would say, I want someone who's thoughtful and they think things through and they, um, you know, hire really smart people around them who can help them.

We want to find out work, right?

With our boss or leaders.

But in the minute, you know, in the moment of a debate where you have like two seconds to say something and you just want to hear the answer.

Um, I think a lot of people, I think that people very often aren't comfortable with uncertainty and they want to believe that the, you know, the world is kind of a scary place and this person is going to make it all better and they know what the answers are.

I don't think that that's what somebody wants in a room in a company with their leader.

You know, the people in the room, they aren't, they aren't the populace who's saying, please tell me that everything's going to be okay.

And that it's all going to, you know, they're, hopefully you're all sort of in the process together trying to come up with the right answer.

And, and then I'm not sure is a really good thing, you know, in some sort of way, like you want to be expressing that in some sort of way.

I honestly wish we were training this kind of thinking earlier and getting people to understand probabilistic thinking better so that you could, instead of saying, I know that, well, I mean, from the 2016 election, instead of the pundits who were saying, I know that Clinton's going to win, that people

would be more accepting of like a Nate Silver saying she's 65 % to win and that he would be seen as the more believable character in that play, right?

But I think that you need to be training this much younger.

And I don't think that we get people to think this way very well, but certainly I think in a company as a leader, you're much more believable when you're modulating your beliefs to the people on your team, to the people that you're working with.

So I think that's the first thing is like, wrap your arms around uncertainty.

It's going to make you a better decision maker because when you acknowledge, I'm not sure, you naturally view your beliefs as under construction.

And so then what happens is that you end up being much more open -minded to dissent.

You end up being much more hungry.

You're more likely to start googling things.

You're more likely to ask people their opinions and try to start working with other people to share in this process of constructing your beliefs, which you view to be under construction.

And that's the power of I'm not sure.

So I think that's number one.

Number two is it's really hard to do it on your own.

So get yourself a group and really...

Find one or create one.

Create one, find one.

Maybe you can plug into something at like a YPO and get yourself a pod there, or find people in whatever enterprise you're in and the company that you're in could be friends that are like -minded.

Go find some people to be in a group with you, because you really need people to watch your back and you can help them by watching theirs.

So that's number two is this is a group process.

If you really want to get better at this, you need other people.

And number three is start being a really good time traveler.

What does that mean?

Well, going back way back to the beginning of this conversation, remember we're talking about, well, we have this trade -off, right?

We can kind of feel the pain in the moment, that's right.

The pain is good in the long run if we use it well, because we're going to be better decision makers in the long run because we're experiencing the pain.

But the pain in the moment is pain.

It doesn't feel good.

So we have these competing problems.

What's best for me now, present Annie in terms of the way that I feel versus what's best for future Annie in terms of how her life turns out.

I think that we can agree that the better my decisions, the higher the probability that I have a life that turns out in a way that's good.

It's almost like letting the hindsight of your future self become the foresight of today's self.

I like literally need a pen to write that down.

That was really good.

You should write that down and just post that somewhere because that is exactly right.

It's getting the future version of you to get involved in the decisions of the present version of you.

That's exactly what you're doing when you say, okay, I'm going to take some short -term pain right now.

I'm going to admit that I had some part in the way that this outcome turned out.

Or maybe I'm going to explore the possibility that this great thing that just happened was because I got lucky.

That's a painful thing to do.

I don't get to celebrate and have a party for myself.

But I can only do that if future Annie is involved in the conversation.

Future Annie has to be saying, hey, I know it doesn't feel good right now, but this is really good for me later.

That perspective allows you to go through that short -term negative pain to get to a better place.

Even it might not even be painful.

It could just be the willingness to look different in the short -term, the willingness to look like an idiot.

Buffett was on the cover of Time Magazine 1999 as being out of touch.

We're still talking about him today in 2008 because he was willing to look like an idiot and set out something he didn't understand.

And I think that that was key to what's made him so successful.

Right. I think that it's what allowed Pete Carroll to make that call.

It's what allows you to take the shortcut to the dinner party is that you're saying, I understand like there might be some pain in the short run, but that's okay.

Because I think long run, this is way better for me.

And I've had a conversation with future Annie and we've talked to each other that when we think about how do we create a good group to make decisions, we can think about actual physical other people, but we can also think about other versions of ourselves.

We can think about future me.

I could bring past me into the decision because past me has a lot of experience with this kind of stuff.

So I could bring past me into it.

When I was talking to James Altucher, he brought up the different versions of yourself, meaning like there's me that gives advice to my children.

Maybe that would help me think about a decision that I have to make in business.

Well, if I were giving advice to somebody else, what's the advice that I would give them?

Maybe that version of me can help this version of me make a decision.

So we could think about some situational traveling, traveling from one situation to the other.

So I think of that as another way to create a group.

It's just a group of different versions of you and that you can bring all of those different versions of you into the decision that you're making now.

I like that a lot. I get an email from somebody who I'd be remiss if I didn't ask this question.

And I think one of their children was trying to walk down a professional gambling career.

So I wanted to ask you about what your take on the morality of gambling is.

Oh, I wasn't expecting that question.

So what would you tell your children about the morality of gambling?

Well, I have my own values around that.

For me, it's the same as what's the morality of being an options trader.

It depends on how do you take it to the extreme.

Like what's the morality of the fact that if I have a restaurant and I'm selling food, that you are giving me money in exchange for that.

I think that that's a perfectly moral thing to do.

And so that's the lens through which I view it is that there's a market and everybody's playing in the market and they're voluntarily playing in the market.

So nobody's being forced to enter.

And people are getting different things out of it.

So I could be playing with somebody who is there for pure entertainment purposes and they're viewing whatever money they might have at risk as the same as if they were going out to a nice restaurant.

And they may actually not have an expectation or a value in actually coming out ahead at the end of the night.

I certainly don't know that.

But we've all voluntarily sat down at the table and we're doing this thing where we're using our minds in order to try to best this, to try to beat the market.

So I don't really view it as that different.

That being said, I mean, I can understand why somebody would have moral objection to gambling.

And if they have that, what I would say is don't gamble.

I get it. I have different moral views about lots of things than other people do.

So there are some people that think that owning a bar is immoral, for example, or a variety of, I mean, there's just many things.

So some people think that speeding is immoral.

Going five minutes over the speed limit, maybe they think that's immoral.

And what I say to someone like that is, okay, I literally can't argue with you.

Those are what your values are.

So then I guess you won't speed.

I mean, then you're not going to gamble, but hopefully I can gamble because we're making different choices.

And on a lot of moral questions, I just really revert to the direct harm principle.

Right. So I like that.

Let's finish with this one.

If you can go back and talk to your 20 year old Annie Duke self, what would you tell her?

What would you want her to know about this version of yourself?

What information would you pass her to create a better version of yourself?

Maybe that's a better way to word this question.

Okay. Well, I'm going to tell you, I'll tell you a story.

My 20 year old self is actually very specific.

It's very funny. For some reason when I turned 20, it was like the worst birthday I've ever had, much worse than any other subsequent birthday.

For me, there was something so significant about not being a teenager anymore.

Oh yeah. And I might actually say that I have a little bit of the opposite problem of a lot of people, which is I actually live in the future and very often the far distant future a little bit too much.

So literally having a collapse on my 20th birthday would be an example of living in the future a little bit too much.

I mean, I just felt like, my God, I'm not a teenager anymore and this means I'm going to die.

This whole confronting of mortality, it was this really bad.

20. Most people do that at 80s.

I think that my 20th birthday party was probably most people's 40th birthday party.

I think I had a midlife crisis at 20 years old that it was the complete facing death down.

So I can tell you that what I would say is it's okay.

It's all right. You're just, you're going to move through life, you're going to live life.

It's not really, you know, being 20 is no different than being 19.

And there's lots of great things that come with being older and it's okay.

I think I would have just said like, calm down, stop having a midlife crisis.

You'll be fine. Yeah, you'll be fine.

Annie, thank you so much.

This has been a phenomenal conversation and I really appreciate you taking the time.

Yeah, well, I'm happy that we made it work out and that you got to be in my hometown.

Philadelphia, yes. Of Philadelphia, yes.

Thank you. Hey, guys, this is Shane again.

Just a few more things before we wrap up.

You can find show notes at fernamstreetblog .com slash podcast.

That's F -A -R -N -A -M -S -T -R -E -E -T -B -L -O -G .com slash podcast.

You can also find information there on how to get a transcript.

And if you'd like to receive a weekly email from me filled with all sorts of brain food, go to fernamstreetblog .com slash newsletter.

This is all the good stuff I found on the web that week that I've read and shared with close friends, books I'm reading, and so much more.

Thank you for listening.

[불확실성 속에서 현명한 결정을 내리는 법: 애니 듀크와의 대화]-[#37 Annie Duke: Getting Better by Being Wrong]

📋 Summary

불확실성 속에서의 의사결정: 포커와 삶의 공통점

1. 결과주의(Resulting)의 함정에서 벗어나기

2. '나는 잘 모르겠다'는 말의 힘

3. 학습을 위한 그룹의 필요성

4. 미래의 나를 의사결정에 참여시키기

결론: 확률적 사고의 중요성

🎯Key Sentences

📝Key Phrases

📖 Transcript