#146 Barbara Tversky: Action Shapes Thought｜The Knowledge Project

I'm listening to when the boiling of the rice, I'm smelling the olive oil on the saucepan, and I'm picking up the danger sign, got to lower the heat.

That's only from a great deal of experience and paying attention to the smell and the sound of those cues, and that isn't in the recipe.

The abstraction you get either in a diagram or verbal instructions isn't going to enable you to be a great cook.

You can make satisfactory things and you need to know, not just adding salt, you need to know when to throw it in the trash and start over again.

Welcome to the Knowledge Project podcast.

I'm your host, Shane Parrish.

The goal of this show is to master the best of what other people have already figured out.

To do that, I sit down with people at the top of their game to uncover the useful lessons you can learn and apply in life and business.

If you're listening to this, you're missing out.

If you'd like special member -only episodes, access before anyone else, hand -edited transcripts, and other member -only content, you can join at fs .blog membership.

Check out the show notes for a link.

Today, I'm speaking with Barbara Diversky.

Barbara is a professor of psychology at both Stanford and Columbia.

Barbara has published more than 200 scholarly articles about memory, spatial thinking, design, and creativity.

She's the author of Mind in Motion, How Action Shapes Thought.

If you're interested in cognitive psychology, how the brain works, the way that language shapes thought and thought shapes language, you're in for a real treat with this episode.

We cover her nine laws of cognition, why movement, not language, is the foundation to thought, how language changes how we think, gestures, and the relationship to language, why she dove into the work of Leonardo da Vinci, and the importance of perspective taking.

It's time to listen and learn.

What are you interested in how the mind works?

I think everybody is puzzled by the mind.

Whether to pursue it or not is a choice and how to pursue it is a choice.

When I got into newly birthed cognitive psychology, I had been interested anyway in the visual -spatial world.

We are constantly experiencing it, so it's a large part of our lives.

Most of the theories at that point were based on what philosophers called propositions on language -like representations and attempting to reduce everything in the visual -spatial world to these propositional statements, which are minimal statements about truth conditions.

And that struck me as odd because the visual -spatial world precedes language evolutionarily.

It takes up far more of the cortex than language and it has to have its own logic, not a logic or a structure based on language.

If anything, I thought maybe language is based on the way we perceive and interact, perceive space and interact with things in space.

So I began looking at various aspects of the visual -spatial world, looking for its own structure and own logic.

And that brought me back to language because we use language to describe and talk about the world and the spatial language seems both primitive and primitive in the sense of its origins, both ontologically and phylogenetically.

We talk about where things are, where to look, what to do.

Those are all about our interactions with space and they seem crucial in life.

So it brought me back to language, but it also got me interested in understanding how we perceive and understand space and interactions with things in it.

And if you think about actions in space, the most basic action is whether to approach something or avoid it.

To go forward, we like something, we're attracted to it, we must interact with it, or we avoid it in one way or another.

And that simple basic action that even one -celled creatures must take seems inherently value -laden, emotional.

So is it fair to say that space and movement have changed how we describe things?

Is that fair to say?

Yes, and in fact, I think it's fair to say that all our spatial representations depend on interactions.

They're a coupling of what we do and how we perceive it.

And those studies go way back 150 years when Stratton and later others put lenses on people's eyes that moved the visual system or the visual perception left words or right words so you can do it upside down.

And people adapted by moving, by grasping things.

And if they didn't grasp things and recalibrate how space was represented in the brain, then they didn't learn to behave in the new situation.

And it's undoubtedly something we're doing continuously all our lives is recalibrating our many visual spaces to the way they allow us to behave in the world.

Has language changed how we think too?

The language that we use to communicate is also I would assume somewhat the language that we use to think in.

Right, and that I think goes deep on how, so there's beautiful work on rodents and other primates on how space is represented, the larger space that we move in, not space of hand interaction, but the larger space we knew.

And this won the Nobel Prize for the Mosers and John O 'Keefe, the combinations of their work.

So O 'Keefe with Nadel and other people had found in hippocampus of rats, single cells that respond when the rat is in particular places.

And what's beautiful about these cells is that a single cell brings together information about vision, smell, touch, the movement of paws on the ground and so forth.

All this multi -sensory information into a single cell and that encodes a place.

They thought, in fact they wrote a book called They Have the Campus as a Cognitive Map.

However, those place cells weren't arrayed spatially.

So the problem was still open.

How do you get all these place cells related one to another?

And the Mosers working in O 'Keefe's lab found grid cells, one synapse away in entorhinal cortex that arrayed the place cells relative to one another, essentially a spatial array.

And later work, work that came quite a bit later, beautiful work, showed that place, the single cells in hippocampus also in cold people, events in time, conceptual relations, and that grid cells mapped them in temporal space, conceptual space, social spaces.

So that shows quite beautifully, elegantly that the foundation for spatial thought is also the foundation for conceptual thought, for conceptual relations, because what the grid cells are doing is mapping relationships among single items.

So that is one cornerstone of the claim I've been making that spatial thinking is the foundation of all thought.

And I qualify that by saying it's just the foundation, not the entire edifice.

So language, I'm going back a long way to your question.

Rats don't talk. They communicate but not abstract ideas or so we think.

And even more, what we would call advanced primates don't seem to communicate in language or gesture more abstract ideas.

So chimpanzees, gorillas will point, but members of the con species won't look at where they're pointing.

And nobody so far has seen a gorilla or an orangutan or any of these other primates make a map in the sand to explain to somebody else how to get somewhere or even gesture directions.

So language does affect in humans, does affect the way we conceive of space.

We're all wandering around now with Google Maps trying to get us to all kinds of places.

And we tell people how to get to places.

So that gives human beings another layer or two layers if we include three layers, if we include maps, gestures and language for understanding space and for understanding conceptual relations that are spatial.

So our language reflects our spatial thinking.

We say somebody's on the top of a heap and we gesture up or we say someone's fallen into a depression and we gesture down and our body goes down to indicate up and down, correspond to being happy versus being depressed.

We've created a dimension, it's vertical.

And so again, the way we spaced to communicate to other people to think ourselves is layered with these gestures and language and diagrams that themselves have a basis in spatial thinking.

But they give us additional tools to navigate the world, to do things in the world like set a table or operate a bicycle, all the more manual activities that humans engage in, play a piano.

I wanna get into some of the implications of that in a second.

The question that came to mind is that people tend to think of themselves as a visual or verbal or mathematical thinker.

Is there any evidence behind that, or can you be good at more than one of them or how does that work?

Yeah, I mean, the disappointing is that you can be good or bad at any of them more or less independently.

So that's discouraging in some way.

You think if you're a visual person, if you're not very good at visual, maybe you're better at verbal and maybe you are because we do tend to compensate and use media in channels that we feel more at home and we're more adept in, but they do seem to be pretty independent.

And I should say that a lot of our spatial thinking is really not visual.

The taxi drivers in London that get you to any of these obscure streets aren't imagining a map as they're navigating.

They simply have a spatial sense of where things are relative to each other that may not, you might not necessarily always have a conscious experience of them.

Blind people navigate quite well, many of them, and it's spatial, not visual, and they'll navigate by other cues, acoustic cues, smells, wind, the texture of the ground.

These are all kind of landmarks and cues that keep people without sight oriented.

And I know many people who've been blind from birth or blind later who have amazing spatial abilities.

They can be excellent mathematicians and get around space better than I can.

So space is not just visual, and in fact the areas of the brain that capture space are multimodal.

They get input from many different modalities and output to them.

Now each modality is gonna have its special characteristics, but to some extent they're compensatory.

Let's talk about some of those modalities, I guess, in the sense of like, how does this change how we write or how we present material, and maybe we'll keep Zoom separately and some of the implications for that, but when we're writing an email or we're writing a document at work or we're giving a presentation,

what are some of the impacts of this?

I don't have direct information.

I can describe some studies that we've done where you're learning and how you're learning, and there we put people alone in a room.

They're not talking to anyone.

And they're alone in a room, and we've given them complicated material to learn, and they're going to be tested, and these are students, so they're used to learning and being tested, and they're good at it.

And in the material we gave them as varied, we started with descriptions of environments, so describing where for or landmarks were in a gym or a small town or a museum, and describing incidental information about the environments to make them more vivid.

And we told them that they would be tested, and they could read these descriptions four times, and we filmed them as they were reading, and as they were reading, they were making gestures.

So they're not talking, the gestures are for themselves, and for the environments they were drawing lines for the paths in the environments and points, dots for the various landmarks.

And they were doing it over and over the way you might rehearse a phone number, so they were rehearsing that, checking it, and doing it in different ways.

But for the environments, just about 70 % of the people did this spontaneously, we never suggested that gestures would be helpful, we never gave them a hint that we were interested in their gestures.

But about 70 % made lines for paths and dots for places, the way you would do if you were drawing somebody a sketch map of how to get from one place to another.

And when they did it, they remembered better.

And when we took a new group of people, we've run this now four or five times, a new group of people had them sit on their hands, and that group did worse on the memory test, and some of them even said, I can't think without my hands.

So this was spontaneous, we tried it with other material, how a car brake works, it's quite complicated.

And you can again see people making models of how the car brake works.

And here, a smaller percentage did it, but it again helped their memory, those who did it remembered better.

We've tried it with time, scheduling multiple events that happen at the same time, and people made matrices, sometimes with the knuckles on their hands, sometimes on their tables, sometimes in the air.

So people model them differently, but they make models.

Again, it's somewhere around 60 % of the people make these models spontaneously.

And when they do, they remember better.

So those models are a special motor.

For the most part, people weren't looking at their hands.

Some were every once in a while, but for the most part, their eyes were on the screen reading the material and their hands and their body were translating what they were reading into thought.

Or that's the way we conceive of it.

And when you watch it, you can't help but feel you're watching somebody think.

I mean, you see them tilt their head, think, look at the screen, gesture, going back and forth in that way that we do when we're thinking.

But the models were special motor.

So they were a separate channel from language.

And they're a redundant channel.

They're another way of representing the same information and the gestures are representing and they're representing in a more direct way than language.

A line on a map is analogous to a path on the road or a line that your fingers draw.

And a point is analogous to a place.

And so that that representation is more direct than saying path or road or saying place or landmark.

So we think it gets to meaning more directly.

And the fact that we use spatial language to describe these things like line and dot it seems to again indicate that our thinking about it is spatial visual and based on the spatial visual that we're representing abstract ideas in these simple geometries of lines and dots and boxes.

So you see that in diagrams as well.

I mean, people make, if I take the rudiments of a sketch map that it's lines and places, well those are lines and nodes or links and nodes.

And we can use links and nodes to describe decision trees or conceptual hierarchies or phylogenetic hierarchies, office hierarchies.

So we use that same way of conceptualizing, which is spatial.

For describing, for representing abstract things in a diagrammatic form, but also in language.

So does that have implications for how we present?

Like should we be presenting in more graphically in terms of bullet points and maps and diagrams and relationships, explaining them in a spatial sort of way to people?

I think it's a good idea and there's more and more of it.

Now that pixels don't care if they're representing words or diagrams, and we do have access to many, many, many pixels, so that making diagrams, it's certainly easier digitally.

And then it's easier in printing books before books were typeset.

And that gave an advantage to language, whereas diagrammatic things had to be done differently and integrated in a different way.

In a far more work -intensive way.

So there's still a bias, I think, in publishing towards language rather than toward diagrams.

The bias isn't simply the ease of printing.

There are all kinds of reasons.

But I mean, language is great, so I use it quite frequently.

So but I do think that diagrams go a long way to conveying conceptual information.

There is reams of experiments supporting that.

Now, diagrams in science, even in history for showing time and dynasties and so forth.

They can be well -crafted, just as texts can be well -crafted or poorly crafted.

So you have to take that into consideration, but well -crafted diagrams are superior to language for explaining many, many, many kinds of information more directly, more succinctly.

Diagrams also allow huge numbers of inferences.

So I mean, Venn diagrams for logic, you have intersecting circles, and you can make a great deal of a huge number of inferences just from that diagram that would take many sentences to describe.

And even those sentences, you have to put into a conceptual framework and the Venn diagrams, the intersections give you that conceptual framework, just like a phylogenetic tree will give you those inferences directly.

They're compact ways of incorporating huge amounts of information.

So certainly on the teaching side, diagrams can help enormously in almost any field.

Again, they have to be well thought through, certainly in data, in this line, bar charts and line diagrams and so forth go a long way to allowing many inferences about data and to allow the direct, more direct way of expressing it.

But we've found that producing diagrams is a good way to learn.

A study with Eliza Bobick looked at junior high students learning chemical bonding, and she taught them, she was a teacher in the school, she asked many other teachers who were teaching the same sort of material to participate.

So she taught the material in the normal way, you have text and diagrams and animations and explanations, teaching chemical bonding.

After a couple of days of learning, she divided, she tested the class to see how they were doing and then divided them into two groups.

The groups were equal on that test.

Half the students were asked to give a verbal explanation of chemical bonding, which is the normal way of testing.

The other half were asked to give a visual explanation, and then everybody was retested.

What's interesting is that everybody does better on the second test, even though they're not learning new material, the fact that they have to create an explanation helps consolidate and make sense of the information in their own minds, so that everybody's doing better on the second test than the first

test, even though they haven't had access to study materials.

But what interested and excited us was that the group that had made visual explanations did far better than the group that had done verbal explanations.

And we expected this for the structural information.

We divided the information into information about the structure of the molecules and the function, the action, the causation.

And we thought making a visual explanation requires you to put down the structure.

Surely students will put more of that information into the visual explanation than the verbal, and we found that.

But to our surprise, we also found better representations of the behavior, of the function, of the causality in the system in the visual explanations than the verbal.

So we started thinking, why does this happen?

And one reason we think is that making a diagram gives you a platform for inference.

You can make inferences about the behavior from the structure.

So you've got the structure, you can make inferences.

Another is that it gives you a check for completion.

You know all the parts are there because it is, if they weren't, it wouldn't be coherent, it wouldn't make sense.

So having the diagrammatic representation gives you a check for the coherence and the completeness of the information.

One word after another doesn't do that as easily, the check for coherence and completeness.

So we thought those were the reasons that the visual explanations did so well.

There are a check for coherence, a check for completeness, and a platform for inference.

They also force you to abstract, to find the essentials and put those down.

Now, in some sense, language can do that too, but it apparently doesn't do it as well.

That getting things in this diagrammatic form really does force you to be more abstract.

So that abstractness has been found by other people in other situations.

Dan Schwartz did a number of studies again with junior high students, and they were supposed to be diagramming something like the nitrogen cycle.

And many of them put in lily pads and frogs and that sort of decorative features.

They made these lovely little diagrams.

You often see them in books.

But when two people were working together, those individual decorative features disappeared.

And the more abstract stayed in, the diagrams can get you to the essentials better than a verbal explanation.

This is so interesting.

There's three rabbit holes I'm gonna follow up with here individually before we even talk about Zoom.

So the first sort of rabbit hole that I wanna explore a little bit is you talk about relationships and used Venn diagrams as an example.

And some of the relationships that come to mind that I think you also used vocabulary around are structure and comparisons.

What are the other type of relationships that visual tends to draw out for people in a way that's sort of almost 3D and allows us to see something that was previously invisible?

Yes, structure is important for physics, for chemistry, for social relations, for corporate relations, for business, getting the structure down and only the structure can be really important.

The relations, you're right, compare and contrast, and we do that quite nicely with gestures.

We can set up a spatial schema on the one hand, on the other hand, and then I can keep pointing left or right, and you know that I'm referring to that sort of compare and contrast.

I can order things, so I'm deciding which automobile to buy or which vacation to take, and I can order the criteria, the most important things at the top, and less important things lower down.

I can order things left to right in reading order, and we typically do that.

We put time going in reading order, which is left to right in most Western languages, right to left in some other languages.

Talked about them in more traditional versions of Chinese and Japanese, so and I don't really know how they, they also have horizontal ways of ordering time.

So we can order things, we can order things, we can group things into these networks, categories and subcategories.

We can make dimensions, and we do that.

So we can have ordinal scales, which are just ordered.

We can have interval scales, where the difference between items is meaningful, and we can certainly do that on a line or either vertical or horizontal, and then more absolute scales, where there's an absolute zero, as in temperature, money, and we can make, so we can make scales.

So we can diagrammatically represent many of the ways that we order the world, organize the world, and measure the world.

I think probably all of that can be done in a diagrammatic way.

Certainly math and statistics do it that way.

If you look at even formulas, in math they have a diagrammatic aspect, things like equal signs are arbitrary characters.

We put two sides that balance on either side of the equal signs.

So that's a spatial array of equality.

So we see those spatial visual aspects, even in writing we indent for paragraphs.

We space between words.

We make more important ideas in bigger font, so that we're using those features that are visual spatial, even in print, where it's almost strictly linguistic.

Of course, there are analogies in the way we speak.

We use intonation, we raise our voices, lower them.

We pause in dramatic places.

Those are all kind of almost spatial devices that we use to convey meaning more effectively.

That's fascinating.

I think as you were talking about the text, it's really interesting because we use the space to draw attention, maybe not to relationships, but to heighten our focus on what's important or what the author thinks is important and wants to convey or communicate with bold, with sort of fonts, with spacing.

Right, and we make tables, even in text.

And the tables again are diagrammatically displayed.

Truth tables, this was an observation by Revio Netz, who's a really brilliant, creative, classic scholar, has studied Greek mathematics.

But he pointed out that truth tables in logic, which were developed by the ancient Greeks, are tables, that things are lined up in columns and rows, and that that enables you to see the relationships much more clearly than if they were in continuous text.

So does that mean that when we're writing in continuous text or we're writing a book or a document that we should use some sort of visual to summarize or reinforce what we've written, not only allowing the reader to sort of scan and find the relationship and then reinforce that with maybe more detail and nuance

in the writing, but also to allow a bit of spaced repetition between what we just told them in words and allowing them to see it visually?

Oh yes, and again, it's there anyway.

Chapter headings are usually lower, and paragraphs have some sort of parallel structure in them, or we're taught to use paragraphs in that way.

So the visual devices are there anyway, but using them more to advantage is probably a good idea.

I've become a huge fan of graphic books, exactly because they integrate so beautifully.

Text of various forms, not just words.

There are all kinds of textual components to comics that aren't traditional words, but also visuals of many types, and they can be integrated seamlessly.

And I've tried to do that when I write articles to not have table one, table two, each with a caption, but to have it integrated into the text and editors never let me do it.

So it's gonna happen soon because there are more and more graphic books, and I think people are seeing how beautifully they can be designed and what artistry, both conceptual and visual go into it.

So one of my favorites is Larry Golnick, who's written cartoon guides to various forms of math, to the environment, to chemistry, to calculus, to physics and so forth, anything where there's an accepted body of knowledge, he'll do a graphic guide to.

And the books are enchanting.

He uses diagrams in ways that textbook authors never thought of, building them up from concrete to more abstract than using bits of the diagrams as icons throughout the book to remind you and bring you back.

There are mnemonic devices that are conceptual and bring you back.

So I think more and more books will be crafted this way.

I see more and more young people learning the rudiments of drawing because the drawing that you need to do for explanations or even for narratives are quite different from the paintings you see on the wall or the drawings you see on the wall that are standalone.

They have to be integrated and they have to, again, abstract the essentials, convey it in an appealing way.

So what graphic books do is what ordinary conversation has.

And that ordinary conversation is multimodal.

We are using words, we're using gestures, we're using intonation, pauses, all many, many devices beyond words in a row to convey meaning.

We interrupt ourselves, we interrupt each other.

And all of that multimodal conveys a message.

And if you look at people's spontaneous explanations in conversation and take out the visuals, so you're taking out the gestures, a lot of the meaning goes.

If you take out the intonation, a lot of the meaning goes.

If you turn it just into words, a lot of the meaning goes.

And because we're so used to learning from words and explaining in words, written ones, not ones in conversation.

And we get quite good at that.

I think we become blind to the multimodal ways that we convey information.

And graphic books use that.

They can convey music, they can convey smells, and they do it all through kinds of symbols and so forth.

I can't do it now, but I've collected along with John Bresman, who was a former student who was a comics geek, just brilliant with comics.

We've got a huge collection of innovative ways that graphic books can convey, meaning they're poetic and inspirational and multimodal in this way.

Did gestures override language too?

Like, I'm thinking of a scenario where somebody says yes, but they're shaking their head no.

Right, and my husband, who was a paratoper and had an uncanny sense of direction, right and left in any language eluded him, as it does many people.

People confuse right and left for good reason.

And he would point one, he'd say, go right and point left.

And I knew, follow the gesture.

It's not, the language is not, it comes later, gestures precede language.

And they're more in tune because they're more direct.

Susan Goldenmeadow and some of our colleagues, she's done gorgeous work on gesture, especially in children, looked at children on the cusp of conceptual thinking.

They're going from this sensory motor.

So you get two cups, one is thin and one is fat and pour water from one to another.

And children who are small think that pouring the water changes the amount in the glass.

And you asked them to explain.

And children on the cusp of getting that idea will gesture something like pouring before they can articulate it.

So they've got the concept of conservation, that pouring water doesn't change the quantity.

They've got the concept, they can't say it in words, but they can show it in gesture.

So their understanding is ahead of their ability to articulate it, which makes sense even for us, not just for children.

The articulation is an additional layer that's really quite difficult.

Do gestures also help us remember?

And I'm thinking like the last time I was lost when somebody explained directions to me, I'm just remembering that the two things that happened when I was lost, I turned down the volume to increase my focus sort of on what I was doing.

But the second thing when somebody explained directions is I was actually like, okay, so up and right.

And sort of like I was using a gesture to help me remember.

No, absolutely they help you remember.

And part of it is a different code.

And adding codes adds redundancy and that adds retrieval, ways of retrieving information.

And so certainly adding another code.

But again, because your gestures are directly representing going up and turning in a way that language doesn't.

And I found getting lost in many places and asking for directions in languages that I don't really know, watching the gestures helps.

And people don't say things in words that they do in gesture, like going up or turning slightly that a road bends, they won't say that.

They might neglect saying that.

But it'll be there in their gesture, but certainly mimicking.

And there's some evidence from the neuron mirror work that we're mimicking even without externally showing that we're mimicking.

So there's a lot of work on understanding through enacting.

I mean, there was old work on language that people understood language by evoking it internally in themselves.

I have a good friend who now repeats what I'm saying in order to understand it.

Now he doesn't do it out loud.

It's very softly. And I've seen other people do that.

That as they're listening, they're doing a little bit of talking.

So they're externalizing that internal information.

But there's beautiful work, brain work on watching dance, videos of dance.

And you can take experienced dancers either say ballet or capoeira, this very acrobatic South American dance, and have them watch videos of ballet or capoeira.

And in both cases, the motor areas of the brain will be active.

But to be far more active for the dance that you can enact than the one you can't, again, indicating that something in the body, in the brain and probably in the muscles, there is evidence from other work in the muscles, is resonating with the action that you see and incorporating it into your body.

And probably when you're watching someone giving directions, you're doing that.

Certainly for direct physical activity, you know, a tennis serve or a dive that you're going to be incorporating that into the muscles and your brain.

One of the things that as you were saying that gesture sort of override language, when you were using the example of giving directions, but and your late husband in terms of pointing one way and saying a different way and realizing that gesture was more right, it was more true because it was lower language

sort of in your sense or your body or your brain than language, so it comes before language.

And that speaks to the whole notion of body language when people are communicating, when they're saying one thing, but their physicality is saying something different.

To some extent, we do learn not to express certain emotions because we know that people can read them and people read them very quickly, and in many cases without awareness.

So we do learn to suppress some of that, not to show that we're afraid or unhappy or upset or depressed, we learn to suppress that at some cost probably.

And how much, how good we are at it varies with people, how good, how much we want to suppress it, but there is an awareness, I think on the part of most people that the body language does communicate.

And sometimes we don't want to communicate certain aspects of the body language.

Some of it, I become more and more aware of it as I study gesture and someone wants a meeting to end, they stand up and that's a signal.

If you're around a table, who you look at next gets the floor and women aren't looked at.

So this is, so there are aspects like that of that are social, of how we interact socially, that again have effects, in the body language has effects on the conversation, on the speaker or the communicator and on the communique.

So as you were saying that two things stuck out, one you said that we don't tend to look at women, I'm interested in sort of exploring that a little bit.

And the other thing is how does Zoom change this?

So what are the positive and negative aspects of communication over something like Zoom versus face -to -face.

And how does communicating over video, which sort of changes our spatial perceptions, affect what's communicated and what's understood?

Like face -to -face communication uses our whole body and even the language we use, right?

We say this or that, or we look at something to draw attention to it.

Yeah, no, I'm with you completely.

And the research needs to be done and probably is being done.

And obviously it depends on what's being communicated.

A big thing for me is that we're only seeing face and shoulder so we don't have a shared workspace.

And if you look at designers or planners or any kind of committee, you often have a whiteboard or something that's central on the table that everybody can point at and add to and subtract from and that a great deal of the communication and the joint understanding is externalized in that way.

And we've done a small amount of work and others have too on collaborative design and found that when people can work over the same diagram, the communication is much more successful.

People are looking at each other's hands on the diagram, not at each other's faces.

The words are annotating that diagram.

They aren't independent.

So people are more, the communication is better, the result is better, the product is better.

People are happier with the collaboration when they can do that, then when it's just over the phone and we have separate diagrams.

So I think for any kind of collaborative work and teaching is often collaborative.

Conversations certainly are, but in many cases, we need that external representation of our shared understanding.

And there's a huge amount of research and practice dissolving conflict that way, that if you take conflicting parties in an organization, between organizations, put everything up on a whiteboard with a facilitator.

And then you realize the points where you misunderstood each other.

You use the same words to refer to different things.

You can come to a consensus much better and feel better about it than if you don't have this external space that you can all, your joint thinking space out there in the real world.

So that's a real drawback of Zoom.

Now I can talk and lecture with my PowerPoint, but I can't animate it as well as I can in a classroom where I'm standing and moving and pointing and gesturing big.

I mean, you make big gestures and they look like these giant hands and they're disturbing.

Whereas you don't have that in real life.

So these are some of the disadvantages, not seeing the nuances of facial expression, not hearing the nuances of vocal expression.

But I also found in teaching students, I have many students for whom English isn't the first language.

So it was easier and being in a classroom with masks, I couldn't have understood them.

They couldn't have understood me.

So Zoom for the pandemic had that help.

I could also put up transcription so that if I didn't understand their accent and they didn't get all my words, they at least had that other code.

I mean, transcription is funny sometimes, but it is self -correcting, it's quite remarkable.

So that helped. What also seemed to help interestingly, my classes were around 20.

So everyone, if I wasn't speaking from a PowerPoint, everybody was there on the screen.

We're each a box, we're equal.

So it seemed to encourage more discussion from the students.

Everybody could see everybody's faces, whereas in a classroom, everybody's facing me.

And I'm facing them, but they don't see each other.

And I'm standing, they're sitting.

So I get a primacy as a teacher that I don't have if I'm a box in a screen.

So it seemed to have equalized, democratized, encouraged more students to just treat me as another box.

I think other instructors had a similar experience.

If your class is 140, that doesn't work.

The breakout sessions worked again for teaching.

I don't know in corporate situations, they might work, but it allowed the students, in fact, it gave them more opportunities to interact with each other than being in class would.

And that was enormously facilitate the breakout.

So there were features in Zoom that I could use teaching.

Again, I can't speak for these corporate experience.

I should ask my sons, both of whom had, lived from one Zoom meeting for another for several years.

But yeah, it is fascinating.

And I think the need for gestures and a shared workspace is something that I hope designers of these media know about and will make efforts to correct.

There's only so much you can smash into a screen.

So then just having the smaller space is, until we can do it surround and in VR, then that will go a step further.

Sounds like one of the important aspects of Zoom, I guess the two sort of things that we can take away from that are one, it hopefully it encourages more diversity of opinions to be expressed rather than the same people expressing opinions over and over.

And the second thing that I took away from that was that if we're going to communicate or facilitate over Zoom, then a shared visual space is super important, allowing people to not only focus on something, but also see and view representation.

Right. The second rabbit hole that I wanted to, I promised three rabbit holes before we get to the sort of like next big theme, but we did explore Zoom, which was good.

You mentioned that when we create a diagram or a visual representation, what we're doing is we're consolidating, I'm trying to use your words here, we're consolidating and we're creating an explanation.

And what that drew attention to in my mind was the process by which we learn, I call it the learning loop.

And so my belief on this is that we don't learn from experience, we learn through reflection.

So we have an experience, we reflect on that experience.

And from that reflection, we draw abstractions or relationships and those abstractions and relationships become an action that we take the next time we're in a certain situation.

The underrated part of that is the reflection, because the reflection is the distillation of all of this information into its salient points, into the relationships that matters, the variables that matter.

And maybe this sort of goes to my third point, which was a third rabbit hole, which was it forces us to abstract.

I'm curious as to how you think about that reflection component in terms of not only learning, but in terms of how we encode this information in our minds.

Yeah, no, I think all your observations are astute and couldn't agree more.

And there has been some research on that.

So sheer rehearsal of material we're trying to learn isn't as helpful as trying to draw implications, make comparisons, do the sorts of reflections that you're referring to and there again, just as you asked me, what can you put in diagrams?

And I said, categories and hierarchies and themes and one to one correspondences and much more orderings, those are the sorts of reflective processes that we might undertake.

There has been research showing that if we want to transfer learning, and of course, that's one of the goals of learning is taking more than one case that are similar and abstracting what's similar about them is going to help you transfer, single case isn't gonna help you, two cases are better, and two

cases and a diagram indicating the relationships is even better.

So that in making those abstractions that can then get applied to new situations, it's hard to go from abstractions to instances, it's even hard to go from instances to abstractions, but probably from abstractions to instances is even harder.

And I've seen so many phenomena in cognitive psychology be shown over and over again in particular situations because until they're in a particular situation, no one believes them and I see them and I say, it's just the same as X what's new here because I've got the abstraction, but other people need

the concrete showing that particular thing in this situation, same in politics, same in history, we have it all over.

So going from instances to abstractions is one thing, it's not that easy and more than one case, drawing the similarities and it diagramming the abstraction helps you get to that abstraction.

But getting from the abstraction to the particular is still a challenge and I can't think of people who've worked on that on teaching.

I wanted to say one more thing about education.

Most of our education is vicarious.

We're not encountering, you said we learned from situations the situations for us, school people is classroom in many cases, sometimes it's actually experiences and of course classrooms that create experiences through laboratories or field trips or whatever are more effective because learning from experience

is effective, but so much of our learning is vicarious and through language or other math, other abstractions.

So making it concrete is important.

Just to riff on this for a second, experiences don't need to be a physical experience and experience can be reading somebody else's experience, I'm experiencing it.

A conversation is an experience.

Reading a textbook is an experience.

Doing something like building a sauna or a shed or something is an experience.

So these are all different types.

The way that we have an experience is very different from how we reflect on it and how we abstract it.

And I think that the experiences don't necessarily need to even be firsthand.

I can learn from your experiences, but this gets to a different point here, which is when we consume information, a lot of the times we're consuming the abstraction.

So these diagrams are the abstraction, they're the salient points, but they're one person's idea of what the salient points are.

That's not my encoding of it, that's their encoding of it.

And I think of consuming other people's abstractions as the illusion of knowledge.

I'm the line cook, not the chef, right?

To use an example from a friend of mine, Tim Urban, which is when things go right, I look like a genius.

I follow the recipe and everything works out.

And because I've consumed the abstraction, the recipe in this example, everything turned out right, I look like I know what I'm doing.

But when things go wrong, the chef who has distilled the experience, the reflection to create the recipe that has gone through the thousands of trial and errors will taste it and be like, oh, this needs more salt.

You use too high of heat.

And instantly they will know what the problem is because they've got the abstraction, but they've got the, it's almost like an earned abstraction versus a learned abstraction.

And that's really interesting because it changes when we're consuming information, we're consuming a lot of these abstractions, we assume their knowledge and we assume there are knowledge and we put them into practice.

And when they don't work, we don't know why.

And the way to go back up this chain is to actually decompose it and ask people, well, how did you come to these conclusions?

What variables were important to you?

And you need firsthand knowledge, not second, third or fourth hand.

So you wanna get as close to the person who had the original experience or created the abstraction.

And often when you pull back the onion here, we're two or three layers deep and it's a very sort of weak relationship.

No, no, you're absolutely right.

And you think about teaching a sport or piano or diving tennis or of any of that, you can give verbal instructions, but you've got to do the serve over and over and over again and the dive over and over again because translating that language, that abstraction into the body, into your body is going

to be very different.

And a good coach can watch you and see the tiny place where you're leaning forward too much on the dive or breaking the jump too quickly so that you're not jackknifing high enough.

A coach can see those, my new things and tell you and then you have to work to incorporate it into your body.

And a similar thing will happen on cooking.

I'm listening to when the boiling of the rice, I'm smelling the olive oil on the saucepan and I'm doing as it's heating and I'm picking up the danger sign, got to lower the heat.

That's only from a great deal of experience and paying attention to the smell and the sound of those cues.

And that isn't in the recipe.

And you will go crazy if you had to pay attention to those details in a recipe or an instruction to dive, you'd give up.

The abstraction you get either in a diagram or verbal instructions isn't going to enable you to be a champion diver or a great cook.

You can make satisfactory things and you need to know, not just adding salt, you need to know when to throw it in the trash and start all over again.

And you know, potters know that.

Something doesn't rise properly and it's hopeless, go and start again.

And I think any creator knows when things simply can't be corrected and when they can be corrected.

And again, a good coach can tweak, can use language to tweak and abstractions in language to, how does it feel in your body?

You know, your body should feel this, the muscle tension, because a lot of that isn't visible.

I mean, I learned a bit of yoga and that sort of thing.

You have to really feel it in your body and which muscles are working and which ones aren't to know are you doing a proper stretch and are you really stretching the muscles you need to stretch.

Weightlifting is similar.

You have to get it into your body in that way.

So an example I use sometimes in navigation.

So I talked about sketch maps, getting you to one place or another.

So even though sketch maps could be analog, they aren't.

They don't give the exact distance and they don't give the exact angle of turn.

They're approximate.

And they're approximate in the way that verbal instructions are.

You say make a right, turn right.

You say grow to how you get to this street and then turn.

We don't specify either in a sketch map or verbal instructions, the exact angle or the exact distance.

But those are meant to be used in the environment.

So if you come to an intersection and it isn't 90 degrees, it's less or more, you're gonna go the way the road goes.

So you use that information as if it were approximate and you know it's being used in coordination with the world.

Same with distance.

You're not measuring the distance.

It's until you get to that next intersection or highway cutoff and you'll go to there and you know approximately what the distance is, but you don't need that complete information because it's meant to be used in a context.

And I think most instructions have to be that way because again, the detail would drive you crazy and you have to assume some knowledge of a cook or some knowledge of a piano player, for a piano player to read the notes and translate them into actions of the hand.

And again, where it goes wrong, a good teacher can supplement.

Or hearing yourself.

You can be your own good teacher, right?

You know that dive didn't work out quite right.

What were you doing?

And the piano, and you do it over and over again.

Until you get the right sound.

And we do that speaking, you know, learn a foreign language.

What's the accent? And I can make words and sentences in languages that I don't know that well.

And I know they sound off.

And can I get the accent right?

And I go over and over.

And I can even hear it in my head that I don't have it right.

Or the dive right. There's something missing.

But then you're reflecting, right?

So you're creating your own abstractions, which I think is, I think one of the takeaways if maybe you can challenge or push back if you don't agree is that abstractions are necessary.

We need to consume other people's abstractions.

We would get overwhelmed in the detail.

And a different way to think about that is earned versus learned knowledge.

So this is learned knowledge versus earned knowledge.

Earned knowledge involves, you know, you can't necessarily be taught all the details.

You have to sort of do it or embrace, you know, dive into the details.

And I think where this becomes important in life is to think of decisions, for example, and cost of failure.

So when the cost of failure is low, it's okay to use learned knowledge.

And when the cost of failure is high, you wanna make sure that you're not using learned knowledge and you're using earned knowledge.

Or at least if you think of those things on the continuum, you wanna be more towards earned knowledge.

The more, the higher cost of failure.

But I'm just throwing that out there and sort of pontificating on the spot, but I'd love your reaction.

So I wonder about that.

Some of the high cost decisions we make, we can't have had experience.

Choosing a spouse. Oh, but trust, hold on.

You have thousands of experiences with trust with people all over your lifetime.

Sure, but it's more than trust.

It's really a huge amount of trust, a lifetime of trust.

It's trust about your physical, your body.

It's trust about joint decision -making, how we're going to grow each of us in life.

If we haven't had a family, it's how we're gonna do that.

You can't accumulate enough experiences, same with getting a job.

You can't try many, many different jobs because jobs are so complicated, what you do, how you relate to the people around you, how you relate to your boss, what happens when things change.

So, and there I think that I'm curious.

I mean, people talk about gossip as being informative and hearing about troubles that other people have gotten into in either in work or love.

And trying to map that on your own life is even babies.

After our first kid, a friend came over, a guy deeply ingrained in economic theory came over and wanted to tell us details of how to care for this baby and diapers.

He said, you learn so much, you wanna pass it on.

But so all of that and new mothers, new parents talk endlessly about crying and feeding and diapering and so for things that ordinary adults would find boring.

So I think even on big decisions, the vicarious, how to raise children, who you should get involved with, when you should leave, all of those things, we need other people's stories as a way of making the big decisions.

As you're speaking, I'm thinking you're probably right.

And there's a huge body of knowledge that comes from avoiding problems that are learned knowledge and not necessarily earned knowledge because you don't want to, you don't have enough time in life to earn all the knowledge from all the problems.

And so you have to adapt and incorporate learned knowledge.

And yeah, thank you.

I think you're on to something there.

I wanna switch gears to part of your book, you talk about the nine laws of cognition.

I think they're more like nine generalizations, but let's start with the first one.

There are no benefits without cost, meaning creativity versus learning.

How do they offset each other?

So the big example there I think is, one big example is categorizing.

So if we awoke a new as a newborn each day and didn't know how to categorize things into animate, inanimate, parent, friend, chair, table, these sorts of things that we do, that our brain does quite quickly.

If we didn't know how to make categories very quickly and had to depend on new learning every day, we'd never advance beyond that new learning.

And people wonder when they have a baby, what are they looking at?

What are they thinking about?

They're doing a huge amount of learning about what the world looks like and the acts like and sounds like and smells like and what happens in time.

These are huge things that a child needs, a baby needs to learn.

Some of it, the proclivities are built in and even maybe some of the kinds of knowledge, but a great deal of it, words have to be learned.

And if we had to go through that learning every day anew, we'd remain helpless infants.

So we need to categorize, but any kind of categorization has error in it.

We can miscategorize, I've mistaken twins for their sibling and said hello and wondered why they greeted me so coldly.

I think that's an experience many people have had.

The extreme leads to biases that I'm a small woman, so alone at night, I'm very alert to any kind of danger.

And because I'm hyper -alert to any kind of danger, I can miscategorize what might be dangerous to me, but it's something I need to survive in the streets.

So we can miscategorize, but we need to do it.

And so that's one of the costs and benefits of anything in cognition.

And there's a reason for every one of these biases that is, there's an underlying reason that's deeper, more abstract or broader that where these things have been useful in one way or another, but their use is highly limited and can lead you into trouble as when somebody pulls a hand out of a pocket

and another person thinks it's a gun.

So it can lead to error and sometimes catastrophic error.

You can't live with them, you can't live without them.

And there are just many cognitive and social phenomena that are like that, that the efficiency of say enacting and action over and over and over again, that you get very good at it, but you can apply that action, misapply it to situations where it's not appropriate, and then it'll get you into trouble.

I think that's something people often don't see.

The second generalization is that action molds perception.

Right, and there I gave the example of the distorting lenses, that we don't accommodate to lenses that distort unless we're doing the actions and the actions actually mold perception, mold our perception in that sense, they do shift the world if we're acting in a shifted world.

I mean, it can happen in, in a more general form, if we decide that somebody isn't worth interacting with because they seem like, remind us of somebody who hurt us in the past, then we don't act, we don't interact with them.

So we're taking that, our very action, not only changes, it changes our perceptual world, we won't include that person.

And that's another kind of error that we can make from the first law of cognition that we get experienced at knowing what kind of people we can trust, what kind of people might hurt us, what kind of people to stay away from, but we can make mistakes.

And that's a speed versus accuracy trade -off.

Yep, and sometimes we have the opportunity to correct that mistake, that certainly happened to me where somebody whom I dismissed and somebody else said no, and I had new encounters and found that I was wrong and correct my impression.

But if I never had that opportunity to correct, I would have been left with that misimpression for a long time.

So what we, our actions affect the world we experience in big ways.

We move to a new city, we take a new job.

All of those things, we have a baby.

All of those actions change our world completely.

The third generalization is that feelings come first.

So that's work of Bambi Zions and many others after that that he showed in what seemed like random shapes to people and got affect, liking and memory judgments.

And people had, the more people saw a strange stimulus, the more they grew to like it.

So familiarity is good and, or at least a modicum of familiarity too much might be harmful again, but they couldn't remember them.

So the systems are somewhat independent.

Our affective reactions and our memory are somewhat independent and the affective seems to work faster.

And that again might be tied to danger and avoiding danger that if there are things in the world that are dangerous to us, we better know it quickly and act quickly before we might remember or not that we might think someone's dangerous and withdraw before we remember, no, no, that person looks dangerous,

but is fine. In just because there's such a high cost to the danger, responding first to the emotional is probably adaptive and keeping the system somewhat separate is probably adaptive in any case, the brain does it.

And the fourth, which was one of my favorite ways of putting it anyway was that mind can override perception.

Yeah, and that's where we know and that'll be kind of system two as opposed to a system one that we then the person that looked dangerous, we then next time we encounter, next time we remember that as an exception.

When we're, this isn't perception, this is action.

When we're repressing emotions, because we don't want, we think revealing them will harm the other person or harm ourselves, we're using our mind to overcome our first reaction, which is to burst into tears and bursting into tears or screaming, neither of them is gonna help us at this point, so that repressing

it is probably a good idea.

Does this work in reverse too, which is like that we don't wanna see information that disconfirms our existing beliefs, so our mind blocks out.

Yeah, so it's, you're pointing out nicely that everything has an advantage and disadvantage and that mind overcoming can also lead us to self -delusion because we're trying to fit the world into our beliefs.

I should have said that.

I think I got it from one of your other interviews for what I was doing research for this.

The fifth one was cognition mirrors perception.

Yeah, and there are all of these things other people have pointed out, they aren't my invention.

Many of the biases in perception and perception we categorize and cognition we categorize so that many of the phenomena that we see in perception end up being in cognitive phenomena as well.

So studying both is useful.

What happens in sheer perception, what happens in cognition, the borders are murky anyway, so yeah.

And let's just briefly go through the other ones.

I mean, we can spend a little bit more time on the ninth one because I thought that was super interesting.

But six was spatial thinking is the foundation of abstract thought.

So there I think I spoke at length at the beginning on that, that the brain structures, the brain strata that encode spatial or information and code relational information or spatial information cause relational information can be specialized in for time, for social relations, for abstract.

And again, we've seen it in language.

So it's there in brain, it's in language.

Somebody's at the top of the head, people have grown close together and we see it in language, we see it in gesture, we see it in diagrams, which again are spatializations of thinking.

And seven is one I think we're sort of all familiar with but it's also invisible to us which is the mind fills with missing information.

The information we get from the world is incomplete.

I mean, the work on change blindness where you show two photographs in rapid succession and large things are missing from one like the engine of a jet airplane and people don't even notice it.

So it takes a long time to notice, even when you know it's an engine it takes you a while to notice it.

So the world that we see is in the head, not out there.

We have the sensation that it's out there but it's in the head and we're filling in.

When we make errors, we know that.

So my vision is going and I will fill in shapes with a uniform color that it isn't uniform at all.

When I get closer and get better acuity, I can see that it isn't, but the mind forms, the visual system makes shapes and the colors are added the way you fill in the blank.

And so I'm aware of those errors.

But we're, and again, it's one of those things we couldn't function if the mind didn't fill in.

We don't know what's behind us.

I mean, I do because I can see myself in that, right.

But I, if I'm in my office and my books are behind me I can reach for a book, I know where it is.

So the mind is filling in constantly for what isn't there.

In understanding sentences, we can miss a lot of words and fill in.

Again, making errors makes us aware of how much we're filling in.

The eighth and penultimate one was when thought overflows the mind, the mind puts it out into the world.

So there we put our mind and the world in many ways, in language, in action, in gesture, in making lists.

We put the things we need to bring to work or to wherever we're going by the door so that we remember them, David Kors pointed out, we lay out the materials for a recipe in the order that we'll use them on the counter so that it's convenient to make, to enact the recipe.

So there are many, many ways that we put the mind in the world.

Again, I think it's probably uniquely human.

I don't think that gorillas make trail markers for how they should go in a path.

I grew up in a small town in a town that had been occupied by Native Americans and the park I walked through to get to school had various trees that had been shaped as trail markers.

So there was a huge oak tree that was an H and there was another one that was bent into the ground and they were trail markers so that people wouldn't lose their way.

So we do it, I mean, birds mark territory.

This I know because I used to run into the territory of a bird and it would attack me, which was rather disconcerting.

A friend who was an a theologist told me where I had.

That worked, but birds mark territory.

But so there are small bits of that kind of externalization of mind.

Bees tell each other where to go, but they're very limited.

But they're still extremely interesting that other creatures do put their minds into the world and partly to communicate.

They just don't do it in this enormously plastic way, an inventive way that we do.

The ninth generalization was we organize stuff in the world the way we organize stuff in the mind.

So we've talked a little bit about how we organize things in the world.

We categorize, we make hierarchies of categories.

We order things, we make one -to -one correspondences in giving in place settings and other situations.

We organize by themes, our houses, that way everything for cooking is in the kitchen, everything for cleaning is in the bathroom.

So these are things from different categories, but they're organized around a common function or a common goal.

So, and we do that in the world.

So we have buildings that have one -to -one correspondences, windows for each apartment.

We have, we draw lines, which are roads in the world for the paths we can go on and make intersections.

We even diagram the world so that if you look at an aerial photograph of a city, you can see there are lanes for people, lanes for buses, lanes for bicycles, where people can go, where buses can go, where you have to stop, where you can turn, where you can't turn, where you can park.

The world is communicating to us not just indirectly in these categories and hierarchies.

Furniture stores are arranged by theme.

But so the world is communicating to us not just in categories and hierarchies and themes and orders, cues for getting into the theater, but it's also telling us how to behave.

So it's controlling our actions by the way it's diagrammed in the world, turn signals and turn arrows.

And if you think about the world that we grew up in and that our children are growing up in, it's so different from the world of hunter -gatherers.

I mean, we've designed the whole world.

You fly in an airplane and look down and there's hardly a spot.

Humanity hasn't designed, touched and designed.

Now other creatures designed too, there are beaver dams and territorial rights of birds and so forth.

But we've diagrammed it and we've designed it in much more detail and we've designed so much, almost all the inhabitable world.

So how about the inverse of that?

How does the physical environment influence how we think?

And if the physical environment does influence how we think, how can we design that to improve our thinking?

So diagrams are one way to improve our thinking in the small.

There's certainly one and even written text is a way of improving our thinking, making lists.

We have so many, all those ways of externalizing mind can help bring it into the mind and improve it, especially if it's well -designed.

When you think about the world, I can go into a supermarket in any country, buy food and put it on the counter.

And that's a communication to the person at the cash register that I wanna purchase those items.

I don't have to talk.

It's the way the world is designed and it's universal.

Now I have to go to the automatic cash registers and find the barcode and scan it.

And I can do that now in any country except China because I can't pay because I don't have the right app on my phone.

It does change our actions in many ways that makes it more efficient.

I can't help but think that babies growing up in a world where their homes are organized by themes and where books are on a shelf in an order, where toys are kept in different categories and so forth, that that can't help but influence their minds.

And I wish someone would study that.

I remember my own children and if they're in the living room, everyone's talking and playing and I'd pick up a child and walk toward the bedroom.

They'd scream because they knew they were being taken away from the social environment and put to bed.

And so they knew from the organization of the house what was going to happen.

If they were brought to the kitchen, they would calm down because they knew they were going to be fed.

Children growing up with that kind of organized, designed world are certainly using the way the space is designed, the way that hunters and gatherers used the natural design of the world.

And people use the environment as a way of organizing their behavior, of what to approach and what to avoid.

But the design world now seems to communicate so much more.

Where's the door to a building?

Where do you put your car or your bike or your stroller?

And where is the food and so forth?

So all of that, it seems to me, is changing or is affecting the cognition of everyone around.

So I have had the pleasure of visiting Japan many times and there are huge roads that intersect.

And at first you sort of go along them, but they're unpleasant.

And then you learn everything's done in quadrants and quadrants inside quadrants.

So you start walking inside the quadrants.

And it's a completely different world.

And then in Japan, you see a torii gate and you know that's gonna lead to a temple.

Now that won't happen in New York City or Paris.

So then you have to navigate differently.

You spent a lot of time diving into the work of Leonardo da Vinci, I'm curious as to what you learned.

That was a wonderful opportunity.

An incredibly talented musician friend wrote an opera on da Vinci for the 500th year of his birth.

Unfortunately, the performances, the pandemic interfered as it did with so much of our lives.

It's Jonathan Berger and Jonathan's at Stanford.

And whenever he has a new opera and his operas are nonfiction based on real lives and really enormously moving, was one on hallucinations that stays with me years later on personal experiences with hallucinations that were gripping.

And he turned them into opera.

And one of the people whose experiences he turned into opera said afterward, Jonathan conveyed my anguish better than I could have.

So very moving, he made an opera on da Vinci.

And whenever he does an opera like that, he has a symposium where he invites people with different views on the topics to speak.

And I had the honor of being invited and use the opportunity to talk about da Vinci's sketches.

And there are millions of them, not millions, but thousands of them.

You can find them thankfully online in many cases, because in many cases they're in libraries all over the world.

And he invented uses of sketches that weren't reinvented until many years later, because his sketches were unavailable.

So compare and contrast, you've talked about that.

He did a great many like that of heads and then skulls and then the brain structures inside, he was looking for the soul.

And that was the kernel of the opera, was looking for the soul and looking for the soul in the brain.

So, and thought he founded at the confluence of sensory pathways.

So there are cutaways, there are compare and contrast.

You see him and this is a phrase I'm borrowing from a catalog on an exhibit of Raphael that's currently at the National Academy of Arts in London where somebody pointed to Raphael's sketches as thinking out loud.

And that's what da Vinci's sketches were.

You can see in the same page, many sketches from different perspectives.

He's thinking out loud, how to prepare the painting.

So he used these sketches in especially the cutaways and the compare and contrast in ways that again, medical illustrators didn't come up with till centuries later and were truly inventive.

Now I haven't seen sketches of other creators at the same time, but da Vinci was unique in being a creator of art and a creator of architecture, a creator of implements and so forth.

So he was an engineer and an artist, and he created events, the talent was explosive.

So and similarly, the use of sketches to inform him because they were for him, not for show, although in many cases, they're as beautiful as the paintings because of - It was his way of understanding.

Exactly, thank you.

Yeah, it's his way of making sense.

And you can certainly see that today, many people making sketches, it's their way of making sense of things, of data.

You display it in different ways to make sense of it.

Yeah, and then eventually how to show it, how to use it to communicate.

In your writing, you've used the term cognitive collage.

What did you mean by that?

So people have talked about cognitive maps and a term that came from Ptolema who showed that rats learning roots could put those roots together into a map of where things were relative to each other.

And of course, the beautiful work on place cells and grid cells has shown that conclusively that rats have ways of organizing places in a space that's viewpoint independent and then navigating in it or using that to navigate.

So we studied along with many other people various errors that people make systematically in making judgments about spatial relations, distance and direction in particular.

So people think that Philadelphia is north of Rome when it's actually south of Rome.

And it seems to be because people line up Europe and North America and Europe is actually farther north than North America.

And anyone who goes to Europe at this time of year knows the sun sets very late in Paris and Rome compared to the Northern American cities.

So people line them up and then they see that Philadelphia is a northern part of America and Rome is sort of south in Europe and thinks use that to infer the direction between Philadelphia and Rome.

People think that an ordinary building say Jacques house is closer to a landmark Eiffel Tower than Eiffel Tower to Jacques house.

And that violates any metric assumption.

And it seems to me because the Eiffel Tower defines a neighborhood.

You might say I live near the Eiffel Tower.

You wouldn't see, unless somebody knew Jacques you wouldn't say I live near Jacques house.

So landmarks in any city will tell you, I live near NYU or I live near Columbia.

Columbia I can say from New York and people will know about where I'm living.

And so Columbia and NYU for academics define a neighborhood.

So ordinary things are drawn into the space of a neighborhood of a landmark and that defies any metric representation.

Eleanor Roch has shown that in judgments of color, magenta is closer to red than red to magenta.

And again, it's similar, magenta is a form of red, red includes many shades of red.

So she'd shown that in prototypical categories that the things are more similar to the prototype than vice versa.

My husband showed that in social political situations people think a son is more similar to a father than a father to a son.

People think or thought years ago probably think again now that North Korea is more similar to the PRC than the PRC to North Korea.

So those asymmetry, so that you have those asymmetries in conceptual relations as well as in spatial relations.

So if in there are many other errors like that, Milgram famous for his work on punishment spend a sabbatical in Paris and found people straighten the sand which makes them take longer routes in many cases cause the sand actually forms a kind of upside down crescent in Paris if you orient the map North.

So there are many errors like that that have been documented.

And if you put them all together you don't get a coherent map, you can't.

If you add to that maps that we know about environments from maps and use them to navigate or memory of maps to navigate, we use verbal instructions go up in that street until another street turn right.

We were mixing all that information and coming up with a judgment of how far things are what directions things are, how long it will take me to get there which is confounded by means of transportation.

So any of those judgments we're pulling together disparate information.

How do we combine it?

And it's not gonna be combined in a two dimensional or even three dimensional map nor do we have them stored in our ads.

We have partial information stored in our heads and making it coherent isn't by no means easy.

And so I called it, I said the analogy is probably to a collage, it's multi -modal.

It isn't coherent in a metric sense but maybe more beautiful than a map.

I wanna explore one final topic today which is the importance of perspective taking in terms of how we interpret things.

And I know you've done some work where you got people to think creatively and one of the ways that you got them to think creatively was to ask them to think about different roles of people.

So like how would a doctor use this?

How would a gardener use this?

I wonder if you can talk to me a little bit about that.

Yeah, sure. We've done a lot of work on perspective taking starting with the map situation.

Are we taking an overview perspective or embedded in the environment?

And that is really hard.

We work with Google maps and we're seeing something.

How do we reconcile them?

There's been beautiful work on the brain on that and how the brain might do it.

It's a russepstein pen and under the others.

So there's perspective taking there.

There's perspective taking in, should I take your perspective or my perspective when I'm describing things?

And if you're in front of me, I generally take your perspective because I understand and your cognitive load is higher.

But so changing perspective is key in many, many places in our lives, we found that even in situations that are non -communicative or where you and I have the same perspective and we're looking at somebody else acting, we take the perspective of the actor instead of our own perspective.

And I think it's a way of understanding their action is taking their perspective.

And we do that more quickly and automatically than taking our own perspective.

So that was surprising to many people who assumed that an egocentric perspective was primary and overcoming it was difficult.

But by now that experiment has been replicated by many people in many cases.

We talk about social perspective taking and taking the perspective of others.

It's probably, it seems to be different from spatial perspective taking.

And there we found it useful in creativity.

So there's been a lot of work on how to get people more creative.

And some of the work has been done by a group advocating mind wandering.

Just let your mind go.

And we thought thinking about creative people and how they work with their own minds that they don't completely let their mind go.

And that mind wandering can help you get out of fixation which is a real problem in design that you keep designing the same way.

You can't see new uses for old objects.

And even experienced designers fall prey to fixation, mathematicians, product designers, architects fall prey to that.

And it's partly that first cognitive law.

Those old design tricks work and they've worked over and over again.

And learning depends on learning those old tricks.

But if you wanna do something differently then you have to get out of those old tricks.

So there all your learning has a cost and you wanna get out of it.

So mind wandering can release you from fixation partly because it brings in new stimuli with new responses.

But it doesn't give you a productive way of coming up with new uses.

So the test that's typically used in creativity and even in engineering design courses and graphic design courses is to come up with new uses for a brick.

And it's a kind of warmup exercise like doing scales on a piano.

And it's often used in design situations.

Come up with new uses, how do you do that?

So we started thinking deeply about how people come up with new uses.

We looked at sketches and that is one way, active way that designers come up with new uses is they make a sketchy sketch.

They look, for one reason, they look at it.

And because it's full of ambiguity they can reinterpret it and they see new things in it.

So they're designed for one reason and the messiness allows them to see new things.

So we knew that would help.

And the new things turn out to be new perspectives.

New really ways of organizing the visual, whatever you're seeing visually.

So we thought maybe that would be a way of increasing finding new uses that just letting your mind go might help release you from fixation on these various objects, bricks, umbrellas, ping pong balls, and so forth.

But they wouldn't be a productive way of coming up with new uses.

And then we started thinking, what are natural experiences in life?

So we experience people.

We experience places.

We experience events.

And all of those seem to lead to new uses.

But what seemed to combine all of them are roles.

So a doctor is a different person or a particular person in a particular place for particular events.

A musician, same thing.

A gardener, same thing.

That each of these roles in life is associated with particular events, particular equipment, people, places, and forms of interaction.

So roles capture a huge amount of information that we know about just from living.

Children are asked from an early age, what do you want to be when you grow up?

So their books are filled with firemen and policemen and doctors.

Roles they might encounter.

So we know people knew a lot about roles.

So we gave people different objects, asked them to think of new uses.

They were things, again, like umbrellas and ping pong roles.

We pretested them to make sure that people could think easily of new uses.

And one group, we said, just let your mind go.

We used the instructions from the previous work.

Another group, we just told, think of new uses.

We didn't give them any particular instructions.

That's the normal task.

And the third group, we said, think of different roles.

Think of what an artist might do with it.

Think of what a gardener might do with it.

Think of what a policeman might.

So we gave them a list of roles and said, use these, think of new uses, think of their roles.

And then we looked at what we got.

We ran this on mechanical turks several times.

And that gives us a broader range of humanity than college students.

And we did get a broader range.

And the most effective strategy was roles by far.

In fact, the mind wandering didn't differ from the no instruction group.

And when we asked people, how did you do it?

The no instruction group said, well, I just let my mind go.

But the roles group came up with far more new uses.

They came up with far more new uses that were unique or rare.

So we tabulated all this work with Julia Chow and she was just a beautiful eye for data.

So she tabulated all the uses.

She looked at the number of people who came up with them.

And for the uses that only one person or only three came up with, they were in the role group.

The role group persisted more.

Their ideas came later, the unique ideas.

And doing the task, it's almost as if you have to get rid of all those ordinary ideas before you can think of the unique ones.

And then once you're thinking of the rare or unique ones, you need a way to go back and try again and a way to go back and try again.

And Juliet was able to show that at least half of the ideas they came up with were connected with the roles that we gave them.

They also invented roles.

They got the idea, think of new roles and they invented roles, often imaginary creatures or creatures from stories or animated cartoons or movies and thought about how those creatures might use the new objects.

So they were doubly productive.

It wasn't just the roles, it was inventing roles and using roles to come up with new ideas.

So in that perspective taking, it seems to underlie a great deal of creative behavior, not just in design, but Phil Tedlock has studied super forecasters.

These are people who year after year are able to predict outcomes of world events better than most people.

So they have certain skills that other people don't have.

It isn't like predicting the stock market where the good predictors one year aren't the good predictors the next year because there really aren't many skills there, but they are skilled.

So what are their skills?

They are knowledge junkies.

They love information.

They are numerate. So they can distinguish between 10 % probability and 15 % probability.

And they can change their predictions by a percentage point or two percentage points.

So they're well calibrated in their use of probabilities.

But most of all, they seem to change perspective.

They challenge their own perspective.

So this is, I think what you were saying about confirmation bias and looking only at evidence that conforms to your views, you wanna look at evidence that challenges your views in order to be calibrated with the world.

Otherwise you fall prey to wishful thinking and terror.

So they challenge their own perspective.

They also deliberately think of perspectives of other people.

So if you're a liberal, how would a conservative see this?

If you're Zolensky, how would Putin see it?

And how would the chancellor Germany see it?

How would different viewers see it?

Challenge your perspective, take other perspectives.

So we see that in social decision -making, in political prediction, in economic prediction.

So taking other perspectives seems to be a useful way of coming up with other ideas.

I sometimes worry though, that certain stubborn people don't take the perspective of others.

Some of them are bullies.

We can all think of examples.

And I wonder if that doesn't allow them to get their way, the fact that they will not change perspective nor will they see yours.

And so that worries me.

But I choose to think that on the whole, perspective taking enlarges the mind and increases possibilities.

If you think about a lot of the cognitive biases, they sort of come down to frame of reference.

And we only have our own frame of reference.

And the example that sort of comes to mind is standing on a train and holding a ball and the train is moving at 60 miles an hour.

And relative to you, the ball is not moving because you're standing on the train, but relative to somebody watching the train go by, the ball is moving at 60 miles an hour.

And we have so many blind spots because all we see is what we see in front of us.

We think that's all there is to the world.

And one of the advantages to perspective taking or changing your frame of reference into a problem is that without any additional sort of information that you're coming across, you can put yourself in somebody else's shoes and you can change your lens into the situation.

And that will reveal a little bit of your blind spots that you have from your own perspective if you let it.

And by walking around a problem almost in a three -dimensional way, if you wanna conceptualize it, you can sort of begin to eliminate the blind spots.

And if you think of decision -making, the source of all bad decision -making is blind spots because you're only acting on things that you feel are rational.

So you're blind to some piece of information or some implication or some sort of future that you can't see.

And by walking around and changing your frame of reference or taking a different perspective, you can learn to reduce those blind spots quite a bit.

No, absolutely. And it's again the benefit of vicarious stories, of vicarious learning, is that's one way of getting different perspectives, reading the newspaper and not just reading your news but other people's news.

Perspective taking allowed Einstein to take the different temporal and spatial perspectives that allowed Newton to take.

So it also helped for leaps of advances in science and math as well as these small sorts of things of taking a spouse's perspective on your own behavior instead of getting defensive and screaming but just saying, oh, or your child's perspective, they see it that way.

And of course, if they see it that way.

So it's useful on a personal level and national level and creative level just everywhere is frame of reference taking different ones and seeing, and it's not always easy.

I mean, we didn't jump to the insights that Einstein did.

So it's by no means easy.

But as you say, with examples like your ball on the train, it can be really insightful.

You can say all of a sudden, wow, I get it.

And then how much transfer that is, we again talked about how you transfer and telling people to take different perspectives probably can help in some situations.

Providing them with perspectives to take can help even more.

So it's capturing a deep phenomenon, but instantiating it can still be difficult.

Although I think it's midway between really abstract and really specific perspective taking changing is something in the middle that people can grasp.

I think that's an excellent place to sort of wrap this up.

One final question.

What does success mean to you?

Oh, it's so personal, isn't it?

And it keeps changing.

And it's dependent, am I successful as a parent?

Am I successful as an academic?

Am I successful as a friend?

There are different.

Have I professionally achieved what I hope to achieve, but what I hope to achieve keeps changing?

I certainly went beyond what I hoped for.

When I was a small child, I hoped for an interesting life.

I certainly had an interesting life.

And I hope for a life that had friends and intellectual excitement.

I certainly wanted to be a parent and have a family.

So I hope for all those things, I think I've been very fortunate to have even more, but there's more I could have had and time is running out.

So at some point, I think life cycles do that too.

There are periods when you're hungrier and there are periods when you need to reconcile.

I think being female, I look at what opportunities are open to women now and they're incomparable.

Both familial, my parents had no expectations for me.

They thought being smart was dangerous.

It led to insanity.

And they certainly didn't have any aspirations for their two daughters other than successful marriages.

They certainly had other aspirations for their sons.

So, and that was there all along.

And I think, again, opportunities for women.

Now I look at my children, I look at my grandchildren, I see what's expected of them.

And what they can aspire to is if climate change doesn't kill us all.

That's the pessimist in you, Kim again.

Well, you look at the data.

You can't be self -deluding.

But I am hoping for even more enriched lives.

I've been very lucky to have an extraordinarily rich life.

And I can see that they're even better prepared for a rich life than I was.

Both the world is better prepared for them and they are better, they're enabled in wonderful ways.

Thank you so much for taking the time to speak with me today, Barbara.

Thank you, it's been a real pleasure.

Thanks for listening and learning with us for a complete list of episodes, show notes, transcripts, and more.

Go to fs .blog slash podcast, or just Google the knowledge project.

Until next time.

[공간적 사고와 인지: 바바라 트버스키가 말하는 사고의 근원과 도구]-[#146 Barbara Tversky: Action Shapes Thought]

📋 Summary

공간적 사고: 모든 사고의 근간

제스처와 모델링: 사고를 외부화하는 도구

인지의 9가지 일반화와 삶의 지혜

관점 전환과 창의성

요약: 학습된 지식과 체득된 지식

🎯Key Sentences

📝Key Phrases

📖 Transcript