Hello everyone, I'm Stephen West. This is Velocity. If you're new to the podcast and haven't
listened to last episode on chat GPT and large language models, please listen to that one first
before this one. In fact, if you're new to the show, you may consider going all the way back
to the beginning of this series we've been doing on the philosophy of mind. Episode 178 on whether
consciousness is even something worth talking about. Thanks to everyone supporting the show on Patreon,
patreon.com slash philosophize this philosophize this dot org. It's a website. I hope you love the show
today. So I want to start out the podcast today, but they question that will return back to near
the end of the episode. You know, you talk to some people about technology in today's world.
You know, something goes wrong, something horrible happens in the news, and you ask people what they
think about it. And they say, you know, technology itself is not a bad thing. Technology is just a tool.
It's neutral. Whether it's good or bad, that just depends on how someone's using it.
An hour job as a people all we can do is try to incentivize the good actors of the world and
deter the behavior of the bad. We've all heard people say this kind of stuff before, but the question
is, should we really be thinking about it that way? Should we be thinking of technology as this
class of neutral things that can't be judged as good or bad? Or is it possible that each piece
of technology carries with it a type of latent morality, just given the capabilities it has to
affect the society it's a part of? You have to ask yourself, is TikTok a neutral piece of
technology or nuclear weapons neutral? Do we have the luxury anymore of thinking about technology
in this way? Hopefully these questions and me being awkwardly dramatic right now make a lot
more sense by the end of the episode. But for now, let's build a bit of a foundation for the
discussion we're having today. Last episode we talked about chat GPT and we got past what's an
initial intuition that someone might have who's just getting into these conversations about AI,
that when chat GPT talks to me and it sounds like it's an intelligent person that it must in
its processing unit be doing the same kind of things that intelligent people are doing in their
brains when they have a conversation. Now clearly that's not what chat GPT is doing, but it should
be said, the fact that it's not doing all of that, that doesn't make chat GPT unintelligent
and it certainly doesn't make large language models not scary. There's a type of person that comes
to the defensive chat GPT here, a really really smart person. They say, well how do you know chat GPT
isn't doing exactly what we're doing? Do you know for sure? Hmm, maybe all we do is people is just
probabilistically interject the that's not the best argument if you want to defend chat GPT.
You know to try to make a case for why it's actually doing what human beings are doing. No,
the better argument is to say what makes chat GPT scary is that it doesn't have to be doing what a
human being is doing because we're not actually aiming for reproducing a human intelligence at all
necessarily. Let me explain what I mean. Two points we got to talk about and then we'll bring
them together here in a couple minutes. The first point is this, when John Surra last episode talks
about syntax and semantics and how the person inside the Chinese room can't possibly understand
Chinese from just sitting there in a room manipulating symbols. Someone could say back to that and
they'd be evoking the most common response back to the Chinese remark and it called the systems
response. They could say, yeah, you're right. The person in the room doesn't understand a word of
Chinese. But neither does any single subsystem of my brain understand the language I'm using right
now. The neurons that make up my brain don't understand language, the single parts of my brain don't,
but the whole system does. Could it be that the leap from syntax to semantics actually goes on
at the level of the entire system instead? That's the first point. Hold on to it for a minute.
Here's the second point I want to make. If we're making the case that we're not actually aiming for
creating a human intelligence anyway, well then what sort of intelligence are we trying to create?
Seems important to ask the question, what is intelligence more fundamentally? And what you quickly
realize is you start trying to answer that question, is it coming up with a rigid definition of intelligence?
It's about as fruitful as it was for Socrates when he was harassing people back in ancient Athens.
Nobody ever agrees fully on a definition in these conversations, but there are a couple important
things to take from the people that have tried. One thing that certainly seems to be the case,
very important, is that the definition of intelligence is clearly not just whatever human beings are doing.
Meaning human intelligence is not the only kind of intelligence. Intelligence is something more broadly
definable than that. Intelligence exists in animals. It exists in complex systems in nature.
And it's not controversial to think that it may be able to exist in machines as well.
So how do we broadly define intelligence if we were going to try? And again, a lot of people in
these conversations that do their best to give a definition here, I hope people are satisfied
enough with this one as a preliminary definition. I just want to be able to have a conversation today.
Is it too far off to say that intelligence is the ability of something to understand, learn,
solve problems, adapt to new situations, and then generate outputs that successfully achieve its
objectives? Hopefully that's close enough to get started. Well, under that definition,
then something like chat GPT is already intelligent. The question just becomes a matter of degree.
And when philosophers and scientists that are in the business of building machines that are
progressively more intelligent, when they talk about this stuff, they typically divide intelligence
into three broad categories, narrow intelligence, general intelligence, and super intelligence.
Narrow intelligence is something like chat GPT or image recognition technology or a chess
computer. Look, a chess computer can outperform every human being on the planet. It can beat the best
players at chess in the world if all it has to do was play chess against them, which is to say
if its narrow intelligence is confined to a closed system with a set of rules like the game of chess,
that's narrow intelligence. But creating a general intelligence like we have, one that's navigating
the open world, setting goals, learning, adapting, that's something most computer scientists think is
just entirely different. That it's a whole different level of intelligence, a whole different
architecture of how the information's processed, and now to bring these two points together.
A person coming to the defense of chat GPT here might say that just because chat GPT hasn't
achieved that general intelligence yet, doesn't mean it's not a major step in that direction.
This person might say it doesn't take much to imagine this narrow intelligence of chat GPT
that by the way is doing tons of stuff it wasn't even optimized to do. Imagine it linked together
with 50 to 100 other narrow intelligences, each performing different functions, all of them
communicating with each other within a larger system. Could it be that that is where general
intelligence emerges? Consider the similarities here to the theories of consciousness we've already
talked about. To Daniel Dynett's multiple drafts model in our consciousness as an illusion episode,
where multiple parallel processes, all communicating with each other, create the illusion of our
phenomenal consciousness. Point is could it be that our mind and our system of general intelligence
is made up of various subsystems, and that no one of these subsystems ever understands everything
that the whole brain's doing, like the person in the Chinese room, but that in aggregate,
is it possible these things could work together to produce something that's capable of navigating
the open world in a general intelligence sort away? No matter what side of this debate you're on,
at this point your answer to that question has to be that we just don't know. And on top of that,
consider the fact that in order for general intelligence to be a thing, we don't even need one of these
machines to have consciousness or a mind. All we need is general intelligence. So there's people
out there who will say, oh, we got nothing to worry about with AI, because these things are still
decades away from ever being able to do what a human mind is doing. And those people are probably
right, but human intelligence is a straw target people say in these conversations. That's not even
what the people working in this area are aiming for anyway. Where the money is, the quadrillions of
dollars it's estimated, and where the scientific prestige and political power is in this field,
is not in cracking human intelligence, but in solving general intelligence, which just to say it
another way, we're not trying to create a person here. Get that idea out of your head. We're trying
to create an entirely different species. And that's what makes artificial intelligence in its current
form a pretty scary thing. The people at the top of these conversations about the risk of AI,
these are not people who typically are screaming from the rooftops. The robots are coming man,
come on down into my basement with me. I got canned peaches we can live here forever. No, these people
are usually very reasonable. What they're saying is that given the impoverished state of our understanding
of how the mind or intelligence even operates, we don't know. We wouldn't know how close or far
away we are from a general intelligence. Our predictions certainly on a very good sign because we
keep on being wrong. And unfortunately, it's not good enough on this one to just wait it out and see
what happens. Because on this one, the person on the AI risk side of things would say that if we don't
have these conversations now and understand the unprecedented states of the situation we're in,
we may not have a world to be able to fix this situation in as we go. There's a lot of philosophers
out there that have tried to paint a picture for people as to what it would look like to live in a
world with super intelligence. And if for somebody skeptical of this whole possibility bear with me for
a moment, as we do on this show, we'll talk about the counterpoints, but I just think it's important
to visualize what this world might feel like to someone who's living in it. I think it helps to make
this whole discussion a little more real for people. But one philosopher who's focused on painting
this picture in recent years and absolutely brilliant communicator among all the other things
is Sam Harris. Now, Sam sets up this thought experiment by asking people to accept what he
thinks are two very simple premises. The first premise is that substrate independence is a reality,
meaning that there's nothing magical about the meat computer of the brain that allows for this
general intelligence. That a general intelligence could be run on something like silicone,
mirroring a neural network. That's the first premise. The second premise is simply that we just
keep making incremental progress. As he says, it doesn't have to be Moore's law. It just has to be
consistent progress in the direction of a general intelligence. And eventually he says, we will be at
human levels of intelligence. And by necessity, far greater levels of intelligence, super intelligence
eventually. It's easy to picture all the good things about having a super intelligence on your side.
Picture a world without disease and equality, even death. It all sounds so wonderful.
It's also easy to picture all these super intelligent ways. It would be coming for your canned
peaches if it wanted to. So maybe the more interesting thing to do, what Sam Harris asks people to do,
is to imagine yourself in that initial place of ambiguity. Try to imagine yourself standing
in the presence of one of these super intelligent beings. Try to picture it in your head right now.
What do you picture? What does it look like? How do you feel? Well, one thing to keep in mind is
you're forming that picture in your head is that this thing, whatever it looks like, would not be
constrained by biology in the same way that you were IR. It doesn't need to have two legs and two
arms. Remember, this is not a person. This is an entirely different species we've created. This
thing could look like a fire hydrant. It could look like a floating orb if it wanted to. It doesn't
even necessarily have to take a physical form. But let's say that it did for a second. And let's
say that it had eyes that are expressive of how it's feeling in the same way that a mammal's eyes
are expressive. How do you think this thing would look at you? How does something look at a creature
that's thousands of times less intelligent than it is? How does a general intelligence without
biological restrictions, with cognitive horizons, your brain can't actually even begin to comprehend?
How does that thing look at you? Well, it wouldn't look like a lion would look at you,
like a predator that wants to eat you. This thing doesn't want to eat you. It wouldn't be like looking
into the eyes of a bear. You know, when a bear looks at you, at least black bears up here in the
northwest, they don't usually look at you like they want to eat you. A bear looks at you with eyes
that are kind of curious. Like they got nothing to worry about in the entire world. And they're just
kind of looking through you. You're like a Netflix show they're mildly interested in right now.
That's how they look at you. But when it comes to a super intelligence, it seemingly wouldn't
even look at you like that. Give it enough time to study you. It would already know almost everything
about you. Maybe the closest comparison is that a super intelligence might look at you. The same
way we look at something like a honeybee living in a hive. Quick question for you about honeybees.
Do you think honeybees worry about racism that's going on in the public school system? I mean,
why not? Why not? It's an incredibly important moral issue that's facing us right now. Are honeybees
just uncaring or something? No, obviously a bee doesn't have the cognitive capacity to care about
stuff like that. What it cares about is what's going on in the world of its hive. And while these
moral dimensions clearly exist at other levels of intelligence for other types of intelligent creatures,
there is no amount of explanation that is going to bring a honeybee up to speed on why this is an
important issue that needs to be addressed. Point is, we have to assume that the moral dimensions
of a super intelligence would be similar in comparison to our level of intelligence. We have to
assume this thing would be progressively learning about its surroundings, progressively adapting,
coming up with new goals, and that these goals could be of a scope that are literally impossible
for our brains to fathom. Sam Harris compared it one time to the relationship between birds and
human beings. How if you were a bird and you had the ability to think about your relationship
to human beings, you are essentially living every day of your life. Just hope that human beings
don't find something that they like more than the existence of birds. And that to a bird, our
behavior has to be pretty confusing. Sometimes we look at them, sometimes we don't. Sometimes we
kill huge numbers of them for what to them must seem like absolutely no reason at all.
Picture you are now living alongside a being that is like that to you, where you're looking at
this thing, and you really are like a cat that is watching TV. It all looks very familiar to you,
but you can't possibly comprehend the full depth of what's going on.
Now the simple place for the brain to go here is to say, well that's terrifying. What happens if
this thing decides one day that it doesn't like us and just kills everyone? Maybe I'll be really
nice to it. You know, I'm kind of charming when I want to be. Maybe if we're all really nice to
this thing, it'll cure cancer for us and make us some new shows to watch. Why would this thing be
mean to us if we never give it a reason to be mean? But the more interesting thing to consider,
Sam Harris says, and this is an adaptation of an idea from the computer scientist Stuart Russell.
But he says that an AI that is on the level of a superintelligence wouldn't even need to have
malicious intent towards humanity in order for it to be dangerous to us. He says, you know, in the same
way, you buy a plot of land and you want to build a house on it, and that as you're building that
house throughout that whole process, you kill hundreds if not thousands of bugs. And it's not because
you have any evil feelings towards bugs. You just don't consider their existence because the goal
you're trying to accomplish is of a scope and a level of importance where the bugs are going to
have to kill is not even something that crosses your mind. A superintelligence wouldn't need
an aferious motive to be able to do things that makes its existence dangerous to people.
Now somebody could say back to all this, wow, wow, that's a wonderful picture that you painted
there. A great fantasy world where a superintelligence exists. But this stuff isn't going on in the real
world yet. And there's a lot of assumptions you have to make to be able to get to this point in
your little fantasy. This person might say it starts to almost feel like you guys are larkin
over here. You guys are heard of larkin, right? Those dudes that go out into the woods and wizard
costumes and create a mythical world where they can fight and shoot magic missiles at each other.
I'm sure you've all seen the videos of people larkin on YouTube. If not, God, what does my algorithm
say about me there? Point is, is that what's going on here? Are these AGI risk people?
Just a bunch of people creating a fantasy world where they can argue with each other about this
stuff, playing a bunch of different characters where anybody else who wants to play in their
little game has to accept a whole ton of premises before they can participate. Are these people
just larkin in the woods? Well, let's test that criticism. And let's do it in the form of a
dialogue, classic format in the history of philosophy. And as you do in a dialogue, let's hear from
both sides of the argument. On one side, we have the AGI risk person who's alarmed by the
possibility of a general intelligence emerging. And on the other side, we have a skeptic who's
not buying any of it. Let's start with them. The skeptic could say, for you to be scared about
the possibility of an AGI in today's world, you have to be making a ton of assumptions about
how this thing's going to behave. And most of those assumptions, from what I can see, come down
to the fact that you're projecting your human way of thinking onto this thing that's not going
to be thinking like a human being. That's your problem here. For example, just a name one of many,
the survival instinct that we're all born with. How can anyone in their right mind assume we're
going to turn this AGI on for the first time and that it's going to be thinking like a survival
oriented creature like we are? How can you worry about something like a preemptive strike against
people because the AI scared is going to get turned off? Look, that sounds like science fiction.
I mean, you have to remember this thing's essentially a circuit board. You give the thing a goal
and it executes that goal. And if we don't program all the nasty survival instincts into it,
it won't ever have them. But somebody on the AI wrist side of things might say back, well,
you're right that people often project their humanity onto these things way too much.
But what you're saying about survival there is just not true. And we know it's not true
because we've run the experiments to test it out. The concept's called instrumental convergence.
The idea is simple. Whenever anything, even an artificial intelligence is given some sort of primary
goal that it has to carry out, there are always certain sub-goals, lower level goals,
that naturally are required to be able to carry out the primary goal. Survival almost always ends
up being one of them, though there are a lot of goals that instrumentally converge. It turns out
the philosopher Elia Zaryudkowski writes quote, whether your goal is to bake a cheesecake or fly to
Australia, you could benefit from matter, energy, the ability to complete plans, and not dying in
the next five minutes. End quote. So the person on the AI wrist side of the argument could say that
even if an AGI doesn't have a survival instinct programmed into it, it will still have a desire to
survive so that it can carry out whatever goal is programmed into it. And by the way, you can find
this same exact distinction between Nietzsche and Schopenhauer a hundred and fifty years ago.
But then the skeptic could come back at them. They could say, okay, point taken. But honestly,
survival is the least of it. How about all the other things people start to assume about AI?
That it's going to be hostile. That it's going to want to be in charge. That it's going to be
thinking like a chimpanzee in terms of hierarchies like we do. Why are you assuming that? Look,
just because your dad had toxic masculinity, doesn't mean my robots got to have it. Because look,
this is where you guys on the AI wrist side of stuff talk about all the time. Something having a higher
level of intelligence does not necessarily mean that it's going to be hostile towards everything
that's less intelligent. For example, a deer is far more intelligent than a predatory insect like a
spider. But when you're out camping in the woods, nobody ever worries about a deer coming and
attacking you in the middle of the night while you're asleep. This person might say this is
transparently just another example of us being scared of our own worst tendencies as people
manifesting in something else that's way stronger than us. But we can program this AI to not have
those tendencies. And we have no reason to assume this thing wouldn't have a moral framework
that's compassionate towards other living things. Where it uses that super intelligence to come up with
creative ways to accomplish its goals that don't mess with anything else. And I think somebody on the
AI wrist side of things would say, okay, I guess you're right that we can't know whether we turn
this thing on and it's just the nicest being we've ever come across. And we can't know for sure
whether as this thing extends into its practically infinite cognitive horizons, if for thousands of
years, it'll just always have our well-being as its number one priority. It's not that any of what
you just said is impossible. It's just, as Sam Harris says, a strange thing to be confident about.
We don't know if it'll go that way. We don't know if it won't, which is why everything you've
been talking about so far, by the way, the survival instinct, the chimpanzee level of hostility. In other
words, whether or not this intelligence is aligned with human values, this is one of the biggest
conversations that's going on in this area. If we accept that this thing's more intelligent and more
powerful than we can possibly comprehend, the question is, how do we make sure that its values
are not only aligned with our values right now, but how do we make sure that goes on endlessly
into the future even as our values change? This is often called the alignment problem by people
having these conversations. And it can be tempting to think from the other side, hey, haven't we been
doing moral philosophy now for thousands of years? Don't we have some common sense values we can
program into this thing and not have to worry about it? I mean, isn't that what all those hours
rambling about trolley's was supposed to be about? Objective moral ideals, where did those go?
But it's not that simple. As Elias R. U. Kowski puts it, we don't know how to get internal
psychological goals into systems at this point. We only know how to get outwardly observable
behaviors into systems. So if there was some sort of domino effect that happened that we didn't see
coming, where a super intelligence emerged over the course of the next 30 days, we wouldn't know
how to program these values in even if we had them, which we don't. And even if all human beings
could agree on which values to put in, which we can't, we'd still be in a place where we're at
the mercy of every unintended consequence that you can possibly imagine and even all the ones
you didn't imagine. There's a famous thought experiment in these conversations that illustrates
this point by the philosopher Nick Boastram. You'll hear about it if you're talking to people about
AGI. It's called the paperclip maximizer. So imagine a world where people are trying to align
an AGI and they want to play it safe so they give it what seems like to them to be a totally
simple goal to accomplish. They tell it to make paperclips. And by the way, use that super
intelligence of yours to get as efficient as you can at making those paperclips and make as many
of those paperclips as you possibly can. Go ahead. And it starts out great. The things make
in paperclips. But then it starts to improve. It starts mining the earth's resources to make more
paperclips. Eventually develops nanotechnology. Eventually, it starts mining human beings as
sources of carbon to find ways to make more paperclips. The point of the story is even something
as seemingly innocuous as making paperclips. When the stakes are as high as when you have a super
intelligence executing the stuff that's outside of your control, even making paperclips can have
devastating unintended consequences. And if paperclips are too theoretical for you, how about a real
world example? You could command a super intelligence to do something undeniably good. You tell it to
cure or eliminate cancer. And while you're doing it, don't kill any human beings in the process.
It seems like a totally reasonable set of parameters nobody could be mad at them. But then imagine
the thing does a bunch of research and finds a really cool way that it can decapitate human
heads and keep people alive inside of jars. But hey, now they don't have a body anymore, right?
And I was looking at the stats, it says. Most of the cancer goes on in the body. Cancer numbers
are way down since the heads have gone in the jars. I don't understand what you're so mad about.
Now it should be said, there's a lot of people currently working on solutions to the alignment
problem. The field seems to be moving away from the area that we're ever going to be able to come
up with strict protocols that account for every unintended consequence imaginable. Both Stuart
Russell and Eliezer Yudkowski are people doing good work in this field if you're looking for
somebody to read more. But that said, let's get back to the skeptic of all this AGI stuff that
was in the conversation before. Because another popular skeptical opinion people like to say back at
this point when they're confronted with just how complicated it is to align a system like this
is that maybe that's true. But maybe we don't really need to be perfect when it comes to alignment
anyway. I mean, given how complicated the alignment process seems to be, maybe our real goal should
be to just be able to control the AGI. And that seems easy. Trust me on this, as cousins of chimpanzees,
we are experts in the realm of violence. We know this thing would have to run on electricity at least
at first, right? So when the thing starts to show signs of getting out of hand, can't we just
unplug it? And that's a silly way of saying, can't we just launch an EMP at it? Can't we just
shut down the internet or shoot missiles at it? I mean, is this really that complicated? We just
never let the thing get too powerful. That's the solution. People on the AGI wrist side of things
might say back. Do you actually think that's how it's going to go down? Stuart Russell says,
just from a game theoretical perspective, that's kind of like he's saying that I'm going to play
chess against this supercomputer that's way smarter than me. It's way better at chess than I am.
But don't worry, the second it starts to beat me too bad, I'll just checkmate it. It's like,
if you could checkmate it on command, you wouldn't need to be playing against it in the first place to
benefit from its intelligence. It's a strange thing to be confident about that we'll just nuke the
thing if it gets out of hand, problem solved. To which the skeptic could say back, look, I was joking
about the missiles. But aren't there other ways to control an AGI? Ways that involve us preventing
it from ever getting out in the first place? Wouldn't that be a viable strategy? And yes, it is,
in theory. That's why next to the alignment problem, another big area conversation that's going
on in this field is known as the control problem or the containment problem. This is why a lot of
private companies that are developing this technology keep this stuff locked up inside of a black box.
But even with black boxes, the same problems start to emerge. Just imagine a super intelligence
trapped inside of a black box. How do you know this is a perfect black box with zero way of escaping?
Software has bugs sometimes, okay? Do we really put ourselves in a spot where the future of humanity
is going to rely on a computer coder being perfect once? More than that, how do you know that you're
not going to get persuaded by the super intelligence to let it out? Stuart Russell says, if you want to
think about this more, try to imagine yourself being trapped in a prison where the warden and all
the guards of the prison are five year old kids. Now, you're way more intelligent than them. You know
things they can't possibly come up with on their own. And you'll look at the way they're acting
and they're acting in a way that's totally dangerous to themselves, to everyone around them. They're
about to burn the whole prison down. Do you, from inside of your prison cell, try to teach them
everything they need to fix their situation? Do you teach them how to read through the bars,
teach them how to use power tools, teach them why it's important to share? Do you do it that way?
Or is the more responsible thing to do to convince one of these dumb five year olds to let you out of
your cage and then help them all once you get outside? Again, the thing doesn't even need to have
malice towards human beings for it to not want to be contained. So both the alignment and the
containment problem are far from things that are solved that we can just wait around on.
And look, all these criticisms from the side of the skeptic, the person concerned about AGI wouldn't
see all this as antagonistic. At the end of the day, from their point of view, this is the skeptic
participating in the discussion about AGI alignment and containment. They're just at an earlier
stage of their understanding of the issue and welcome to this level of the discussion.
Now, we started this whole section of the episode by accepting a point from Sam Harris,
that there's only two major premises that you need to accept, substrate independence and
continued progress, and that if we accept those, we will certainly, with enough time,
be able to produce a general intelligence one day. But it needs to be said, there's people out there
that disagree with that. These people would say that he's making a lot of assumptions by making those
assumptions. They may say he's smuggling in the computational theory of mind that he's assuming
intelligence is something reducible to information processing, that he's glossing over some big
possibilities of embodied cognition. But I think they'd say that even if we're just going to stick to
the two premises he mentioned, those two things are still filled with tons of ifs that we have no
idea whether they're ever going to become wins. For example, we have no idea whether the massive
progress we've seen in the last few years in the field of AI is just a bunch of low-hanging fruit
that we're going through an initial phase with the software we're seeing tons of development.
But that very soon we're going to hit some sort of a wall where there's a point of diminishing
returns. Seems likely that could be a possibility, they may say. I mean, the only example of general
intelligence that we have to look at took billions of years of evolution, which is not an argument
that it has to take that long. It's just an argument trying to illustrate the potential complexity
we're dealing with when it comes to organizing concepts, the ability to create new concepts,
link them together into existing concepts in a meaningful way. That's a pretty big task that's
not yet being done. Another if this kind of person would say is when it comes to hardware.
You know, maybe we solve the software side of this AGI thing in the next couple years,
but maybe it takes centuries or is impossible to have the hardware that can run this kind of
software. Or maybe it's just so expensive to run, it'll never be scalable. This kind of person
could say back to Sam Harris, look, all you're doing here is indulging in a very self-justifying
loop of arguments. You're saying, hey, it's possible for a superintelligence to be among us one day.
So now let's have a bunch of conversations about how the world might end if that actually happens.
But under that logic, why not just be worried about everything? Why not be worried about genetic
engineering and chimeras taking over the planet? Why not worry about facial recognition technology?
Why not worry about gain a function research? And to Dr. Sam Harris, I mean, I'm not trying to
speak for him here. This is just my opinion as to what he might say. I've created this weird
dialogue between hypothetical people this episode. But I think he might say back to this person,
yeah, we should be worried about all those things he just mentioned. And not worried like we're
being doom sayers about this stuff, running around panic. But of course, we should be concerned
about our relationship to science and technology. Keep in mind, this is the same Sam Harris
the back in 2015 is doing an interview. And at the time, it may have just seemed like a throwaway
comment to many. But he's talking in this interview and he's saying how when we're facing the upcoming
election in the United States at the time, when we're trying to decide who we want at the helm of
the ship in terms of candidates, that we have to consider the possibility of something drastic
happening, something like a super bug or a pandemic breaking out in the future that we're not
prepared for that causes a global catastrophe. That's all the way back in 2015. This is clearly a
man doing his work that is self aware of this moment we're living through where we have to reexamine
our relationship to technology because there's never really been this much at stake when a new
piece of technology comes out as in 2023. And this brings me back to the question we asked at the
beginning of the episode is technology simply neutral? Is it just a tool that can be used for good
or evil depending on the person and that it's our job to incentivize the good actors and eliminate
the bad or has technology become something that we no longer have the privilege to have a reactionary
policy towards? Because that's typically how this works, right? A new technology is developed
usually in the business world where the only gatekeeper between the public and a new piece of tech is
some ambitious CEO that's driven by a profit motive. And then the widespread use of this new product
is seen as a sort of experiment our society's running in real time. And then any negative effects
that start to show up, the government's supposed to take a look at them and pass regulations that
try to protect the public after the fact. Fantastic strategy when the stakes of technology are low
back when we're making vacuum pumps in the 1600s back when someone's buying a blender or a TV set
for their home. But how about with AGI with nuclear technology autonomous weapons bioengineering
technology is not a neutral thing. You know, Foucault says to always be more skeptical of the things
that have power over you that claim to be neutral. People in science and technology studies have
said for decades now that a new piece of technology always carries with it certain affordances,
meaning when something new comes out, there's always new things that it allows people to do that they
couldn't do before. But it also takes away things that people used to be able to do. And to the skeptic
in this episode saying that it's silly to worry about the future possibility of AGI that anybody worried
about it is buying into a whole lot of ifs that haven't yet become wins. Someone on the AGI
wrist side of things might say that given the stakes of the technology that we're producing,
we don't have the luxury of waiting for those ifs to become wins. We have to have these conversations
about AGI now because this is the new level of mindfulness we need to apply to new technologies
before they're released to the public or used in an unregulated way where people get hurt.
And look, even if all this stuff with AGI never happens, say we hit some sort of a wall in 10
years where AGI is impossible and this era's look back on as a bunch of dumb, tech-obsessed monkeys
larpen in the woods. These conversations are still producing results when it comes to reexamining
our relationship to tech. For example, think of the alignment problem and how that same line of
conversation may have saved us from some of the perils we've experienced with social media over
the years. Meaning if your algorithms on a social media feed are aligned to the goal of maximizing
the engagement of the viewer, then if misinformation and political antagonism is what gets the
population to engage with stuff the most, then that's what people are going to end up seeing.
Think of the conversations around the containment problem of AGI and the nuclear power
disasters of the 20th century. Could any of that have been prevented?
Next time, we're going to be talking about narrow AI and its potential impacts on society.
Some that are even going on right now. I know I said last time that was going to be a part of
this episode, well this is more for you to listen to. I'm certainly grateful you decided to
listen to a podcast about this. It just seems like a useful conversation for people to be familiar
with being alive today. But I want to close today by acknowledging one point that we haven't
brought up at all this episode. And it may at first glance seem like an obvious solution to all this.
Why not just ban AGI? I mean, we put moratoriums on other tech that's dangerous. It's illegal to
clone people. It's illegal to build a nuclear reactor in your garage. Why don't we just do the
same thing with AGI? Well, funny. You should ask that's another thing that makes this a particularly
complicated brand of existential risk. We can't stop at this point. We are living in a world with
quadrillions of dollars up for grabs to anybody who has the insight and the free time to crack this
thing. And you couple that with the fact that this is not like nuclear technology or cloning where
you may need a team of scientists to be able to monitor and regulate something like that.
No, with AGI, GPUs are everywhere. Knowledge about algorithms and how to improve them. You don't
got to be in a secret sisterhood to get access to that kind of stuff. So the only thing some people
feel like we can do is to just win the race to the finish line. There's talk about maybe putting
a temporary pause on this kind of stuff. But everybody knows even that would only be temporary.
What seems certain here is that this is nothing like share noble or three mile island where
everyone's gung ho about nuclear technology for a while until there's a couple major accidents
and then governments around the world start to pump the brakes and the typical reactionary policy
sort away. And if you're someone out there just frothing at the mouth to win this technological
arms race, you know, cheerleading the techbearance of the world all the way to the finish line. That's fine.
But there may come a day where you find yourself face to face with the kind of species we talked
about earlier in the episode, staring down the barrel of something that looks at you far more like
you're an elementary school science project than a human being. Thank you for listening. Talk to
you next time. Patreon shout out this week. We got Francisco Aquino Serrano, Donald Manas, Brett Clark,
Jera Brown, and Dylan Breninger. Thank you to everyone that supports the show in any way that you do
could never happen without you. I hope you have a good rest of your week.
