You know, I grew up as a Catholic and I remember the guardian angel was the thing that I really
loved that concept when I was a kid.
But then when I got to be, I don't know, maybe around seven or eight, like your guardian
angel is always watching you, at first it was like comfort and then it turned into kind
of like a, are they watching me, if I pick my nose, do they?
And are they watching out for me or are they just watching me?
Exactly, like are they my guardian angel or my surveillance angel?
Surveillance angel.
I'm Andrea Valdes, I'm an editor at The Atlantic and I'm making Barbara writer at The Atlantic.
And this is how to know what's real.
I just got the most embarrassing little alert from my watch.
And it's telling me that it is quote, time to stand.
Why does it never tell us that it's time to lie down?
Right, or time to just like go to the beach or something.
And it's weird though because I'm realizing I'm having these intensely conflicting emotions
about it because in one way I appreciate the reminder.
I have been sitting too long.
I should probably stand up.
But I don't also love the feeling of just sort of being casually judged by a piece of
technology.
No, I understand.
I get those alerts too.
I know it very well.
And it tells you, stand up, move for a minute.
You can do it.
You can almost hear it go and bless your heart.
Bless your lazy little heart.
The funny thing too about it is like I find myself being annoyed.
But then I also fully recognize that I don't really have a right to be annoyed because
I've asked the watch to do the judging.
Yes, definitely.
I totally understand.
I mean, I'm very obsessed with the data.
My smartwatch produces.
My steps, my sleeping habits, my heart rate, you know, just everything about it.
I'm just obsessed with it.
And it makes me think, well, I mean, have you ever heard of the quantified self movement?
Oh, yeah.
Yeah.
So, quantified self.
It's a term that was coined by Wired Magazine editors around 2007.
And the idea was it was this movement that aspired to quote unquote, self-knowledge through
numbers.
And I mean, it's worth remembering what was going on in 2007, 2008.
You know, I know it doesn't sound that long ago, but wearable tech was really in its
infancy.
And in a really short amount of time, we've gone from, you know, our Fitbit to, as you said,
Megan, this device that not only scolds you for not stating up every hour, but it tracks
your calories, the decibels of your environment.
You can even take an EKG with it.
And you know, when I have my smart watch on, I'm constantly on guard to myself.
Did I walk enough?
Did I stand enough?
Did I sleep enough?
And I suppose it's a little bit of accountability.
And that's nice.
But in the extreme, it can feel like I've sort of opted into self surveillance.
Yes.
And I love that idea.
In part because we typically think about surveillance from the opposite end, right?
Something that's done to us rather than something that we do to ourselves and for ourselves.
Watches are just one example here, right?
There's also smartphones and there's this broader technological environment and all of
that, that whole ecosystem, it all kind of asks this question of who's really being watched
and then also who's really doing the watching.
So I spoke with Deb Rajee, who's a computer scientist and a fellow at the Mozilla Foundation.
And she's an expert on questions about the human side of surveillance and thinks a lot
about how being watched affects our reality.
I'd love to start with the broad state of surveillance in the United States.
What does the infrastructure of surveillance look like right now?
Yeah.
I think a lot of people see surveillance as a very sort of out there in the world, physical
infrastructure thing where they see themselves walking down the street and they notice a camera.
Yeah, I'm being surveilled.
Which does happen if you live in New York, especially post-9-11.
Like you are definitely physically surveilled.
There's a lot of physical surveillance infrastructure, a lot of cameras out there.
But there's also a lot of other tools for surveillance that I think people are less aware
of.
Like ring cameras and those types of devices.
I think when people install their ring product, they're thinking about themselves.
They're like, oh, I have security concerns.
I want to just have something to be able to just like check who's on my porch or not.
And they don't see it as surveillance apparatus, but it ends up becoming part of a broader network
of surveillance.
And then I think the one that people very rarely think of, and again, is another thing
that I would not have thought of if I wasn't engaged in some of this work, is online surveillance.
Faces are sort of the only biometric.
It's not like a fingerprint.
It's like we don't upload our fingerprint to our social media.
Like we're very sensitive about like, oh, this seems like important biometric data that
we should keep guarded.
But for faces, it can be passively collected and passively distributed without you having
any awareness of it.
But also, we're very casual about our faces.
So we upload it very freely onto the internet.
And so immigration officers' eyes, for example, has a lot of online surveillance tools where
they own monitor people's Facebook pages.
And they'll use sort of facial recognition and other products to identify and connect
online identities across various social media platforms, for example.
So you have people doing this incredibly common thing, right?
Just sharing pieces of their lives on social media.
And then you have immigration officials treating that as actionable data.
Can you tell me more about facial recognition in particular?
So one of the first models that she built was a facial recognition project.
And so I'm a black woman and I noticed right away that there were not a lot of faces that
look like mine.
And I remember trying to have a conversation with folks at the company at the time.
And it was a very strange time to be trying to have this conversation.
This was like 2017.
There was a little bit of that happening in the sort of like natural language processing
space.
Like people are noticing, you know, stereotype language coming out of some of these models.
But no one was really talking about it in the image space as much.
But oh, some of these models don't work as well for darker skin individuals or other
demographics.
We audited a bunch of these products that were these facial analysis products.
And we realized that these systems were working very well for those minority populations,
but also definitely not working for the intersection of those groups.
So like darker skin female faces.
Wow.
Some of the ways in which these systems were being pitched at the time were sort of selling
these products and pitching it to immigration officers to use to identify suspects.
Wow.
Imagine something that's not 70% accurate and it's being used to decide, you know, if
this person aligns with the suspect for deportation, like that's so serious.
Yes.
You know, since we published that work, we had just this, you know, it was this huge moment
in terms of it really shifted the thinking in policy circles, advocacy circles, even
commercial spaces around how all those systems worked.
Because all the information we had about how well these systems worked so far was on data
systems that were disproportionately composed of lighter skin men.
Right.
And so people had this belief that all these systems work so well, like 99% accuracy.
They're incredible.
And then our work kind of showed like, well, 99% accuracy on lighter skin men.
And could you talk a bit about where tech companies are getting the data from to train their
models?
So much of the data required to build these AI systems are collected through surveillance.
And this is not hyperbole, right?
Like the facial recognition system, they're built on top of, you know, millions and millions
of faces in these databases, millions and millions of faces that are collected, you know,
through the internet or collected through identification databases or through, you know, physical
or digital surveillance apparatus.
Because of the way that the models are trained or and developed, it requires a lot of data
to get to a meaningful model.
And so a lot of these systems are just very data hungry.
And it's a really valuable asset.
And how are they able to use that asset?
What are the specific privacy implications about collecting all that data?
Privacy is one of those things that we just don't, we haven't been able to get to federal
level privacy regulation in the states.
There's been a couple states that have taken initiative.
So California has the California Privacy Act.
Illinois has a BIPA, which is sort of a biometric information privacy act.
And that's specifically about, you know, biometric data like faces.
In fact, they had a really, I think BIPA's biggest enforcement was against Facebook and
Facebook's collection of faces, which does count as biometric data.
So in Illinois, they had to pay a bunch of Facebook users a certain settlement amount.
Yeah.
So, you know, there are privacy laws, but it's very state-based.
And it takes a lot of initiative for the different states to enforce some of these things versus
having some kind of comprehensive national approach to privacy.
That's why enforcement or setting these rules is so difficult.
I think something that's been interesting is that some of the agencies have sort of stepped
up to play a role in terms of thinking through privacy.
So the Federal Trade Commission, FTC, has done these privacy audits historically on some
of the big tech companies.
They've done this for quite a few AI products as well, sort of investigating the privacy
violations of some of them as well.
So I think that that's something that, you know, some of the agencies are excited about
and interested in, and that might be a place where we see movement, but ideally we have
some kind of law.
And we've been in this moment, this, I guess, very long moment, where companies have been
taking the ask for forgiveness instead of permission approached all this.
You know, so airing on the side of just collecting as much data about their users as they possibly
can while they can.
And I wonder what the effects of that will be in terms of our broader informational environment.
The way surveillance and privacy works is that it's not just about the information that's
collected about you.
It's like your entire network is now, you know, caught in this web and it's just building
pictures of entire ecosystems of information.
And so I think people don't always get that, but yeah, it's a huge part of what defines
surveillance.
Do you remember surveillance camera man, Megan?
Ooh, no.
Now I'm regretting that I don't.
Well, I mean, I'm not sure how well it was known, but it was maybe ten or so years ago.
There was this guy who, he had a camera and he would take the camera and he would go
and he'd stop and put the camera in people's faces.
Oh, wow.
And they would get really upset.
And they would ask him, why are you filming me?
And you know, they would get more and more irritated, you know, and it would escalate.
And I think the meta point that surveillance camera man was trying to make was, you know,
we're surveilled all the time.
So why is it any different if someone comes and puts a camera in your face when there's
cameras all around you filming you all the time?
Right.
That's a great question.
And yeah, the sort of difference there between the active, active being filmed and then
the sort of passive state of surveillance is so interesting there.
Yeah.
And, you know, that's interesting that you say active versus passive.
You know, it reminds me of the notion of the panopticon, which I think is a word that
people hear a lot these days, but it's worth remembering that the panopticon is an
old idea.
So it started around the late 1700s with the philosopher named Jeremy Bentham.
And Bentham, he outlined this architectural idea.
And it was originally conceptualized for prisons.
You know, the idea was that you have this circular building and the prisoners live in cells
along the perimeter of the building.
And then there's this inner circle and the guards are in that inner circle and they can
see the prisoners, but the prisoners can't see the guards.
Oh my goodness.
And so the effect that Bentham was hoping this would achieve is that the prisoners would
never know if they're being watched.
So they'd always behave as if they were being watched.
Mm.
And that makes me think of the more modern idea of the watching eyes effect, this notion
that simply the presence of eyes might affect people's behavior.
And specifically images of eyes, simply that awareness of being watched does seem to
affect people's behavior.
Oh, interesting.
Beneficial behavior, like collectively good behavior, sort of keeping people in line in that
very Bentham-like way.
We have all of these eyes watching us now in, I mean, even in our neighborhoods and at
our apartment buildings in the form of, say, ring cameras or other cameras that are attached
to our front doors, just how we've really opted into being surveilled in all of the most
mundane places.
I think the question I have is, where is all of that information going?
And in some sense, that's the question, right?
And Devrajji has what I found to be a really useful answer to that question of where our
information is actually going, because it involves thinking of surveillance not just as an act,
but also as a product.
For a long time, when you, I don't know if you remember those, you know, complete the
picture apps or like spice up my picture, they would use generative models.
You would kind of give them a prompt, which would be like your face.
And then it would modify the image to make it more professional or make it better lit.
Like sometimes you'll get content that was just, you know, sexualizing and inappropriate.
And so that happens in like a non malicious case.
Like people will try to just generate images for benign reasons.
And if they choose the wrong demographic or they frame things in the wrong way, for example,
they'll just get images that are denigrating in a way that feels inappropriate.
And so I feel like there's that way in which AI for images is sort of led to just like
a proliferation of problematic content.
So not only are those images being generated because the systems are flawed themselves,
but then you also have people using those flawed systems to generate malicious content on
purpose, right?
One that we've seen a lot is sort of this deep fake porn of young people, which has been
so disappointing to me, just young boys deciding to do that to young girls in their class.
Like it really is a horrifying form of sexual abuse.
And I think like when it happened to Taylor Swift, I don't know if you remember.
Someone used the Microsoft model and generated some non consensual sexual images of Taylor
Swift.
I think it turned that into like a national conversation.
But months before that, there had been a lot of reporting of this happening in high schools.
Anonymous young girls dealing with that, which is just another layer of like trauma because
you're a coody.
You're not Taylor Swift, right?
Right.
So people don't pay attention in the same way.
So I think that that problem has actually been a huge issue for a very long time.
Andrea, I'm thinking of that old line about how if you're not paying for something in the
tech world, there's a good chance you are probably the product being sold.
Right.
And using how outmoded that idea probably is at this point because even when we pay for
these things, we're still the products.
And specifically our data are the products being sold.
So even with things like deep fakes, which are typically defined as using some kind of
machine learning or AI to create a piece of manipulated media, even they rely on surveillance
in some sense.
And so you have this irony where these recordings of reality are now also being used to distort
reality.
Yeah.
It makes me think of Don Falles, this philosopher who talked about the epistemic threat of
deep fakes.
And that it's part of this pending info apocalypse, which sounds quite grim, I know.
But I think the point that Falles was trying to make is that with the proliferation of deep
fakes, we're beginning to maybe distrust what it is that we're seeing.
We talked about this in the last episode, you know, seeing as believing might not be
enough.
And I think we're really worried about deep fakes, but I'm also concerned about this concept
of cheap fakes or shallow fakes.
So cheap fakes or shallow fakes, it's, you know, you can tweak or change images or videos
or audio just a little bit.
And it doesn't actually require AI or advanced technology to create.
So one of the more infamous instances of this was in 2019.
Do you remember there was a video of Nancy Pelosi that came out where it sounded like she
was slurring her words?
Oh, yeah.
Right.
Yeah.
But really the video had just been slowed down using easy audio tools and just slowed down
enough to create that perception that she was slurring her words.
So it's a quote unquote cheap way to create a small bit of chaos.
And then you combine that small bit of chaos with the very big chaos of deep fakes.
Yeah.
So one, the cheap fake is it's her real voice.
It's just slowed down again using like simple tools.
But we're also seeing instances of AI generated technology that completely mimics other people's
voices and it's becoming really easy to use now.
There was this case recently that came out of Maryland where there was an athletic director
at a high school.
And he was arrested after he allegedly used an AI voice simulation of the principal at
his school.
And he allegedly simulated the principal's voice saying some really horrible things.
And it caused all this blowback on the principal before investigators, you know, they looked
into it.
They determined the audio is fake.
But again, it was just a regular person that was able to use this really advanced
seaming technology that was cheap, easy to use and therefore easy to abuse.
Oh, yes.
And I think it also goes to show how few sort of cultural safeguards we have in place
right now, right?
Like the technology will let people do certain things.
And we don't always, I think, have a really well agreed upon sense of what constitutes
abusing the technology.
And usually when a new technology comes along, people will sort of figure out what's
acceptable and what will bear some kind of social cost.
And will there be a taboo associated with it?
But with all of these new technologies, we just don't have that.
And so people, I think, are pushing the bounds to see what they can get away with.
Yeah.
And we're starting to have that conversation right now about what those limits should
look like.
I mean, lots of people are working on ways to figure out how to watermark or authenticate
things like audio and video and images.
Yeah.
And I think that that idea of watermarking, too, can maybe also have a cultural implication.
You know, like if everyone knows that deep fakes can be tracked and easily, that is itself
a pretty good disincentive from creating them in the first place, at least with an intent
to fool or do something malicious.
Yeah.
But in the meantime, there's just going to be a lot of these deep fakes and cheap fakes
and shallow fakes that we're just going to have to be on the lookout for.
Is there new advice that you have for trying to figure out whether something is fake?
If it doesn't feel quite right, it probably isn't.
A lot of these images don't have a good sense of like spatial awareness, like it's just
pixels in, pixels out.
And so there's some of these like concepts that we assume is find really easy, but these
models struggle with, I advise people to be aware of, sort of trust your intuition if
you're noticing weird artifacts and the image, it probably isn't real.
I think another thing as well is like, who posts?
That's a great one.
Yeah.
Yeah.
I mute very liberally like on Twitter.
Any platform.
I definitely mute a lot of accounts that I notice.
Either caught posting something either like a community note or something will reveal that
they've been posting fake images or you just see it and you recognize the design of it.
And so I just mute that kind of content.
Don't engage with those kind of content creators at all.
And so I think that that's also another successful thing.
On the platform level, deep platforming is really effective if someone has sort of three
strikes in terms of producing a certain type of content.
That's what happened with the Taylor Swift situation where people were disseminating this,
you know, Taylor Swift images and generating more images.
They just went after every single account that did that, you know, completely locked down
her hashtag, that kind of thing where they just really went after everything.
And I think that that's something that like we should just do in our personal engagement
as well.
Andrea, that idea of personal engagement I think is such a tricky part of all of this.
I'm even thinking back to what we were saying before about ring and the interplay we were
getting at between the individual and the collective.
In some ways, it's the same tension that we've been thinking about with climate change
and other really broad, really complicated problems.
Yeah.
You know, connection between personal responsibility, but also the outsized role that corporate
and government actors will have to play when it comes to finding solutions.
And with so many of these surveillance technologies were the consumers with all the agencies
that that would seem to entail, but at the same time, we're also part of this broader ecosystem
where we really don't have as much control as I think we'd often like to believe.
So our agency has this giant asterisk and, you know, consumption itself in this networked
environment is really no longer just an individual choice.
It's something that we do to each other, whether we mean to or not.
Yeah.
You know, that's true.
I do still believe in conscious consumption so much as we can do it.
Like even if I'm just one person, it's important to me to signal with my choices what I value.
And in certain cases, I value opting out of being surveilled so much as I can control
for it.
You know, maybe I can't opt out of facial recognition and facial surveillance, you know,
because that would require a lot of obviscating my face.
And I mean, there's not even any reason to believe it would work.
But there are some smaller things that I personally find important.
Like I'm very careful about which apps I allow to have location sharing on me.
You know, I go into my privacy settings quite often.
You know, I make sure that location sharing is something that I'm opting into on the app
while I'm using it.
I never let apps just follow me around all the time.
You know, I think about what chat apps I'm using, if they have encryption, you know,
I do hygiene on my phone around what apps are, you know, actually on my phone because
they do collect a lot of data on you in the background.
So if it's an app that I'm not using or I don't feel familiar with, I, you know,
I delete it.
That's really smart.
And it's such a helpful reminder, I think, of the power that we do have here and a reminder
of what the surveillance state actually looks like right now.
It's not some cinematic dystopia.
It's, sure, the camera's on the street, but it's also the watch on our wrist, the phones
and our pockets, the laptops we use for work.
And even more than that, it's a series of decisions that governments and organizations
are making every day on our behalf.
And we can affect those decisions if we choose to in part just by paying attention.
Yeah, it's that old adage who watches the watcher and the answer is us.
That's all for this episode of How to Know What's Real.
This episode was hosted by Andrea Valdes and me, Megan Garber.
Our producer is Natalie Brennan.
Our editors are Claudine Bade and Jocelyn Frank, fact checked by Anna Alvarado.
Our engineer is Rob's Merciac.
Rob also composed some of the music for this show.
The executive producer of audio is Claudine Bade and the managing editor of audio is Andrea
Valdes.
Next time on How to Know What's Real.
And when you play the game multiple times, you shift through the roles and so you can
experience the game from different angles.
You can experience a conflict from completely different political angles and re-experience
how it looks through each side, which I think is something like.
This is what games are made for.
What we can learn about expansive thinking through play.
We'll be back with you on Monday.
