You're listening to TIP.
Hey, everyone.
Welcome to this Wednesday's release of Infinite Tech.
Today's episode is a deep dive into the early foundations of conversational AI and what they reveal about today's language models.
My guest is Dr Richard Wallace, a pioneering chatbot creator and three-time Lobner Prize winner, best known for building Alice and the AIML language that powered early conversational systems.
The Lobner Prize was an annual competition designed to implement Alan Turing's imitation game, awarding the chatbot that could most convincingly carry on a human-like text conversation.
Simplicity beats scale in the early AI race.
How supervised rule-based systems differ from modern LLMs, what the Turing test actually misses and why combining symbolic reasoning with neural networks may matter more than raw model size.
This is surely an episode you will not want to miss.
So without further ado, let's jump right into the conversation.
You're listening to Infinite Tech by the Investors Podcast Network. hosted by Preston Pysh.
We explore Bitcoin AI robotics, longevity and other exponential technologies through a lens of abundance and sound money.
Join us as we connect the breakthroughs shaping the next decade and beyond, empowering you to harness the future today.
Hey, everyone.
Welcome to the show.
I'm here with Richard Wallace.
And wow, this is really exciting for me to talk to such a pioneer in this space, in the chatbot AI space.
And first of all, welcome to the show.
Excited to have you here.
Thank you.
Thank you, Preston.
It's a pleasure to be here as well.
So where I want to start is I'm just super curious how people kind of fall into their field of expertise.
And when I look at what you accomplished very early on back in the 1990s, I'm curious what drove you or motivated you to be paying attention to chatbots and the Turing test And all of that.
It's such an early phase because I think for most of the listeners they know that all this stuff has really come to fruition in the last five or 10 years.
It's gone on everybody's radar.
But you were doing this literally decades before.
Anybody was even aware of these ideas of chatbots and whatnot.
So what was your initial motivation to get into this kind of stuff?
That's absolutely right.
You know, I like to say that nobody knew what artificial intelligence was until a couple of years ago.
Yeah.
And now I'll be sitting in a restaurant somewhere and I'll hear a conversation at the table next to me, and they're talking about AI.
Well anyway, there are several threads that came together that inspired me to work on the chatbot Alice.
And I'll just pull on a couple of those threads here.
One is that in 1990, I read an article in the New York Times about the first Loebner Prize contest.
Now, the Loebner Prize was an annual Turing test, an annual contest based on the Turing test, funded by a rather eccentric philanthropist, Hugh Loebner.
And the story with the very first contest was that none of the programs competing came close to passing the Turing test.
They were all just terrible chatbots.
But Loebner awarded a bronze medal every year to the chatbot that was ranked highest by the judges in terms of being the most human.
And that first simply based on the old ELISA psychiatrist program which, if you're familiar with that, was a very primitive chatbot developed by Joseph Weizenbaum in 1966.
And it had very few responses.
But it You know, it could sort of match keywords in the input and add canned responses associated with those keywords.
It could invert prepositions.
So, you know, if I said I came here to talk to you, then it would repeat back.
You came here to talk to me.
So it did that sort of pronoun swapping trick.
But when I was in graduate school in the 1980s, this ELISA program was basically considered kind of a dead end or at best kind of a hoax in AI.
And not only that the inventor, Joseph Weizenbaum ended up pulling the plug on it because he thought it was too dangerous.
He thought that people were reading too much into it and was actually there.
It was a psychiatrist program, so people were trusting it with their personal issues and problems.
They were surprised to find out that Weizenbaum could read all the transcripts of their conversations.
Wow.
And so he wrote a whole book after that where he criticized the whole field of AI and his ELISA program in particular.
It's really hard to imagine this now that someone would come up with a new AI application that's very engaging and popular.
People are using it.
Then they would say, oh, no, this is too dangerous.
We have to put the genie back in the bottle.
I think most people now would run out and try to find venture capital to start a company to commercialize.
True.
Very true.
But isn't this fascinating that the thing that he discovered very early on in the 90s he started playing around with this in the 60s, which is mind-blowing to me.
But what he found in the 90s was that there was a huge centralization, concern with privacy and what people were putting into these discussions, which is now a major talking point with AI.
And it doesn't seem that I know I'm generalizing here, but it doesn't seem like the population really cares too much or even thinks about these issues.
That caused him to shut down his entire effort behind this.
I don't know.
I find that really fascinating that he discovered this.
What four decades before it became like, or three decades before it became something that the rest of the world should be very concerned about?
Well, he really discovered it in the 1960s when he first created the program.
The other ironic thing about Eliza was that, up until very recently I would say well, let's say 20 years distributed, popular and well-known AI application.
If you knew anything about AI up until maybe the year 2000, then you would know about ELISA.
I'm curious.
When you read this I think you said New York Times article in 1990 did you ever think that you would be the winner of this Lobner Prize a decade later?
Well, that planted a seed in my mind, and I didn't really do anything about it for about five years.
Okay.
So another thread that led to the development of Alice, or the inspiration for Alice, was around that time in the early 90s.
It was the end of the Cold War.
And so there was decreased amount of government funding available for AI and robotics research compared to the 1980s.
And so a number of us in the robotics field I was working in robotics at the time got interested in the idea of minimal demand, robot minimalism.
And basically that was the idea that we could build robots with very simple, inexpensive sensors and actuators, very commodity microprocessors.
And as a result of that, you could actually get more lifelike behavior out of these robots than you could with.
You know approaches people had tried in the past with much larger computers and so forth.
One of the interesting inventions that came out of that period was the Roomba.
Yeah, I think of the Roomba rolling around and bumping into things and changing its direction.
It's all basically just a stimulus response application.
We call that stateless.
So it's sensing something and then taking an action based on what it's sensing, you know, changing direction, for example.
Yeah.
So that that whole approach of minimalism was also in my mind at the time.
The very simple approach of the ELISA program, which was also kind of a stimulus response.
You know, it was so simple that it could respond very quickly.
It didn't have to go and do a lot of computations to come up with the responses.
What was your inspiration for thinking that simplicity was going to lead you to better results?
Was there something in your life or something that you were reading at the time, or you know what drove you to that intuition?
Like I said, we were working on the minimalist philosophy of robotics and at that time I was working on the development of a robot eye, and by that I mean a visual sensor that's from a TV camera, in the sense that a TV camera is basically a square grid of square pixels.
But the human eye is more like concentric rings of pixels with higher and higher resolution towards the center.
We call that a log map.
And so we developed a sensor that had that log map log map pixel organization.
In order to use a camera like that effectively, you have to be able to point it.
So we developed a little motor, high speed pointing motor based on a direct drive design.
And that motor could point directions very, very quickly.
And again it was a very simple kind of actuator, simple sensors, and it could move very quickly, could move actually faster than the human eye.
So you'd sort of see this thing whipping around and looking at different things.
And it was very lifelike.
So just for the audience to understand.
So in 2000, I believe, 2001 and 2004, Dr Wallace won the Lobner Prize, which is this Turing test with his Alice protocol or chatbot that he had created.
And I guess for me, what was the major insight that you think that you had back then?
You talk about this idea of simplicity, but what would you say was the major insight that you had to outperform everybody else that was competing on what is I mean for anybody listening the most complex, challenging problem you could ever try to go after right?
What would you say was your keen insight that you had that allowed you to do this?
Well, it was basically the idea that I could build on the ELISA program.
So the ELISA program had about 200 rules, 200, you know, stimulus response rules.
And you can think of that as a pattern and a response.
And my idea was to kind of was to build kind of a super ELISA where instead of 200 rules you had thousands and thousands of rules.
And in fact, I
By the time I was entering those contests, I got Alice up to about 50,000 patterns and responses.
Preston Pyshko Wow, amazing.
So Richard, one of really fascinating about you back at this time was that you came up with this artificial intelligence markup language.
You effectively.
For all intents and purposes and correct me if I'm mischaracterizing this you had to come up with your own language in order to kind of build efficiency into how this chatbot was working, which is, you know, as a person who's not very good with languages, I'm much more of a math person.
I'm reading this and I'm thinking this is mind.
Talk to us about this.
And what was this insight that you had to come up with, the AI ML artificial intelligence markup language at the time that you did this?
Well, AIML is based on XML, and XML was very popular at the time.
One thing that appealed to me about XML for the purpose of writing chatbots was that I always say XML has an implicit print statement.
So when you write the responses, you don't have to put in an expression that says print blah blah, blah something, because the XML already just provides the text inside the markup.
So the response is just the text inside the markup.
And then basic unit of knowledge.
In AIML I call the category, which is like the rules I was talking about a second ago.
So the category consists of a pattern that matches some input, some natural language input.
It's not exactly the answer, but it's a template for the answer that can be populated with various other things.
And then there was also a recursive element to it, where the response could actually simplify the input into a kind of simpler input.
So the example of that is, I want you to that by removing the right now.
So I want you to tell me who you are.
And then you can remove the I want you just so it reduces the just tell me who you are.
And then that reduces to who are you.
So there was that recursive element built into the responses as well.
So in general, you're just you were taking language and you were making it way more efficient.
And they're like, where do you even start with something like that?
I mean, you literally have to go through.
There's just so many different variations of language.
And I think of the complexity of this.
I wouldn't even know where to begin to start writing something that makes it more efficient.
Like, right.
Yeah.
So how did you how did you think about solving that problem?
Well, it all goes back to the conversation logs.
So just like Weissenbaum, you know, I can read the transcripts of conversations people were having.
By the way, this would have never worked without the internet.
Without the World Wide Wide Web, I could start to accumulate conversations from a very large audience of people.
And by looking at the transcripts of those conversations I could basically program responses to the things people were saying.
Later on, I realized that there was kind of zip distribution over the things people were saying.
So, you know, there's kind of a most common thing people say, which is hello.
And then who are you and how are you?
And I like something.
So you can create the responses in order of how frequently people say particular things.
Let's take a quick break and hear from today's sponsors.
Have you ever been interested in mining Bitcoin?
As a miner myself.
I've been using Simple Mining for the past few months, and the experience has been nothing short of seamless.
I mine with the pool of my choice, and the Bitcoin is sent directly to my wallet.
Simple Mining, based in Cedar Falls Iowa, offers a premium white glove service designed for everyone, from individual enthusiasts to large-scale miners.
They've been in business for three and a half years and currently operate more than 10000 Bitcoin miners.
Based in Iowa, their electricity is over 65% renewable, thanks to the abundance of wind energy.
Not only do they simplify mining with their top-notch hosting and on-site repair services, but they also help you benefit financially by running your operations as a business.
This approach offers significant tax advantages and enhance We'll see you next time.
Visit simplemining.io slash Preston to get started today.
That's simplemining.io slash Preston to get started today.
With Simple Mining, they make it simple.
Imagine scaling your business with technology that understands your customers, literally.
That's the story behind Alexa and AWS AI.
Every day, Alexa processes over 1 billion interactions across 17 languages, all while reducing customer friction by 40.
It's not just about making life easier.
It's also about transforming customer engagement and generating new revenue streams.
Behind the scenes, AWS AI powers more than 70 specialized models working together to create natural conversations, proving how enterprises can deploy AI at scale with confidence and security.
Alexa's AI capabilities were battle-tested across Amazon's massive operations, delivering real, measurable impact at scale.
These same innovations now give other businesses a proven framework to boost efficiency, unlock new revenue streams and gain a lasting market edge.
Discover the Alexa story at AWS.com slash AI slash R dash story.
That's aws.com slash ai slash r dash story.
The more your Bitcoin holdings grow, the more complex your challenges become.
What started as a simple self-custody now involves family legacy planning, sophisticated security decisions and navigating situations where a single mistake could cost generations of wealth.
Standard services weren't built for these high stakes realities.
That's why long-term investors choose Unchained Signature, a premium private client service for serious Bitcoin holders who want.
And I'll see you next time.
Unchain's collaborative custody model is designed to provide the same security posture as the world's biggest Bitcoin custodians, but for those who prefer to hold their own keys.
Learn more about Unchain's signature at Unchain.com slash Preston.
Use code Preston10 at checkout to get 10% off your first year.
Bitcoin isn't just for life.
It's for generations.
All right, back to the show.
My question for you is and I'm struggling to find a way to frame this but do you write these thousands of rules and you're also working on a way to compress or make the English language more efficient.
What did you fundamentally learn through the experience of writing these thousands of rules and rules of thumb of compression?
Because, when I think about it, we look at these LLMs and machines are doing all of this really hard and complex work.
But I would imagine what you were doing there in the 90s and early 2000s was exactly what all these LLMs are doing today.
I hear these people that say well, we have no idea what's behind these ones and zeros and all these LLMs, which I guess is a true statement, right.
But if a human was going to maybe be able to understand what it is that it's doing, I think you would be one of the very few people on the planet that could maybe help us understand what that is, because you did this manually for so many years.
Right. things wrapped up in that question.
So let me see if I can pull it apart.
Yeah.
So there's always been a kind of tension in the history of artificial intelligence between, let's say, supervised learning and unsupervised learning.
So what I was doing was what we call supervised learning, because I was playing the role of a teacher or a guide.
So whenever I added a new response, it was manually added, as you're saying, driven by a particular input that I saw in the conversation logs.
And so the way that I'm teaching the robot is by acting as its teacher basically, and saying you know, when you see this, you should say that.
And, you know, that's in contrast to unsupervised learning, which is what these LLMs are doing.
They're basically, you know, trying to accumulate a lot of inputs and, you know, find the neural network weights that match it to particular outputs.
And so with that technique you can get phenomenal results.
Obviously but, as you're saying, it's difficult to know how the LLM came up with particular responses, whereas in the supervised learning case, where it's all a symbolic process, it's very easy to trace back through the logic of the program and see what caused a particular response to be generated.
And I always say that people who do supervised learning approaches spend all of their time doing creative writing, which is what I was doing with the Alice bot.
But people who do unsupervised learning spend, And that's sort of what's going on with the LLMs now is they're having to put a lot of work into filtering to make sure they don't say anything inappropriate or offensive or political.
And that ends up being a lot of manual work as well.
Yeah.
And I guess my understanding is that everybody that's on the cutting edge of AI today that's the holy grail for them is to get the human out of the loop and for it to be completely AI-generated and filtered and just like there's no humans there.
As a person who deeply understands this and the way that you frame that is this back and forth, and there's consequences to one side and the other.
Is there a moment where you complete removing the human out of the loop and it progressing in a way that's actually beneficial?
Or do you think that, the more that they lean into removing the human out of the loop, that they actually are setting themselves up for a systemic failure, because it's going to spiral into this AI slop, if you will, or it's creating and generating content in a direction that's so fast and so extreme that they get away from human filtering altogether?
And it just kind of turns into this almost like a runaway virus, if you will.
Is that how you kind of see this, that it needs to be balanced?
Or is it even possible for it to go in that direction without humans?
Well, it's so hard to predict the future.
I would have never expected this whole LLM development to come along in the first place.
But I always think of a child learning language, and there are big differences here between a child learning language and an LLM.
A kid doesn't have to scan the whole internet to learn how to speak a language.
In fact, they're pretty good at what we call one-shot learnings.
Say to a kid this is a dog, then they can instantly recognize every dog in the world as a dog.
But what also comes into play here is the supervised, unsupervised learning dichotomy, which is if you are a kid and you have a good teacher and good parents, you'll learn to speak very well.
But if you're a kid who has to pick up language on the street without any supervision, then your language learning won't be nearly as good.
And so the LLM is more like the kid out on the street learning language without any supervision.
And that's why they learn so much inappropriate and offensive material and so on.
What did the wins teach you back in the day when you were winning this, about how humans judge intelligence?
Well, you know, I can say the same thing about LLMs now that I said about my chatbot back then, which is that people say well, these chatbots are becoming more, and about that, which is that what it's really showing us is that people are more like robots than we would like to think we are.
Because it's not that they're robots becoming more like a human.
It's that it's revealing to us that most people, most of the time, are saying things that they themselves have said before or that they've heard other people say before.
And even when they're writing, they're basically synthesizing thoughts and ideas that are not necessarily original.
And all of these chatbots work because language is predictable, and predictable means robotics uttering an original line of poetry with every sentence we spoke.
Then these chatbots would never work because they're based on language being predictable, not original re-utterance.
Yeah.
Is it fair to say that you would suggest that humans judge intelligence by their flow or by this response of like?
Most people are looking at that and they're saying oh, that's intelligence.
But then you're looking at it and you're saying that's not intelligence.
It's just repetition.
I think that's kind of what you're getting at.
Repetition.
Right.
I just read something it was like last week and I think Google came out with this many months ago.
But for them to do this long-term learning, where it has much more of a memory, it's highly based on whether something's novel or not relative to its index of everything that it's been trained on.
And when it sees this novel thing that it wasn't predicting or expecting to come next, that it then stores that in its long-term memory, or I apologize for the terminology here, Dr Wallace, but it flags it as something that is worthy of being remembered because it's novel and so different and outside of what would have predicted to be the next thing.
And it's interesting that it's in keeping with Claude Shannon's information theory.
That and how it's all aligned.
I'm curious if you have any opinions on that in particular and whether you think that that has a key component to intelligence or how new things are discovered in knowledge in general.
Well, that really gets to the heart of what I think the difference is between humans and robots, which is that, like I said, I think most people, most of the time, are acting like robots.
They're just acting in kind of a stimulus response fashion.
Just as an aside, I always used to say that most human conversation is stateless, meaning that what I'm saying to you right now only depends on the question that you just asked me.
And we can forget the whole history of our conversation up to this point.
You know, one of the pieces of evidence for that is if you can imagine yourself having a casual conversation with someone at a party say, and then you say oh, where did you go to college?
And they say, oh, I went to Harvard.
I already told you that.
You kind of forgot that you had already talked about college earlier in the conversation.
And you're just responding to the most recent thing you heard and most recent input.
But what really gets to the difference between humans and robots is even though most people, most of the time, are speaking in this to have original thoughts and be creative.
And it's almost like a muscle that you need to exercise in order to build it up.
If you want to break out of that robotic mold, then you have to put some effort into trying to be creative and original with your thoughts and thinking and ideas.
Let's take a quick break and hear from today's sponsors.
You know what sets the best businesses apart?
It's how they leverage innovation to turn complexity into growth.
That's exactly what Amazon Ads is doing, powered by AWS AI.
Every day, Amazon Ads processes billions of real-time decisions, optimizing ad performance across a 31 billion advertising ecosystem.
The result is campaigns that run 30% faster and deliver measurable business impact at scale.
Argentic AI transforms marketing from a resource-heavy process into an intelligent, autonomous system that maximizes ROI and empowers marketers to focus on creativity and strategy.
Amazon Ads is proving that AI-driven advertising isn't just the future.
It's the new competitive advantage.
And, better yet, every enterprise can apply the same innovation playbook that Amazon perfected in-house.
See the Amazon ad story at aws.com slash AI slash R dash story.
That's aws.com slash AI slash R dash story.
Startups move fast.
And with AI, they're shipping even faster and attracting enterprise buyers sooner.
We'll see you next time.
Plus, Vanta scales with you, backed by support that's there when you need it, every step of the way.
With AI, changing regulations and buyers' expectations.
Vanta knows what's needed and when, and they've built the fastest, easiest path to help you get there.
That's why serious startups get secure early with Vanta.
Our listeners get $1,000 off at vanta.com slash billionaires.
That's V-A-N-T-A dot com slash billionaires for $1,000 off.
It's the new year, which means that it's the best time to finally start the business you've been dreaming about.
Just a couple of years ago, I launched my own e-commerce business, and Shopify was exactly the tool I needed to get started.
While many people continually push off their dreams until the next year, I am here to tell you that now is the time to capitalize on the opportunities right in front of you.
Shopify gives you everything you need to sell online and in person.
Millions of entrepreneurs, including myself, have already made this leap from household names to first time business owners just getting started.
Choose from hundreds of beautiful templates that you can customize and use their built in AI tools to write product descriptions or edit product photos.
And as you grow, Shopify grows with you every step of the way.
In 2026, stop waiting and start selling with Shopify.
Sign up for your $1 per month trial and start selling today at shopify.com slash WSB.
Go to shopify.com slash WSB.
That's Do you think that the Turing test actually measures intelligence, or is it something else entirely?
I'm so happy you asked me about the Turing test.
So the Turing test most people understand the Turing test as sort of game where there's three players.
You have a person who's called the interrogator or the judge, and then they're communicating through a teletype, a text only medium.
You know, much like texting on your phone, but without any audio visual, just typing.
And then the two entities that the judge is talking to, one is a human and one is a machine.
So then the judge has to decide which one is the human and which one is the machine.
And if they misidentify the machine as the human, then it's said to pass the Turing test.
You see, this has a big problem as a scientific experiment, because it's not really clear how often the interrogator has to misidentify the human.
Is it 50% of the time?
75% of the time?
100% of the time?
What does that even mean?
The robot is more human than a human.
So in Turing's 1950 paper, test or the game.
And earlier in the paper he described something called the imitation game which, as far as I understand, was based on a real parlor game that people played in Victorian England.
And in this game, again, there are three players, the judge or the interrogator.
The other two players are a man and a woman.
And let's just set aside the gender issues and the context of writing in 1950 here.
So there's a man and a woman sequestered away in the Victorian England case in different rooms.
And then the judge is sending them handwritten questions back and forth.
And the judge's job is to decide which one is the man and which one is the woman should always tell the truth and the man should always lie.
So now, if you, if you ask the man, are you a woman?
He would say yes, because he has to lie.
Okay, and then you know.
The judge's job is to try to figure out which one.
You replace the man with a very crude chatbot like Eliza, or even Alice, then the judge could identify the woman correctly 100 of the time, because it's clear that only one of the players is a human at all and that has to be the woman.
Experiment with 100 judges and 100 men and 100 women.
I don't know exactly how many are needed for statistical accuracy, but let's just say we did a random sample where we collected the results of this game for a large number of players.
Then you could measure a certain percentage of the time that the judge would identify the woman correctly.
And let's say that's 70% of the time.
Now, if you replace the lying man with a computer and the computer is a very good AI that can actually play the role of the lying man then you should get closer and closer to that actual 70 measurement.
So that's actually a better scientific experiment than the Turing test.
Very interesting.
Yeah, yeah.
The Loebner contest was really based on the original standard Turing test.
Turing test, okay.
Yeah.
And you know the rules change from year to year depending on you know who is hosting the contest, but
Loebner's rule was basically if 50 of the judges usually there were four judges, so if two out of four judges misidentified the robot as a person, then he would award the silver medal for passing the Turing test.
That's so cool.
It was never awarded, by the way, a time machine right now and go back to your days, call it 2000.
When you had done this, what would be the thing that you would whisper to yourself as a hint as to how to improve the chatbot that you had back then?
I would probably tell myself, don't even do this.
Because you know how hard it is.
No.
There was no money to be made from chatbots until very recently.
You were very early.
The Loebner contest was always the domain of you know, hobbyists and amateur programmers.
There were a few, you know, academic entries, but no big companies ever got involved in it.
And then in the 2000s I organized a number of chatbot conferences.
You know international chatbot conferences and we have a hard time getting 25 people to attend.
Oh, really?
Okay.
Yeah.
So you know, after many years of And I did co-found a company called Pandora Bots which is, you know, based on attempting to commercialize the AI, now bots.
But you know, after a while in the early after, in the early teens, I should say, I just decided to get out of the field completely.
And I went to work in health care.
Yeah.
But then in the past five or six years, I've gotten back into AI as it's lucrative, I should say.
It seems like in 2017, Google came out with this paper.
It was called Attention is All You Need.
And this seemed to be a very seminal breakthrough in how to, for all intents and purposes, do what you were doing in a very manual way and let machines do it way faster and with way more horsepower and more data right.
I'm curious when this paper came out.
Did you read it when it first came out and were you kind of aware of this, or did it kind of hop on your radar a couple years after, when we started seeing the breakthroughs?
Yeah, I was really not paying attention to it at the time.
Like I said, I was working in healthcare.
I don't think the LLM... industry really came to my attention until we started hearing about GPT.
Do you think that that paper was kind of like a really important seminal piece of work for people to kind of understand how to start doing this in a mechanical machine kind of way?
Yeah, obviously.
Obviously, that was a breakthrough.
Yeah.
Wow.
And so in your own words, what would you say?
I mean, we know attention is a big piece of it.
But I think for somebody that just kind of hears that label, it's like okay well, what does that mean?
If you were going to try to explain to somebody in a very simple way, like what is that paper saying that has enabled, you know, machine learning to do what it does?
Well, in a way, I'm reminded of the work we talked about earlier, which was the robot eye in the early 90s, because that was also an attention-based mechanism.
Hmm.
I described how, in order to make use of that log map arrangement of pixels where there's high resolution towards the center, you have to be able to point the camera so that the high resolution can be aimed at something interesting.
Well, how do you know it's interesting?
It's by, if you see something in the periphery, for example movement you want to move your eye towards the thing that you're seeing in the periphery and place the attention on that.
So attention has to do with focusing your highest resolution sensor sensory capability on whatever seems most interesting in a scene.
I think there's an analog for that in the LLM version of attention as well.
You know, they're sort of swinging in the direction of where the gaze of the robot is looking, depending on what they see in the periphery.
Okay.
So this is super.
I love this example because it's very physical and you can kind of make sense of it very simply, because it's dealing with vision.
And so when you're changing your attention and you're able to zoom in, because you have the capacity to zoom in on something, how are you filtering or knowing what's novel in that broader perspective picture in order to know to adjust the focus to that thing?
What gives us that capacity to know?
Oh well, I'm looking at you and now I'm focusing on the tree back behind you and I'm zooming in on that and I'm putting my attention there.
What would be that insight in order to say, oh, that's different.
That's something I need to dial in on or pay more attention to.
Yeah.
A long time ago a guy called Hans Moravec, who's very interesting.
We should talk about him some more.
He came up with an attention mechanism called an interest operator.
And this is for computer vision again.
Okay. that have high variance, a high ratio of dark to light, more interesting than other things.
So that would typically be edges, like the edges of the tree you just described, or corners of things, or just any sort of bright spot against a dark background or vice versa.
And then recognizing those in the periphery of your visual field would cause you to move the center of your visual field towards whatever the interest operator is highlighting.
Fascinating.
Okay, here's an odd question for you.
Do you think your real subject of study ended up being humans rather than machines?
Oh well, you know i'm a computer programmer, so i was always more interested in the machine side of it.
I think i did learn a lot about human conversation from monitoring those conversation logs.
The reason i asked this question is, you know, in kind of research and preparation for the interview, it seemed to me that you have this opinion.
I suspect and correct me if i'm saying any of this wrong but It seems like you were not convinced that any of these chatbots were actually saying anything intelligent.
It was just It was this canned response that was coming back.
And then the reaction that humans had was like, wow, this thing is real.
And there's like something behind it.
And so I guess that's the impetus for the question is because you were, I suspect you were fascinated at the response of people and how duped I guess they were by interacting with some of these chatbots.
So I guess that's more of the impetus for the question.
And would you agree with everything that I just said?
Well, I used to categorize the users or the clients.
I call them into three categories A, B and C, And A.
Clients are abusive.
So they're going to say, how can I put this?
Very inappropriate things to the chatbot.
And you see those in the conversation.
So you always have to wonder if someone is saying, you know, I hate you or I love you even.
Is that what they really have in mind, or are they just, you know, trying to get a response out of the robot?
Testing the limits.
Yeah.
Category B people were the ones who could suspend their disbelief and they would be very engaged with the bot and have, you know, very long conversations, come back and continue their conversations, and so on.
And so that would be the group that you know, as you're saying, would be kind of reading more into the bot than was actually there, because they're engaged with the critics, which are people who know something about computer programming and AI, and they just think this thing is terrible and they walk away after a few interactions.
Yeah.
Well, I'm curious to hear your thoughts on where we're at now and where you see some of this going next.
You have some really smart people in this space that have demonstrated their knowledge through the things that they've built.
And I think, if we back up the tape three years ago, many of them were very suspect as to whether AGI could ever be possible today.
And I have a hard time knowing if this is them trying to get more capital or they actually believe that we're on the cusp of AGI.
I don't know which one of those two it is, but I'm just curious to hear your general thoughts on where you see us today and what the next five years might bring.
As exciting of a next five years as we've seen in the past five years.
Kind of just give us.
Your AI has always been a goal that's just out of reach.
And you know we have an imagination of what it is based on, seeing science fiction movies and that sort of thing.
You know, Hal and R2-D2 and all those examples give us a template for what we'd like to see in an AI.
And so it seems kind of odd that they've come up with a new term, H2O2.
A very simple answer to this question, which a lot of people I know would not agree with, is that God gave human beings a soul, but machines don't get a soul.
So, in the sense that human beings have freedom of thought and self-reflection and creativity, I don't think those things will be reproduced in a computer anytime soon.
Yeah.
And I think I'm with you 100% on what you just said.
And I know there's a lot of people that want to argue these ideas and we're not here to do that, but I'm with you 100.
I think that there's something very special and unique about just any living being, not just humans.
I think any living being has this special connection from a higher source.
And I don't think that we're necessarily going to see these humanoid robots have whatever that is.
And I have no idea how to define that.
But I do think that some of these humanoid robots call it five or 10 years from now, are going to do things.
And it goes back to some of your earlier comments about these chatbots and how people were just like oh my God, I feel like, And I think that some of these humanoid robots are going to feel like real humans to a lot of people.
But that doesn't mean that it's the same thing as us.
I think we are something very hard to define, very different.
But oh, my goodness, Richard, I really enjoyed this conversation.
Anything else that you want that you think is super important on this particular topic that you see right now or kind of going into the future that you think is worthy of highlighting or that the audience should know? yeah well the company i work for right now franz okay it's actually a very old ai company founded in 1985 and franz started out as a company selling uh lisp compilers but then you know by the end of the 1990s very few people were paying money for software you know because there's so much free language software available So they pivoted to graph database technology.
And without getting into too much detail about what that is, now that we have the LLMs, we are taking an approach called neurosymbolic.
So in the history of AI I talked about supervised versus unsupervised learning.
Another dichotomy in AI is between symbolic and neural approaches.
So symbolic approaches are things like you know, theorem proving programs or the early chatbots that we were talking about, based on rules where basically, you're manipulating symbols.
Or you can also think of a chess playing program, you know which is very mechanical.
The symbolic approach is in contrast to this neural learning approach.
And now we're basically trying to find the best of both worlds.
So one example of that is in the medical field.
You can make predictions about how likely someone is to be well, their mortality, how likely they're going to be readmitted to the hospital after being discharged within 30 days.
How likely are they to be readmitted or how likely they are to have a stroke and the various other things.
But the medical field has developed these symbolic techniques for making those predictions.
And so in the case of stroke from AFib, there's a test called account criteria like your age and gender, whether you've had congestive heart failure, history of hypertension and various other factors like that.
And when you plug in those values, it produces a number which can then be used to estimate the likelihood of you having a stroke.
And now you could also do that with a neural network, a recursive neural network, where you basically train it by feeding in the, you know, the patient data, the diagnostic data and their medical history, and then just look at whether they had a stroke or not.
So you can train this neural network to take a new patient data and, you know, give some prediction about whether they're going to have a stroke.
And then the third way of doing that is to use an LL, the entire patient chart to the LLM and say how likely is this person to have a stroke?
And so what we've been doing is sort of combining those three approaches together.
You know, we've got the symbolic estimate, we've got the neural estimate and we've got the LLM estimate.
You know, you can potentially display all three of those, and then it's up to the clinician to make a judgment, or you could even put them all back into a different LLM.
And ask the all on which one of these measurements is best, which one of these predictions is best.
So it's an effort to combine the best of the symbolic approaches with these newer neural approaches.
Wow.
Say the name of the company one more time.
I want to make sure I have the name of it in the show notes for people if they want to check it out.
Yeah.
All right.
Well, I'm just so thrilled to be able to talk to somebody who's been in this space for decades.
It's miraculous to see what's happening.
And I can only imagine where we're going to be in five years from now.
But, Dr Richard Wallace, thank you so much for making time and coming on the show and imparting all of this knowledge that you have.
We really appreciate it.
Well, i'm glad people want to talk to me about it after a long time of people not being very interested.
Well, there's a lot of people interested now let me tell you something, but thank you again for making time.
Okay, my pleasure.
It was great talking with you as well.
Thanks for listening to tip.
Follow infinite tech on your favorite podcast app and visit theinvestorspodcastcom for show notes and educational resources.
This podcast is for informational and entertainment purposes only.
Listeners should do their own research and consult a qualified professional before making any financial decisions.
Nothing on this show is a recommendation or solicitation to buy or sell any security or other financial product.
Hosts, guests and the Investors Podcast Network may hold positions in securities discussed and may change those positions at any time without notice.
References to any third-party products, services or advertisers do not constitute endorsements, and the Investors Podcast Network is not responsible for any claims made by them.
Copyright by the Investors Podcast Network.
All rights reserved.
