You're listening to TIP.
Hey, everyone.
Welcome to this Wednesday's release of Infinite Tech.
Today we're talking AI and robotics, and where there's still key areas of development that need some work.
My guest is Ken Goldberg, a leading robotics researcher whose work bridges academic AI, real-world automation and large-scale commercial robotics systems.
One of the things we discuss that's super interesting is the assumption that large language models automatically unlock physical intelligence.
And this is an area where Ken is really well-versed and does a great job explaining what that actually means.
We cover what has improved like mobility and automation. and what's still painfully hard, like dexterity, sensing, and real-world manipulation.
This is a grounded conversation about engineering reality versus expectation, and Ken is a true expert in this field, as you'll see in the conversation.
So without further delay, I hope you guys enjoy this chat.
You're listening to Infinite Tech by the Investors Podcast Network, hosted by Preston Pysh.
We explore Bitcoin AI robotics, longevity and other exponential technologies through a lens of abundance and sound money.
Join us as we connect the breakthroughs shaping the next decade and beyond, empowering you to harness the future today.
Hey, everyone.
Welcome to the show.
I am here with Ken Goldberg and I am so excited to have this conversation because you are an expert in this field of robotics and AI.
And it's something that we talk about all the time on the show.
And it's just exciting to have somebody like yourself here today to talk about it.
So welcome to the show, Ken.
Thank you, Preston.
I'm excited to talk to you, too.
So before we started chatting, you sent over an article.
It to kind of set the stage for probably most of the conversation we're having today.
And it was an article that was in the New York Times and it's talking.
Rodney Brooks has, quote unquote, said the field has lost its way.
And for people that aren't familiar with Rodney Brooks, he's the former director of MIT's Computer Science and Artificial Intelligence.
He's the Rhomba inventor and running that entire company and product line.
And so for him to come out and say something like this, this is kind of a big deal.
And I just really want to kind of capture your take on what is he talking about?
The field has lost its way.
Rodney Brooks Well, I think he's a very, very respected individual.
He's a good friend of mine, and I agree with him very much.
And I think he's provocative.
He's put it in his own words.
But I sent that to you because I think it's very relevant for us to start this conversation about what's real and what's hype in robotics.
And I have to be careful about the word hype, but I want to call it inflated expectations.
You know, I think people are excited about technology.
Everyone, I am too.
We all grew up with science fiction and we love it and we love new things.
And there has been some breakthroughs.
I mean there's no doubt that the advances in artificial intelligence, in particular deep learning and then generative AI with the transformer model, have been transformative in the field.
The AI systems are doing things that no one thought would be possible by now.
Mm-hmm.
And I will be the first to admit they're capable of creativity.
They're immensely valuable.
But people then take the next logical step and say, OK, these systems have solved language.
So therefore, they'll solve robotics, too.
And that is where I have a lot of concerns.
I'm really.
We can get into the details, but Rod and I agree that there is not at all obvious that the advances in language in AI will extend to robotics.
What would you say is the number one thing that you're seeing?
That's just grossly out of touch with reality when it comes to the robotics piece being not as far along or it's not coming in.
The talking point is in five years, we're going to have humanoid robots doing everything right.
So like what are the big chunk pieces that people that aren't intimately familiar with the space that you see are missing on that particular topic?
Okay, so let me tell you, first of all, where some of the advances have been made.
And one of them is in quadrupeds.
That's walking dogs, basically, and bipeds.
That's walking machines and navigation, and I would call mobility.
So the ability to get around with robots with legs has made immense progress.
That's been very exciting, and there's no doubt about it.
Those machines are capable of doing backflips, as you know, side flips parkour, all kinds of things that I certainly can't do.
Also, huge advances in drones.
And so the past decade we've seen drone technology take off from something that was very experimental, but it's been a number of advances that have made that possible.
In both cases, a lot of it has to do with motors and the hardware, but also advances in simulation.
And the ability, for example, for drones is to stabilize themselves and then to be able to control very accurately the motors on the four or six rotors that are there.
And the same is true for robots that have legs or quadrupeds or bipeds.
So where these are big, undeniable and major advances.
And if you just look at the field, you say, OK, all this is coming in now.
The next thing we're going to have a home robot taking care of us.
And, you know, this is around the corner, according to Elon Musk.
And I'm sure I'm going to get some pushback from some of your listeners who are going to say I don't know what I'm talking about.
Okay, I've had that happen from a number of very confident quote experts from Silicon Valley tell me that
But I've been working in this field for 45 years and I've studied very closely and understand where the gaps remain for particular manipulation.
And manipulation is being able to pick things up You know all kinds of things that just happen to be in your environment and then being able to manipulate them.
You know, do things.
That skill is very, very nuanced and tricky.
And it's not clear that the current methods for doing AI is going to get us there.
I've heard in interviews Elon in particular say that the hand, mimicking the hand and the tendons and being able to have that tactile ability is extremely difficult is the way he has said it in interviews.
But I think, I suspect, that when it really comes down to it, what you're seeing is a lot of demos online that you see, like this video somebody picks up a pen and the robot did it.
But What's actually happening there behind the scenes?
Whether that was a programmed publicity stunt or something that the robot can just do quite well, there seems to be a large gap there.
So talk to us about where you see that gap and what it is in reality in your humble opinion.
Okay.
And this is understandable.
Again, I don't want to say people are naive.
I get where they're coming from.
They see something and it looks human-like.
And so they attribute human-like qualities to it and skills.
I understand that.
And, by the way, when Ilan says the hands are hard, we can produce.
People are designing hands that look very much like human hands, that is, they have 22 degrees of freedom and they can move all these joints independently very quickly and they look almost identical to human hands.
So we can reproduce that.
In fact, there's like a hundred different hands that are being produced by different companies in china right now.
Okay,
So the advances in the hand itself are very sophisticated, but the control of the hands is where the challenge is.
And this is where, if you have this hand doing this, but then get it to actually tie your shoelace, that is where the challenge is.
And this is because we have There's so many nuances in the interactions that we have with these fingers with the environment that we are sensing the environment, we are exerting forces on the environment.
And this is very subtle, very nuanced.
And we perceive this through a variety of techniques.
We have something like 15,000 sensors in our hand, in every hand.
Yeah, I know, it's remarkable.
We don't even think about it because it's subconscious.
But then we also have sensors in our joints, every one of our joints, so we are able to perceive very subtle forces, slip and, in particular, when very, really interesting way, that those pads are extremely helpful.
If you put on, let's say, thimbles on your finger, like you're doing when you're sewing, that makes it much more difficult to do anything.
You can imagine.
Or just actually heavy gloves.
Gloves?
Yeah well right, but we can do these things very, very subtly.
We have learned this ability to interact with the forces of objects, that the objects are constantly being moved and deformed.
So if you think of the shoelace right that's, the object is being deformed.
The fingertip is being deformed as well.
This mutual deformation is something that's really nuanced and subtle, and we don't even know how to simulate it accurately, So we can't even simulate the forces and torques and deformations that are occurring.
And then we don't have the sensing capabilities to perceive these nuances.
Like you can feel a shoelace if it's a little bit slipping out of your fingertip.
No robot can do that.
So what happens is that when you now have this hand and you actually try to execute something, sometimes it works, but a lot of times it doesn't work.
And now you have the issue of reliability.
Yes, that is where we're seeing and, by the way, you can see robots all day long picking up stuff off a table and moving it somewhere.
That's actually not so difficult if you just want to pick up, especially a stuffed animal.
By the way, stuffed animals are very easy because you almost can't go wrong.
You put your gripper anywhere near them and close it and you'll pick up that thing.
Okay, those are like you know.
They're sitting ducks, right.
They're no, this is super easy, low hanging fruit, let's call it.
And that makes it very easy to pick up and move things.
But now, when you want to start doing things like inserting things, like repairing a stuffed animal by opening it up and getting things and you know, pulling out the stuffing or sewing it back up, this is totally different and much, much more difficult.
Your example of a shoelace is really profound, because until you take a step back and just think, if I had to design or build a robot to tie a shoelace, I can't even imagine how incredibly difficult something like that would be, because it is such a complex task.
And I've never even thought about it, how difficult something like that is.
Well here's, here's what i know because uh shoelace, we all do.
We learn when we're young and we kind of do it without even thinking about it.
It's a subconscious right.
I can be on the phone tying my shoelace, don't even think.
But think about this one.
I don't know about you, but do you know how to tie a bow tie?
Uh, a tie, but not a bow tie.
Yeah.
Bow tie.
Okay.
I thought you might, cause you seem like a fashionable guy.
I have tried it.
It's very tricky.
Yeah.
It's very tricky business and it's subtle.
You have to be able to feel and pull in all these different directions.
Yeah.
Forget it.
There's no robot that's going to be able to do that for a long time.
I would love to have it happen, because that would be something I would love to have a robot tie my bow tie.
And here's another one that's very simple.
It's just buttoning your shirt.
It's actually a little tricky for humans if you think about it, how you have to kind of fiddle with it a little bit to button on and off, especially a small button.
So that's way beyond robotics.
You'll never see a demo of a robot buttoning up a shirt.
We're take for granted.
Now, when you get into solving that problem, it seems like you mentioned this earlier that it's almost a sensing issue that we need a lot of developments on the whatever type of sensors you have in the fingertips or whatever you're using for the manipulation.
Is that the biggest hurdle right now is in just kind of replicating how our fingertips can have so much sensing capability.
Okay, so that's one, but here's something that's somewhat encouraging, for me at least, which is that if you look in the realm of robot surgery and, by the way, there's a lot of misconceptions about that
I give talks where people say, well, robot took out my nephew's appendix.
And I'll say, that was not a robot, that was a surgeon using a robot as a tool to do that operation.
So they call it a robot, but it's really a telerobot, or more literally, a puppet.
A very, very important and very useful and expensive puppet, but it's a puppet.
And so that's very important to understand.
So surgeons can do remarkable tasks.
So you sew up a wound, they can take out an appendix or a gallbladder with these tools.
Now, they do not have tactile sensing.
Actually the very latest versions of it.
They started to introduce some, but for many years they didn't.
And surgeons are still able to do amazing things.
So this is evidence that maybe we don't need to know how to do tactile sensing.
It's just a hypothesis, but it says that we have an existence proof.
That manipulation, dexterous, complex manipulation with very complex deformable surfaces, right?
I mean, it's harder than tying a bow tie to take out an appendix.
That you can do that without tactile sensing.
Now, what's fascinating is the way surgeons seem to do.
It is essentially accommodating the lack of tactile and then using vision their eyes.
They have cameras in there and they're watching what's happening and they're seeing what happens and they have a feedback loop based on vision, so they can see very small deformations of the tissues and they infer what's going on.
This is remarkable because this, I think, is the most exciting path to getting to manipulation, which is, rather than trying to reproduce tactile, which is extremely difficult for all kinds of reasons.
But I think it's interesting and worth pursuing.
But there's another path, which is to understand the visual tactile interactions.
And that I think if we can do that, we might be able to get away with just using cameras.
Interesting.
So this week or last week, I'm sorry I saw an article that was talking to the difference between Elon's approach, particularly on the hands of between him, and what Figure AI is doing, where they put right here in the palm, they put a camera to your point, they put a camera right here in the hand and Elon is refusing to put a camera in the hand.
And the person who posted...
Most of this was saying.
This is akin to him not using LIDAR in the cars because in the end it's going to come down to a cost thing.
And he wants to force his team to figure it out without additional sensors and, for all intents and purposes, costs for manufacturing.
And he's playing this longer game.
Yeah.
What are your thoughts on that?
Well, that's a brilliant point, Preston.
I'm really glad you made it.
It's very good.
The analogy really works there.
Elon is very, in some sense, he's very confident.
He's done amazing things, understandably.
He should be confident.
But sometimes that can blind you.
So, in this case, his decision not to use LIDAR has really, I think, put a limitation on the Tesla driving systems.
It can be very helpful for filling in the edge cases with certain conditions, when vision cameras can be distorted or blinded by light flares, or especially in rain.
So LiDAR actually is a great addition there.
And also the cost, I don't know.
I think it will come down over time.
I'm not in the car business, so I defer to his expertise there.
But in the same way he has.
You know originally, you might remember, when he first started Tesla, he wanted all the cars to be made with robotic factories.
And he had a decree, we will have no humans.
Everything must be done by robots.
And I remember engineers coming in from Tesla to my lab and saying, can you help us?
We're trying to do this thing and we can't get it to work with a robot.
And he was just unrelenting.
And then finally he said, I was wrong.
He was wrong.
He said, I was mistaken.
Humans are underrated.
Do you remember that?
Yes.
Yeah, so that was really interesting because it was one of the rare times he admitted, but it was also.
It was a great example of the idea that you can't do everything right with robots and even if if you will it, you know he can will things into existence right by demanding this.
It doesn't always work that way, and so the lidar story is very analogous.
And i think you're right about the cameras.
You know, having cameras in the hand makes a lot of sense.
It's not how humans, for animals work, right.
But cameras are something we understand very well.
We have very high quality cameras.
They're very fast.
They're very accurate.
And they're really low in cost comparatively.
So I'm for more cameras.
Put a lot of cameras in there.
Because the other issue is when you walk, it's one thing you have a camera on the head.
You can sort of see what's around you, right?
Or drive.
By the way, driving.
I should have mentioned this earlier.
But driving is much easier than manipulation, because driving you're just trying to avoid.
Avoid hitting anything in manipulation.
You must make contact with objects.
You must manipulate them.
Right.
So it's very different to your point.
This is really fascinating because, to your point, when you talk about this idea of you know if I'm holding a shoelace and the tip of my finger is indented, or I can see the compression of that, I can feel it.
I'm relying on that touch.
Like I'm tying my shoe.
I'm relying on that sense of touch.
But if you were going to try to build a sensor that can do that and you're kind of hitting a roadblock or can't find something that can provide that tactile feedback, I could look at a camera and say okay, it went in by a half a millimeter.
Therefore, it's about this much pressure.
And you can substitute that sensing capability through an image or a video of being able to see it.
So it is kind of interesting that Right.
We have figure going that path.
And yeah.
Well, OK, so let me let me add on to this.
So you just made a very nice nuance point.
You said, if you weren't just looking at your fingertip and you saw the shoelace pressing into it by looking at the shadow structures and others.
But if you, you could probably figure out that it was.
Yeah, slipping away or it was firmly grasped.
Absolutely.
That's what surgeons do.
And they, by the way, work with surgical thread, which is really thin and they have to use a needle.
It's very complicated.
Right.
But they're doing a lot of this with their eyes, with their intuition.
Now it's not just a matter of putting cameras around, because it doesn't.
That doesn't solve the video and you need to interpret that.
And that's also extremely difficult, because humans have this incredible ability and we can't underestimate it.
It's just amazing what humans can do.
So from an inference standpoint, as far as like, if I hold the shoelace this way, I can also kind of just intuitively infer the That if it was held 90 degrees from that, that it's going to have this same slipping sensation.
And that's something that's really hard to train a robot on versus humans can just kind of like figure it out like very easily.
Is that what you mean by that?
That's what I mean.
Yeah.
And here's the thing.
We have we don't have good language for describing this.
Right.
We're trying to because it's all intuitive for us.
Yeah.
No.
If you ask me, tell me how to tie a shoelace.
Right.
I'd be like this.
It's not easy, right?
We don't have language, and that's part of the reason.
By the way, this is the other issue I'll come to is the data gap, the gap between the amount of data we have for language versus robots.
Maybe this is a good time to- Yeah, let's talk that.
Yeah, let's talk this.
Okay.
All right.
Well, there's a way of quantifying all this, and this is something that I call the robot data gap.
It's the following that if you put together all of the data that was used to train language models now, it's vast, but it's hard to wrap your head around.
How much data is that?
Well, my students and I were able to calculate that if you actually look at it.
And actually there's another, Kevin Black, who's a researcher at Physical Intelligence.
Very, very smart guy.
He had the first insight about this.
And then we've been...
Basically it's that if you added up all the hours it would take you to read a human, average human, to read all the text that's used, that's available to train the language models.
Right.
So it's all the books that are out there.
It's all Wikipedia.
It's everything that's on the Internet.
If you add up all those tokens, if you will, and then to figure out well, a human can read at the average speed of 238 words per minute.
Right.
You can do the math and you end up with 100,000 years.
Right.
Okay.
So you could sit down and read everything that's used and be a hundred thousand years later.
You'd be done.
Okay.
Now, We don't have such data for robot manipulation.
It doesn't exist.
It's not like we can just find it on the internet.
The data is very different.
There, we want to start with vision images and then end with control signals to the robot.
This doesn't exist.
So we have to start and basically generate this data.
But what we're up against, right, is it's 100,000 years.
We're 100,000 years behind the language model.
So again, I'm sort of exaggerating and to make a point which is certainly there's a number of ways to accelerate that.
And I think we can eventually get there.
By the way, I'm not saying this will never happen.
Please don't get me wrong.
I believe it will happen.
But my big question is when.
I think it's really important to be prepared for the reality.
There's a lot of people say, hey, this makes sense.
I want this.
We should have this soon.
And, you know, but remember, there's a lot of cases where people have talked about that in the past.
Fusion energy, nuclear fusion.
Right.
Makes a lot of sense.
Sort of the technology is pretty obvious, but you have to contain this plasma.
It seems like a technical issue.
We can figure that out.
Well, 50 years later, we're still working on it.
And it's hard.
It's a very one of these very, very nuanced problems.
Another one is curing cancer.
When I was a kid, they used to say we're going to have a war on cancer, just like we got to the moon.
Ten years will solve cancer.
We haven't solved it.
So there are problems that are extremely difficult and they take much longer than anyone expects.
Yeah.
It seems like robotics is like that.
We don't know.
And listen.
I'd be the first to celebrate if we, if someone wakes up, I wake up and I read someone has solved it.
Right.
It could happen.
Yeah.
And then you'll look back on this podcast and say Goldberg was completely wrong.
No, it couldn't.
Totally happen.
But I want to be a voice to say, hey, it might not happen.
And let's just think about that and be a little bit realistic, because I know how a lot of people are thinking that it's inevitable.
It's going to happen in any you know.
Hopefully by next year, according to Elon and many of his followers.
But they have to be ready for that maybe not to happen.
And I'm worried about a backlash that people will say hey, this whole robotics thing is you know was, you know hocus pocus.
And, you know, we're going to move out of this field in droves.
And I don't want to put words in your mouth.
So correct me if I'm stating this wrong.
I don't think you're saying it's not going to happen.
You're just really suspect on the timeline that everybody seems to be.
Yeah, that's it.
That's it.
Exactly.
And, you know, that's where I line up with Rod Brooks, because for very similar reasons, we have experience.
We've both been working in this field for like 40.
He's been working longer, slightly longer than me, 60 years.
50 years.
But you know, we have a lot of experience with trying to solve these problems and they're much more nuanced than they seem on the surface, especially because a child can pick things up and manipulate it.
It seems obvious.
Why can't robots?
It's very counterintuitive.
But when you work with these things and you really see their limitations, you start to understand that this is a very, very complex problem.
Let's take a quick break and hear from today's sponsors.
Have you ever been interested in mining Bitcoin?
As a miner myself.
I've been using Simple Mining for the past few months, and the experience has been nothing short of seamless.
I mine with the pool of my choice and the Bitcoin is sent directly to my wallet.
Simple Mining, based in Cedar Falls Iowa, offers a premium white glove service designed for everyone, from individual enthusiasts to large-scale miners.
They've been in business for three and a half years and currently operate more than 10000 Bitcoin miners.
Based in Iowa, their electricity is over 65% renewable thanks to the abundance of wind energy.
Not only do they simplify mining with their top-notch hosting and on-site repair services, but they also help you benefit financially by running your operations as a business.
This approach offers significant tax advantages and an.
Do you ever worry about the complexities of maintaining your mining equipment?
They've got you covered for the first 12 months.
All repairs are included at no extra costs.
If you experience any downtime, they'll credit you for it.
And if your miners aren't profitable at the moment, simply pause them with no penalties, when you're ready to upgrade or adjust your setup.
Their exclusive marketplace provides a seamless way to resell your equipment.
Join me and many satisfied miners who have simplified their Bitcoin mining journey and That's simpleminingio.
Slash Preston to get started today.
With Simple Mining, they make it simple.
Imagine scaling your business with technology that understands your customers, literally.
That's the story behind Alexa and AWS AI.
Every day, Alexa processes over 1 billion interactions across 17 languages, all while reducing customer friction by 40.
It's not just about making life easier.
It's also about transforming customer engagement and generating new revenue streams.
Behind the scenes, AWS AI powers more than 70 specialized models working together to create natural conversations, proving how enterprises can deploy AI at scale with confidence and security.
Alexa's AI capabilities were battle-tested across Amazon's massive operations, delivering real, measurable impact at scale.
These same innovations now give other businesses a proven framework to boost efficiency, unlock new revenue streams and gain a lasting market edge.
Discover the Alexa story at AWS.com slash AI slash R dash story.
That's aws.com slash ai slash r dash story.
The more your Bitcoin holdings grow, the more complex your challenges become.
What started as a simple self-custody now involves family legacy planning, sophisticated security decisions and navigating situations where a single mistake could cost generations of wealth.
Standard services weren't built for these high stakes realities.
That's why long-term investors choose unchained signature, a premium private client service for serious Bitcoin holders who want expert guidance, resilient custody and an enduring partnership.
With Signature, you're paired with your own dedicated account manager, someone who understands your goals and helps you every step of the way.
You get white glove, onboarding same day, emergency support, personalized education, reduced trading fees and priority access to exclusive events and features.
All right, were you know you're a program manager.
You're kind of looking at all the different swim lanes to get there.
The hand seems to be like one of the you know, critical path, if you will, for getting there.
Is there anything else that you would define as being on that critical path or is the hand so far out there as far as difficulty goes compared to everything else, that that's really kind of the limiting factor.
Okay, great question.
I would say it's not only the, when you say hand, it's the manipulation of the hand, right?
Because, by the way, I do have another thing to say here, another opinion, which is that we will get much more out of very simple grippers than we will out of hands that look like human hands.
Again, if you look at surgery, the tools that surgeons use to perform an appendectomy are very simple grippers like this and they can do immensely complicated things.
So i believe you don't need complex hand.
So i'm not saying that's the path to go.
I believe you can do simple grippers.
In fact, my company ambirobotics, uses an even simpler gripper, which is a suction cup, And you can do incredible things with them.
So it's not necessarily the hardware, but it's the software.
It's a control of this nuanced interaction that is very, very challenging.
I think many of the other aspects are addressable.
We have the ability to tell a robot go pick up the orange.
You know, the orange jumper off the table.
We can solve that now.
Computer vision systems are good enough to know that a jumper is a sweater and there's an orange one, and it will pull that up.
No problem.
But it's being able to actually pick it up and maybe put it on you and then button it up for you.
That's where it's difficult.
Yeah.
Yeah.
I mean, maybe what we see in the interim is robots that go to market, humanoid robots that go to market that have simplified the hands or have the range of activities or things that they can actually perform is very limited relative to a real human being in there and being able to do it.
I don't know if that's how they go to market or not.
I want to talk a little bit more about your company, Ambi.
So this is really fascinating.
So you guys have gone the market, primarily focusing on logistics and warehouse type activities for robotics.
Is that correct?
Correct.
So this started about seven years ago.
We had a breakthrough in robot grasping, and that was just simply the ability to pick things out of a bin.
So it's not manipulating doing surgery, but it's just picking things out of a bin.
That was a very old problem.
It's been known as the bin picking problem, and people have been looking at that for decades.
But we made it in advance, and this was especially the work of Jeff Mahler, who was the PhD student of mine, who was the lead researcher on this.
And we can go into more details on the technical aspects of this, but the system was called DexNet dexterity network and it was based on collecting data, lots of examples and it was analogous in many ways to ImageNet, which was a breakthrough for computer images.
So we did something similar.
We synthesized this data set, we added noise in a very specific way, but the system started working remarkably well.
And so it could pick up almost any object that you put into a bin.
It would just pull it out and you would throw in the whole pile of objects.
We were digging around in our garages and closets and throwing everything we could into it.
And it was consistently just being able to pull these things out.
And so that was a very exciting moment for us.
We got some publicity.
It was in the New York Times and other places.
And then we were approached by a number of companies and we decided to form our own company.
That's awesome.
I'm curious where you've seen just good old fashioned engineering matter more than additional data or larger models.
And then to the converse of that, when did you have data actually really surprise you?
Okay, good.
Well, that was a great example.
That was a case where data really did surprise us.
We were able to generate 6 million example grasps over because we had collected 10000 object models and then we could generate grasps on those models.
And then we had all these and we trained a network to be able to learn essentially where to grasp an object.
So that was a data-driven approach.
But I will tell you that when you take that and you have to move that into an experimental system or into a commercial system, Then you need a huge amount of what I call good, old-fashioned engineering.
And this is where you have to really sweat the details.
You have to make sure that the sensors are calibrated correctly, that your robot arms are calibrated and accurate.
You have to be able to do the computation to move the arms very quickly.
You have to control the surfaces of the grippers, the suction cups.
Myriad of details like that.
The lighting that we had a little scale underneath the system that would recognize when an object was removed from the bin.
It was like a digital scale.
It was just another piece of engineering that had to go in there.
So lots of all that.
That was just our demonstration system in the lab, experimental system.
But then when we moved into Ambirobotics and, by the way, I want to give credit also to the other students who are involved
Matt Maddow was another computer scientist working closely with Jeff Mahler.
And then I also had two other PhD students from mechanical engineering, and one of them, Steve McKinley and David Geely.
Brilliant.
All four of these guys, extremely, extremely brilliant engineering students.
And so they really knew how they were very good friends.
They remain good friends.
They all worked very closely and spent a huge amount of time camping and hanging out together, too.
But they were perfectly complementary because we had the computing skills and the mechanical skills.
And mechanical guys knew how to design machines that could work reliably over a great period of time.
And that's when we moved into building the Ambi sort system, which sorts packages for e-commerce.
And this was a little bit.
We didn't go in with this plan, but what we saw very quickly was that e-commerce was growing and we needed there's a huge demand for sorting packages right.
It's very challenging to get packages out to the customer fast.
So we started using that technique DexNet, we evolved it and made it, it commercialized it, and then we can make it work very fast.
And then all kinds of other elements had to come in.
We had another gantry system that would drop it into, pick it out of bins.
Pick an object out of bin had to be scanned for its zip code.
Figure out which bin to go into.
Then put it into the right bin.
Avoid jamming the whole time.
Make the system reliable, safe and easy to use right.
All this is what I call good old-fashioned engineering.
And so I become a big advocate for this because, after all, this is a body of research and ideas and insights that have been developed over 400, 500 years in engineering.
It's still what we teach at Berkeley and all the major universities.
We teach the engineering principles.
And my point is, let's not forget about those.
Those are still extremely valuable.
Or engineering and getting and for robots and getting them to actually work in practice.
Yeah.
And anyone working in robotics, I think, will acknowledge that, although the public perception is oh, it's just now.
You know we're using AI and that's solving everything.
It's not.
It's solving certain little pieces of it.
And, as I said, there's certain pieces that are very, very difficult with, that still remain very difficult.
So this goes back to what I was saying earlier Preston, about my fear, which is that because there's so much expectation around humanoids right now that if the companies can't deliver on that ability, then there might be a big backlash.
And that's going to hurt companies like Ambi who are not trying to do that.
Ambi is trying to solve a real practical problem and do it efficiently and cost effectively and actually you know, basically something that's very valuable for every, every one who shops at Amazon or any of the online companies, right?
We've sorted 100 million packages so far.
And I'm very proud of that, because these machines, as we're talking, are out there sorting packages and they're very reliable.
They're not featured in the videos about there's no humanoids doing this, by the way.
Although some have said that, you know, we'll have a humanoid doing that.
But humanoid with hands.
It's going to be a long time before that's Even close to the efficiency of the systems that we have with suction cups.
Let's take a quick break and hear from today's sponsors.
You know what sets the best businesses apart?
It's how they leverage innovation to turn complexity into growth.
That's exactly what Amazon Ads is doing, powered by AWS AI.
Every day, Amazon Ads processes billions of real-time decisions, optimizing ad performance across a 31 billion advertising ecosystem.
The result is campaigns that run 30% faster and deliver measurable business impact at 6%.
Argentic AI transforms marketing from a resource-heavy process into an intelligent, autonomous system that maximizes ROI and empowers marketers to focus on creativity and strategy.
Amazon Ads is proving that AI-driven advertising isn't just the future.
It's the new competitive advantage.
And, better yet, every enterprise can apply the same innovation playbook that Amazon perfected in-house.
See the Amazon ad story at aws.com slash ai slash r dash story.
That's aws.com slash ai slash r dash story.
Startups move fast.
And with AI, they're shipping even faster and attracting enterprise buyers sooner.
But big deals bring even bigger security and compliance requirements.
A SOC 2 isn't always enough.
The right kind of security can make a deal or break it.
But what founder or engineer can afford to take time away from building their company?
Vanta's AI and automation make it easy to get Plus.
Vanta scales with you, backed by support that's there when you need it every step of the way.
With AI, changing regulations and buyers' expectations.
Vanta knows what's needed and when, and they've built the fastest, easiest path to help you get there.
That's why serious startups get secure early with Vanta.
Our listeners get $1,000 off at vanta.com slash billionaires.
That's V-A-N-T-A dot com slash billionaires for $1,000 off.
It's the new year, which means that it's the best time to finally start the business you've been dreaming about.
Just a couple of years ago, I launched my own e-commerce business, and Shopify was exactly the tool I needed to get started.
While many people continually push off their dreams until the next year, I am here to tell you that now is the time to capitalize on the opportunities right in front of you.
Shopify gives you everything you need to sell online and in person.
Millions of entrepreneurs, including myself, have already made this leap from household names to first time business owners just getting started.
Choose from hundreds of beautiful templates that you can customize and use their built in AI tools to write product descriptions or edit product photos.
And as you grow, Shopify grows with you every step of the way.
In 2026, stop waiting and start selling with Shopify.
Sign up for your $1 per month trial and start selling today at Shopify.com slash WSB.
Go to Shopify.com slash WSB.
That's Shopify.com slash WSB.
Hear your first... This new year with Shopify by your side.
All right, back to the show.
After shipping robots that work every day in the warehouse.
What's one belief that you held earlier in your academic career that you've had to revise based on that?
Lots.
I would tell you.
One of the things that you know is very interesting is that you think OK, I have this great new technology.
That's the breakthrough that really solves an important problem.
Therefore, I can rush out into commercial work.
It turns out that technology is only a very small core part.
It enables.
But then there's all these things that have to come around it, that are equally, if not more important.
And actually when you go to the customers and you say hey, we have this new AI thing, they're like wait a second, I don't care about that.
How much money is it going to save me?
That's all they care about.
And that's business.
That's business.
Both my grandfathers were entrepreneurs and so was my father.
So I grew up in these kind of environments and it's tough stuff out there.
One grandfather was very successful in electronics and my other grandfather was in the housing business, building homes.
But my father struggled.
He was a metallurgist and he had a company doing chrome plating and it was very, very difficult and you know he was buffeted by things way behind his control, like you know, recession of the 70s.
Actually really He hurt his business very badly so he struggled.
So there's a lot of factors, and it has to do with competition and timing.
What I would also say in industry is that and this is gonna come back You can do things in a lab that you think are you've really explored the full range of a problem.
So let me give you this example.
We were addressing the bin picking problem, remember?
And we were dropping all kinds of objects in there.
In fact, when people would come to the lab, they would visit and I'd say well, where do you have your car keys?
Drop them in here.
I said, if the robot will pick it out, we'll keep the car.
And then, but it would always do that.
It was no problem picking out someone's car keys right, and we tried it on all kinds of things again toys we made, 3d printed, weirdly shaped objects, all kinds of things we could think of.
We try to basically consider everything and we were just trying to push the envelope right.
Well, the envelope was the key word, because it turned out that one thing we didn't ever really experiment with was bags.
Oh, and bags are extremely common in shipping.
You probably, if you know this, if you receive bags from e-commerce, from Amazon or others, you get bags of all kinds of forms.
Now, bags are often plastic or paper, but the problem, the issue with bags is they're loose.
And so they have objects in them, but there's a lot of slack and they tend those really in the lab.
That wasn't something that we would have thought about too much, but that's so much more common in real shipping.
So my point is we had to adapt all of our systems to the reality of the consumer market, which in this case is bags.
And that was something we didn't have a lot of data on.
So we had to adapt our systems to work on data on real bags.
And real bags are very difficult to actually even simulate and model because they fold again.
And, by the way, the folding matters, because if you go to pick up with a suction cup right on top of a fold, as you lift it the fold will unfold and you'll lose the suction and drop the object right.
So we started collecting data as we started putting these robots to work.
So, as our customers were putting these systems into production at Ambi, we also had an agreement that we would maintain these systems at high performance levels, because we were constantly monitoring them.
So we have a dashboard at the central headquarters in Berkeley where the team keeps an eye on every machine that's in operation out there.
And so what we do is we get data on every single PIC operation what happens, how long it takes, whether it dropped the object, whether it was classified correctly, all kinds of things like that, right?
And we use that so that we can immediately tell when the performance let's say the picks per hour performance, that's how it's often measured drops.
We can spot that early and say, and we call the company and we say, what's going on?
Did something change?
Did the camera get knocked?
Is the suction cup getting worn?
And so we're constantly on top of it.
Part of it is that that's a source of big pride for us.
We're really customer focused and we want to make sure our machines work completely reliably.
Mm-hmm.
But the nice, amazing side effect of this is that we've been able to collect data from all this real systems in real environments over the last four years.
And we now have elapsed on 22 years of robot data.
So remember I talked about the 100,000 years?
Yeah, yeah.
Now we have 22 years, though, to start, okay?
But it's real robot data.
It's extremely valuable.
It's a high quality.
It's a gold standard for data.
And so we're now using that to refine our systems and then make them better and more higher performing, more reliable, but also allowing us to now branch out into new related types of products.
So we now have, we introduced a new product called AmbiStack that stacks boxes very efficiently, very densely.
And that's a new product that we sold out our first batch of these systems this year.
Amazing.
On this idea of robot data or the covering this hundred thousand year gap that you're talking about For a company that would be trying to overcome this, because the data just isn't there?
Are they just having to construct a bunch of physical real world?
Going back to the hand example, would they have to have a bunch of physical hands with just a bunch of physical objects to then just be doing this?
Or is this something that you think we could simulate in a virtual environment to accelerate that speed, or kind of a combo of both?
Good.
So...
For grasping.
It turns out, simulation works pretty well, because there you just need to know the geometry of the environment fairly accurately, and then the geometry of the object and the gripper.
And then you can actually model that fairly well.
So now grasping is just lifting an object off a table, okay, or out of a bin.
Mm-hmm.
That's very different than tying the shoe that we talked about earlier.
There, it turns out that we can't simulate that so well.
As I mentioned, we don't know how to model and simulate the deformations, the minute forces that are going on in the process of interacting with that object.
So that's a challenge.
This is a little nuanced.
And I know that your audience might say, what is Goldberg talking about?
He said this couldn't be solved.
Now he says it can be solved.
Well, it depends.
There's certain categories of problems that can be addressed.
And I think that picking objects out of a bin is something we've made enormous amount of progress in the last five years, so i'm very optimistic about that.
I think we're going to get.
We're getting faster, more reliable, and those systems are.
You know, that's the cutting edge of robotics and it's real yeah, but then Tying shoes and doing things around a home, by the way, or in a factory where you're actually trying to put together electronic parts or car bodies, or installing upholstery and wiring inside a car.
These are extremely difficult, by the way.
And even in Detroit or anywhere in the world, there's still humans doing those jobs because they're very, very hard.
So those are hard to simulate.
And I do think everything's pointing toward this deformation is a key obstacle to doing it.
And I've talked with people who are physicists and experts in deformation and they agree this is a very, very hard problem.
We don't even understand the physics of friction and deformation very well.
Interesting.
You've said your views on AI creativity have changed.
Walk us through some of the timeline and what's changed and just kind of your overall opinion today.
Okay.
Well, on a very different note, I have been working as an artist in parallel with my work as a researcher and engineer.
You know, I like to say my day job is teaching at Berkeley and running a lab there.
But I have another passion, which is making art.
And I've worked on this for almost the same amount of time.
And I make installations and projects.
We did a project called the TeleGarden, where we had a robot that was controlled by people over the internet.
And the robot could tend a garden, a living garden.
We put this online in 1995, which was the very early days of the internet, and i'm very proud of that project because it stayed online for nine years.
Huh, 24 hours a day.
People can come in and explore this garden and plant seeds and water them, so it was a very interesting.
It was an artistic project, but it was also a engineering proof of concept and it had to work reliably, And so you know it really pushed us.
I sometimes say people think doing engineering is hard.
Try art.
Because you have to deal with the public and they're going to interact and do all kinds of crazy things.
So we had to really spend a lot of time designing that system.
But I continue an interest in art, and I have a new show coming up.
It's a joint project with my wife, Tiffany Schlain, who's an artist.
And she and I are collaborating on an exhibition that's going to open in San Francisco in January 22nd.
Okay.
All right.
So this is a big passion of mine and it's using technology like AI and robots to ask questions about technology.
And I'm very interested in this contrast between the digital and the natural world when they seem very symmetric and similar, but there's very profound differences between them.
So that's what I, I think about, or I try to express in my artwork.
And so your, your question about the creativity.
So I always said, you know, AI won't be creative in the sense that it will, you can ask it questions, but it's not going to actually come up with original ideas.
But I actually have shifted my view on that.
And I give this example where I asked chat GPT in the early days Hey um, give me a hundred uses for a guitar pick.
And uh, I just thought it would get you know it would start repeating the same thing over and over again.
And it did.
You know, it started with a screwdriver to scrape ice off a windshield, things like that, which are all made sense.
But then it started listing these like as fast as I could read them or faster.
And then it came up with one that I was like, ah, it was a miniature sail for a toy boat.
And when I saw that, I was like, oh, my God, that is a genius idea.
And I would not have thought of it.
And you know immediately when you see something that's original and creative like that, you spot it and you say ah, why didn't I think of that?
Those are those rare ideas.
And AI is capable of that now.
And so it's very exciting.
Yeah, it is exciting.
It's super exciting.
So I'm not I'm not negative about AI at all.
I love it.
I use it.
I advocate for it.
My daughters, my wife, everyone uses it.
And so I'm 100 percent for it.
I do think it's going to help with robotics.
But the question is, is it going to do everything that people are hoping?
And that's that's where I hope that this conversation Preston, will put things into context for your audience.
So I don't know if you're going to like this question or not, but I'm going to throw it over because I'm curious what you would think of this.
So Figure AI recently sold their humanoid robot to put into the home.
And there was a lot of pushback from at least the comments that I saw online that this was just a giant marketing scheme, or maybe they're trying to raise their next round?
I don't know.
But for the audience, I'll just kind of frame it.
It's a humanoid robot.
All the demos that I've seen to date are extremely suspect as to its ability to actually do anything in the home.
When you dig into it more, they were using.
You're putting this thing in your house and then I guess that it has this ability to go back to a human that would actually be manipulating the robot inside the house, which I think has all sorts of security, privacy issues and everything else right.
But The reason I bring this up is because I don't know if he's... I'm pretty sure he's the founder and CEO of the company.
He was suggesting that, in order for AI to really start to accelerate its learning, that it needs to start being embodied into the physical form and to put itself into a challenging learning environment.
And what he means by some of this, and Ken correct me if I'm wrong in the way that I'm describing it, but what he's getting at is there's all these ambiguous situations that happen in the household with respect to social dynamics, the way that the family would interact and what they would ask of the robot, like hey, go get me a cup of water.
And then the person who's asking for it always likes it half full, or they like it warm, or they like it cold or whatever.
And so that learning that the robot would be forced to kind of undergo from a social dynamic would assist in its ability to get smarter collectively, because I'm sure all this information is then going back up into the mothership and getting networked.
But his argument is the point of my question, which is do you also agree that for AI to kind of take this next step, or this next quantum leap from where it is today, that it really needs to be able to immerse itself and basically partition itself into the physical form?
I think that is helpful.
And certainly understanding the dynamics of human interaction, especially in a home, is the social dynamics are very, very subtle.
And as you said, very, very important.
Just understanding tone of voice, gestures, like my daughter will say, you know, does this, right?
Which is, don't bother me.
She's a teenager, okay.
Yeah.
Or just rolls her eyes right like that oh yeah, the world's right.
Oh yeah.
Yeah, the subtle body language is super complex.
When you think about complex yeah, we pick up on it in myriad of ways we don't even recognize right, like i can pick up.
If one thing i always notice is when i'm teaching, i can pick up if students are starting to lose interest or get tired or bored.
Right yeah, i just feel it.
You know, i look around but i'm always watching.
That's why it's too tricky to teach online.
But all these things are very nuanced.
Body language can tell you a lot of what's going on.
And just interpreting what's the dog doing and what's the dog, you know, how's the dog feeling?
Right.
There's a lot of nuance there.
So all that is you're going to you have to be in real homes to be able to do that.
And I think that's actually that makes sense.
I'm not opposed to having, let's say, a humanoid in a home.
That might be helpful for doing certain things, like maybe fetching water or being able to pick up things around the house.
Remember grasping?
I said that is actually something I think robots can do.
So if you said, hey, you know, pick up all the things that are on the floor.
We would all like that.
We have a Roomba.
You know, you mentioned earlier the vacuum cleaners.
But the next step is to be able to actually pick things up and put them away.
And that, I think we can get there.
I do.
I actually think that's going to come, that can happen in the next decade.
And it's very valuable because, by the way, if you're a senior citizen, you really want things off the floor.
And if you're a young parent, you have a lot of kids.
You have kids.
Or if you have a teenager, there's like can you clean your room?
You know what I mean?
Great to have a robot go in and just clean off the, pick up all the clothes.
We call that the teenager's problem, by the way.
We have a paper on this, which is how to get a robot to efficiently pick up clothes.
And it's not the obvious thing, because you know if you just program a robot and go in and pick up, it'll.
Pick up one sock, take it to the bin.
Pick up the next sock, take it to the bin.
You actually need to be able to pick up lots of socks together.
And so how do you do that?
That's called multi-object grasping.
It's a very complex and nuanced topic.
And so we're studying that in the lab.
So, just coming back to your bigger point, I think that robots will do some things useful in the house.
I think that's possible.
They could be useful for security.
Also maybe in some form of companionship somewhere down the road or as you get older, and I can appreciate this more and more that I might want to have a robot that might help me shower or get changed or help me get out of bed in the morning.
I think that would be nice.
I'd rather have that than a stranger in my house.
Let me put it that way.
I think you can relate.
Just not one that's networked back to some other person on the controls.
Yeah, right.
I mean, well, the privacy issues are huge.
And that's something also a lot of engineers don't appreciate that.
I faced this at Berkeley a few years ago, where I was talking about privacy and I made an art project about privacy.
And we had cameras, surveillance cameras.
We did a whole installation about this.
And some of my friends were like, I don't care about privacy.
I have nothing to hide. recommendation you wrote for the last 10 years?
Oh, no, you know, I'm not going to share those.
I said, okay, how about all the research proposals that you're working on?
Oh, no, I can't share those, right?
It's all kinds of stuff that you don't want to share.
It's not that you're hiding, you know, it's not even doing something criminal or embarrassing, but you, just you, confidential.
So same as in your home.
You know, you just, it's not that I'm going to, you know, it's going to catch me naked.
But in the morning, you know, I have bad hair day and I don't necessarily want that to be transmitted widely.
So it's all kinds of things like that.
So I'm not opposed to humanoid robot.
I think it's going to be interesting to see what's happened in the next few years that we will probably start to see these.
It'll be very interesting to see that roll out from figure.
And Berndt is very, you know, he's a very compelling businessman like Elon.
You know, he has a lot of optimism, a lot of confidence and he's definitely building something that's working to some degree.
So it'll be interesting to see.
And, you know, I'm not a naysayer.
I'm not saying that All this is going to fail.
I just say that.
Be patient.
You know, it's going to take longer.
The real science fiction stuff is going to take longer than we think.
Last question I got for you, Ken what's the most exciting or surprising thing that you've seen in the lab or just in the space in general that you almost gasped when you saw it in the past?
Call it year.
OK, so actually, I have a good answer for that.
You know I'm so proud of Ambi for being able to sort packages around the clock at a very high speed.
Right.
But I recently saw a company called Dyna, dino robotics okay, and i'm friends with the founders so i'm maybe slightly biased, but i have to tell you they demonstrated folding napkins with a robot okay, and they did it for 24 hours.
So they just had the camera set up and they had and now this is, by the way, just two clippers, two Okay.
Right.
No head but it has cameras, but there are two grippers basically holding a stack of napkins over and over again.
And they did it for 24 hours.
And I showed you the whole process.
Now that to me as a roboticist is a big deal because they were able to do it fairly fast, reliably.
The napkins were, you know, often get tangled up and it would figure out how to untangle them and keep going.
And the folds were actually pretty nice.
So that's impressive.
And then I got to see a live demo of the new version of that, which can now fold shirts.
And it worked really well.
I saw this in a.
They had a booth in a conference in Korea in September and it was folding shirts as it just round the clock.
And it was fantastic.
Even you could bring your own T-shirt and it would fold it.
So that to me, that's very exciting that that shows.
And again, it's a specific task.
I do think we're going to make progress there.
And by the way, everyone wants to have something for their clothes.
That's for sure.
That is for sure.
I'm so bad.
I'm so bad.
I got one of those little folding.
Oh, you do?
Yes, I do.
Because I'm so bad at folding it.
But when I use that, I'll actually do it.
Oh, you have a folding board.
Okay.
Yes, I do.
And I'm sure that you're going to be a great customer for this.
But you know, you have high standards, right?
Because you want the things just right.
And that's where it gets tricky, right?
But the Dyna Robotics guys it's Jason and Lyndon who are the leaders of this company.
They really are pulling something off.
And I think it's very interesting to keep an eye on there.
And a lot of the other robotics companies are now trying to emulate that, which is to show one task, one special task, doing it very reliably making coffee or folding boxes.
That's really exciting, and I think that is actually going to be important.
Rather than trying to do general robotic, do everything in a home, which is, I think, going to take a long, long time.
But if you get it to do certain tasks like folding laundry or maybe making coffee, certain things like that, that's a way to sort of bottom up from certain tasks Learn other tasks rather than top down.
I think that's going to be a path to getting progress.
But again, it's going to take longer than most people think.
Ken, I can't thank you enough for making time.
Your expertise is just off the charts.
And I know the audience is going to love this.
If you have anything else you want to highlight or point people towards, that we can put in the show notes.
You know?
Just let us know what that is.
Okay, I'll send you some links, because I have a bunch of things online I can link to that follow up on this in various ways.
And no, I think it's great.
Thanks for doing this.
I'm really glad you're also going to connect with my good friend, Rich Wallace.
Yes, yes.
Because he is fascinating.
You know, he's a very, very original thinker and very I would say very much an unsung hero around chatbots.
People don't know But he was really a pioneer in doing this very early.
And he still has a lot of great, really interesting insights and ideas.
So you'll appreciate.
Amazing.
All right.
Well, Ken, thank you so much for making time and coming on the show.
My pleasure, Preston.
Thanks for listening to TIP.
Follow Infinite Tech on your favorite podcast app and visit theinvestorspodcastcom for show notes and educational resources.
This podcast is for informational and educational Entertainment purposes only and does not provide financial, investment tax or legal advice.
The content is impersonal and does not consider your objectives, financial situation or needs.
Investing involves risk, including possible loss of principle, and past performance is not a guarantee of future results.
Listeners should do their own research and consult a qualified professional before making any financial decisions.
Nothing on this show is a recommendation or solicitation to buy or sell any security or other financial product.
Hosts, guests and the Investors Podcast Network may hold positions in securities discussed and may change those positions at any time without notice.
References to any third-party products, services or advertisers do not constitute endorsements, and the Investors Podcast Network is not responsible for any claims made by them.
Copyright by the Investors Podcast Network.
All rights reserved.
