Hello, and welcome to the NVIDIA AI Podcast.
I'm your host, Noah Kravitz. One of the forces that shaped AI and machine learning over the past five years in particular is the democratization of the technology.
As the tools for training and deploying models and the data sets that power them have become more widely accessible, the growth of the field as a whole has accelerated.
Along with access to the tools and data has come an abundance of blogs, tutorials, YouTube videos, and even podcasts. folks can use to learn how to train ML models and keep up with the industry.
Today's guest has taken a novel, somewhat old-school approach to helping educate the machine learning community.
Santiago Valderrama, a machine learning engineer, has created a following by posing one machine learning question every day on his website, binomial.com.
As Santiago wrote on a LinkedIn post, I got together with a couple of friends and built Binomial, a site with a simple goal, a non-BS simple way to learn something new as fast as possible.
We publish one machine learning question every day.
That's it. You load the page, answer the question, and return the next day.
Rinse and repeat. It's a cool, fun and educational site that's taken off since launch, and we're lucky enough to have Santiago here to tell us all about it.
Santiago Valderrama, welcome, and thanks for taking the time to join the NVIDIA AI podcast.
Hey, thanks for having me on. So why don't we start out?
Why don't you tell us about the site? How'd you get the idea for Binomial?
Man, this is, believe it or not, I think this idea is 22 years old right now.
So back when I was... In my college days, I wanted to do my dissertation in multi-choice questions, in a system to ask multi-choice questions.
For some reason, I've always been fascinated about these type of questions because, you know, depending on how you design them they can pack a punch.
You can teach a lot and know a lot about the person who's answering those questions.
So obviously back then, you know, 22 years ago, the idea of machine learning questions, that wasn't what I have in mind.
But I think for the longest time I've been thinking about this, I'm 2021, you know, like you mentioned in the introduction, Fran and I got together and we decided to give it a try.
We are both machine learning engineers. And I started writing questions first on Twitter.
And people responded very well to this format, right?
So people engaged, people started submitting answers.
So we said, you know, why not? Let's, let's Let's put it together.
Did that over a couple of nights. And here we are.
Do you have a teaching background at all?
I don't want to say yes, because I would not be.
So I was a teacher a long, long, long time ago.
But for a short amount of time, So I wouldn't consider myself a teacher, but I do have a passion for it, yes.
I have somewhat of a similar story, so I appreciate the way you answered the question.
Call myself that either. So when you started out posting the questions to Twitter, Was it the same kind of multiple choice format?
Because I think it's interesting that you started by saying there's so much you can learn.
You have this fascination with that multiple choice format.
So unfortunately, Twitter has this restriction on their polls that you can only answer one question. that's not what I wanted.
You know, it's forcing me to only make questions that have exclusive choices, like you cannot answer more than one choice.
So that's one of the reasons why I wanted to move away from Twitter and do it on a specific website.
Also to have the ability to explain what the answer is.
So if you go online, you're going to find a thousand websites with machine learning multi-choice questions.
That's not, there is not a lack of that.
What you don't have a lot is a detailed explanation of why something is the answer and why something else is not the correct answer.
And I think that's what makes Binomi a little bit different and why so many people are showing up every day to answer these questions.
You want to throw us some examples of a question and answer?
Sure. Like yesterday, we were talking about PyTorch Liming.
It's a new library. And one thing that I really like about these questions and a good intersection, a good overlap with what I do as an engineer is is that I'm trying to make these questions in a way they're practical.
Like if you read the question, if you answer it and you read the explanation, hopefully you're going to get something that will be immediately useful for you.
So yesterday's question, for example, was... a team that's migrating from TensorFlow to PyTorch Lightning.
And basically you had to select which are the challenges that that team is going to have to face or is going to face by doing that transition.
You know, there is always a trade-off. So, you know, you needed to know what are the possibilities with PyTorch Lightning, why you're leaving behind, et cetera, et cetera, in order to answer the question correctly.
Is there a skill level, sort of career level of experience, skill level that you're targeting?
Are there certain... topic areas you're targeting, or is there even kind of an overarching, you know, that you're really hoping that people who come to be no meal every day for a year. throw a time like that there.
We'll walk away with, you know, a specific type of knowledge or skill.
It sounds like it's practical knowledge India, which is great.
Yeah. I wanted to make, you know, when you're writing so many questions, obviously there's going to be a little bit of everything.
So number one, I want to make them practical.
Number two, I'm not trying to target a specific audience.
So it doesn't matter where you are in your career.
Hopefully, you're going to get something. out of one of these questions.
What really matters to us is the habit. The five minutes that you take every single day to do something, right?
You know, personally, I'm a huge fan of the idea of 1%. better every day, right?
How that compounds over a year. And I think this is one piece of that, right?
That for me, it is And I want for other people to be as well.
If they spend five minutes every single day after a year I can promise you, you're going to be so much better in so many things about machine learning for sure.
How long has the site been up? It's been out for eight weeks.
So, yeah, it's pretty new. And how's the feedback?
I mean, we know it's catching on, but yeah, what have you... Yeah, so just to give you some metrics, when we started, about 1500 people are answering the question every day, up from 800 about when we launched.
The first time we started sending emails out for, you know, people can go unsubscribe so they get the question by email.
I remember we sent 700 emails. Last night we sent 2,300 emails.
So that's about three times in about four weeks.
So that's growing quite nicely. But this is the one that really gets me, right?
The real reason I have I'm really excited about this is because the people that are reaching out over email, telling me stories that are crazy to me.
Like there is this guy who said, Dude, listen, I have a bet with my friends, $20 bet to see who misses a question first, right?
Or a high school professor telling me that he's going to show the website to his high school students so they get something new every single day in class.
Those are the stories, right? Plus the people telling me, hey, this is helping me or the conversations that are happening on Twitter, not involving me.
But I'm sort of like an expectator. People telling other people, hey, I'm just answering the question every single day.
It takes five minutes. You should try. Those are the ones that make me keep doing this.
Yeah, yeah. It's like the Wordle for the machine learning community.
Yeah, that's an interesting comparison. When we decided, by the way, what the idea was not originally... hey, let's do something daily.
That was what we had in mind. It was a much more complex idea.
I think a cool idea, but much more complex.
When my friend and I realized that there's got to be a lot of work to get this out, we didn't have the time.
We both work full-time jobs. So we decided, hey, let's put something together.
That is just a question of date. I don't remember if I had Waterloo in mind back then.
But Wordlet was very popular. So it's probably my subconscious had Wordlet in mind and let's do this daily thing.
And it turned out that it was a beautiful thing.
It was a perfect thing. Like we are afraid right now if we give you access to more questions, we are afraid that you're going to get tired of answering questions. questions and you're never going to show up again.
Right, right. Right now it's the opposite problem where people send me messages saying, hey, I keep repressing the page, even knowing that the question is not going to show up.
I'm just waiting. There's a cheat code. If I type in the right caption, I'll unlock all the questions.
That's a great idea. Are there particular kinds of questions that you find either attract more interest or just that you're more interested in, or, you know, you'll pose certain types of questions and get excited for what the community response might be.
Yeah. So, so far, we have not found any patterns on questions that attract people or not.
So at least I haven't been able to pinpoint anything like that.
However, I do you know, I'm usually more excited about questions that I was successfully painted a practical picture, something that is not necessarily theoretical. but something that is gonna, that I know right away that you can use that, right?
Like the other day, for example, we talked about, it was a question about rotated images and How can you actually process rotated images in a deep learning model That question came from a real example, from pain.
We had to do this in real life. Let's just ask a question about that.
Those are the questions that, you know, I can get a kick off of them for sure.
Yeah. Are you pulling the questions from your own Were you and your partner, I guess, from your own daily life or from other sources?
Where do you get the idea? Some of the questions, yes.
From what we do every day, you know, every time I have an opportunity to see something, I...
Now my brain is thinking, how can I ask that, right?
But I also read a lot. So just like, we were talking about small habits.
One of mine is writing a question every single day, right? it forces me tremendously to to study the material to to you know to be up to speed so It's just, I read a lot and I tried, I'm looking at it from that lens, right?
So from this thing that I just learned, how can I get a question or two questions out of this?
That's that's the process. Yeah. Do you get your readers, community members submitting questions?
So, so far we've gotten ideas and we are fine with that.
Like there is this line with the, you know, the, creative license of a question.
We're selling some of these questions, so I don't want to get into the area where somebody submits a question.
Who owns that question, right? we have solicited ideas.
Some people have posted ideas like, you know, hey, why don't you ask about this situation here.
But we still design the question and come up with the explanation and the references, et cetera, et cetera.
Yes. Right. So the site is free to use.
And I looked at it, and I may have missed something.
Wouldn't be the first time I missed it, but...
It looks like, and you kind of alluded to, you can only see the current day's question.
Yes. And so there's not an archive, but there was something about a book.
Correct. So what we did was, so the site is free and it's going to remain free.
That's our goal is that. That's the main goal.
We are paying development and infrastructure and whatnot.
We're selling sponsorships to the site. and also by selling a book that contains all the questions.
So after April was over, We published these all 30 questions from April.
You know, we revisited the questions and expanded on the explanations.
And we created a little book We generated the cover using AI and it was this whole thing.
It was fun. And we are selling that book.
And that's the way we're planning to, for now, we have other ideas, but for now, that's the way we're planning to sort of like add. sustain this.
Pay for your costs. Yeah, got it. And is the books available through the website?
It is available through the website. So we have a small little a banner on the website that says if you're looking for previous questions, you can click here and people go there and buy the book if they want to.
Very cool. So are you going to do one per month?
So, yeah. So for now, yes. We're going to keep doing one per month.
So we're releasing them like There were comics like volume one and volume two and volume three.
We set up a subscription, which it's interesting because most people have opted for the subscription.
So it's going to be cheaper to have, if you wanted all of the questions, right, and you like this style of learning, You can subscribe and get the book sent to you whenever it's ready.
But we're also planning to create special volumes, like when we have like a hundred questions, we're going to publish one with the 50 hardest questions on the website or topic. books where, you know, 50 questions about neural networks or 50 questions about decision trees.
These are just hypothetical examples, but just to give you an idea, we're going to be doing that.
Yeah. That actually makes me wonder, do you have metrics to keep metrics on what percentage of people get the questions right?
That is correct. So we keep metrics so we know when a question is hard or medium or easy.
Yes. What's kind of the average percent who get the question right?
So right now on the website, the last time I checked, If 100% is perfect, the average right now, it's around 70-something.
Oh, that's great. 71, 72%. Yeah. Yeah, knowledgeable audience.
That's good. Yeah, yeah. I mean, it totally makes sense.
You know, if you're going to go and answer questions about machine learning, you probably have some sort of like foundation on it.
Yes. Yeah. Yeah. I'm speaking with Santiago Valderrama.
Santiago is a machine learning engineer who has created or co-created with his partner, A website called binomial.com.
One question about machine learning each day.
With the answers explained, it's fun. It's a great tool to learn and keep up and expand your knowledge on machine learning and AI.
And as Santiago said, you know, developing that everyday habit is one of the best ways to just over the long haul, really build your base of knowledge.
Apparently, it's also a great tool for bets with your friends as well.
Santiago, to shift gears a little bit and talk about your own work, How did you first become interested and get into the field of ML and AI?
Yeah, well, I remember, you know, early 2000, I had a couple of classes back in college about AI, but I'm from Cuba originally. back there, AI didn't have a lot of applicability outside of a classroom.
So I really didn't connect to it. I'm a very practical person.
I really like to do stuff with the things that I learn.
Yeah. So, you know, took the class, was fine, never looked at it again.
Back in 2015, I decided to go back to school, went to Georgia Tech for my master's.
And I took this machine learning class. There wasn't anything else available for that semester.
So, you know, this machine learning class, okay, it's fine.
I guess I'm going to go through it. Professor Charles Isbell, it completely changed the way I looked at engineering up until that point.
I completely fell in love with it. And that day I decided to completely change my master's around machine learning.
So I started taking more classes, reinforcement learning, and all of the classes that they had available, I started taking them.
So graduated, and it was even comical that now looking back, it almost seemed like every day I was taking a class, I found a way to apply what I was learning Back in my job, as a software engineer, I had been working with many, many different clients. different applications.
And it was this super nice thing that every day I had fresh ideas.
I felt like every day I came with fresh ideas of things that we could do.
That was the way I started. Yeah, that's 2015.
Yeah. That's the best. When that clicks in, it sounds like you really just had this moment where it all shifted for you.
That's great. Is that what you do now? That's what I do, yeah.
Basically, it's mostly related to computer vision.
So that's where my focus is, yes. What are...
I don't know, through your work, through the website, through the questions or the conversations that even, as you said, you've just observed since Binomial kind of took off.
Where are some of the places that you see the field headed that you're excited about?
That's a great question. So a lot of what I do is helping people do a better job, whatever that means.
That's what my day job is, right? So it's helping companies something a little bit faster or with a little bit less waste. or a little bit more cost effective.
So I'm a huge believer in the union of AI
And people, like usually when I talk to other folks, the idea of AI is replacing people.
That's not the way I look at it. The way I look at it is augmenting our abilities with machine learning.
I'm a huge believer that AI is going to unlock our potential.
And you know, in five years, I usually tell people that I think it was 1492 when we invented the telescope and it took 200 years for the next major invention.
And now it seems like every year we get something completely revolutionary.
Right. Like in 2017, it was the attention is all you need paper from Google that, you know, we came out with transformers and now transformers are, completely revolutionized the way we do NLP and even computer vision.
Man, I don't know what's going to happen in four or five years, but I can tell you I'm super excited about the ways that we're going to use this power to help humanity.
Is there anything that you found surprising, good, bad, or otherwise, but just kind of unexpected, surprising? from working on Binomial in particular?
Surprising. I think, well, number one, the amount of people that send us messages.
To be honest, I told my wife when we were about to release Binomial, is if the first month We make it to, I think our numbers were 500 answers per day.
This is a huge success. I would eat three times that.
So I think that amount of people that are looking for something new to me, is very, very surprising.
Even in a field that it's machine learning, which is, you know, it's difficult and you don't get, it's not, you know, it's very difficult for a lot of people.
So, I think that's the most surprising thing that I found when building Binomials.
So many people with so much hunger for knowledge.
Right. Are there any particular advice or just kind of tips or tricks that you would give to folks who are looking just hungry for more ways to hone their skills in the field.
You mentioned that you're a big fan of The Daily Habit, which is huge.
That is correct. Yeah, yeah. Yeah, that's what I would say.
So don't worry about like people usually look at a book and say, wow, that's a thick book. concentrate on just one page and that's it.
But it's like when I'm doing a workout, I look in front of me, I have an hour in front of me.
I just feel pain just looking at that hour.
I'm concentrating the next rep, writing the next set.
And that's what I do with a book. And if you do that repeatedly and you show up, you're going to look back at some point and say, wow, so much progress.
That's incredible, right? That's my main advice for my children and for everyone that asks is just do one small step and make sure you do it continuously.
And that's it. I don't know if you can see these metrics or just from people who, you know, reach out to you in conversation, but are you getting questions from, you're in the United States, you're in Florida?
I am. So are you getting questions mostly from the states or one other area or are they worldwide?
Do you have any view into that? I do. And if you give me a second, I can tell you right now where people are coming.
OK, so here I have it in front of me. So United States is number one, India and the UK are two and three.
And then you get Spain, Germany, France.
But these are the two that really gets me.
Nigeria and Brazil are up there in the top 10.
Yeah, specifically Nigeria. It's incredible, the software community that they have in there. met so many people from Nigeria and it's great to see them in the top 10 countries accessing the website.
But yeah, United States is by far you know, the one that sends the most visitors.
Yes. Do you get any sense of, is it mostly, you know, professionals and students reaching out?
Do you have any sense of the kinds of folks who, you know, not the kinds of people working in the field in particular, but the kinds of people with that that hunger that drives them to, you know, do the daily question and even to reach out to you or to have conversations online that you see talking about ML.
Is there a, I don't know, a certain type of user of the site that you're seeing.
So I don't know how to generalize. But I can tell you, like the author of a book that I'm looking at right now here when I'm sitting here, He's answering the question and that makes me so happy.
He actually reached out yesterday. because I made a mistake on one of the questions.
You know, I had a type on one of the questions and he reached out.
But people that I admire are showing up and that makes me so happy.
But at the same time, people that don't have any credentials that they're studying, they're also trying it out.
Again, we try to make the questions Not too hard, not too easy, but it's sort of like trying to find a sweet spot that can teach anyone something.
That's fantastic. We talked about publishing the books and you've got the subscription.
Any other plans? I mean, the site's still pretty new, so I'm not trying to to put expectations or pressure on you, but any plans you might want to talk about for the site going forward.
I think the community that we're building, which is a very knowledgeable community, is obviously machine learning and data scientists here.
I think we have some ideas. to help provide more value to that community and monetize that way, not by charging the community, by charging the people who wants to access that community.
Right. And, you know, obviously we're only two people doing this on weekends, so... we don't have a lot of time, so I'm not going to announce anything that we might not be able to deliver.
Sure, smart. But that's where our head is at, right?
Like, how can we provide more value to this community by giving companies access to them.
Right, right, right. Very cool. Well, the website, we've been saying Epinomial.
I haven't actually spelled it out. So it's B-N-O-M-I-A-L.com.
And I think it resolves to today.binomial.com so you can get today's question.
I also mentioned up top a LinkedIn post when you announced the site.
Are there other places online that listeners can go to connect with you, to follow your work beyond, you know, following the website itself.
Yeah, my Twitter account. That's where I usually hang out.
It's S-V, like in Victor. Pino, P-I-N-O.
S-V Pino, okay. SVP, no? Yeah. So that's, you know, if you have a question, if you really want to engage or say hi or whatever, just go there.
Yeah, excellent. A couple of things that I would like to say is like, you know, again, if you're looking for something that you can do in a very short amount of time, just give Binomial a try.
And again, it's just the power of compounding.
And that's going to do a lot for you. And number two is to thank everyone that has sent those emails, because that's actually the fuel that keeps us going.
Right. Like it's not the money. We're not making a lot of money with these or anything.
It's just hearing from the community and hearing the stories.
So I would like to thank everyone that has done that so far.
It's been incredibly helpful. Yeah. Awesome.
Well, Santiago Valderrama, congratulations on what you've done so far.
All the best of luck to you and your partner on keeping it going.
It's super cool to hear about and hearing about it from you it's clear you know from the get-go it's clear that uh what you just said you know hearing from the people whether it's somebody who wrote a book you're reading or somebody in their side of the globe who is interested and came across the site and is learning from it every day.
Like you said, that's what it's all about.
Super excited to hear about it. Wish you all the best of luck and mostly thank you for taking the time to come on the show and talk about it.
Yeah. Thank you for having me on. Thank you.
Thank you.
