So I love to say that APIs are the dendrites of the internet.
So if you think about the brain being a collection of neurons that, by firing together, you know think, you can think of the internet as a collection of servers or programs running on servers that, by interacting with each other, think or do.
And that connection, the synapse, that's almost tautologically, almost always an API of some kind.
Realize that every company is becoming an API company.
And every API company needs an API platform with SDKs and docs and API versioning and just a million different things.
That amazing team that I worked on at Stripe all collaborated on.
People want to be able to focus on their core capabilities.
Be thoughtful in exposing the interfaces to users and not have to worry about the low-level details.
If you've been building with APIs, you know they're the connective tissue of modern software, powering everything from payments to cloud services.
But as APIs multiply, so do the challenges developers experience.
SDK quality, documentation, versioning, and now a new frontier.
Interfaces not just for humans, but for AI agents.
In this episode, Stainless founder Alex Rattray joins A16Z partner Jennifer Lee to share how his time at Stripe shaped Stainless' mission to bring world-class developer platforms.
SDKs, docs, and more to every API company.
They discuss why SDK is often the API for developers, how MCP Model Context Protocol reframes APIs as interfaces for large language models, and the new design challenges of treating AI agents as first-class users.
It's a conversation about the future of APIs, agent experience and what it means to build developer platforms that are robust, idiomatic and built to scale.
Let's get into it.
So I started working on API tooling when I was at Stripe.
I had been working on other teams and I just saw how important the API platform was.
And so I moved over to that team and built out a redesign of the Stripe API docs, which was an incredible experience.
And through that, I saw how important the SDKs were, even internally at Stripe.
People, I think, didn't quite see that to the developers who use the Stripe API every day.
The SDK is the API.
And those were kind of falling behind.
So I built a code generation system there that can take the Stripe open API spec and turn it into a great Python library and a great TypeScript library and so on.
And that was really fun to build.
You know, the computer scientist in me just had a phenomenal time building sort of the compiler aspects of that and the library design nitty gritties.
And so after I left Stripe, people kind of kept asking hey, you know, how do I get good SDKs or how do I build a great developer experience for my API?
And it was very painful for me, actually, because you're saying, hey, that's a great question.
You really need SDKs.
It's a core baseline part of the developer experience at this point.
Everybody interacting with your API is going to expect a well-typed SDK in their language.
Unfortunately, the bad answer to that great question is that there's no way to do it.
You know the open API generator, that's open source.
At the time, certainly it was pretty far from the Stripe quality bar.
In terms of the output it would generate.
A lot of great work has gone into that, but it's a very hard problem.
The maintenance burden, the headaches involved in trying to set this up.
It's just too much for almost any company other than something like a stripe at 2000 people, which was when I worked on that.
And so it's very dissatisfying as an engineer to tell people a bad answer like you need this thing and also you can't have it.
And so eventually I started saying, hey, would you pay real money for this if I built it for you?
People both said, hey, yes, I would.
And also can you help us with everything else around our API, and realized that every company is becoming an API company and every API company needs an API platform with SDKs and docs and API versioning and just you know a million different things.
That amazing team that I worked on at Stripe all collaborated on.
And so you know, can we bring that Stripe developer platform in a box and bring it to every company?
And so that's sort of the way that I came to start Stainless API as a company.
That prediction you had or that statement you had for 2022 definitely turned out to be true today.
Every company is an API company, and we're going to dive into MCPs too.
Can you explain sort of how did you approach the sort of SDK generation aspect of the product?
And Stripe experience, of course, is very critical to building this product because TO today, is still one of the North Stars of what developers look to as great developer experiences and great SDKs.
Yeah, we had a lot of experience with this at Stripe and you know, saw a lot of the good and a lot of the bad and a lot of the ugly.
There's so many details to get right, whether it's making a client library that's really robust.
And when things get weird on the Internet, your SDK is not the thing that's blowing up or causing problems.
It can handle scale.
It can handle great error messages and everything.
It can work well with telemetry and all this stuff.
In addition to being robust, an SDK also needs to be really polished.
And that means if you're building something for a Python developer, that it's idiomatic to them, it's familiar.
If they hover over on a thing, there is documentation that pops up in their editor telling them about that thing, whether it's coming from the REST API or from the core of the library.
And crafting that is really quite a journey.
You know, it's something that I just personally love.
Years ago I created a small programming language and it was all about kind of polishing those nitty gritties and finding the best way to do every little detail.
And so that's one of the things that I most enjoy at Stainless.
There's an interesting thing here where, you know, we're going to be talking about MCP.
And crafting.
I think the Python library that we do is a great example of this, because it's Python was the first programming language that I learned.
No offense, Visual Basic.
It had evolved a lot since I had last used it, when I was sitting down to create our Python generator.
There's types in Python, and there's multiple type systems.
There's async.
There's pattern matching.
It's really incredible to see.
And what's idiomatic through all of this world?
And that's a moving target.
I think the industry spent years basically figuring out how to make an idiomatic Python library and figuring out what it looks like for a Python library to be intuitive and legible to a Pythonista.
And that's continued to evolve.
And I think that we've moved the industry forward a little bit.
And I'm proud of the work that we've done there.
When you come to MCP, which is this kind of new frontier where, instead of exposing your API to a Python developer writing Python software, one of the things that you can do with MCP is expose your API to an LLM agent using an MCP client.
In that world, what's intuitive to an LLM?
What is usable and ergonomic and idiomatic to a large language model, to an AI in interacting with a large API.
And the world is starting from scratch there almost.
So there's a lot of interesting work to do and develop and figure out.
So for us, it's kind of this really fun exploratory research problem.
Maybe zoom out a little bit.
You called APIs being the dendrite of the internet.
Now that MCPs are everywhere, like you know, occupying a lot of the brain bandwidth for developer tooling companies like Stainless, what do you call MCPs?
And how did that have impacted your business, how you think about the landscape?
Yeah, so I love to say that APIs are the dendrites of the internet.
And that connection, the synapse, that's almost tautologically almost always an API of some kind.
So it's a really critical part of the internet software ecosystem.
And seeing that kind of like CPU brain, you know, extend into the GPU brain is a really exciting moment, I think.
And you need the right interface.
So MCP kind of is that interface.
I don't know if I have a biological metaphor for it. off the top of my head for that.
I kind of want to compare it to eyes or ears or a new sense.
When a software system is communicating internally okay cool, but it's all in your head.
In some sense it's all in the cloud, it's not doing anything in the real world.
And how does you know all the software that the technology industry has been building for the last you know, 30 and beyond years?
How does that kind of become real in the real world and do something meaningful?
And one of the long-standing ways of doing that is sort of i'm going to call it through the eyes in the form of user interfaces.
So you'll have someone log into a web application.
You know, I used to work at Stripe, and so I use this example a lot.
Someone at a small business might log into their Stripe dashboard, refund a customer, create a new payment, add a customer to the system, anything like that.
In the days before APIs, everything was this way.
Then we saw that we needed to automate things.
Maybe instead of eyes, I don't know, now we're going with fingers, right?
And there's some part of your where there's a communication between your mind and the neurons and something tactical in the real world where all of a sudden these computers can do things in an automated fashion.
I don't know if you would call MCP and AI, like they're the new ears or something like that, but it's a new sense or it's a new way of interacting, getting information from the world and also putting something out.
You're sort of like growing this new limb.
I definitely imagine a hand raising up in the air of like here you can use me and like I can do this.
So definitely, you know, having more of the touch points and surface areas for AI to explore and also to leverage a lot of the existing capabilities from software.
So I like that analogy a lot.
And just maybe going down that train of thought.
You have been developing for decades tools for developers and as a developer for a long time.
But now we have this like new persona that's large language models and AI, and they are part of the design aspect of how to serve large language models as a user and building sort of tools, thinking about this new persona.
What are the changes and challenges in just like the thinking of designing tools when taking in this sort of new interface and new requirements, that has impacted how You know you're building and building the products.
Yeah.
So if you think about a Python library being the interface between an API and a Python program or Python developer, I would think about MCP as being the interface between an API and an LLM.
And so to us, it's all SDKs.
Right now, again, you can do other things with MCP and these other things are important.
Documentation is really a big example.
And I'd love to talk about that, too.
But in terms of sort of these, like core operational, you know, let me go refund a customer in Stripe dashboard or let me go find out when this last transaction occurred, or anything like that.
You know, as you say, there's a lot of difficulties and new challenges and problems in making this work.
And a Python developer might really want to make sure that async works without too much rigmarole.
Or a Java developer might want to see that the classes that they're interacting with are type safe, but at the same time may not want them to sort of blow up at runtime if something is unexpectedly null.
And that's a really hard problem.
Right.
You know, yesterday I was chatting with someone who was on the Java team back in the day and helped design some Java languages.
And I was telling him about this.
He was like, yeah, that's an unsolvable problem.
Right.
And you know that's something that may be solved as the wrong answer, but that we've tackled this at Stainless and I'm proud of.
There's really difficult challenges for LLMs.
And so some of those challenges that you're asking about, a lot of it comes down to the context size.
And so if right now, with MCP, that the naive way, the typical way of taking an API, usually described with an open API spec and presenting that to an NLM through MCP is, you basically take the JSON schema for every single endpoint, every single operation in your API.
You turn that over into a tool in your MCP server and say, if you want to create a charge in Stripe, you know, use this, create charge tool.
And inside of that tool, here's every request parameter.
So here's how you specify the amount of the charge.
Here's how you specify the description, the currency, so on and so forth.
And that's going to have the same descriptions and documentation that you would see in Stripe API docs.
It's all going to be there in the tool description.
You can get accuracy there.
But if you want to describe every single endpoint in the Stripe API with every single parameter, fully documented, congratulations.
You've just used up your entire context window.
And that basically just doesn't work.
Even if you don't use up the whole thing, LLMs tend to kind of get overwhelmed.
MCP clients tend to kind of get overwhelmed at seeing too many tools in use or available.
And all these request parameters, like you may not need them.
You know, maybe you're only trying to, maybe you only need three parameters from this endpoint, not all 50.
Or maybe you only need one or two endpoints from this API, not all 500 or 50 or what have you.
So that's a really difficult challenge to figure out how to gradually unveil.
What are the tools available, what are the parameters available, what are the descriptions of those parameters.
And you want to be able to do that without too much back and forth because every turn takes time.
And takes tokens too.
Yeah, exactly.
Yeah.
So do we have a solution for the context window challenge as well as the limited tool calls?
Is there a solution yet?
There's several things that we do to help with this today at Samus.
And there's several things that we're doing soon.
But the most simple thing that we do today is we can generate all of the tools in your API.
And when someone wants to spin up this server, they can use command line flags in the CLI or query parameters in the remote URL to basically say hey, I only want to interact with these resources.
So I only give me the you know payment intents part of the Stripe API, or only give me the customers part of the Stripe API.
Or I only want reads or I only want writes.
And that limits things down a lot.
And the end user can decide what they want to do.
The API provider can decide what they want to do.
And that nipping and tucking is really useful at sort of a first approximation.
That could be a little bit clunky.
And so maybe you don't know in advance what the limits you want to do are.
And so a more dynamic approach that we also provide is the ability to say hey, give me this in dynamic mode.
And instead of, say, 50 or 500 different tools, we only expose three.
You have a tool to describe a single endpoint and get all those request parameters.
And you have a tool to execute an endpoint.
And so you've gone and made these tool definitions totally dynamic.
Now there's some trade-offs there.
It takes three turns to do one operation here instead of just one.
And so at runtime, that's a little bit slower, but it saves context window.
And if you end up not interacting with you, know again say, the Stripe API, then you don't burn any of your.
You know hardly any of your context at all, which is terrific.
So that's another thing that we offer today.
A third thing that we do currently, I think, is really fun and interesting.
This is something that we shipped recently.
It actually solves a slightly different problem, which is okay.
Let's say you make your API request and you say hey Stripe, you know, tell me about the customer, Jennifer right.
And it'll make a retrieve request to the customer endpoint.
It'll find Jennifer, or actually a list request.
It'll come back with a Now, especially if it's a list request.
So maybe you make a request to Stripe API.
It tries to find the right Jennifer.
There's a lot of Jennifers in there.
So it's going to have to sort it through.
And the size of the API responses here can be very large.
In fact, especially for like a list request like this, they'll typically be so large that your MCP client, like you know cursor or cloud code or anything like this will just refuse to look at it at all.
You know, you just literally can't interact with it.
Even if it can, it's just too much data to sort through everything.
What we've done is we've added sort of this JQ filter to all of these requests.
And so JQ stands for JSON query, and it's a commonly used CLI for saying hey, out of this large JSON object, give me only these properties and filtering down the data that you're getting.
So sort of like the SELECT clause in SQL, for those who are familiar with that.
LLMs are great at JQ.
So just giving them that tool has enabled them to say, out of this large API request, I only need Jennifer's name and email and description and I don't need to know all these other details.
And that keeps, again, the impact on the context window light, and it keeps things really focused.
That's incredible.
That sounds really useful.
It reminds me another question of.
There's a misconception or misconceived notion that now you have MCPs and that's how LLMs are accessing and interacting with APIs.
The quality or interface of APIs or SDKs are not that important anymore.
But I think that's like the polar opposite.
It's exactly because there's more usage you need strongly typed, very polished SDKs as well.
Do you mind elaborating on that?
Yeah, absolutely.
You know.
So I was chatting about this morning actually, with someone who leads developer relations at a large financial technology company.
She was talking about this problem where today, if you ask a coding agent to integrate with this company's REST API, the first thing the coding agent is going to do is install this company's SDK.
That sounds great, but what she's seen is that it's going to install a version from 10 months ago, an old version, and then it's going to hallucinate the whole interface, right.
And I think a lot of people have had this experience where they say hey ChatGPT, can you help me integrate with ChatGPT?
And it gets the version of the SDK a little bit wrong, right?
And so something I'm really excited for us to ship very shortly is the ability to get comprehensive reference documentation for the API of your library into the hands of an LLM.
So that you know when people integrating with this financial technology company say you know hey, make me an integration, it'll know the right version and it'll have access to the full API of that version's SDK and it won't make mistakes.
So that's one of the things that I'm really excited about all this kind of coming full circle.
And I'll note.
You know again, there's still a reason that the it sounds bad that the coding agent kind of got the SDK usage wrong in the first place.
But it's important to note that the agent preferred to use the SDK.
It didn't want to just make a fetch request, given the option.
And the reason is that with the SDK, you have type checking.
So if you have an LLM, that's kind of like trying to one shot a whole big, crazy API request.
That's not so easy to do with 100 accuracy.
You really want those red squiggly lines saying, hey, this property has a typo.
Or hey, you're assuming that this string is always going to be string and it's never going to be null in the response.
You know, a human reviewer is just not going to catch a mistake like that. for an untyped interface.
They need to see their type checker kind of giving the quick thumbs up that the LLM is in the right direction, so that they can basically quickly review the business logic that was produced by the coding agent.
You know I think a lot of people are seeing this that the more declarative and clean and dry the code that you're writing and the more type safe the code that you're writing is, the more you can trust LLMs to do the coding part of your job for you.
And think about the higher level stuff and keep an appropriate eye on things.
Totally.
So it's not just about keeping the accuracy of the LLM outputs.
I mean, it certainly helps with that when you have a more constrained exploration space but also just debuggability.
Like it's so much better when you have strongly typed SDKs and very I would call it like you have the tools but also have the reference docs and menus in place with the docs aspect of it.
Like it's going to be a more holistic experience given to the AI agent to really perform the task.
Exactly.
Your coding agent is going to be able to try something out press save, see if there's a type error and iterate by itself automatically, rather than trying to make an API request.
It'll need an API key for that.
It might be a destructive API request.
Having an LLM test in production is...
You know, with a foreign API, with a foreign system, that is not what you want to be doing, right?
And so you know, for any company that ships an API that they want their users to be using in production, whether it's a big business user, you know an enterprise doing a key integration, or whether it's a developer trying to move quickly.
In either case, it's really critical to make it easy for people to move fast at that integration so that it can happen before some other project comes in, and for them to do that with confidence.
I love to just think about maybe the role of API developers in five to 10 years.
You have been a developer, also developed really well-known APIs SDKs as well.
If you imagine the future where the agents are going to be as common as human developers And we're going to have this agent experience instead of just dev experience.
What do you think that experience is going to be looking like down the road?
Today, we're still continuing designing for both of these two personas or users.
I still call it like LLM a user.
What do you think it will look like in that time horizon?
I mean, I think it's a really interesting and exciting question.
What I was saying a minute ago, where an LLM, a coding agent, writing your code means you want your code to be more declarative and more dry.
You've always wanted that, but it becomes more important to be able to quickly and accurately review code.
And so, for people building APIs, you want a way to write your backend code that is going to be not like spaghetti.
And that's how a lot of people unfortunately have to write the REST APIs today.
And furthermore, you're going to want more clear and prescriptive standards around what API design should look like, so that the LLM can follow a set of rules that the company has put forward around how they want to do their API design.
And again, the LLM can follow the slot into maybe, the API framework that the company is using and just focus on.
Okay, let's declare the endpoint.
Let's declare the parameters.
Let's write the business logic.
So a human can quickly review that.
I'm imagining really high-taste developers.
You're kind of crafting the endpoints of how you're designing both the APIs themselves to the documentation with it and SDKs in a very detail-oriented way.
But now we have a lot of, let's say, co-gen tools, and we have even more advanced frameworks and also much higher-level tooling that you can define the specs maybe i wouldn't call it natural language but just more of like a spec out way but not having to worry too much of the like polishing on the details and that's exactly you know what i feel like as soon as its work is doing is to maybe elevate the developers to think about more of the higher level tasks because of what you want the functions to be and we want the users to interface with instead of like caring about the errors and the details and pagination or like debuggability, observability, these type of nuances.
The most painful and difficult part around building APIs.
There's many painful and difficult parts, unfortunately today for people who are shipping APIs to their customers and their users and their partners, which is unfortunate.
But one of the worst is sort of the design part.
You know, and dealing with all the bike sheds and thinking, you know, you just want to be able to say We need an API that does this thing right, that manages this kind of resource.
You know the humans should be thinking about, as you say, the high levels of what the business needs and letting the robots do all the less exciting stuff.
You want AI to be writing your maybe business logic.
You don't really want it to be writing your platform code, those core things, that sort of operate as all the glue between everything else.
For that, you want a good platform.
So you want a good framework.
You want good libraries.
And that's something that Stainless is looking to bring to everything around the API so that the humans can say the high-level stuff, the LLMs can build that media layer and that Stainless can take care of the low-level infrastructure.
The last question I want to ask is you work with some of the most cutting edge logos and customers like OpenAI, Anthropic, CloudFlare.
Without disclosing any confidentials?
Is there any learnings that's worth sharing as the audience on these collaborations, of how they're thinking about their SDK design, their API platforms and what they want to offer to developers?
Yeah, I mean, I think there's so many lessons that every API team learns again and again.
Everyone in this space sees how important it is to be thoughtful, how important it is to consider the user.
The exact way you consider.
That is changing a little bit, you know, just like it did when people moved from writing JavaScript to writing TypeScript, or when people went from having slow connections to fast connections on the internet.
The fundamentals I think are still fairly universal.
It's exciting to see people really think deeply about how to expose their capabilities, whether those capabilities are a large language model that can do all sorts of you know text and audio, or whether that's the ability to spin up new servers in the cloud instantly, or whether that's the ability to move money around the world.
People want to be able to focus on their core capabilities.
Be thoughtful in exposing the interfaces to users and not have to worry about the low-level details.
Thank you so much, Alex.
This is wonderful.
Thank you, Jennifer.
This has been fun.
We've got more great discussions on the future of software, infrastructure, and AI coming soon.
Thank you.
Thank you.
Thank you.
