It's become kind of a joke how the era of AI has led to the demise of every job in tech. In fact, so many jobs have died this year, I'm surprised I haven't been invited to more funerals.
Product management is dead, user research is dead, and most egregious of all, software engineering is dead?
I'm probably preaching to the choir here, but anyone who really believes that software engineering is dead because your friendly neighborhood LLM can write code is definitely not an engineer themselves.
My guest today, Cortex .io founder Anish Dhar would even argue that engineering is definitely not dead.
It's just growing up.
Formerly an engineer at Uber, Anish founded Cortex to make it easier for engineers to understand complex code bases.
As an engineer himself and someone whose users are engineers, what he's seeing in the space is actually an evolution in engineering excellence and a disconnect between new and old ways of measuring it.
He shared next -gen thinking for measuring and evaluating excellence in engineering and a hot take on how the Vibe Coding craze fits into the conversation.
Let's jump in. Oh, by the way, we hold conversations like this every week, so if this sounds interesting to you, why not subscribe?
Okay, now let's jump in.
Welcome back to the Product Manager Podcast. I'm here today with Anish Dhar.
He's the founder of Cortex .io.
Anish, thank you for making time to talk to me today.
Thank you so much for having me.
Yeah, so can you tell us a little bit about your background and how you arrived at your role today?
Yeah, absolutely. So I'm the co -founder and CEO of Cortex .io.
We started the company about six years ago, but before that, I used to work as an engineer at Uber.
I really started my career there.
And a lot of the problems that I faced as an engineer at Uber actually inspired all of the reasons we started Cortex. So two really close friends of mine.
Uber has this massive internal service architecture.
And as an engineer there, it was really difficult for me to understand different parts of the code base, especially when I joined.
There were so many different services that were being built, it added this intense complexity.
I was talking with a really close friend of mine, who was an engineer at a very small startup called Lando, and they only had 100 engineers while Uber had over a thousand, but we were both facing similar challenges around organizing and understanding our service architecture.
That just rang these alarm bells that, okay, well, if Uber is on one end of the service complexity scale, and this other company that's just starting their journey on microservices has the same problems, it's clear that this is a big problem in industry, So we ended up starting the company, we went through the winner 20y comm meter batch and then fast forward to today, we just figure there's Series C and now work with a few hundred different enterprises use Cortex to manage their complexity.
Cool well that's an amazing journey and it's always wonderful to hear when a company kind of comes out of the ashes of an issue that you know intimately and speaking of which we're gonna be talking about engineering excellence and what that looks like in today's tech landscape on this episode, so of course this is an issue that you've been very close to throughout your career.
So to kick us off how do you define engineering excellence in 2025 and why is it becoming such a critical focus for CTOs and VPs of engineering right now?
Yeah absolutely, that's a great question.
So what we found at Cortex is that for a long time the conversation is really focused on developer experience.
And developer experience is a really critical part of any engineering organization, right?
It's simple things like making sure that developers when they join a company it's really easy to get their internal system set up and they connect to GitHub and the various tooling that they have, or when they're maybe deploying a service.
The infrastructure is setup in a right way so that there's not a lot of troubleshooting or steps to get there.
But what we found over the last couple of years especially is that the conversation has shifted a lot more from just developer experience to what we call engineering excellence.
And I'd say the big difference between the two is engineering excellence is really the focus of different teams within organizations.
You can think like SRE, security, But it really allows them to act feel business outcomes.
And I think that's the key difference here where a lot of engineering excellence is thinking about how does the work I'm doing actually impact the business?
And how does it actually move forward the goals that we have all the way from the CEO's personally organization all the way down to the specific SRE, for example, that's working on it.
You know, a good example of that could be as an organization, you know, we're focused on improving our customer experience.
And we want what we want to do better.
customer to use their product to be reliable and a better customer experience leads to more revenue because people are using our product more.
So from an engineering excellence initiative, your SRA team then might have a productive readiness checklist that they're trying to implement.
Because before services are deployed, they want to make sure that all services are meeting kind of the standards of the organization.
And so you can see how an initiative that starts with the SRA team kind of drives back to this real business outcome that the organization cares about.
And I think it's really critical for teams to think about their initiatives in this way, because it kind of reaffirms the value and aligns the technical initiatives is something that business cares about, which is, which is I think what engineering excellence is all about.
Interesting. So, and this I'm sure you described many times is that kind of like a never ending journey, which involves many disciplines kind of working in tandem.
Tell me about this framework that you've developed.
Like, what are the key pillars look like?
How do you kind of look at this from your own organization?
You're absolutely right.
It really is a never ending journey.
I think that a lot of companies that we work with, especially, you know, especially the size of the company, or if they're a large enterprise with a lot of legacy infrastructure versus maybe a newer company that's kind of built on newer technologies and maybe even AI -first, there's still different initiatives that revolve our engineering excellence and really acquire, I think, a really thoughtful approach across all these different teams about what is the work and how does it kind of impact ultimately the business excellence goals we have. And so from a framework perspective, I think one of the things
that we have really worked on is, yeah, how do you define entering excellence for your organization?
And I think the way that we think about it is it starts with business excellence, right?
There is different goals that you have as a leadership team.
And they can be around things like unlocking innovation and reducing time to market.
It could be maybe lowering costs and increasing efficiency.
And then usually the third one we see, which I just mentioned is how do you improve quality and customer experience?
And then underneath that is really the pillars of entering And these are the different teams and practitioners that kind of make up the initiatives that drive these eventual goals.
And you know, that's things like velocity, efficiency, security, reliability.
And even within those subcategories, you know, you'll have initiatives like maybe there's a security migration or approach readiness checklist. Maybe there's an incident management process that you're trying to implement.
Or just something as simple as we want to track door metrics to understand from a productivity standpoint, how is our engineering team actually performing?
Then the foundation of any engineering or exposed initiative really comes from what we call, we like to call them the 4Cs.
It's essentially like complete visibility, continuous improvement, consistent developer experience, and of course clear ownership.
Because without ownership and understanding the different parts of your code base and all the services, it's really hard to actually drive these initiatives forward. Typically, we find that without that foundation it's really hard to drive any initiatives.
Typically we also see IDPs, or internal development portals are a really strong way to kind of build a foundation.
But they can also be done through internal tools.
But you just need some sort of system to be able to understand what people are building so you can thrive these engineering initiatives for.
Okay, so I want to dig into something that you mentioned there about measuring performance, because I know that there's a little bit of tension around measuring developer productivity and metrics like lines of code can be a little bit controversial between engineers.
So how should engineering leaders think about measuring productivity in a more holistic way that kind of takes into account all these kind of Cs and that kind of thing.
Yeah, absolutely. You know, I think the interesting thing is a lot of normal productivity over the last few years has been about lines of code or door metrics and there's a lot of different frameworks that I think have been created to simplify if I want to think about productivity and there's some truth to how those metrics are calculated, right?
Yeah. Lines of code isn't a good indication of if someone is being productive or not, but if you're delivering zero lines of code consistently, quarter over quarter, There's something clearly wrong with the output or even from comparing team over team, it's interesting sometimes to see those data points.
But I think the conversation has really shifted from, well, I have this data to, how do I actually get engineers to think about that data or improve it?
If you really break it down, it's a completely different problem and a way more difficult one.
Because any way you can go in, hit your GitHub API and get these metrics and get a snapshot of how your team is doing.
but just because I show a set of metrics to an engineer, and try and describe, hey, we have to improve this metric, as an engineer, that doesn't really mean anything to me.
I'm focused on building software for the business, and I'm focused on doing it typically in the most efficient way possible.
But I think the conversation has really shifted, especially with the CTOs that we work with, I have these metrics, now how do I translate that into something that an engineer cares about?
I actually think that's where engineering excellence play such a critical role, because I think engineers, especially ones who work at fast -forward companies, they want the business to grow.
They're building products because they want to see the impact of their work with customers.
I think developer productivity has really shifted from, here's just a bunch of metrics to, well, as a business, these are the things that we care about, and these metrics tell a story as part of that.
But for an engineer, it's how do you translate that into something then that the work that I'm actually working on, it means something.
And so I think that's the big shift that we've been seeing.
So have you noticed that there are some kind of outdated evaluation methods or KPIs that people are starting to move away from?
What would you say is kind of like the new school of evaluation?
Or do you have any specific examples you could share?
Yeah, absolutely. So I would think about productivity as sort of input and output metrics.
So output metrics are all of the classic frameworks that you see today to track job productivity.
One of the most famous and one of the most popular ones is DOR metrics.
Which is a few difference set of metrics that give you this holistic view of, or they're supposed to give you a holistic view of how is my engineering team performing.
I would say that most engineering organizations today while they want to see these metrics, these output metrics and try to capture them, it goes back to what I was saying a little bit earlier around.
Well, then how do I actually influence those metrics and see them move?
So what we've been seeing, especially with our customer base, and I think why we've seen this interest in Cortex grow over the past few years is because there's a full cost of input metrics and influences of the metrics.
For example, let's say something like deploy frequency.
Deploy frequency is a great metric to look at because the rate at which your engineers are deploying software is probably a good predictor of how fast you're shifting product, which at the end of the day is then how you'd be your competition to market and how you just move faster as a business.
So maybe as an organization where you've decided that deploy frequency is the main metric that you want to track.
Well, if I have a dashboard of deploy frequency and I can pull this off, but I try to show the whole engineering team, hey, everyone, we're at deploying two times a week, we want to get that to four.
As an engineer, how am I actually supposed to think about, as it relates to my work, my part in the business or the services that I own, obviously, deploying faster means maybe I have to ship more.
But does that lead to increased bongs?
Because I'm shipping more, is reliability going to go down?
There's so many different variables that go into this, and I think that's where input metrics become really critical because the input metrics ultimately influence the output metrics.
Maybe for deploy frequency, maybe there's a process that you put into place to actually see that go faster.
I can give you an example, like one of our customers, O 'Reilly, they had a very similar initiative like this and they were tracking deploy frequency using one of our engineering intelligence dashboards.
But what they found was that, okay, we want our engineers to deploy faster.
But the way we're going to do that is by putting really important guardrails in how engineers deploy and giving them really clear guidelines on, this is what a good deploy looks like as it relates to reliability guidelines.
Because what was happening is engineers were currently deploying faster, but it would lead to bugs or things would break, and so there was this hesitance to actually move as fast as quickly as possible because, well, there was all this customer impact that was happening.
Going back to those input metrics, they basically came up with this production readiness checklist, and it was a list of eight or nine different input metrics that as a whole gave a really good understanding of, is our deployed process actually healthy or not?
These are metrics like, it's our own call setup correctly.
Is our build process actually passing?
Do you have tests that are passing on your services?
What that did by putting those input metrics in place, was it gave engineers a really good guideline I know, I own these 10 services, this is the process and these are input metrics that actually means something to me because they represent the services and how they work.
What they saw was a gradual increase in deployed frequency from two times a week to three to four, and especially with the critical services, and also a reduction in things like incidents and things like that.
Going back to your original question, I think a lot of enterprises are thinking about, we have those output metrics, But then how do I translate that in something engineers care about and they kind of feed into each other in a very meaningful way?
I think you have to be thinking about your developer productivity and metrics in general, as like a complete story between the two.
Yeah, that makes it a lot more, I think holistic is a really good way of looking at that.
Switching gears just slightly, but we're kind of on the same topic of deploying more quickly.
We can't have a conversation about engineering in 2025 without talking about vibe coding.
Let's talk about these AI tools that are transforming the coding workflows right now, I've heard you say before that you can't vibe code your way to a million users per day.
Maybe a hot take, maybe not.
So what would you say is the reality versus the hype when it comes to using AI for production -scale environments?
Yeah, I mean, it's certainly a very hot topic, and I think that every engineer organization is thinking about AI or has adopted some sort of AI coding assistant, for example, or if there's several really popular ones, including Cursor out there in the market, even the engineering team at Cortex, almost all of them are using some AI coding assistant to help them with their day -to -day.
But what we found talking to the engineers in our team and also working with most of the customers that we work with, have been thinking about similar initiatives.
I think that AI coding assistance are great for when you have an initial idea and you want to quickly validate something.
Or even if you're a front -end engineer and you want to quickly mock out something very quickly from an idea to how something could look and feel like, What we've seen is from an engineering development process, it's perfect for something like that.
You want something quick and dirty, something just show people how something can work.
Or if you have an idea as an entrepreneur, you want it quickly validated.
I think you're seeing such amazing growth with things like that.
But the reality of where coding assistance and Vibe Coding is today, is that you could never trust a code that is shipped from vibe coding to actually power production system that is then being used by millions of different users right and that's just experience from what we've seen right like at the end of the day I think vibe coding is at best a junior engineer who has really just learned how to code versus like I think that's where senior and staff engineers who understand system design who understands how infrastructure is actually kind of deployed at scale, we're just not nearly there.
I'm not saying that it can't get there, I think the rate at which AI is developing is unbelievable, and I think it'd be stupid to say that maybe there's not a world in which AI systems can understand actual production instances.
But the reality of it is, today, it's just, I mean, I can tell you for a fact there's no enterprise out there that's relying on vibe coding to power any production system that handles millions and billions of users, Because it's just in the level of technical expertise you need to kind of send up those systems and diagnose them and make sure that they're scaling, it's just not even close to that.
But I mean, as a whole, I would say productivity has improved with quoting assistance.
It's just in different areas than I think maybe the market likes to talk about, or is sexy talking about five quoting, but the reality is it's not already for enterprise production systems. I think that's very relatable to a lot of professionals who feel like folks who are outside of their profession and are getting excited about the accessibility of their profession using AI tools.
That doesn't mean that it's necessarily, you know, coming right behind you.
This is an important thing to talk about.
We have, you know, leaders listening to the show, so for engineering leaders trying to balance investment in AI tools versus our headcount, from your perspective, what should they be considering?
Like how do you evaluate the actual impact that AI is having on a team's productivity?
And how do you kind of take that into account with your budget?
It's a great question.
I mean, I think there's a lot of different facets to this question, and I mean, at the end of the day, and you didn't like, even just taking it from a very macro point of view, any engineering teams or engineering leaders that prevent their teams from looking at these tools or prevent their teams from accessing, you know, like for example, things like cursor or GitHub code pilot or whatever, I think it's a big disservice to, I think the overall health and quality of your engineering team over the very long run, because I think the reality of it is in the next 10 years, a lot of the systems or at least
initial code that people will write will be AI -assisted.
Just because going back to what I was saying earlier, if you're just starting a company or you're starting an idea, or you want to quickly iterate and test something, the reality of it is it's just 10 times faster using something like Cursor because you can just so quickly iterate on your ideas and you don't really need to think about scale, or how things work.
And so even if you take a very long -term view of it, I think that's why engineering teams that are adopting these AI tooling and are learning how to use it in different parts of their coding life cycle, even like the largest enterprises where you need production scale systems, like those engineers at a collective hole are going to be at an advantage compared to kind of, people who are saying, like, oh it's not really, it's just hype, you know.
I think just starting there, I think it's very important that engineering leaders let their teams explore with these types of things and even give their non -technical users access to these tools.
Because that's probably the most interesting innovation that I've seen, especially from our customer base is like product managers and TPMs, and data scientists who understand technical concepts, but maybe didn't have the expertise to code, who can take ideas and share them with the engineering team in a lot more powerful way because they can actually spin up or pick code and things like that.
I think from that perspective, it'd be really foolish not to create budget, these tools.
Now, I think the million dollar question is, how much productivity actually are we getting from this?
And I think honestly, every single enterprise is trying to answer this right now.
We see it all the time, right?
A lot of times customers will buy our product in conjunction with something like get a co pilot.
And then the first question they ask us as Cortex is, okay, well we have this tool that is supposed to, you know, 3x, 4x the output of our engineering team, we'll actually want to figure out is actually happening.
I think it actually just comes back to the earlier system of input and output metrics I was talking about.
It's not enough to just pull up, deploy frequency and say, does GitHub CodePilot have an impact on that?
Because maybe GitHub CodePilot is making your engineering shift faster, but it's really bad code.
That leads to level reliability issues.
You have to really take this almost 360 view of it and look at different metrics.
I think it comes back to engineering access at the end of the day.
It's as a business, forget vibe coding, forget everything else.
What are we focused on as a business right now?
What are the things that are blocking us from getting our next skill of growth or whatever it is as a business you care about?
I think you have to think about is at the end of the day, the introduction of the coding system or AI tool actually making a difference or impact from that perspective.
The reality of it is for a little bit of time, you might not really see it impact or difference.
Because I do think it takes some time to understand where these kinds of tools will have impact on your business.
I think maybe the mistake that I see a lot of enterprises doing is buying into the hype, immediately just buying 5 ,000 licenses of whatever tool, like a cool tool they see, and then like six months go by and they're, okay well actually is there being an impact?
You might have a very negative reaction.
I think without having a very thoughtful strategy on why we're introducing these type of tools.
Maybe the strategy is what we were talking about earlier.
We don't want to be left behind.
We want engineers to continue to feel like this is a cutting -edge place to work, and we just think that it's advantageous for us to have AI -assisted tools in our engineering team.
Even if it's something as simple as that, at least you know the intentions of why you're buying it.
I think maybe that's the mistake is just understand your intentions and then you can do gut checks along the way, like is it actually doing what we thought?
I tend to agree. I think that right now there's so much pressure to kind of have mastered these tools and figuring out kind of where they fit into the workflow.
That's helpful. But yeah, I tend to agree that right now, what we're kind of running into across many departments is the difference between quantity of output and quality of results is like vast. And we're like really having to be so intentional, not just in the engineering department, but are we applying these tools in the right context in order to kind of facilitate and kind of empower the best of our head count rather than just kind of blindly trying to account for it as like, you know, next year's head count is going to be lower because it's AI tools to be able to replace.
So I think that right now it's interesting to kind of see everybody figuring this out in different ways at the same time.
And it's a very disorienting time to be in tech. But, you know, speaking of the AI tools and them becoming more prevalent in development workflows.
And speaking of quality also, like how do we kind of think about kind of finding that balance of maintaining code quality and security standards and reliability while also wanting to make sure that we are being, you know, cutting edge workplaces and taking advantage of these tools to the best of their functions?
So in your organization, for example, how have you guys approached that?
It's a really good question.
And just really briefly, I think I will say I think the idea that AI is replacing engineers is so far fetched and silly.
And I think what is true is maybe when you're first starting out, instead of hiring like 15 engineers, maybe you can hire a few less than that, just because the early days of just iterating and trying to find product market fit, you can just do so much more with a tool like cursor than you previously ever could.
It could like try out 10 different ideas really, really quickly.
And I think That's like speed of iteration is really powerful, but yeah, I think once you're actually deploying production systems, in my opinion enterprises that are saying, we don't have to hire as many engineers because of AI, is just like trying to get some nice media coverage or something.
There's just no way that's true.
I know that for a fact.
No naming names. Yeah, all of them are going to continue to hire.
Engineers will never not be in demand.
I'm going to ask you a question about what does AI tooling and AI coding systems impact some of these key pillars around like reliability and security?
Honestly, I think that's the big open question right now, and I think that's probably the biggest thing that scares these teams, and scares teams like SRE, or security, or operational excellence.
Because I think the introduction of these tools ultimately creates more surface area because people are shipping just more code.
The more of your system that is built with AI, the more likely it is that you don't really understand how everything works internally.
What that means is, when there's inevitably a reliability issue, because that is forever constant.
No matter how much you prepare, one day there will be something that happens, Whether it's an influx of users that you didn't anticipate or a part of your code base that you didn't fully understand that goes down.
The more your system that was AI -assisted, I mean, you're just in a much more difficult place.
Because as an engineer then, how do you actually go and traverse all the different parts of your codebase if we don't fully understand it?
And I think that's maybe the biggest downside I see right now and why actually I think it becomes more critical than ever.
I was talking about that foundational layer of engineering excellence.
Like, complete visibility and clear ownership are really key pillars of any engineering excellence initiative.
And if the more of your systems that are created through AI, honestly, you have less coverage on those things, because who actually owns it, who actually understands and has that visibility?
I think those are kind of the big things that scare a lot of security teams and reliability teams. And honestly, we see with some of our customers, we even see it with our own engineering team, right?
Like, you know, when we do publish code with AI or something that's created with AI, engineers are very careful to mention that, hey, some of this was 100 % generated through cursor or whatever.
And we take an extra look at those systems. You know, I've seen this huge train around AI -assisted testing.
So there's a lot of companies right now that are actually creating an AI engineer that will review your tests and stuff.
Sometimes I see that and it's a little bit dicey, because at the end of the day, I think AI systems are extremely powerful and are doing more good today than in any harm in any team.
But it's just scary to think about a world where 80 percent of my system is ready through AI.
Honestly, it is going to lead to reliability and so this would take longer to resolve security incidents might go up because you just have less visibility.
I think that's the thing that you have to watch out for.
I tend to agree, and I think that that's kind of the less talked about logistical problem that a lot of teams are dealing with when it comes to enhanced output, which also means an enhanced demand for oversight.
We just had a conversation last week with the SVP of product management at Mastercard Gateway, and he was talking about how a lot of AI tools are really accelerating the ability to complete things like forms in order to enter new markets, you know, in an inordinate amount of paperwork that you need to do in order to enter a new market and kind of comply with all the regulations.
AI is allowing a lot of that to be completed in a really, really rapid pace.
But then there also has to be that level of oversight to ensure that there is ownership over that over those submissions.
And so there's that kind of push and pull of like you can complete it quicker, but you also need to be able to be accountable at the same speed.
And that, I think, is a huge logistical bottleneck for...
Yeah, sounds like a lot of engineering teams as well as other kinds of folks who are kind of using these tools to accelerate.
So I think that's kind of a layer that kind of bears underscoring a little bit is, you know, like, sure, we can shit fast. But can we also at the same speed say that, yeah, I'm like accountable to that work.
So yeah, this is this is so interesting to talk about, you know, we've talked now to leaders in all kinds of departments and all kinds of disciplines, and it's very fascinating to see how a lot of these concerns are so parallel to each other, despite the fact that the discipline itself is different.
So I really appreciate all the insights you shared today, that concludes our episode.
Where can folks follow you online, Anish?
Anish, yeah I would say that you can follow myself on LinkedIn or Twitter, if you just search by name you'll find me and then Cortex as well, you can find us on LinkedIn, we're always posting.
We are actually hosting Andrew and Epson Summits all across the world and so if there's one city near you definitely come through, we just like to have a community of people who think about these types of things and uh every company is thinking about them.
So it's really cool to see such a great community come from that.
And then we also are hosting our conference called IDPHon and it's really centered on engineering excellence.
And so we have leaders from, you know, different types of enterprises who come and developers who are just interested in connecting with other developers on engineering excellence.
A lot of great talks.
It'll be in New York City in October.
So hopefully we'll see you there as well.
Yeah that sounds fantastic.
Well, thanks for letting us know.
Make sure to plug that in the description box.
And thank you so much for making the time to join us.
Yeah, thanks for having me.
It was great. Thanks for listening in.
For more great insights, how -to guides, and tool reviews, subscribe to our newsletter at theproductmanager .com slash subscribe.
You can hear more conversations like this by subscribing to The Product Manager wherever you get your podcasts.
