In today's episode, we're going to talk about a potentially controversial principle that I want to share with you, and we're going to frame it specifically for you as you grow in your career.
As you become a more senior engineer, as you become a staff level, director level, if you're an IC manager, it doesn't really matter.
This is still going to hold true.
And it's a little bit controversial because it requires that you take more responsibility for what's happening.
All right, and we'll talk about why that's the case.
We're going to talk about building systems that work.
work now specifically when we say systems in this case we're not necessarily talking about technical architecture okay you can apply some of these principles to technical architecture but that's not really uh the meat of what we're talking about in this episode instead we're talking about the kinds of events the kinds of policies that you have in your organization the many different kind of like the multitude of potential reasons why something is behaving the way it is, why some people are behaving the way they are, right?
Like why some resources are being funneled the way they are, okay?
There's, you know, so many possible system effects for you to pay attention to.
So, when we say system, in this case, we're talking about the more kind of systems design, systems thinking than we are talking about architectural technical systems. So these are processes, for example, that your team is following.
These are hiring processes.
This is cultural effects.
This is interpersonal effects.
There's a whole variety of things that you might include when we talk about systems. Okay, so let's make this more concrete with an example.
Let's say you're in an organization and you want to examine your quality control system.
them and specifically you want to be able to catch bugs before they get released to production it's a great goal to have right we don't want to release bad software so instead we want to catch them we want to deal with them before we release we want to have a process in place we want a good system for finding and squashing bugs ahead of time upstream we want to prevent these things from going to production.
And let's imagine that you've figured out what you think is the perfect system for doing this.
Maybe there's some kind of review required, maybe for particularly sensitive things or for new code, you're going to require some coverage, some kind of automated testing, integration testing.
You're going to do all of the, you know, kind of industry standard things.
Okay. So you write this perfect system, you set it into motion, and then things are not working.
And you have a meeting, you and your boss and the whole QA, you know, guild or whatever it is that's responsible for trying to make this, this happen.
Bunch of, bunch of senior engineers maybe in the room.
And the outcome that you come away with is that the system is designed perfectly fine the problem is the talent we're missing talent so the reviews that we're putting through the quality of the review is not very good and so they're not catching the bugs because they don't have enough experience and so you all walk away accepting the idea that you've developed and designed a good system but there's some other problem something that you got to deal with in order for for the system to work.
Here is the principle that I want you to take away and why this thinking is unequivocally wrong.
You're incorrect about your system being good.
Your system is perfectly designed for the results that you are getting.
Your system as it is now is perfectly designed for the results you are getting now.
What does that mean?
What it really means is that your system can't have arbitrary boundaries.
We choose to think about systems with boundaries, but when we're talking about whether our quality control system is working, if we say that the system is good, but the thing we care about is failing, then our boundary for what is a good system is missing a critical factor.
We are struggling to incorporate talent into our system.
We imagine that, for example, we have these discrete systems like our QA system and our talent management or our recruiting system.
system. The truth is that if your talent has any impact systematically on your ability to catch bugs, which clearly it does, then your quality assurance system should take talent into account.
And this very simple concept, but so often we, I might even say, delude ourselves into believing that our systems are discrete from each other.
Why do we do this? Probably the most likely reason is because it allows us to assign responsibility more cleanly.
So in this case, if we had that QA guild, it's likely a bunch of senior engineers.
The senior engineers are, broadly speaking, not usually responsible for talent development of the more junior engineers, right?
So if you're going to give some kind of system design task to your senior engineers, like go figure out our QA process, and you tell them develop the system in the best way possible, they're going to look at the things that they are responsible for, that they own, that they have some agency over, and they're going to consider their system to live, kind of live at that boundary.
It's going to end at that boundary, okay?
So what do we do about that?
First of all, we need to evaluate as if the responsibility is not a factor in our system design.
We should consider the system regardless of the responsibility or the domain lines or whatever those kind of arbitrary things are.
Why do those things exist?
We want to make sure that we limit the scope of responsibility for a given person.
Otherwise, if we're all responsible for everything, then none of us are responsible for anything.
So in this case, what is the diagnostic that we would use.
First, we need to correct our thinking about systems. Our system is perfectly designed for the outcomes, for the output, for what we are getting.
Now, I want to put a little bit of a caveat here because you've heard us talk about resulting a lot on this show.
If you're not familiar with what resulting is, judging the quality of a decision based on the outcome of that decision.
This seems intuitively correct, but it's wrong.
Statistically, most often it's correct, but it's not technically always correct.
Why? Because the quality of your decision doesn't know about the outcome.
You're trying to make a decision in order to optimize for an outcome, but you don't have certainty about whether your decision will achieve that outcome, okay?
So, in other words, let's say you had information that if you were to go route A, you're going to get a 60 % chance of a good outcome.
Route B is a 40 % chance of a good outcome.
Any good, reasonable thinker is going to choose route A, all things being equal, right?
It's a higher chance of success.
If a bad outcome occurs on path A, which when we made that decision, again, we had a 60 % chance of having a good outcome.
If a bad outcome occurs, which by the way will happen 40 % of the time, if a bad outcome occurs, we have a tendency to judge ourselves negatively as if we made the wrong decision.
decision. Something about that decision, you know, because of the outcome, because of uncertainty, because of something that we didn't know, we judge ourselves negatively.
So very often what we will do is we will try to develop systems that reduce that uncertainty so that the decisions that we're making have a higher likelihood of adhering to what we're trying to get, right?
So let's go back to our example.
We have a talent pool of engineers.
We want to have a solid ability to identify bugs in 100 % of cases.
Now, if you've been doing this career for very long, you know that that's not possible even the best engineers the world's best engineers have encountered bugs that were quite literally impossible to predict okay so our systems we want to develop them in such a way that we're catching as many bugs as possible all right how do we do that how how can we ensure that we're going to catch as many bugs as possible Well, we've created all of these protocols through our testing, through our review, through all these validations.
There's a lot of things that we can do to reduce the risk, but we won't ever get to 0 % risk.
However, one of the things that might increase the risk is if our talent pool is limited.
if we have limited experience on the team for engineers, especially for those who are reviewing code, looking for those regressions.
So what do we do about that?
There's a lot of different strategies.
We're not going to go into every strategy.
You may, for example, instead of requiring one review, you may require two reviews.
You may require review by someone who has a certain amount of experience or has expertise in this particular area.
You may require, you know, more rigorous testing.
You may require, maybe you change, and this is really where it gets interesting, right?
Maybe you change your hiring procedures, okay?
We're talking about avoiding resulting and developing systems that reduce the likelihood likelihood that we're going to have a bad outcome, that the decision that we're making is going to produce a bad outcome.
So the system that we have to catch bugs, if it's failing and we've diagnosed the reason that it's failing as something dealing with talent, then we need to develop our understanding of our system with talent in mind.
Now, this may include training, right?
It might include a different kind of hiring procedure.
We might include, maybe we start doing some more kind of focused discussions in our teams about testing and sharing some common knowledge, sharing pathways about testing.
Maybe we reemphasize the importance of quality.
Maybe we create incentives for people to have higher quality.
There are a lot of different things that you could do, but if you're avoiding or if you're ignoring the talent aspect, right?
If you're ignoring this factor because it's not part of the system, then you could have this irreducible risk, right?
Or a very expensive risk that's hard to reduce without looking at that system or that subsystem of talent.
So if instead we We approach this from as many very varied locations or vantage points, and we collaborate on our systems so that we're not drawing arbitrary lines of responsibility where we're actually creating these subsystems that are not necessarily having the effects that we want.
We're trying to accommodate.
Imagine that we are adding more review steps to accommodate for the fact that we can't change our talent.
That's not as efficient of a change, not as efficient of an intervention in the system as going and actually fixing the talent pool, fixing that recruiting process, fixing whatever the thing is that's requiring that talent to be improved, that talent pool to be improved.
You could also look at it from another angle.
You could say, well, we need to reduce the complexity of our systems such that the talent that we have can work effectively against it.
Once again, you need to be able to introduce this talent aspect into your systems thinking in order to make that decision in the first place.
© BF -WATCH TV 2021 Thank you.
Thank you. We'll be right back.
I'll see you next time.
Thank you. Thank you.
Thank you.
