I Don’t Care What You Build (And Neither Should You)

We need to have an uncomfortable conversation about design reviews. Not the kind where we debate whether to use PostgreSQL or MongoDB, or argue about microservices versus monoliths. I mean the fundamental question of what engineering leaders should actually be paying attention to — and why most of us have been looking at the wrong things for years.

Here’s a confession: I used to love design reviews. There’s something deeply satisfying about diving into architecture diagrams, debating trade-offs, and demonstrating your technical chops by spotting potential issues before they become production problems. It feels like leadership. It feels like adding value.

It’s also, I’ve come to realise, largely theatre.

As the economist Thomas Sowell observed, “It is hard to imagine a more stupid or more dangerous way of making decisions than by putting those decisions in the hands of people who pay no price for being wrong.” In our world, engineering leaders who fixate on implementation details are making exactly this mistake — we pay no price for being wrong about whether that service should use gRPC or REST, but we’re spending all our time there anyway.

The Yaron Lesson

My old CTO, Yaron, taught me something I didn’t understand for years. It became a running joke among the engineering managers. You’d walk into a design review with a beautifully crafted presentation — architecture diagrams, sequence flows, tech stack justifications, the works — and Yaron would ask: “What’s the KPI on slide one?”

We never got past slide one.

I remember walking past frustrated engineering managers in the corridor. “Design review with Yaron again?” “Yep.” “Didn’t get past the first slide?” “Yep.”

We thought he was being difficult. We thought he didn’t understand the technical nuances. We spent hours preparing detailed implementations, and he’d wave them away with a single question: “How will you know if this works?”

It took me years to realise what he was actually doing. He wasn’t avoiding technical discussions because he didn’t care about technology. He was focusing on the only thing an engineering leader should genuinely care about: how the team would measure success.

The Mathematical Inevitability of Being Wrong

Here’s the uncomfortable truth that makes Yaron’s approach not just reasonable but mathematically necessary: according to research by Ronny Kohavi at Microsoft, only 10–30% of product changes actually improve metrics.

Let that sink in. When leaders prescribe the implementation — when we tell teams what to build rather than what outcome to achieve — we’re wrong 70–90% of the time.

Those design reviews where we debate architectural choices? Statistically, most of those decisions won’t matter because the feature itself won’t improve the metrics we care about. We’re optimising the wrong layer.

But here’s the thing: if you prescribe the measurement instead of the implementation, something magical happens. The team discovers what works through experimentation. They iterate. They course-correct. The math forces humility about implementation while enabling confidence about outcomes.

As the basketball coach John Wooden put it, “Never mistake activity for achievement.” We’ve built an entire engineering management culture around reviewing activity — designs, code, architectures — when we should be obsessing over achievement.

The Framework That Actually Works

I’ve found that effective engineering leadership boils down to one question asked relentlessly: “How will we know?”

Not “What will you build?” Not “Which technology will you use?” Not “Show me the architecture.” Just: “How will we know if the problem is solved?”

This isn’t abdication. It’s the opposite — it’s the hardest form of leadership because it requires clarity about what success actually looks like. Anyone can have opinions about database choices. Defining what victory means for the business, the user, and the team? That’s the real work.

Here’s what this looks like in practice:

Define measurement, not implementation. What does success look like? Is it improved conversion? Reduced error rates? Faster load times? If you can’t answer this, you’re not ready to start building anything.

Give genuine autonomy. Let the team decide how to solve the problem. They’re closer to the code, closer to the constraints, and — here’s the key insight — they’ll be the ones iterating when the first attempt doesn’t work.

Enable experimentation. Provide the platform and the psychological safety to test and learn. The first solution is rarely the right one, and that’s fine — as long as you can measure whether it’s working.

Trust the data. If the implementation is wrong, the metrics will show it. And because the team owns the outcome, they’ll fix it themselves. You don’t need to catch the problem in a design review; the measurement system catches it in production.

Let course correction emerge. Teams naturally fix problems when they can see them. The leader’s job is to make problems visible through clear success criteria, not to prevent problems through implementation control.

Why This Feels Wrong (And Why You Should Do It Anyway)

If you’re a technical leader who came up through the engineering ranks, this approach probably feels uncomfortable. We built our careers on technical expertise. We got promoted because we could spot the flaw in an architecture, optimise the slow query, design the elegant abstraction.

Letting go of that feels like abandoning what made us valuable.

But here’s the thing the physicist Richard Feynman understood: “The first principle is that you must not fool yourself — and you are the easiest person to fool.” When we dive deep into implementation details, we’re often fooling ourselves into thinking we’re adding value. We feel productive. We feel involved. We feel like leaders.

Meanwhile, the team is building something that might not move the metrics at all, and nobody noticed because we were too busy arguing about whether to use Kafka or RabbitMQ.

The Questions That Actually Matter

Next time you’re in a design review, try asking only these questions:

“How will we know if this works?” If the team can’t answer this clearly, they’re not ready to build anything yet. This isn’t about being difficult — it’s about ensuring everyone understands what success looks like before committing resources.

“What will we measure?” Not “what can we measure” — what will we measure? There’s a difference between metrics we could theoretically track and metrics we’ve actually instrumented and will actually monitor.

“What’s the hypothesis?” Every implementation should be a bet. What specifically do we believe will happen when this ships? How confident are we? What would prove us wrong?

“How quickly can we learn if we’re wrong?” Speed of iteration beats quality of prediction. The team that can deploy, measure, and adjust in days will outperform the team that plans for months.

The CodeBuddy Principle

I recently saw this approach articulated perfectly by a team working on AI tooling. Their process:

Observation — look at dataAnnotate and build evaluations — define what “good” and “bad” look likeHypothesise — understand the whyDesign and run experiments — test the hypothesisMeasure outcomes — did it work?Apply and iterateNotice what’s missing? Nobody prescribed the implementation. Leadership provided clear measurement criteria, and the team self-corrected through iterations. The result was a product that actually improved the metrics that mattered.

This is what Netflix calls “context, not control” — leaders provide the goals, constraints, and metrics, but not control over how work gets done. It’s what the military calls “commander’s intent” — give the mission objective, not step-by-step orders, so people can adapt when circumstances change while preserving the intent.

The Counter-Example

I’ve seen the opposite play out too many times. An engineer pushes back on a feature request with a question that should have been asked at the start: “Before we do anything on this, even refinement, I would want to have some clearer information on what exact metrics we’d be using to verify our changes have improved things.”

That’s the dysfunction. Leadership proposed a solution — improve search — without defining measurement. They prescribed the what without the how-will-we-know. And now the team is being asked to build something with no way to tell if it’s working.

As the management theorist Peter Drucker noted, “What gets measured gets managed.” The corollary is equally true: what doesn’t get measured doesn’t get managed — it just gets built, shipped, and forgotten.

Escaping the Trap

The hardest part of this transition is trusting your team. You spent years building technical expertise, and now you’re being asked to set it aside in favour of defining outcomes and letting others figure out the implementation.

It feels like giving up control. It feels like being less involved. It might even feel like being a worse leader.

But here’s what I’ve learned: the best engineering leaders I’ve worked with share one trait. They’re obsessed with outcomes and relaxed about implementations. They ask “how will we know?” until the team can answer it clearly. They trust that if the measurement is right, the implementation will follow — through iteration, experimentation, and the collective intelligence of the team.

The worst engineering leaders? They’re obsessed with implementations and vague about outcomes. They can tell you exactly which database to use but can’t tell you what success looks like. They feel productive in design reviews while their teams build features that don’t move the needle.

The Bottom Line

I finally understood what Yaron was doing all those years ago. He wasn’t being difficult. He wasn’t avoiding technical discussions because he didn’t understand them. He was demonstrating the one thing engineering leaders should actually care about.

Not the architecture. Not the tech stack. Not the implementation details.

The question that matters: “How will we know?”

If you get that right, the team will figure out the rest. If you get it wrong, no amount of architectural elegance will save you. You’ll just be building the wrong thing more efficiently.

As the educator John Dewey observed, “A problem well put is half solved.” In engineering leadership, a problem well measured is mostly solved — because the team can iterate their way to the answer.

Now, if you’ll excuse me, I need to go prepare for a design review. I’m planning to stop at slide one.

Joel writes about engineering leadership, software architecture, and the organisational dynamics that make or break technical teams. He’s based in Bangkok, where he leads engineering teams at Agoda and occasionally frustrates people in design reviews.