<?xml version="1.0" encoding="utf-8"?>
<feed xml:lang="en-us" xmlns="http://www.w3.org/2005/Atom"><title>Micah Stubbs's Weblog: engineering-leadership</title><link href="https://micahstubbs.ai/" rel="alternate"/><link href="https://micahstubbs.ai/tags/engineering-leadership.atom" rel="self"/><id>https://micahstubbs.ai/</id><updated>2026-01-08T14:00:00+00:00</updated><author><name>Micah Stubbs</name></author><entry><title>The KPI Is Time to Closed Loop</title><link href="https://simonwillison.net/2026/Jan/8/the-kpi-is-time-to-closed-loop/#atom-tag" rel="alternate"/><published>2026-01-08T14:00:00+00:00</published><updated>2026-01-08T14:00:00+00:00</updated><id>https://simonwillison.net/2026/Jan/8/the-kpi-is-time-to-closed-loop/#atom-tag</id><summary type="html">
    &lt;h1&gt;Memo: The KPI Is Time to Closed Loop&lt;/h1&gt;

&lt;p&gt;We keep talking about "using AI." That's the wrong goal.&lt;/p&gt;

&lt;p&gt;The goal is to &lt;strong&gt;remove humans from repeatable loops&lt;/strong&gt; as soon as the work becomes predictable. The moment a model is "good enough," the work is no longer a craft. It's an engineering problem: build the loop.&lt;/p&gt;

&lt;p&gt;Call that moment the &lt;strong&gt;good enough signal&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;Most companies will treat the signal as a stopping point: "Great, the team can do this faster now." The companies that win will treat it as a starting point: "Great, now we can automate it."&lt;/p&gt;

&lt;p&gt;To make this operational, we need one metric that forces the right behavior:&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Time to closed loop&lt;/strong&gt; = time from "a human can do this reliably" to "the system does it automatically, with monitoring, escalation, and learning."&lt;/p&gt;

&lt;p&gt;If that time is long, we're leaving leverage on the table. If it's short, we compound.&lt;/p&gt;

&lt;p&gt;There's a simple ladder most work climbs:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;&lt;strong&gt;Assist:&lt;/strong&gt; the model drafts; the human decides.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Autopilot + review:&lt;/strong&gt; the model acts; humans review every output.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Autopilot + sampling:&lt;/strong&gt; the model acts; humans audit a small percentage.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Closed loop:&lt;/strong&gt; the model acts; humans handle exceptions; the system improves.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;Many teams get stuck at level 1 because it feels immediately helpful. Level 4 is where the economics change.&lt;/p&gt;

&lt;p&gt;What "closed loop" requires (and why it's executive work):&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;Clear acceptance tests:&lt;/strong&gt; not perfect, but explicit. If you can't say what "good" means, you can't automate it.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Instrumentation:&lt;/strong&gt; logs, traces, and outcomes. If it's not measured, it can't improve.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Fallback paths:&lt;/strong&gt; escalation to a human, rate limits, and safe defaults.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Ownership:&lt;/strong&gt; one accountable owner per loop, like a product.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;We should treat automation as a portfolio. Every quarter, we pick a handful of loops that consume the most human attention and move them up the ladder deliberately.&lt;/p&gt;

&lt;p&gt;Candidate loops to start with are the ones that are already semi-mechanical: internal reporting, churn analysis drafts, support categorization, post-incident summaries, compliance evidence gathering, release notes, and the countless "glue" tasks between tools.&lt;/p&gt;

&lt;p&gt;The strategic reframe is this: humans aren't here to be better typewriters. Humans are here to pick bets, handle novel cases, and design systems that make the routine disappear.&lt;/p&gt;

&lt;p&gt;Proposed executive decision:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Adopt &lt;strong&gt;time to closed loop&lt;/strong&gt; as a KPI alongside speed and quality.&lt;/li&gt;
&lt;li&gt;Require a "loop plan" whenever a team claims a workflow is "good enough" on a non-frontier model.&lt;/li&gt;
&lt;li&gt;Fund the boring parts (evals, logging, fallbacks). That's where the advantage is built.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Once we do this, "good enough" stops being a comfort and becomes what it really is: a signal to convert human effort into compounding automation.&lt;/p&gt;
    
        &lt;p&gt;Tags: &lt;a href="https://micahstubbs.net/tags/ai"&gt;ai&lt;/a&gt;, &lt;a href="https://micahstubbs.net/tags/automation"&gt;automation&lt;/a&gt;, &lt;a href="https://micahstubbs.net/tags/ai-strategy"&gt;ai-strategy&lt;/a&gt;, &lt;a href="https://micahstubbs.net/tags/engineering-leadership"&gt;engineering-leadership&lt;/a&gt;, &lt;a href="https://micahstubbs.net/tags/operational-excellence"&gt;operational-excellence&lt;/a&gt;, &lt;a href="https://micahstubbs.net/tags/kpis"&gt;kpis&lt;/a&gt;, &lt;a href="https://micahstubbs.net/tags/organizational-change"&gt;organizational-change&lt;/a&gt;, &lt;a href="https://micahstubbs.net/tags/executive-strategy"&gt;executive-strategy&lt;/a&gt;&lt;/p&gt;
    

</summary><category term="ai"/><category term="automation"/><category term="ai-strategy"/><category term="engineering-leadership"/><category term="operational-excellence"/><category term="kpis"/><category term="organizational-change"/><category term="executive-strategy"/></entry><entry><title>The Good Enough Signal</title><link href="https://simonwillison.net/2026/Jan/8/the-good-enough-signal/#atom-tag" rel="alternate"/><published>2026-01-08T10:00:00+00:00</published><updated>2026-01-08T10:00:00+00:00</updated><id>https://simonwillison.net/2026/Jan/8/the-good-enough-signal/#atom-tag</id><summary type="html">
    &lt;p&gt;I had a conversation recently with a colleague who commented that a certain model from mid-2025 was "good enough" for a code review use case. That got me thinking. What does "good enough" really mean?&lt;/p&gt;

&lt;p&gt;Here's the thing: if a last-gen model is "good enough" for some part of our work, that's not the pat on the back you might think it is. We're not done here.&lt;/p&gt;

&lt;p&gt;It's a &lt;strong&gt;warning light&lt;/strong&gt;. It means the work just crossed the line from "hard" to "&lt;em&gt;mechanical&lt;/em&gt;." And mechanical work doesn't stay expensive.&lt;/p&gt;

&lt;p&gt;In software, "good enough" is usually the moment a task becomes a commodity. Once the quality is acceptable, the &lt;strong&gt;cost collapses&lt;/strong&gt;. Not instantly, but inevitably. The teams that win are the ones who treat that moment as the starting gun.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Here's the rule:&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;&lt;em&gt;When a non-frontier model is good enough to do a task with light supervision, we should assume the fully automated version is close and start building it immediately.&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;This is less about AI hype than about &lt;strong&gt;leverage&lt;/strong&gt;. Human attention is our scarcest resource. Compute is not. If we spend human hours doing work that is now cheap to buy, we're burning the only thing we can't easily replace.&lt;/p&gt;

&lt;p&gt;The mistake most companies will make is to stop at "AI as a faster employee." They'll keep the same workflows and just have people type less. That's a temporary advantage. The durable advantage comes from changing the shape of the work: turning repeated human loops into software loops.&lt;/p&gt;

&lt;p&gt;Competitors don't need to be smarter than us. They just need to be more automated than us. Once a workflow can be done by a model, someone will do it with a model. That's what "software eats the world" looks like in 2026: not more apps, but fewer humans in the middle of the same processes.&lt;/p&gt;

&lt;p&gt;So what should our humans do?&lt;/p&gt;

&lt;p&gt;Work on the frontier. The frontier isn't "new tech for its own sake." It's the set of things the business needs that machines can't reliably do yet: deciding what to build, talking to customers, setting strategy, designing systems, handling edge cases, and—most importantly—building the machines that do the rest.&lt;/p&gt;

&lt;p&gt;The company should not be a craft shop. A craft shop is proud that humans touched everything. A software company should be proud that humans touched only the parts that still require judgment.&lt;/p&gt;

&lt;p&gt;The core KPI that captures this is &lt;strong&gt;time to closed loop&lt;/strong&gt;: the time between "a human can do this repeatedly" and "a machine does this automatically with monitoring and fallback." Shorter is better. It's the difference between using AI and becoming an AI-native organization.&lt;/p&gt;

&lt;p&gt;What I'm asking my team to do:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;Treat "good enough" as a trigger:&lt;/strong&gt; when we hear it, we open an automation project, not a celebration thread.&lt;/li&gt;
&lt;li&gt;Build the loop, not the demo: define an acceptance test, instrument it, and ship it into production with guardrails.&lt;/li&gt;
&lt;li&gt;Move humans up the stack: as soon as a loop is stable, remove humans from the path and redeploy them to the next unsolved problem.&lt;/li&gt;
&lt;li&gt;Track time to closed loop: for our top workflows the way we track revenue or uptime.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;If we do this, we won't just be using better models. We'll be turning better models into a &lt;strong&gt;compounding advantage&lt;/strong&gt;.&lt;/p&gt;
    
        &lt;p&gt;Tags: &lt;a href="https://micahstubbs.net/tags/ai"&gt;ai&lt;/a&gt;, &lt;a href="https://micahstubbs.net/tags/automation"&gt;automation&lt;/a&gt;, &lt;a href="https://micahstubbs.net/tags/ai-strategy"&gt;ai-strategy&lt;/a&gt;, &lt;a href="https://micahstubbs.net/tags/product-management"&gt;product-management&lt;/a&gt;, &lt;a href="https://micahstubbs.net/tags/engineering-leadership"&gt;engineering-leadership&lt;/a&gt;, &lt;a href="https://micahstubbs.net/tags/llms"&gt;llms&lt;/a&gt;, &lt;a href="https://micahstubbs.net/tags/operational-excellence"&gt;operational-excellence&lt;/a&gt;, &lt;a href="https://micahstubbs.net/tags/competitive-advantage"&gt;competitive-advantage&lt;/a&gt;&lt;/p&gt;
    

</summary><category term="ai"/><category term="automation"/><category term="ai-strategy"/><category term="product-management"/><category term="engineering-leadership"/><category term="llms"/><category term="operational-excellence"/><category term="competitive-advantage"/></entry></feed>