Six Hours or Twenty Minutes: The One Question AI Never Asks Itself

A simple engineering habit that consistently produces order-of-magnitude gains in AI-generated test suites and evaluation harnesses.

Last night I asked an AI to build me an evaluation harness. It produced clean, working code. Then it told me the run would take about six hours.

I asked one question: can this be multi-threaded?

“Oh yeah, I can do that.”

The run finished in under twenty minutes.

That is not a rounding error. That is the difference between a feedback loop you can actually use and one that quietly kills your afternoon.

This is not a one-time fluke

I want to be clear that this was not a single lucky prompt. I have seen this pattern consistently, across every model I have used, since I started building with AI seriously.

Unless you ask, you get a single-threaded test suite. Every time.

The harness I ran last night was built with Fable 5 — a genuinely capable, mature model — for some internal work I was doing. It still needed to be asked. The same thing happens with OpenAI’s models. The same thing happens with Anthropic’s. Fill in the blank on whichever provider you prefer; the behavior does not change.

The models keep getting better at reasoning, at architecture, at writing code that actually works. But concurrency in a test harness does not seem to be something they reach for on their own.

Why the AI does not care and you should

I think the reason is simpler than it looks.

To the AI, six hours and twenty minutes are the same answer. Both produce correct results. Both satisfy the request. There is no experience of waiting, no context-switch cost, no afternoon lost to a progress bar. Wall-clock time is not part of what it optimizes for unless you make it part of the requirement.

It matters enormously to me. I do not want to sit there for six hours to find out whether a change worked. And in a real development loop, that six hours does not cost six hours — it costs every iteration you did not run because the loop was too slow to bother with.

That gap between “technically correct” and “actually usable” is exactly where human engineering judgment lives.

The questions to ask

None of this requires deep expertise. It requires remembering to ask. When a generated test suite or evaluation harness gives you a runtime that makes you wince, try these:

• Can this be multi-threaded?

• Are there parallel processes here that could run concurrently?

• Which of these tests are independent and safe to run in parallel?

• What is the actual bottleneck — CPU, I/O, or waiting on an external call?

• Can we shard this across workers?

These are the first questions any engineer asks about a slow system. They are not clever. They are table stakes. And they consistently produce order-of-magnitude improvements when applied to AI-generated harnesses.

The part that surprised me

I mentioned this to an engineer I work with recently. He was frustrated with a suite that took about an hour to run and had accepted it as the cost of doing business.

His reaction: “Oh my gosh, I never even thought I could ask this. It didn’t occur to me.”

He asked. Five minutes.

That is the thing worth paying attention to. It was not a knowledge gap — he is a strong engineer who knows exactly what parallelization is. It was a framing gap. When AI hands you working code, there is a pull toward treating it as finished code. You evaluate whether it is correct rather than whether it is good.

Human in the loop, doing what humans are for

I keep coming back to the same conclusion in different forms. AI is remarkable at production. It is far less reliable at knowing what to optimize for when you have not told it.

Human in the loop is not about babysitting the output for errors. It is about bringing a set of questions the model does not think to ask itself — questions rooted in what it actually costs a person to work with the result.

Will the models pick this up on their own eventually? Maybe. It seems like a solvable problem, and the trajectory has been steep on nearly everything else.

But right now, you have to ask.

It takes about four seconds. The payoff is measured in hours.