The AI agents that actually reach production

We built nine AI agents for clients over the past year. Four are running in production today. Three were replaced by a much smaller piece of software once we understood the problem. Two never should have been started, and that is the more useful half of this issue.

The four that survived had a narrow job

Every one of them replaced a task somebody was already doing by hand, every day, on a schedule: reading invoices, drafting the first version of a contract clause, triaging inbound requests, reconciling two systems that were never meant to talk.

None of them were asked to be clever. They were asked to be the first draft of something a person would check anyway. That is the shape that works — the human stays in the loop not as a safety net bolted on afterwards, but as the actual design.

The two that failed were asked to decide

Both had the same brief, in different words: let the agent handle the whole thing and escalate when it is unsure. Models are poor judges of their own uncertainty, so "escalate when unsure" quietly becomes "never escalate".

The fix in both cases was unglamorous. We wrote down which decisions the business was actually willing to hand over, and it turned out to be a short list. Everything else became a draft for a person to approve.

What we ask before starting one now

• Who does this task today, and how long does it take them?
• What does a wrong answer cost, and who notices it first?
• Is there a rule-based version that gets us 70% of the value this month?
• If the model were unavailable for a day, what happens?

The third question kills more agent projects than the other three together, and it saves the budget for the ones worth building.

If you are weighing one up

We wrote the longer version of this argument, with the cases where automation does pay off, on the site.

AI automation for business