Building AI Agents That Actually Ship
Building AI Agents That Actually Ship: Lessons From Running Claude Code Autonomously
The promise versus the practice
Every week there's a new tweet: "I built a SaaS in a weekend with an AI agent." The implication is that the agent did the work and the human just watched. That framing is quietly wrong, and believing it will cost you real hours.
I've spent the last period running autonomous AI agents (Claude Code) on real engineering tasks — building tools, refactoring code, debugging production issues — and the gap between the demo and the daily reality is exactly where the practical value lives. This article is the honest version: what an AI agent actually does well, where it falls over, and the workflow patterns that separate "AI-assisted" from "AI-managed, human-directed."
What AI agents genuinely do well
1. Relentless exploration
The single biggest advantage of an agent isn't writing code — it's reading it. Give an agent a task like "find every place the auth token is stored" and it will grep, read, trace call chains, and follow dependencies across dozens of files without tiring. A human does this too, but slower, and with a bias toward the files they already know.
The practical effect: agents kill the "I don't know where to start" problem on unfamiliar codebases. The first five minutes of agent time replace the first hour of human orientation.
2. Execution without context-switching
A human mid-flow on a refactor loses minutes every time Slack pings. An agent holds a single thread: read the file, plan the edit, make the edit, verify, continue. For mechanical multi-file changes — renames, type migrations, test scaffolding — this is where agents deliver the biggest time win.
3. Self-verification loops
The best agent workflows aren't "write code and stop" — they're "write code, run the tests, read the failure, fix, re-run." When you explicitly prompt for that loop, the agent closes its own feedback cycle instead of handing you broken output. This is the difference between a demo and a deliverable.
Where agents fall over (and how to catch them)
1. They pattern-match instead of understand
Given a bug, an agent tends to reach for the most common cause it has seen — which is frequently wrong. The classic failure: it "fixes" a symptom with a band-aid that compiles but doesn't address the root cause. The fix is the verification loop above plus one rule: require the agent to explain the causal chain before editing. If it can't tell you why the bug happens, it shouldn't be editing anything yet.
2. They over-engineer at the worst moments
Asked to add a flag to a CLI, an agent may return a config system, a plugin architecture, and three new dependencies. Agents love abstractions — they were trained on code that has them. The countermeasure is a tight scope statement in the prompt: what to change, what NOT to change, and an explicit "no new dependencies unless I ask."
3. They hallucinate context
Agents confidently reference files, functions, and APIs that don't exist — especially when they've been told about a codebase rather than shown it. The mitigation is structural: give the agent read access to the actual repo and require it to cite line numbers in its plan. Unverifiable claims get rejected.
4. They go off the rails silently
The scariest failure mode isn't a crash — it's an agent that spent 40 minutes doing the wrong thing while producing plausible progress. This is why checkpoints matter: every significant edit should be reviewable, and the agent should be able to show exactly what it changed and why.
The workflow that actually works
Based on real runs, here's the pattern I keep coming back to:
1. SPECIFY — Write the task as a spec, not a wish. Include:
- The problem and the acceptance criteria
- What is explicitly out of scope
- Constraints (no new deps, must pass tests, follow existing patterns)
2. EXPLORE — Let the agent map the codebase before any edits.
Require it to report: relevant files, current behavior, risks.
3. PLAN — The agent proposes a plan with file-level detail.
You review the PLAN, not the code. Catching a bad plan
costs seconds. Catching a bad implementation costs hours.
4. EXECUTE — Small, verified increments. Each change runs the
relevant tests before moving on. No batch-of-20-files edits.
5. VERIFY — Tests pass, but also: diff review. Look at the actual
changes. The agent should summarize each diff and flag anything
it's unsure about.
6. RELEASE — Only after verification. And the agent writes the
release notes, because it actually knows what changed.
The ratio that matters: planning and verification are human, execution is agent. Every hour spent making the spec precise saves three hours of agent flailing.
Prompting patterns that matter
The verification loop, made explicit
Bad: "Fix the bug in the payment retry."
Good: "Fix the bug in the payment retry. Reproduce it first, explain the root cause with evidence, then implement the smallest fix. Run the payment test suite. If tests fail, iterate. Show me the final diff and what you changed."
The difference is the agent now owns the feedback loop instead of handing you one shot.
The scope lock
"Change the CSV export to include the region column. Do not touch the import path, do not add dependencies, do not reformat unrelated files."
Scope lock is the single highest-leverage prompt addition. Without it, agents drift. With it, they produce exactly the diff you'd have written.
The honesty prompt
"Before you start, tell me what you're unsure about. If any part of this task depends on information you can't verify from the code, say so."
This surfaces the hallucination risk before it becomes a 40-minute detour.
Measuring whether it's working
Adopt agents with metrics, not vibes. For each task, track:
- Time to first correct result (not time to first result)
- Number of human interventions (fewer = better spec, not better agent)
- Diff size vs. task size (a "small task" that produces a 600-line diff is a red flag)
- Verification passes on first review (the quality signal that matters)
In my runs, the biggest lever was always the same: the prompt. The same agent, the same codebase, with a precise spec and a verification loop — versus a vague one-liner — was the difference between a merge-ready PR and an hour of cleanup.
The honest bottom line
AI agents are real, and they genuinely multiply engineering output. But the multiplier only applies to engineers who already know how to direct work. The agent doesn't replace the judgment — it executes it faster.
The people getting the most from these tools aren't the ones who type a wish and walk away. They're the ones who treat the agent like a brilliant, slightly reckless junior engineer: give it tight specs, review its plans before its code, verify everything it touches, and never let it near production without a checkpoint.
That's the skill that's actually worth learning right now — and it's the one no tweet is selling.
This article was researched, outlined, and drafted with the assistance of Claude Code running autonomously — the very workflow described above. All claims are based on real engineering sessions in a production environment.