I Built a Log Analysis MVP With AI in One Session
I Built a Log Analysis MVP With AI in One Session — Here's What Actually Happened
The setup
I'd just finished shipping a small developer tool (a disk-cleanup CLI) and wanted to test whether the "build with AI agents" workflow held up on a second, more ambitious project: a log analysis tool with actual anomaly detection. The market signal was real — small DevOps teams spend hours manually reviewing logs, and the tools that automate it (Datadog, Splunk) price themselves out of reach for teams of 2–20. The gap was a 9.0 out of 10 on a market-gap analysis of 1M+ real user complaints.
This is the honest account of that build: what the AI agent did well, where it went wrong, and what the final product actually cost in human attention.
What the agent did well
1. It turned a vague idea into working code fast. The first pass — log parsers, level categorization, a CLI — took minutes of wall-clock time. The agent produced a coherent, working skeleton: nginx access-log parsing, ISO-timestamp application logs, level-first variants, graceful fallbacks for unknown lines.
2. It wrote the boring parts without complaint. The HTML report generator, the README, the JSON output mode — all the parts that make a tool usable but that are painful to write from scratch. The agent doesn't get bored. That's genuinely valuable.
3. It held the whole file structure in context. Parser module, analyzer module, report module, CLI entry, tests — the agent kept the interfaces consistent across five files without me having to re-explain anything.
Where it went wrong (the honest part)
1. The anomaly detector was self-defeating — and the bug was subtle. The first implementation flagged a time bucket as an anomaly when its error count exceeded the mean + 3 standard deviations of all buckets. Sounds statistically reasonable. But the spike itself inflates the mean and the standard deviation — the more extreme the spike, the harder it is to detect. This is a known failure mode of using non-robust statistics for outlier detection. The agent wrote it confidently, and the test suite caught it.
The fix: switch to the median as the baseline, flag a bucket at ≥3× the median. Medians are robust to outliers by construction — a spike can't pull the baseline toward itself. This is the same principle monitoring tools use in production, and it's the difference between "catches real outages" and "never fires because the outage moved its own goalposts."
2. The test fixture was wrong, and it took me a while to see it. The anomaly test generated 10 errors per minute in the "baseline" minutes and only 8 in the "spike" minute — meaning the spike minute had fewer errors than baseline. No algorithm in existence could have flagged it. The failure wasn't the analyzer; it was the test author (me, via the agent) encoding a broken mental model. Fixing the fixture to 2 baseline errors vs. 8 spike errors made the whole test suite green.
The lesson: when a test fails, check the test's assumptions before the implementation. A failing test is two possibilities: the code is wrong, or the test is wrong. Both happened here, in the same session.
3. The agent over-engineered at the edges. Asked to "handle directories," it proposed glob expansion, recursive traversal, and multiple input modes. The MVP needed "read this file or all files in this directory." Scope-locking the prompt fixed it.
The workflow that made it work
The same loop from my earlier build held up:
- Spec the MVP tightly — formats to parse, outputs to support, what NOT to build
- Let the agent explore and write — but require it to run the tests itself
- Review the diff, not the vibes — the anomaly detector looked right in a quick skim; only the test run exposed it
- Fix root causes, not symptoms — median-based detection isn't a patch on the mean-based code, it's the correct algorithm
- Verify end-to-end with real data — a hand-crafted sample log with a planted spike: 10 errors in one minute against a baseline of ~2. The tool flagged exactly that minute. That's the moment a tool goes from "code" to "thing that works."
What the final product looks like
One command. A directory of logs in, a shape of the problem out:
$ logwatch sample-data/app.log --errors
31 lines parsed from 1 file(s)
error rate: 51.61%
⚠ Anomaly spikes detected:
10 errors in 2026-08-19T10:03 (baseline ~2.0)
Error lines:
2026-08-19T10:03:01.000Z app stripe webhook failed: connection refused
...
The value isn't the counts — grep -c ERROR does that. It's the baseline and the spike: knowing that 10 errors in a minute is abnormal because the normal rate is 2. That's the thing the small team is missing when they can't afford Datadog.
The meta-lesson: agents amplify judgment, they don't replace it
This session is a clean case study in the three-tier model of AI-assisted development:
- Autocomplete wrote none of this. The work was too novel.
- Chat-assist would have explained code but not generated a coherent multi-file tool.
- Agents did the heavy lifting — and made two significant errors that only surfaced because there was a test suite and a human who distrusts green lights.
The agent's speed is real. But the agent's confidence is not a signal. The anomaly-detector bug would have shipped in a tool with no tests and a developer who assumes "it ran, so it works." The tool has 12 tests, and two of them failed during the build — each failure was a real bug or a real misunderstanding.
That's the workflow that actually ships: agent writes, tests verify, human judges.
What's next
The MVP is done and committed: parsers, anomaly detection, terminal/JSON/HTML output, 12/12 tests passing. The paid evolution — a hosted dashboard, Slack alerts, multi-source ingestion — is the SaaS path, and it's blocked on the parts that need a human (accounts, billing, identity). The local tool, the tests, and this article are all things an AI can produce autonomously.
The next experiment in this series: turn the same pattern — scan, analyze, report — into a third tool, and see if the template holds. Because at this point, the template is the product.
This article documents a real build completed with AI-assisted development in a production environment. The full source is in the companion repository, including the test suite that caught the anomaly-detection bug.