Month 3: Three Things Broke at Once

Sam Bell reviewing a failure log of his AI ops stack at his desk, Social Ads Mentor branding
The three failures that nearly sank my AI ops stack in month three, and the design decision behind each fix. Real build-in-public.

Month three is where most build-in-public stories go quiet. The wins get posted. The breaks get buried. This is the break.

By month three, my AI ops stack was "mostly built." That phrase should have been a warning. Mostly built means nothing actually runs without you standing over it. Three things broke that month, almost at the same time, and each one taught me something I should have known before I wrote a single line.

Break 1: the agent was over-scoped

Sal's original job description had eleven tasks. Eleven tasks, one agent, no isolation between them. When something failed, I could not tell which task had broken. Debugging meant reading through a 400-line system prompt that touched six different tools at the same time.

The failure was not that the model could not do eleven things. It could. The failure was that I had no way to see which of the eleven had gone wrong on any given morning. One agent doing eleven jobs is eleven points of failure wearing a single name.

Fix: cut to four core tasks. Anything beyond those four became a separate agent with its own scope and its own definition of done. One agent. One job. Clear boundaries. The scoping decision I should have made before I built anything.

Break 2: the data sources were not instrumented

Two sources had rate limits I had not planned for. One had API authentication that expired on a fourteen-day rotation, and I had not built in any alert for credential expiration.

The report ran. The data was wrong. Nobody on the team caught it for a week. A full week of ops decisions made on bad numbers, and the system looked perfectly healthy the entire time because it kept producing output. That is the dangerous kind of broken. Not the kind that throws an error. The kind that keeps smiling while it lies to you.

Fix: every data source now gets logged. Every pull gets a timestamp and a record count. A zero return or an authentication failure triggers an alert within fifteen minutes. The system is now allowed to fail, but it is not allowed to fail silently.

Break 3: the output went to a channel nobody checked

The report was generating correctly and posting to Slack every morning. Nobody was reading it. Wrong format, wrong channel. I had built the output around my mental model of how the team worked, not around how the team actually worked.

This one stung the most, because the agent was doing its job perfectly. The failure was entirely mine. I had automated a thing nobody asked for, in a place nobody looked, in a shape nobody could use.

Fix: I asked three people on the team how they actually check campaign health in the morning. Then I rebuilt the format around their answers. Twelve output fields became three. Read rate went from zero to daily inside a week.

The pattern: design, not capability

After fixing all three, the pattern was obvious. Every failure was a design problem. Not a capability problem.

The model could handle all eleven tasks. The architecture could not. The data sources could deliver the data. The instrumentation could not catch when they did not. The output was structurally correct. The format could not serve the people who needed it.

Design, not capability. Every single time.

This is the thing nobody tells you about building with AI. The model is almost never the bottleneck. The bottleneck is the scaffolding around it: how you scope the work, how you watch it, and who you build the output for. That scaffolding is not glamorous. It is also the entire job.

Month four was the first month the stack ran without a major fire. It held precisely because month three broke. Every design decision that made month four boring was forced by a failure in month three. If you want that reliability, you do not need a smarter model. You need tighter scope, real instrumentation, and an output built for the person who reads it, the same system I described in the automated daily ops report.

Why I am sharing the breaks

It would be easier to post only the wins. The clean dashboard, the report that lands at 8 AM, the time saved. But the wins are not where the lesson lives. The lesson lives in the three things that broke, because those are the decisions you can copy before you make the same mistakes.

If you take three rules from month three, take these. Scope every agent so tightly that you could write its job on an index card. Instrument every input so a silent failure becomes a loud one within minutes. And build every output for the specific person who has to read it, not for the version of your team that lives in your head. Do those three things and your month three will be boring, which is the entire goal.

Real build-in-public means sharing what broke, not just what shipped. The full failure log from months one through six, and the design decisions that fixed each break, lives inside The Mentor Lab. Drop your name and email and I will send you the complete teardown.

Share the Post:

Related Posts