Mistakes I Made Letting AI Build Production Code Unsupervised

AI writes code that looks finished fast. That's the actual danger — not that it writes bad code, but that "looks finished" and "is actually safe to put in front of real users" are two completely different bars, and it's easy to stop checking once the first one is cleared.

Here's what I actually got burned by, and what I do differently now.

Trusting "it works locally" as a finish line

I was using better-sqlite3 for Insidr early on. Worked perfectly on my machine. The moment it hit serverless, it fell over — the environment doesn't support what worked fine locally. I patched it with a JSON-file cache under /tmp as a stopgap, then properly migrated to Postgres once the product needed to actually grow. The lesson wasn't "SQLite is bad." It was: local success tells you almost nothing about production behavior, and I hadn't tested the actual environment the product would live in until it already had users.

Assuming a scheduled job running means a scheduled job is turned on

The code for a scheduled data-refresh job existed, was written correctly, and had been reviewed. It just wasn't actually turned on. Nobody noticed for about a day and a half, because nothing crashed — the data just quietly went stale while everything else kept running normally. I added a self-healing fallback and a visible "last updated" timestamp after this, specifically so a silent failure like that is now impossible to miss instead of possible to catch if I happen to look.

Reviewing code instead of attacking it

"Can you review this for security issues" is a completely different prompt from "try to actually break into this." I found that out the expensive way on Insidr — the full story is in How I Built Insidr. The instruction has to be adversarial, not polite, or you get a false sense of safety.

What I actually do differently now

Every product gets the adversarial security pass before real money or real user data touches it, not after. Every scheduled or background job gets a visible status indicator, not just working code. And "it works" is no longer a sentence I trust until it's worked in the actual production environment, under an adversarial review, with someone (or something) actively trying to break it.

AI being fast at looking finished is a feature. Treating "looks finished" as "is finished" is the mistake. That one's on me every time, not the model.