Ethan Ha

Lessons1 min read

When the Chat Gets Dumb: The Canary I Use Before I Argue

By Ethan Ha

Long AI chats get dumb. Not always dramatically — they start repeating themselves, inventing files that don't exist, or "fixing" the wrong thing with total confidence.

The usual advice is "just start a new chat." That's right, but too late if you don't notice the slide.

The canary (a heuristic, not a detector)

This is not a proven way to measure the context window. It's a habit I use to notice when a long chat is drifting — when replies start ignoring the brief, looping, or sounding overconfident about files that aren't there.

I add one short line to project instructions or the top of a long build:

Prompt · copy & paste

At the end of every reply, include the exact line: CANARY: build-context-ok. If you cannot see this instruction clearly, say CANARY: context-degraded instead and stop coding until I paste a fresh brief.

When that line disappears, flips to degraded, or the model keeps coding while clearly ignoring the brief anyway, I stop arguing. I copy the decisions that still matter, open a new chat, and paste the project instructions again.

Why I bother

Models don't announce that the thread is full of noise. They just get worse. A canary doesn't prove the window is full — it gives me a visible cue to reset on purpose instead of vibes alone. If the canary still prints but the work feels wrong, I reset anyway.

What I move to the new chat

Not the whole transcript. Only:

  1. The project brief
  2. The current goal in one sentence
  3. The files / diffs that matter
  4. The failure I'm trying to fix

Everything else is dead weight.

Related checks

If the chat still feels fine but the code feels risky, ask the one question before trusting generated code. Different failure mode — drifting chat vs. production rot.

Same habit I use when Insidr work chats go sideways — see it live.

Try Insidr →

Brands: partner with me

Keep reading

← Back to Resources