The demo-to-production gap

There's a moment with every AI feature where it works in the demo and everyone gets excited. That moment is a trap. The demo proves the happy path exists. It says nothing about the hundred unhappy paths a real user will find in the first afternoon.

The gap, concretely

The demo handles the input you typed. Production handles:

• the empty input, the 50-page input, the input in the wrong language
• the model being down, slow, or rate-limited
• the answer that's confidently wrong and needs a citation to catch
• the user who pastes secrets into the box

None of that is glamorous. All of it is the actual product.

What closes it

The teams that ship don't have better models. They have better plumbing around the model: input validation, graceful failure, a retry budget, an eval suite that runs on every change, and a human-in-the-loop escape hatch for when the machine isn't sure.

I'll walk through the exact plumbing — with code — in the members how-tos. But even if you never read those: assume the demo is the easy 10%, and budget for the 90% up front.