From POC to production: why most AI prototypes stay in a drawer

The demo went well. The model answered the five questions everyone had prepared, the room nodded, and someone said "let's roll it out." Six months later, the prototype is still on a laptop, and the person who built it is on another project.

Meanwhile the CEO, who saw it work, keeps asking why it is not live yet. We hear this story often in first calls about AI. The problem is rarely the model. It is everything the demo did not have to do.

An AI proof of concept answers one question: can this work at all. It is allowed to cheat. It runs on twenty clean documents, one user, no login, and an API key pasted into a config file.

The trouble starts when the demo is treated as most of the product. The demo is maybe a tenth of the work. The rest is data pipelines, tests, access rights, logs and billing alerts.

None of it shows in a demo, and all of it decides whether the thing ships.

Real data, not a curated folder

The demo used a folder someone curated by hand. Production uses whatever your systems actually hold: scanned PDFs, duplicates, three versions of the same contract, a CRM where half the fields are empty.

The model that looked sharp on clean inputs starts guessing. Answers drift, sources go missing, and users stop trusting it after the second wrong reply. They rarely tell you; they just go back to the old way.

So the first production task is unglamorous. Find where the data lives, who owns it, how often it changes and how it gets to the model. On many projects this takes longer than the AI part, and it should be planned that way from day one.

Evaluation: nobody can say if it is good

Ask the team behind a stalled prototype how accurate it is. The honest answer is usually "it seemed fine in the tests." Nobody can sign off on that.

A product needs an evaluation set: a few hundred real cases, with the answer a competent person would give. Start from questions real users asked during the pilot, not the ones the team prepared for the demo: those are the ones the demo was tuned on.

Every change to the prompt, the model or the data runs against it. Without it, each fix breaks something you will only hear about from a customer.

It also forces a conversation the demo skipped: what error rate is acceptable for this task. A draft email that is wrong one time in ten is useful. A refund decision that is wrong one time in ten is a liability.

What the demo never had to handle

Nobody is watching it

A prototype has no monitoring because nobody depends on it. In production someone will, and the first bad answer will reach you as a screenshot in a support ticket, with no log to explain it. You need logs of what the model received and returned, alerts when latency or error rates move, and a way to replay a bad answer to understand it.

The model you tested will also change under you: providers retire versions and update them, and without the evaluation set you will not notice what broke.

The bill nobody put in the demo

A demo runs a few dozen requests a day. Put it in front of every employee, or on every customer request, and you multiply that by a thousand, with longer contexts than the demo ever saw. Caching, smaller models for simple steps and limits per user are design decisions, and they are cheaper to make before launch than after the first invoice.

Before launch, estimate the monthly cost at production volume, from the real size of requests, not from the demo's bill.

Legal will ask questions the demo cannot answer

Before launch, someone has to answer three questions:

• Who can see which documents through the assistant, and does it respect the access rights of each source.
• Where prompts and answers are stored, for how long, and in which country.
• Whether a user can make the model reveal what it should not, through prompt injection or a badly scoped search.

Your legal and security teams will ask. The prototype has no answers because it never had to. Decide hosting first (Europe, your own cloud or on-premise), because it constrains which models you can use.

Ownership: someone has to run it in production

This is the least technical reason, and the one that kills the most working prototypes. The prototype was built by an innovation team, an intern, or an agency on a fixed budget. When it works, nobody is named to run it.

A product in production needs an owner: someone who reads the logs, updates the evaluation set, handles the model provider's next price change and decides what to improve. Without that person, the system slowly degrades, and the business quietly stops using it.

This is also where most handoffs fail. The people who built the demo leave, the people who inherit it did not make the choices, and the first incident takes weeks to understand. We work the other way: the people on the first call build and ship the product, then run it after launch or hand it over cleanly to your team, with the documentation and the evaluation set that make it maintainable.

What to do with an AI POC you already have

Before anyone writes more code, forward these six questions to the team that built it:

Most stalled prototypes do not need to be rebuilt. They need a written scope that names what is missing between the demo and the product, and an honest answer on whether the use case is worth it. Sometimes it is not: the volume is too low, the data does not exist yet, or a simpler tool does the job.

That is a useful answer too, and it is cheaper to get it before the second budget than after. When it is worth it, the path is short: a scope that says what we build and what we do not, a demo every week, and a first version in production within 3 to 6 weeks of kick-off.

If you have a prototype in a drawer, bring it to a free audit: a recording of the demo is enough, the repo and a data sample if you can share them. One hour on your business, then a short document that says what it takes to ship it, or why it should stay where it is.