AI agents in companies: what works in production
Support, lead qualification, documents, internal tools. Where AI agents hold up in production, where they break, and how to measure that one works.
5 min read
Most agent demos work. The question is what happens on the 400th request, on a Tuesday, when the customer writes in half French, half English and attaches a blurry photo of an invoice.
An agent is a model that reads a request, decides what to do, calls your tools and acts. That last part is what changes the project.
A chatbot that answers badly is a bad answer. An agent that acts badly is a refund issued twice, or a lead sent to the wrong sales rep.
If someone on your board has asked "what about agents" this quarter, this is the short version: four use cases that hold up, four ways they break, and three things to have before one goes live.
AI agent use cases that hold up in production
The use cases that survive have the same shape. The task is frequent, the inputs are messy, and a person can check the output faster than they could produce it.
Customer support. The agent reads the ticket, pulls the order or the account, and either answers or drafts an answer for the team. Order status, address changes, return rules: the questions that fill the queue every morning. It does not need to close every ticket: taking the repetitive part off the queue already changes the team's week.
Lead qualification. A form comes in, or an email. The agent enriches it from what is public, checks it against your criteria and routes it: to sales, to a nurture sequence, or nowhere. Sales reps stop reading every inbound to find the three that matter.
Document processing. Invoices, purchase orders, contracts, ID documents, claims. The agent extracts the fields, checks them against your systems and flags what does not match. This is often the least visible use case, and often the one that pays back first, because the volume is high and the work is dull.
Internal tools. An agent that answers "where is the latest pricing grid" or "what did we agree with this client in March" from your own documents and tools. It works when the sources are clean and the agent cites them. It fails quietly when they are not.
Where they break
Agents fail in predictable ways. Knowing them in advance is most of the work of scoping one.
The first is the long tail. An agent that handles most cases well can still be a bad product if the rest are the expensive ones: the angry customer, the contract with a non-standard clause, the lead that is actually a competitor. You need to know which cases those are before you decide what the agent is allowed to do alone.
The second is data: if the order history lives in three systems that disagree, the agent will disagree with itself. No model fixes that.
Sometimes the first useful project is a clean API on top of the data, and the agent comes second. That is less exciting to announce, and it is what makes the agent work.
The third is scope creep. An agent that does one task well gets asked to do five. Each new tool it can call adds ways to fail, and the evaluation you built for the first task says nothing about the others.
The fourth is cost: each step an agent takes is a model call, sometimes several. A workflow that chains ten calls per request looks cheap in a demo and very different at 20,000 requests a month. Much of that cost goes away with a smaller model on the easy steps and a cache on the repeated ones, but only if someone measures it before launch.
How to measure that an AI agent works
"It looks good" is not a measure. Before an agent goes live, three things should exist.
A test set from real cases. Take a few hundred real requests from the last few months, with the answer a good employee would have given. Run the agent on them before every change to the prompt, the model or the tools. Without this set, every improvement is a guess, and every model update is a risk you cannot see.
A clear line for human review. Decide what the agent does alone and what it only proposes. A common split: the agent acts alone on low-risk, reversible actions, and drafts everything else for a person to approve in one click. Track how often people change the draft, and when that rate drops on a category, let the agent act alone on it.
A cost per task. Not the monthly API bill: the cost of one ticket handled or one invoice processed, counting model calls, infrastructure and the minutes a person spends reviewing. Put it next to what the same task costs today. If that number does not hold up at your real volume, the project does not either, however good the demo was.
Test set, review line, cost per task. If one is missing, the agent is not ready.
Once it is live, keep measuring: log every run, sample a few each week and re-score them. The model provider ships a new version, and the refund question that worked last month now gets a polite, wrong answer. An agent nobody watches gets worse without anyone noticing.
What a first version looks like
A good first agent is narrow. One task, one team, a handful of tools, a review step that the team actually uses.
That is enough time to connect the agent to the real systems, build the test set with the team that does the work today, and put it in front of real requests with a person reviewing. It is not enough time to automate a whole department. The first weeks in production tell you more than any pilot.
Two questions come up early. The first is where the data goes: it can stay hosted in Europe, in your cloud or on-premise, under your rules. For an agent this matters more than for a chatbot, because it reads your orders, contracts and customer records to do its job (more on data, GDPR and the AI Act).
The second is who runs the agent after launch. Someone has to own the test set, the review queue and the cost per task. If nobody on your side has that job yet, plan for it from the start: that is why we run what we build, with the same people who built it.
Start with one task
The useful first question is not "where can we use agents" but "which task does a person here repeat a hundred times a week". The free audit is one hour on that question, followed by a short document that says which tasks are worth an agent and which are not.
Book a free audit and bring the task you would hand off first. If an agent is the wrong answer, the document will say so.
