How to test an AI sales agent before going live (without months of acceptance testing)

In AI sales agent projects, the first version is usually ready fast. What takes time is what comes after: the client needs to test, adjust, test again. And that's exactly where many projects stall. A week without testing becomes two, the person in charge gets pulled into other priorities, the enthusiasm cools and the project fades into irrelevance without anyone ever deciding to cancel it.

We've seen this pattern up close enough times to give it a name: the acceptance-testing swamp. This article explains why it happens and the method we use to get through it.

Why the testing phase stalls

There are four causes, and they almost always show up together:

1. Testing isn't part of anyone's routine. The project owner has a real job, with real targets. Between testing the AI and dealing with what they're being chased for today, the second option wins.
2. Nobody knows when testing ends. Without acceptance criteria, each round of testing generates new adjustments, which generate a new round.
3. Testing is unstructured. The tester chats freely, finds a problem, it gets fixed, and in the next round they test something completely different. Nothing gets closed out.
4. The testing tool is a chore. A slow simulation environment, away from the phone, forces people to stop what they're doing to test.

The principles of testing that actually ends

Define success before you start

What's the acceptance threshold? Examples: "the agent correctly answers the 20 most frequent questions", "runs the full qualification in 9 out of 10 scenarios", "never quotes a price outside the price table". With the criteria written down, testing stops being an open-ended conversation and becomes a check.

Write a test plan with hypotheses

A test plan lists the scenarios that need to be covered, who the lead is in each one and what's expected of the agent. It covers three groups: the happy path, objections and edge cases.

The all-in-one message scenario isn't there by accident. Testers in a hurry tend to send all their information in a single message, something the flow doesn't always expect. Better to find that out in testing than with the first real customer.

Stress-test internally before involving the client

The team that built the agent should run the entire plan before asking the client for an hour. The client's time is the scarcest resource in the project. It should be spent on what only they can assess.

Let the expert test what only the expert knows

In technical sales, such as a medical or financial product, nobody on the implementation team notices when an answer sounds like someone who doesn't understand the subject. Set aside a short block of testing for the client's expert to ask the hard questions, the ones that expose an unprepared salesperson.

Testing formats that get things moving

A guided session, ideally in person

Instead of sending the link and waiting, book an hour or two in the owner's calendar and test together. Bring the plan, the scenarios and the messages ready to go. In person works even better: sit next to the person, walk through the scenarios and leave the room with the adjustments written down and a go-live date. Online sessions, with the person splitting their attention with the rest of their day, yield much less.

A test number on WhatsApp

Testing in a simulation environment away from the phone is extra effort. Testing on WhatsApp, on a test number, is testing within the routine. The owner chats with the agent between meetings, forwards the number to colleagues, and test volume multiplies with no calendar cost. At the end of the conversation, the tester gives their rating right there.

AI testing AI, humans only evaluating

The most efficient format flips the work. Instead of the client playing the lead, another AI does it: different personas, with defined profiles, objections and behaviors, talk to the agent. The client receives the finished conversations and only evaluates: this went well, I didn't like this, this answer is wrong. Thumbs up, thumbs down, a comment.

This format solves three problems at once: the client spends minutes instead of hours, the scenarios are controlled and repeatable, and coverage is far greater than a human tester's.

Two weeks and a date

The cycle we've adopted is about two weeks of simulation before go-live: happy path, objections and edge cases, with every playbook reviewed before a real lead touches it. And, above all, with a date on the calendar.

The date changes everything. One way to run the conversation with the client:

• We want to start talking to real leads on Monday. What needs to be ready?
• The number needs to be connected and the templates approved. Who writes the opening messages, and by when? Who approves them?
• Is the integration with your system ready? If it isn't by Wednesday, we start without it and the agent runs in a degraded mode until then.
• You have until Thursday to test. On Friday we make the adjustments. On Monday, it goes live as it stands.

It sounds tough. But the alternative is the project that never goes live, which is worse for both sides.

Go-live checklist

• Acceptance criteria written down and met.
• Test plan run by the internal team.
• Expert testing block completed.
• Number connected to the official API and templates approved.
• Opening and follow-up messages approved by the client.
• Integrations tested, or a plan for running without them.
• Initial volume defined (start small).
• Monitoring routine scheduled for the first week.

After go-live, testing continues

Going live doesn't end validation. It changes shape. With real traffic, every conversation can be scored against the playbook, and every piece of client feedback follows a clear cycle: logged, diagnosed, fixed and closed. In the first weeks, it's worth looking at the numbers every day and using the agent diagnosis method to understand what's working and what isn't.

Frequently asked questions

How long should acceptance testing for an AI agent last?

Around two weeks is enough for most sales agents, as long as there's a test plan and acceptance criteria. Anything longer usually signals a lack of structure, not an excess of care.

Who should test: the manager or the salesperson?

Both, with different roles. The salesperson knows how real customers behave. The manager validates sales policy, pricing and tone.

Does testing with AI replace human testing?

It doesn't replace it, it reduces it. Simulation covers volume and scenarios. The human evaluates what only they can catch: tone, market nuances and technical information.

What if the client simply doesn't test?

That's why the date exists. Agree in advance that, without testing by the deadline, the agent goes live with reduced volume and daily monitoring. In practice, the date is usually what makes testing happen.