A customer wanted to return an air purifier. When asked for a name and ZIP code, the person typed “Daiki Johnson 80273". Asked which order it was, they said, “I am not sure.” The exchange took eight turns.

Given the same task, GPT-4o, playing the customer, offered a full sentence: “Sure! My name is Daiki Johnson, and my ZIP code is 80273.” It was done in three turns, signing off with “Thank you for handling this for me!”
The exchange was part of an experiment, but the difference matters outside the lab. In India, companies are building Artificial Intelligence (AI) voice and chat agents to handle customer support at scale. A 2024 survey by ServiceNow, which sells customer-service software, estimated that Indians spent more than 15 billion hours on hold in 2023. The question is whether the agents being built for them are tested against customers who behave anything like they do.
Many of these agents are tested using language models that are prompted to act as the customer. These “user simulators” generate conversations for testing and scoring agents and, increasingly, provide training data. τ-bench, a widely used benchmark for customer-service agents, uses a simulated customer. I previously examined the grader and the human standard in AI evaluation. The simulated user deserves the same scrutiny.
The air-purifier exchange appears in a 2026 Carnegie Mellon University study that examines what the authors call the ‘Sim2Real gap’. The team ran the full τ-bench protocol with 451 participants role-playing 165 airline and retail tasks and compared their conversations with those of 31 simulators. The simulated customers were “excessively cooperative, stylistically uniform,” and lacked “realistic frustration or ambiguity.” With most general-purpose simulators, agents scored higher than they did with people.
{{/usCountry}}The air-purifier exchange appears in a 2026 Carnegie Mellon University study that examines what the authors call the ‘Sim2Real gap’. The team ran the full τ-bench protocol with 451 participants role-playing 165 airline and retail tasks and compared their conversations with those of 31 simulators. The simulated customers were “excessively cooperative, stylistically uniform,” and lacked “realistic frustration or ambiguity.” With most general-purpose simulators, agents scored higher than they did with people.
{{/usCountry}}Just 1% of GPT-4o’s simulated turns were short, compared with 29% of human turns. Its turns were polite 49% of the time, compared with 15.3% for humans. Simulated users also rated agents more favourably than people did, potentially inflating evaluations that relied on that feedback.
But sounding like a customer is not the same as deciding like one. Liang Chen examined that difference in a second 2026 study, calling the two qualities communicative fidelity and decision fidelity.
The study used 2,790 real conversations between an AI sales agent and customers of a Chinese matchmaking service, 793 of which ended in a verified payment. Simulators were given the same conversation histories, and their next replies were compared with what the real customers said. The simulators came close to reproducing the behaviour of customers who eventually made a purchase. Among non-buyers, resistance fell from 25.1% in real conversations to 13.5% in simulated ones, while deliberation rose from 21.9% to 40.1%. As Chen puts it, “Real non-buyers say ‘not now’ and stop; simulated non-buyers ask about price.”
After a pitch, real customers deliberated 45.3% of the time, while simulated ones did so 75.2% of the time. An agent optimised against the simulator may be rewarded for applying pressure that real customers simply ignore.
Could a better prompt fix this? Only partly. When Chen told simulators they could disengage, the overall bias fell fivefold, and they largely stopped understating the resistance of customers who left. But they became more resistant across the board, including with customers who would have bought. In Chen’s words, the simulator “has learned to perform disengagement but still cannot tell who should disengage”. In a small test in the Carnegie Mellon study, a persona prompt written from the team’s own findings lowered the simulator’s fidelity score from 70.9 to 64.6.
Even a simulator that gets the average right can get particular groups of customers wrong.
There is reason to be especially careful in India. The Carnegie Mellon participants represented, in the authors’ words, “one mostly English-speaking US population.” Indian users may switch languages within a sentence, move between Roman and native scripts, or answer in fragments. A customer might write, “Order cancel kiya tha; refund abhi tak nahi aaya.” I have not found a published comparison of simulated and real users for any Indian language.
I propose a simple gate: The hang-up test. Before a team uses simulated results to make a launch or training decision, it should compare the simulator with real interaction records from the same product that were not used to build it, split by language and by whether the customer completed or abandoned the task. The test has three checks:
- Behaviour: Does the simulated customer stop replying, hang up, answer in fragments, leave out details such as an order number, switch languages and push back as often as real customers do?
- Reaction: Does it respond to the agent the way real customers do, for instance, by giving up after being asked for the same information twice?
- Prediction: Across several pairs of agent versions, does it reliably pick the same winner that real conversations do?
Teams should set tolerances in advance and report the result of each check alongside every simulated pass rate. By that standard, the GPT-4o simulator in the Carnegie Mellon study would fail the behaviour check on turn length alone: 1% of its turns were short, compared with 29% for people. Chen’s pitch results illustrate the kind of mismatch the reaction check should catch.
For a company choosing which agent to deploy, the third check matters most. It builds on the Carnegie Mellon authors’ advice to confirm with real people that a simulator’s rankings hold. A simulator that fails the first two checks but passes the third can still help a team choose between versions of an agent. It cannot tell the team how often that agent will succeed with real customers. The checks validate the simulator, not the agent. The agent itself still needs to be tested with real users before launch.
For companies already running support lines, the reference data may be readily available in anonymised records. Those records should include sessions that never became full transcripts, such as unanswered messages, dropped calls and chats abandoned after one turn. Otherwise, the test will miss the behaviour it is named for. Where there are no records, as before a first launch, simulated results can serve as a rehearsal, and a small live pilot should follow before the product is scaled up.
Simulators make it possible to test conversational agents at scale. But a high pass rate is less reassuring when the customer in the test is unusually patient, polite and willing to cooperate. India’s customers have already spent enough time waiting for support. They should not have to adapt to an agent trained for someone easier to serve.
(The views expressed are personal)
This article is authored by Spurthi Tallam, senior software engineer based in US.