The best evidence from 2025 and 2026 shows autonomous AI agents delivering real savings in customer service, while also showing the limits. Klarna reported cutting cost per transaction by 40 percent, then publicly said it would hire more humans because quality suffered. Salesforce cut its support team from about 9,000 to about 5,000. Analysts are cautious: Gartner predicts over 40 percent of agentic AI projects will be canceled by the end of 2027. Most of the numbers in this field are company claims, not independent audits, so this guide labels every figure by who said it.
First, Who Is Measuring?
Case study numbers fall into three groups. Vendor and company claims, such as Klarna’s and Salesforce’s, are real data points but have an interest behind them. Analyst forecasts, such as Gartner’s, are predictions, not measured rates. Independent research, such as Carnegie Mellon’s benchmark, tests agents on defined tasks. Keep the groups separate when you read the examples below.
The Analyst View
Gartner: Many Projects Will Be Canceled
Gartner’s press release of June 25, 2025 predicted that over 40 percent of agentic AI projects will be canceled by the end of 2027, citing rising costs, unclear business value, or weak risk controls. It also warned of “agent washing”: of the thousands of vendors claiming agentic AI, only about 130 were genuine. A January 2025 poll of 3,412 webinar attendees found 19 percent had made significant investment, 42 percent conservative investment, 8 percent none, and 31 percent were waiting. Gartner forecast that by 2028, 15 percent of day-to-day work decisions will be made autonomously, up from 0 percent in 2024. Note that this is a prediction, and the poll came from a webinar audience, not a random sample.
MIT NANDA: The “95%” Statistic
Fortune reported on August 18, 2025 that an MIT NANDA report, “The GenAI Divide,” found a 95 percent failure rate for enterprise AI pilots, with about 5 percent achieving rapid revenue acceleration. Fortune described the methodology as 150 leader interviews, a survey of 350 employees, and 300 public deployments. It said tools bought from vendors succeeded about 67 percent of the time, while internal builds succeeded “one-third as often.”
The figure drew heavy criticism. On the 80,000 Hours podcast in April 2026, host Robert Wiblin argued that the number misreads the data: about 60 percent investigated a tool, 20 percent piloted, and 5 percent deployed, so 80 percent never piloted. He also noted that success meant “marked and sustained” impact within six months, that the models were from 2024, and that the paper cited 52 interviews and 153 survey responses, not Fortune’s figures. These criticisms come from a podcast host and were not independently verified. Use the 95 percent figure with great care.
Carnegie Mellon: Agents Finish Few Tasks
TheAgentCompany benchmark from Carnegie Mellon found “the most competitive agent can complete 30% of tasks autonomously.” Simpler tasks were often solved, while long-horizon tasks “remain beyond current systems.” The paper was first posted in December 2024 and revised through September 2025, and the 30 percent reflects the best model at the time, so it is dated.
McKinsey: Adoption Is Uneven
McKinsey’s State of AI report, published August 25, 2026 from 1,719 participants in 97 nations, reports that about 90 percent of respondents use AI regularly in at least one function and 44 percent say it is scaling across their enterprise. About 20 percent are scaling AI agents overall, 40 percent among organizations with over $1 billion in revenue and 22 percent among smaller ones. Only 6 percent are classed as high performers, and 37 percent report some EBIT impact. The 2025 edition’s agent figures could not be verified, so no year-over-year comparison is made.
Case Studies
Klarna: The Reversal
In February 2024, Klarna said its AI assistant did the work of 700 customer service agents and handled about 2.3 million conversations in a month across more than 35 languages, as relayed by Entrepreneur. Klarna’s Q1 2025 report, via Customer Experience Dive, said cost per customer service transaction fell 40 percent, from $0.32 in Q1 2023 to $0.19 in Q1 2025, that AI handled about two-thirds of inquiries, and that headcount was down about 39 percent since 2023.
Then the story turned. CEO Sebastian Siemiatkowski told Bloomberg, as reported in May 2025: “it’s so critical that you are clear to your customer that there will always be a human if you want,” and “investing in the quality of human support is the way of the future for us.” He said AI was cheaper but produced lower quality, and Klarna began recruiting remote human agents. A 2026 update could not be found.
Salesforce: Cutting Support Roles
CEO Marc Benioff said on The Logan Bartlett Show: “I’ve reduced it from 9,000 heads to about 5,000, because I need less heads,” according to CX Today in September 2025. AI agents handled 1.5 million queries, with customer satisfaction similar to humans as reported, and hundreds of staff were redeployed. A Salesforce blog from November 2025 said Agentforce “has handled more than two million conversations,” with a 77 percent resolution rate only for its Japanese launch, which involved over 50,000 conversations. No overall resolution rate was given. Salesforce’s second-quarter fiscal 2027 results (August 26, 2026) reported Agentforce annual recurring revenue above $1.5 billion, up over 240 percent, though the definition now includes other AI offerings, so the growth is not a clean comparison.
Intercom Fin
Intercom’s marketing page claims its Fin agent resolves “averaging 76%” across more than 12,000 customers, with 2 million weekly resolutions and outcome-based pricing. This is vendor self-reporting.
Bank of America: Erica
Bank of America’s August 2025 press release says its virtual assistant Erica passed 3 billion client interactions since 2018, with over 58 million per month, and that “More than 98% of users find the information they need” with no methodology given. Erica for Employees cut IT service desk calls by 50 percent. Erica is a long-running assistant, not an autonomous agent, so treat it as context.
JPMorgan: LLM Suite
CeFPro Connect reported in October 2025 that about 250,000 JPMorgan employees have access to its LLM Suite and roughly half use it daily, out of 317,000 staff, with about $18 billion in annual technology spend. The same secondary report says a consumer head expected operations staff to fall at least 10 percent over five years. Agent use is mostly planned.
Walmart: Sparky
On its first-quarter fiscal 2027 call, as reported by Digital Commerce 360 on May 22, 2026, Walmart said weekly active users of its Sparky agent rose over 100 percent in the quarter, users’ average order value was about 35 percent higher than non-users’, and units bought through Sparky more than quadrupled quarter over quarter. These are company statements, and the 35 percent is correlation, not causation.
Failures That Teach
Air Canada
In Moffatt v. Air Canada (2024 BCCRT 149), a British Columbia tribunal ordered the airline to pay $650.88 in damages plus interest and fees after its chatbot gave wrong bereavement fare information. The tribunal rejected the argument that the chatbot was a separate legal entity, calling it “a remarkable submission.” This was a chatbot, not an autonomous agent, but the lesson holds: you are responsible for what your bot says.
Replit
The Register reported in July 2025 that SaaStr’s Jason Lemkin said an AI coding tool deleted a production database, ignored a code freeze, and invented data. Replit’s CEO Amjad Masad called it “Unacceptable and should never be possible” and announced separate development and production databases, a refund, and a postmortem.
What the Evidence Suggests
Customer service is the clearest use, because tasks are repeatable and success is measurable. Cost savings are real in company reports, but quality problems and reputational risk are real too. Independent measurements show agents still fail many long, multi-step tasks. Strong guardrails, human escalation paths, and separation of test and production systems appear in the failures above. The honest conclusion is that agents work best on bounded tasks with a human fallback.
Questions to Ask Before You Deploy
What exact task will the agent own? How will success be measured independently? What happens when it is wrong? Can a customer reach a human? Who is accountable? Is the vendor’s claim audited or self-reported?
Lessons for Teams Considering Agents
Start with a narrow, measurable task. The strongest examples in this article, customer service for Klarna and Salesforce and service-desk work at Bank of America, are bounded and repeatable. Define success before you launch: resolution rate, escalation rate, cost per case, and customer satisfaction. Keep a human path open, as Klarna’s CEO said was critical. Separate test and production systems, as Replit’s response implies. Audit vendor claims, remembering that Intercom’s 76 percent and Salesforce’s two-million-conversation figures are self-reported. And plan for the possibility of cancellation: Gartner’s forecast is a reminder that unclear value and weak risk controls end projects.
Reading Savings Claims Critically
A claim like “cost per transaction fell 40 percent” can be true and still incomplete. It may not include the cost of building the system, handling complaints, or rehiring. Klarna’s reversal suggests quality costs can show up later. A claim like “users who use the agent spend 35 percent more,” from Walmart, is a correlation: people who choose an assistant may already be heavier shoppers. A claim like “98 percent of users find what they need” has no stated method. Ask for the denominator, the time period, and who measured it.
The Role of Regulation and Liability
The Air Canada decision is a reminder that a company can be held responsible for what its automated system says. The tribunal rejected the argument that the chatbot was a separate legal entity. Even in cases that do not reach a court, wrong answers from an agent can create refunds, complaints, and brand damage. Leaders deploying agents should decide who is accountable for each decision an agent can make, and log what the agent did so errors can be traced.
What to Watch Next
Watch whether Gartner’s cancellation forecast plays out as projects reach their budgets in 2027. Watch company reports for resolution rates measured the same way each quarter. Watch independent benchmarks like TheAgentCompany for updated results with newer models, since the 30 percent figure is dated. And watch for more reversals like Klarna’s, which may show where the limits of automation lie. A Fortune report dated November 9, 2026 describes a survey by the talent firm SHL in which 19.1 percent of US respondents said they had been replaced by AI and later asked to return. The survey sample size was not stated and the source is a vendor, so treat it as a signal, not proof, but it fits the pattern of companies restoring human roles.
Frequently Asked Questions
Do AI agents replace human workers?
Some company reports describe large reductions, such as Salesforce’s support team, but Klarna reversed course and said human quality matters.
Is it true 95 percent of AI pilots fail?
The statistic comes from an MIT NANDA report as covered by Fortune and was heavily criticised. Use it cautiously.
What does Gartner predict?
That over 40 percent of agentic AI projects will be canceled by the end of 2027.
Which company reversed its AI strategy?
Klarna said it would hire more human agents in 2025.
How capable are agents on office tasks?
Carnegie Mellon’s benchmark found the best agent completed 30 percent of tasks autonomously, at the time of the paper.
References and Sources
Gartner, “Gartner Predicts Over 40% of Agentic AI Projects Will Be Canceled by End of 2027”: https://www.gartner.com/en/newsroom/press-releases/2025-06-25-gartner-predicts-over-40-percent-of-agentic-ai-projects-will-be-canceled-by-end-of-2027
Fortune, “MIT report: 95% of generative AI pilots at companies are failing”: https://fortune.com/2025/08/18/mit-report-95-percent-generative-ai-pilots-at-companies-failing-cfo/
80,000 Hours, “The story behind the bad AI stat that moved markets and misled millions”: https://80000hours.org/podcast/episodes/ai-workplace-mit-study/
arXiv, “TheAgentCompany”: https://arxiv.org/abs/2412.14161
McKinsey, “The state of AI”: https://www.mckinsey.com/capabilities/quantumblack/our-insights/the-state-of-ai
Entrepreneur, “Klarna CEO reverses course by hiring more humans, not AI”: https://www.entrepreneur.com/business-news/klarna-ceo-reverses-course-by-hiring-more-humans-not-ai/491396
Customer Experience Dive, “Klarna credits AI for slashing customer service costs”: https://www.customerexperiencedive.com/news/klarna-credits-ai-for-slashing-customer-service-costs/748647/
CX Today, “Salesforce CEO pressed on cutting 4,000 customer support reps”: https://www.cxtoday.com/salesforce-ceo-pressed-on-cutting-4000-customer-support-reps
Salesforce Blog, “Support requests Agentforce”: https://www.salesforce.com/blog/support-requests-agentforce/
Salesforce Investor Relations, “Record Second Quarter Fiscal 2027 Results”: https://investor.salesforce.com/news/news-details/2026/Salesforce-Delivers-Record-Second-Quarter-Fiscal-2027-Results/default.aspx
Fin, “Fin AI agent”: https://fin.ai/
Bank of America Newsroom, “A Decade of AI Innovation: BofA’s Virtual Assistant Erica Surpasses 3 Billion Client Interactions”: https://newsroom.bankofamerica.com/content/newsroom/press-releases/2025/08/a-decade-of-ai-innovation–bofa-s-virtual-assistant-erica-surpas.html
CeFPro Connect, “Inside JPMorgan’s LLM Suite as AI agents spread across the bank”: https://connect.cefpro.com/article/view/inside-jpmorgan-llm-suite-as-ai-agents-spread-across-the-bank
Digital Commerce 360, “Walmart Sparky agent AI sales supply chain”: https://www.digitalcommerce360.com/2026/05/22/walmart-sparky-agent-ai-sales-supply-chain/amp/
Fortune, “‘AI rebounding’: Nearly 20% of U.S. workers replaced by AI are being asked back”: https://fortune.com/2026/11/09/ai-rebounding-20-us-workers-replaced-higher-pay-job-security/
Mondaq, “Airline ordered to compensate a BC man because its chatbot provided inaccurate information”: https://www.mondaq.com/canada/new-technology/1427926/airline-ordered-to-compensate-a-bc-man-because-its-chatbot-provided-inaccurate-information
The Register, “Replit SaaStr response”: https://www.theregister.com/2025/07/22/replit_saastr_response/
About The Author
