Only 18 of 330 stores let an AI agent work out the full price
A research team scanned 330 online stores against one shopping job. At 178 the total check failed, and tax was the usual gap. Everything else on the page scored far better.
A paper posted to arXiv on Oct 12, 2026 measures something most agent studies skip. Instead of testing how good the agent is, it holds the job fixed and tests the store. The title is “Get In, Decide, Prove: A Merchant-Side Readiness Benchmark for Agentic Commerce.”
One result stands out. Under the paper’s main test, the all-in total, with tax and shipping, could be decided at 18 of the 330 merchants. This post goes through what that means, and what it does not.
What the researchers did
The 330 merchants are online stores, nearly all retailers, selling in the US and Canada. They came from three lists: 227 stores on one hosted commerce platform and 13 partner merchants, both supplied by a commercial partner of the authors’ company that runs a consumer shopping agent, plus a list of 90 large North American retailers.
Every store was assessed against the same buyer job. The paper calls it a mandate: quantity one, delivery to ZIP code 10001, a budget of the displayed price plus 25%, delivery within 10 days, and returns within 14 days. The scanner chose one offer page per store, read it and the linked policy pages, and ran 38 named checks.
One point matters for how you read the headline. The statuses were assigned by fixed rules. The authors say no language model read a page. So this is a scan with rules, not an agent that tried to buy. A check comes out as “unknown” when the page could not be read, when no page was found, or when the page was read and the rules could not decide.
Tax was the usual gap
Under the plain web-page reader the paper calls the HTTP client, the all-in total was determinate at 18 merchants. It failed at 178. Of those 178, 137 failed on tax and 41 on shipping.
The paper says tax failed when it was not stated on the offer or policy pages, was stated without a rate, or was deferred to checkout. A page that says tax applies, but gives no amount or rate, leaves the total open. The paper does not use a published tax rate to fill the gap.
The other 134 stores were not a pass and not a fail on this check in that passage. The paper’s count, “determinate at 18 merchants,” is what we can quote. It would overstate things to say the agent “couldn’t” at the other 312 stores. At 178 the check failed, and the rest could not be decided.
The rest of the store was fine
Only 103 of the 330 profiles under the HTTP client had enough evidence to be ranked. The paper’s rule is 90% coverage overall and 80% in every pillar. Among those 103, four of the five pillars scored 95 or more on average. The paper’s five pillars are access and discovery, offer identification, material terms, observable operability, and evidence and recourse. Everything except material terms scored 95 or more.
The fifth pillar, material terms, averaged 62 under the HTTP client. Material terms means total cost, delivery and return terms. The authors call it the one pillar well below the others. Most of its failures came from the all-in total check.
Shoppers feel this too
Baymard Institute keeps a list of cart abandonment research. In its latest quantitative study, 42% of US online shoppers said they had abandoned a cart because they were just browsing. Ignoring that group, the remaining reasons are led by extra costs: 40% said extra costs such as shipping, tax and fees were too high. Another 12% said they could not see or calculate the total order cost up front.
A person who sees “tax calculated at checkout” may shrug and continue. An agent given a hard budget has a different problem. The paper says a gap like this leaves an agent with a budget “unable to proceed, or able to proceed only by assuming.” Whether such gaps lead to failed purchases is not something the pilot can show.
Caveats the authors give
- It is a pilot. The authors say it “does not validate” the measurements or rank merchants.
- One observation of one offer page per merchant and profile. The scan ran from Ontario, Canada. Under the HTTP client, 30 merchants served prices in Canadian dollars or pounds.
- The rules mostly under-score. In an audit of 192 material-terms verdicts, an AI model acting as one rater agreed with the rules on 120 (62%). The authors say the rules mostly failed terms that the page states.
- The authors work for Evolutionairy AI, which they say has a commercial interest in merchant assessment.
- The merchants are not named.
The gap the authors describe is ordinary. They write that none of these gaps needs a new protocol to repair. A store can state its tax basis, rate and shipping cost in text.
What to do this week
You can run a smaller version of this test yourself. Tick these off as you go.
How Shopthru.ai helps
Shopthru.ai is an AI agents team for consumer brands. It asks ChatGPT, Gemini, Google AI, Perplexity and Copilot what your shoppers ask, finds where AI gets your products, prices or policies wrong, and drafts each fix into an approval inbox. It ships only on your yes, then checks again. Instagram and Amazon are next.