Rewriting product copy for AI helped broad searches and hurt exact ones
A new benchmark tested 15 ways to rewrite product descriptions so AI assistants recommend them. All 15 helped when the request was broad. 13 of 15 hurt when the shopper asked for exactly what the product is.
Many tools now promise to rewrite your product pages so AI assistants pick them. A new study asks a simple question. Does the rewrite still help when the shopper already wants exactly what you sell?
In this benchmark, mostly not. Rewrites helped on broad requests. On the most specific requests, most of them pushed the right product down the list.
What the study tested
Researchers at Renmin University of China and the National University of Singapore posted IntentGEO on arXiv on Nov 3, 2026. GEO means generative engine optimization: rewriting a description so AI assistants cite or recommend it.
They used 200 test items from four sets: books, retail, video games and a set called Ragdoll. Each item sat in a list of about nine candidates (9.26 on average). Each item was asked about in 10 ways, for 2,000 requests in all. The main answer model was gpt-4o-mini.
- Five levels of need (P1 to P5). P1 is a broad category keyword. P2 adds a topic, such as “a book about Arctic exploration saga”. P3 to P5 add more and more of the item’s own documented features, like a shopper who wants a lightweight laptop with long battery life but names no model.
- Five decision stages (Q1 to Q5). From “survey the options” down to “give me the single strongest pick”.
The score is rank gain. For the same request, how many places did the rewritten description move the item, compared with the original? Positive means it moved up.
Broad requests rewarded rewriting. Exact requests punished it.
At P1, the broadest level, all 15 methods moved the item up. One method, named Quotes, gained 1.48 places.
At P5, the most specific level, the same method lost 0.18 places. 13 of the 15 methods were negative there. The worst, named LLM Guidance, lost 0.70 places.
Only two stayed above zero at P5: a handwritten prompt called RecGEO (+0.01) and AMO (+0.06). AMO is the authors’ own method.
| Method (paper’s label) | Rank gain at P5 |
|---|---|
| Authoritative | −0.12 |
| Statistics | −0.18 |
| Citations | −0.14 |
| Fluency | −0.27 |
| Unique Words | −0.26 |
| Tech. Terms | −0.22 |
| Simple Lang. | −0.37 |
| Quotes | −0.18 |
| LLM Guid. | −0.70 |
| Content Impr. | −0.05 |
| RecGEO | +0.01 |
| AutoGEO | −0.29 |
| MetaOpt | −0.42 |
| MetaOpt-Prog | −0.42 |
| AMO (authors) | +0.06 |
For these methods, rewriting lowers visibility on average even when the source already gives the assistant concrete reasons to recommend the item.
One fair caveat. At P5 the original item already ranked about 2.5 on average, so there was less room to climb. The authors say this too.
Why the copy drifted
The authors also let an automatic optimizer rewrite its own instructions. They saved 36 updates. Only 3 of 36 kept the instruction to keep every fact. None kept the instruction to write as a recommender. All 36 asked for keywords and a call to action.
In the prompt it chose, “Preserve all factual information” became “Maintain factual accuracy”. The authors call this prompt drift: the copy moves from facts toward generic sales talk.
A separate AI judge (also gpt-4o-mini) scored each method’s rewrites from 1 to 5. On consistency, which checks that the item and its facts stay true to the source, the statistics-adding method scored 2.98. That was the lowest of the 15.
What this means for your product pages
- Keep the specs. The exact shopper is matching on facts. A rewrite that softens or drops them can cost you that shopper.
- Test with specific questions, not only broad ones. Ask the way a shopper who knows what they want would ask.
- Check any rewrite tool’s output against the original, fact by fact, before it ships.
- Publish the facts where agents read them. Shopify’s Fall ’26 Edition says that with knowledge base, agents that ask Shopify about a store get only facts the merchant has published.
- Measure before and after on the same question. A gain on broad asks can hide a loss on exact ones.
Limits of the study
- It is a benchmark, not live shoppers or live stores.
- Most results use one answer model, gpt-4o-mini. The authors also test Gemini 2.5 Flash-Lite, Qwen3-30B-A3B and Doubao.
- The P3 to P5 requests are written from the target item’s own features, so the match is built in.
- Lists held about nine candidates. Real AI answers can draw on far more.
- AMO is the authors’ own method. Its claimed 35% edge over the best earlier learning-based method is their result, not an independent one.
- It is a preprint on arXiv.
How Shopthru AI helps
Shopthru AI is an AI agents team for consumer brands. It asks ChatGPT, Gemini, Google AI, Perplexity and Copilot what your shoppers ask, finds where AI gets your products, prices or policies wrong, and drafts each fix into an approval inbox. You approve each fix. Autopilot is coming. Then it checks again. Instagram and Amazon are next.