Trust ·
Which AI shopping agents are on your bot-protection allow list?
Under an hour ChatGPTGooglePerplexity
Short answer
Ask security for the named agents your wall admits. Blend Commerce found 29 of 200 retailers refused its shopping agent twice, even though every readable robots.txt among those refusals allows ChatGPT, Perplexity and Claude crawlers.
Robots.txt and bot protection are different doors. A store can welcome AI search crawlers in robots.txt and still challenge or blank a browser-style shopping agent at the edge. Blend's AI Shopping Audit 2026 (September 8 to 15, 200 UK and US retailers) recorded that gap: 14.5% refused the agent twice, and of the 24 refusals with a readable robots.txt, all 24 allow those crawlers. The fix is a short conversation, not a rewrite of product copy. Ask who set the rule, which named agents are allowed, and what happens to an honest but unfamiliar automated visitor. Then test one real agent task on your own product page.
Do this
- Open a ticket with security or infrastructure: list which AI shopping agents are on the allow list today, and who owns that list.
- Compare that list with the assistants your marketing team is trying to show up in (ChatGPT, Perplexity, Claude, Gemini and any retailer agents you care about).
- Have someone attempt one product-to-cart task with an agent from a residential network, and record whether the product page loads.
- If the page fails, ask security whether the block is intentional policy or a default rule that never got a revenue review.
- Repeat the check after any CDN, WAF or bot-protection change.
Effort is our estimate for a small brand team. Assistant tags come from the assistants this tip names. See also AI shopping assistants compared.
Read more on the blog
- Shoppers let AI choose but not pay. CPG brands should design for the handoff
- Your robots.txt says AI is welcome. Your bot wall says no.