Bot protection blocking Google Shopping: how to fix it

Google’s help warns that a landing page can load fine for you and still fail for its automated systems, and names firewalls and US IP blocks as causes. Check your bot protection, rate limits and country rules, then test product URLs from outside your own network.

Critical4 min read

By Edvinas Ivanauskas How the guides are reviewed

Why this matters

Google's “How to fix: Landing page not working” page names the trap directly: “Even if your landing page loads successfully for you it might not be loading properly for our automated systems.” It asks you to “Check that your landing pages work properly for all locations, regardless of where you're targeting your item”, points to firewalls, especially US-based IP blocks, and server-side restrictions as things that get in the way, and suggests the URL Inspection tool in Search Console “to verify functionality and check for 403 server errors or firewall restrictions.”

Google's help page on product page access problems goes through the same list from the other side. A password or login in the way: “The URL provided is protected by some sort of authentication protocol that prevents Google from accessing the content.” A server too slow to answer: “The server didn’t respond in time”. A country block: the page says you must allow access from the US “because Google often operates from there”. And the landing page requirements ask for the same product page “regardless of the user’s device, user agent (including bots), browser, location, cookies, your ad targeting choices, or any other consideration”.

Bot protection is not the enemy. A challenge page in front of every product page is. Good bot protection lets verified Google traffic through. The trouble starts with a hand-made rule, a country block, a rate limit set for a spam attack, or an “under attack” mode left on long after the attack ended. None of it shows up from the office, because the office browser passed the challenge weeks ago.

The free scan loads your store from outside your network and reports a store that answers with a bot challenge, a queue page, a rate-limit page, a password or a coming-soon page as a failure. It does the same for a robots.txt that disallows the whole site. For the rules inside robots.txt, see robots.txt and Google Shopping; every access check is listed on the misrepresentation checker.

Typical evidence

A CDN rule can block a public check

A bot-management or WAF rule can treat automated requests differently from a normal browser. Test the public URLs with the documented tools for your stack and follow Google’s published verification guidance; StoreVerifier cannot reproduce private Merchant Center fetches.

request as

browser request

200

page responds

request as

automated request

403

blocked by bot rule

Cloudflare · Bot Fight Mode: ON can challenge automated requests. Review its documented allow-list options and verify requests safely.

Compare browser and documented test-tool responses. A 200 in one context and 403 in another is a configuration problem to investigate; it does not by itself establish cloaking or an account outcome.

The public signals this check looks for:

A security plugin or CDN shows a “checking your browser” page to visitors it does not reco…
A security plugin or CDN shows a “checking your browser” page to visitors it does not recognise, including automated ones.
An “under attack” or high-security mode switched on during an incident was never switched …
An “under attack” or high-security mode switched on during an incident was never switched off.
A country block meant to stop fraud also blocks every visit from the United States
A country block meant to stop fraud also blocks every visit from the United States.
A rate limit answers with 429 or a block page when many product pages are requested in qui…
A rate limit answers with 429 or a block page when many product pages are requested in quick succession.
The store sits behind a password or a coming-soon page while its products are live in Merc…
The store sits behind a password or a coming-soon page while its products are live in Merchant Center.
A queue page set up for a busy launch stayed in front of the store after the launch
A queue page set up for a busy launch stayed in front of the store after the launch.

What it looks like once it is right

The store keeps bot protection on but lets verified Google traffic through, drops the country block, and checks a product page with URL Inspection after every security change: 200, full page, no challenge.

Common mistakes

Common mistake

After a spam attack, the store turns on its CDN’s strictest mode and blocks traffic from outside the UK. The owner, browsing from the office, sees a normal shop; every visit from the US gets a challenge page.

Fix checklist

Test a product page, a collection page and your returns page from outside your own network: a phone on mobile data, an online URL checker, or a friend abroad. You should get the page, not a challenge.
Run the same URLs through URL Inspection in Search Console. A 403, a challenge page or a timeout there is the thing to fix.
In your CDN or security plugin, make sure verified search engine and Google traffic is allowed, and remove any custom rule that challenges every automated visitor.
Review country blocks. Google’s help says to allow access from the US; if you block countries to cut fraud, exempt verified Google traffic from the block.
Relax rate limits for verified Google traffic, so a burst of product page requests does not end in a block page.
Switch off “under attack”, queue and password modes as soon as they have done their job, and put the switch-off in the incident checklist.
After any security change, repeat the outside-network test and the URL Inspection check, and keep the results with the change record.

The fix may live one layer above the site

CDNs and WAFs can have documented settings for verified automated traffic. Review the provider’s documentation and Google’s published verification guidance, then test the public page with an approved tool. StoreVerifier cannot prove a private Merchant Center fetch.

Cloudflare — beforeautomated request blocked

Bot Fight Mode: ON — challenges all automated agents

'Known bots' allowlist: off

Cloudflare — aftertest request responds

Documented verification rule reviewed

Bot rule scoped to genuinely bad agents

$ curl -A "Googlebot-Shopping" yourstore.com/products/x

200same HTML a browser gets — no divergence

If a browser gets 200 and a documented test request gets 403, investigate the configuration. That difference alone does not establish cloaking or a Merchant Center outcome.

Questions merchants ask

Why does Merchant Center say my landing page is not working when it loads fine for me?

Google's “Landing page not working” help page covers exactly this: a page can load for you and still fail for Google's automated systems. It names firewalls, especially US-based IP blocks, and server-side restrictions, and suggests URL Inspection in Search Console to spot 403 errors or firewall restrictions.

Can I use Cloudflare or other bot protection with Google Shopping?

Yes, as long as verified Google traffic gets through and product, policy and checkout pages load without a challenge. Check the security level, custom rules, country blocks and rate limits, then test a product URL with URL Inspection after each change.

Do I need to allow visits from the US if I only sell in the UK?

Google's help on product page access problems says you must allow access from the US “because Google often operates from there”. A country block aimed at fraud can therefore stop Google reaching a UK store's product pages. Exempt verified Google traffic from the block.

Remediation

Risk signal

The hardest access problem to spot is the one that looks fine from your own desk. Test from outside after every security change, and let a scan show you what a stranger’s browser gets.

PriorityTreat this and any other highest-severity findings as first-priority work, then document each fix.

EvidenceRecord the current state before each change, apply the fix, then capture the corrected state so every change is evidenced.

Similar cases

Sources

Last reviewed 23 Sep 2026.

That is one issue. The library documents 134.

The free scan lists what it finds on your store. The paid report adds the affected pages, captured evidence and step-by-step fixes. Start free, with no account needed.