SEOFree · no sign-up

robots.txt for Shopify: rules and tester

Shopify writes your store's robots.txt for you and you change it through the robots.txt.liquid theme template. Start from Shopify's main default rules (cart, checkout, account, sorted collections, search), add your own lines, and test any store URL against them.

Start from a preset

Disallow these paths (one per line)

Paths start with /. Use * for any characters and $ for the end, e.g. /*.pdf$

Allow these paths (exceptions, one per line)

Crawler groups Block AI training crawlers (GPTBot, ClaudeBot, CCBot, Google-Extended…), keep AI search bots Block all AI crawlers, including AI search (ChatGPT search, Perplexity…) Block SEO tool crawlers (Ahrefs, Semrush, Majestic, Moz)

Other bots (one per line: BotName: /path, /other)

Leave the paths empty to block the bot from the whole site.

Sitemap URLs (one per line)

Crawl-delay (seconds, optional)

Google ignores Crawl-delay; Bing and some others respect it.

robots.txt

User-agent: *
Disallow: /admin
Disallow: /cart
Disallow: /orders
Disallow: /checkouts/
Disallow: /checkout
Disallow: /carts
Disallow: /account
Disallow: /collections/*sort_by*
Disallow: /*/collections/*sort_by*
Disallow: /collections/*+*
Disallow: /blogs/*+*
Disallow: /*preview_theme_id*
Disallow: /policies/
Disallow: /search

Upload the file to your site's root so it opens at yoursite.com/robots.txt. robots.txt controls crawling, not indexing: to keep a page out of Google, use a noindex tag instead, and don't block that page in robots.txt (Google must crawl it to see the noindex).

Loading the tool…

How to use the Robots.txt Tester & Generator

1Enter your site to fetch its robots.txt, or switch to Paste and paste a file you're working on.
2Choose a crawler, or type any user-agent token, and enter the path or full URL to test.
3Read the verdict and the deciding rule; the table shows the same path for every common crawler.
4Edit the text to test a change before you upload it, or build a fresh file with the generator.

Frequently asked questions

Can I edit robots.txt on Shopify?

Yes, through the theme: in the theme editor's code view, add a robots.txt.liquid template and add your rules to it. Shopify then serves that version at yourstore.com/robots.txt.

What does Shopify block in robots.txt by default?

The admin, cart, checkout, orders and account pages, sorted and filtered collection URLs, internal search and theme preview links. These pages have no value in search results and waste crawl budget.

Should I block collection filter URLs on Shopify?

Shopify's defaults already block sorted and tag-combination URLs. Block extra filter parameters only if they create thousands of near-duplicate pages, and keep your main collection pages crawlable.

All tools

Then schedule it with Join Postly

Plan, publish and track your posts on every network from one calendar. Free plan, no credit card.

Start for free View pricing

No credit card required Free forever plan Cancel anytime

robots.txt for Shopify: rules and tester

How to use the Robots.txt Tester & Generator

Frequently asked questions

Related free tools

Then schedule it with Join Postly