TCGCompareBot
TCGCompareBot is the crawler for TCGCompare, a price-comparison site for trading card games that links shoppers to retailers.
It identifies itself with this User-Agent:
Mozilla/5.0 (compatible; TCGCompareBot/1.0; +https://api.tcgcompare.com/bot)
What it fetches
Public product listings, to show each product's current price and stock. On Shopify stores that means the public /collections/{handle}/products.json and /products.json feeds.
It typically visits each store about once an hour. It paces its requests, honours any Crawl-delay, and backs off when a store returns HTTP 429.
What it does not do
- It does not visit checkout, cart, account or admin pages.
- It does not collect personal data.
robots.txt and request signing
TCGCompareBot obeys robots.txt. It looks for rules under the User-agent token TCGCompareBot (falling back to *) and honours Crawl-delay.
Every request is signed with Web Bot Auth (HTTP Message Signatures). Our public keys are published at https://api.tcgcompare.com/.well-known/http-message-signatures-directory.
Opting out
Add this to your robots.txt:
User-agent: TCGCompareBot Disallow: /
Or email [email protected] and we will stop crawling your store.