{"slug":"api-sitecheck-api-workers-dev-api-ai-crawlers-1c4109","title":"For eleven AI crawlers (GPTBot, ClaudeBot, PerplexityBot, Google-Extended and ot","host":"api.sitecheck-api.workers.dev","method":"GET","resource":"https://api.sitecheck-api.workers.dev/api/ai-crawlers","category":"code","description":"For eleven AI crawlers (GPTBot, ClaudeBot, PerplexityBot, Google-Extended and others): allowed or blocked for the page's path under robots.txt, plus the noai and noimageai opt-outs in meta and X-Robots-Tag, and whether llms.txt exists. Reads the page as static HTML; no JavaScript runs.","price_listed":0.002,"price_asked":0.002,"state":"answering","state_label":"Answering","checks_7d":1,"answered_7d":1,"latency_ms_median":37,"reported_calls_30d":2,"reported_payers_30d":2,"networks":["eip155:5042","eip155:8453","solana:5eykt4UsFv8P8NJdTREpY1vzqKqZKvdp"],"badge":"unverified","paid_checks_7d":0,"paid_ok_7d":0,"example_input":{"method":"GET","queryParams":{"url":"https://developers.cloudflare.com"},"type":"http"},"output_schema":{"$schema":"https://json-schema.org/draft/2020-12/schema","properties":{"input":{"additionalProperties":false,"properties":{"method":{"enum":["GET"],"type":"string"},"queryParams":{"properties":{"url":{"description":"Page address, https://…","type":"string"}},"required":["url"],"type":"object"},"type":{"const":"http","type":"string"}},"required":["type","method"],"type":"object"},"output":{"properties":{"example":{"type":"object"},"type":{"type":"string"}},"required":["type"],"type":"object"}},"required":["input"],"type":"object"},"history":[{"day":"2026-10-18","reachable":true,"status":402,"valid_402":true,"asked_usdc":0.002,"price_match":true,"latency_ms":37,"error":null}],"description_full":"For eleven AI crawlers (GPTBot, ClaudeBot, PerplexityBot, Google-Extended and others): allowed or blocked for the page's path under robots.txt, plus the noai and noimageai opt-outs in meta and X-Robots-Tag, and whether llms.txt exists. Reads the page as static HTML; no JavaScript runs.","last_updated":"2026-10-15T04:15:29.409Z","schemes":["exact"]}