Crawl Policy Index

Look up

svt.se

Any site on the 100,000 we watch. No account.

svt.se

News · Sweden · Swedish

svt.se blocks OpenAI's training crawler, Anthropic's training crawler, and Google's Gemini opt-out. It still allows Perplexity's crawler, ChatGPT when a person asks, and Google Search.

  • GPTBot OpenAI's training crawler
    Blocked
  • ClaudeBot Anthropic's training crawler
    Blocked
  • Google-Extended Google's Gemini opt-out
    Blocked
  • PerplexityBot Perplexity's crawler
    Open
  • ChatGPT-User ChatGPT when a person asks
    Open
  • Googlebot Google Search
    Open
Other bots this site named
  • AI2Bot Allen Institute for AI
    Blocked
  • Applebot-Extended Apple
    Blocked
  • bingbot Microsoft
    Open
  • CCBot Common Crawl
    Blocked
  • Claude-SearchBot Anthropic
    Open
  • Claude-User Anthropic
    Open
  • meta-externalagent Meta
    Blocked
  • meta-externalfetcher Meta
    Blocked
  • MistralAI-User Mistral
    Open
  • OAI-SearchBot OpenAI
    Open
  • PetalBot Huawei
    Blocked
  • Webzio-Extended Webz.io
    Blocked

Want every site on the list, filters, and a file of today's table?

Subscribe · $49 / month