Disallow: /wp-admin/ User-agent: * Allow: / Allow Googlebot User-agent: Googlebot Allow: / Allow Bingbot User-agent: Bingbot Allow: / Optional: Allow OpenAI’s crawler User-agent: GPTBot Allow: / Optional: Allow Common Crawl (used by many AI tools) User-agent: CCBot Allow: / Sitemap location Sitemap: https://clearshield.in/sitemap_index.xml