## https://dev.classmethod.jp/robots.txt User-agent: * Disallow: /pages/ Disallow: /error/ Disallow: /tag/ Disallow: /category/ Disallow: /search/ Allow: / # sitemap Allow: /sitemap.xml Allow: /sitemaps/* Sitemap: https://dev.classmethod.jp/sitemap.xml # RSS Feed Allow: /feed/ Allow: /en/feed/ # Option Crawl-delay: 1