# Disallow all crawlers access to certain pages. User-agent: Baiduspider User-agent: BaiduMobaider Crawl-delay: 600 User-agent: * Disallow: /styles Disallow: /images Disallow: /js #other allow Allow: / # Sitemap files