User-agent: * Disallow: /wp-admin/ Disallow: /wp-includes/ Disallow: /wp-content/plugins/ Disallow: /wp-content/cache/ Disallow: /wp-content/themes/ Disallow: /feed/ Disallow: /trackback/ Disallow: /comments/ Allow: /wp-admin/admin-ajax.php Sitemap: https://www.tumanly.com/sitemap_index.xml # 屏蔽 SEO 分析工具爬虫 (Ahrefs, Semrush, Moz等) User-agent: AhrefsBot Disallow: / User-agent: SemrushBot Disallow: / User-agent: SemrushBot-SA Disallow: / User-agent: SemrushBot-BA Disallow: / User-agent: SemrushBot-SI Disallow: / User-agent: SemrushBot-SWA Disallow: / User-agent: SemrushBot-CT Disallow: / User-agent: SemrushBot-BM Disallow: / User-agent: SemrushBot-SEOAB Disallow: / User-agent: DotBot Disallow: / User-agent: MJ12bot Disallow: / User-agent: MauiBot Disallow: / User-agent: BLEXBot Disallow: / User-agent: ZoominfoBot Disallow: / User-agent: ExtLinksBot Disallow: / User-agent: hubspot Disallow: / User-agent: leiki Disallow: / User-agent: webmeup Disallow: / User-agent: MegaIndex.ru Disallow: / # 屏蔽 AI 训练爬虫 (防止内容被用于大模型训练) User-agent: GPTBot Disallow: / User-agent: ChatGPT-User Disallow: / User-agent: Google-Extended Disallow: / User-agent: Claude-Web Disallow: / User-agent: CCBot Disallow: / # 屏蔽其他常见垃圾/低价值爬虫 User-agent: YisouSpider Disallow: / User-agent: PetalBot Disallow: / # 允许正常的搜索引擎爬虫 (可选,明确列出以确保不被误伤) User-agent: Googlebot Allow: / User-agent: Baiduspider Allow: / User-agent: Bingbot Allow: / User-agent: Sogou web spider Allow: /