User-agent: * Disallow: /wp-admin/ Disallow: /wp-login.php Disallow: /cgi-bin/ Disallow: /search Allow: /wp-admin/admin-ajax.php # basic Disallow: /tag/ Disallow: /?s= Disallow: /search?*restrict[]= # advanced Disallow:/safe_search/ # Google - Allow all bots to crawl your site content User-agent: Googlebot Allow: / User-agent: Googlebot-News Allow: / # Crawl delay for specific bots (optional) User-agent: Bingbot Crawl-delay: 10 Sitemap: https://en.edudaily24.com/sitemap_index.xml Sitemap: https://en.edudaily24.com/news-sitemap.xml