Why should I NOT block /wp-content/ or /wp-includes/?

Old tutorials used to recommend blocking /wp-content/ and /wp-includes/. However, modern search engines render pages like a browser. Blocking these folders prevents Google from loading stylesheets and JavaScript, triggering ranking penalties.

Where does Yoast or Rank Math put the sitemap?

Both Yoast SEO and Rank Math generate their sitemap at 'https://example.com/sitemap_index.xml'.

Can I edit robots.txt directly in WordPress?

Yes. If using Yoast SEO (Tools > File Editor) or Rank Math (General Settings > Edit robots.txt), you can paste this generated code directly into the plugin settings.

WordPress Robots.txt Generator

Generate an optimized robots.txt file specifically for WordPress sites with Yoast and RankMath support.

TL;DR: WordPress Robots.txt Generator is a free, browser-based tool that lets you generate an optimized robots.txt file specifically for WordPress sites with Yoast and RankMath support.

WordPress Robots.txt Generator

WP Best-Practice Rules

WordPress Site URL

SEO Plugin / Sitemap Type

Included WordPress Directives:

Disallow: /wp-admin/ (Protects admin backend)
Allow: /wp-admin/admin-ajax.php (Keeps AJAX frontend functional)
Allow: /wp-content/uploads/ (Allows indexing media)
Keeps CSS & JS assets unblocked for Google mobile rendering

Generated WP robots.txt

What Is the WordPress Robots.txt Generator?

WordPress Robots.txt Generator creates the ideal robots.txt configuration for WordPress installations, disallowing `/wp-admin/`, `/wp-includes/`, and `/trackback/` while allowing `admin-ajax.php` for dynamic frontend scripts.

Step-by-Step: Using This Tool

Enter your WordPress site domain and sitemap location.
Toggle protection for sensitive directories (`/wp-admin/`, `/wp-includes/`, `/author/`).
Click 'Generate WordPress Robots.txt'.
Upload the file to your server root via FTP or copy into your SEO plugin (Yoast / RankMath).

Maximize Your Results

Allow `admin-ajax.php`: Crucial rule: `Allow: /wp-admin/admin-ajax.php` prevents breaking dynamic frontend plugins.
Block Duplicate Author Archives: Disallow `/author/` on single-author blogs to eliminate duplicate content penalties.
Block Internal Search Results: Disallow `/?s=` and `/search/` to save crawl budget.

Related Guides & Tutorials

What Is Schema Markup Json Ld — Learn in-depth concepts, best practices, and expert tips.
Robots Txt Rules Best Practices — Learn in-depth concepts, best practices, and expert tips.

100% Client-Side Privacy Guarantee

All processing runs locally inside your browser using JavaScript and HTML5 APIs. Your data, files, and inputs are never uploaded to any remote server. Complete privacy by design.

Frequently Asked Questions

: Old tutorials used to recommend blocking /wp-content/ and /wp-includes/. However, modern search engines render pages like a browser. Blocking these folders prevents Google from loading stylesheets and JavaScript, triggering ranking penalties.
: Both Yoast SEO and Rank Math generate their sitemap at 'https://example.com/sitemap_index.xml'.
: Yes. If using Yoast SEO (Tools > File Editor) or Rank Math (General Settings > Edit robots.txt), you can paste this generated code directly into the plugin settings.

Cite This Tool

Referencing this tool in a paper, article, or bibliography? Copy a ready-made citation below.

NexLove.org. (2026). WordPress Robots.txt Generator [Software]. https://nexlove.org/tools/wordpress-robots-txt.html

“WordPress Robots.txt Generator.” NexLove.org, 2026, nexlove.org/tools/wordpress-robots-txt.html.

NexLove.org (2026) WordPress Robots.txt Generator. Available at: https://nexlove.org/tools/wordpress-robots-txt.html (Accessed: 2026).

@misc{nexlove_wordpress_robots_txt,
  title = {WordPress Robots.txt Generator},
  author = {{NexLove.org}},
  year = {2026},
  url = {https://nexlove.org/tools/wordpress-robots-txt.html}
}

Embed This Tool

Add the live WordPress Robots.txt Generator tool to your own website with this snippet — it loads the real, working tool in an iframe, not a static screenshot.

<iframe src="https://nexlove.org/embed/wordpress-robots-txt.html" width="100%" height="600" style="border:1px solid #e2e8f0;border-radius:12px" title="WordPress Robots.txt Generator — NexLove.org" loading="lazy"></iframe>

Related SEO Tools

Schema Markup Generator

Free Robots.txt Generator

Browse all SEO Tools