Free technical SEO tool

Robots.txt generator

Create a clean starter robots.txt file and copy it when you are ready. It runs in your browser and does not require an account.

Website address

Paths to block, one per line

Sitemap URL

Generated robots.txt

Review the rules before publishing them.

User-agent: *
Allow: /
Disallow: /admin
Disallow: /private

Sitemap: https://example.com/sitemap.xml

Robots.txt controls crawler access requests; it does not securely hide private content. Protect sensitive pages with authentication.

Audit every important page and receive one prioritized action plan.