Robots.txt Generator

Name: Robots.txt Generator
Rating: 4.9 (128 reviews)

Create a robots.txt file by selecting which bots to allow or block, and which paths to disallow. Preview the output and download the file instantly.

User-agent

allow

disallow

Sitemap URL

User-agent: *
Allow: /
Disallow: /admin/

Sitemap: https://example.com/sitemap.xml

Frequently Asked Questions

What is robots.txt used for?▼

It tells search engine crawlers which pages or sections of your site to crawl or skip.

Does robots.txt block all bots?▼

No. It is a convention. Malicious bots may ignore it entirely.