SEO & Web ToolsFree

Robots.txt Generator

Create a robots.txt file visually. Select which crawlers to allow or block, specify paths to disallow, and set your sitemap URL. Preview the output in real time and download the file ready to upload to your server root. No signup required.

Updated June 2026

  1. 1Set your crawl rules
  2. 2Add your sitemap URL
  3. 3Download the robots.txt file
User-agent: *
Allow: /
Disallow: /admin/

Sitemap: https://example.com/sitemap.xml

How to use Robots.txt Generator

  1. 1

    Set your crawl rules

    Choose which bots to allow or block and which paths to disallow.

  2. 2

    Add your sitemap URL

    Optionally include a link to your sitemap.xml.

  3. 3

    Download the robots.txt file

    Place it in your site's root directory.

Frequently asked questions

Everything you need to know about robots.txt generator.

The robots.txt file tells search engine crawlers (Googlebot, Bingbot, etc.) which pages or sections of your site to crawl and which to skip. It is placed at the root of your domain (yourdomain.com/robots.txt).

Explore more category tools

Convert, optimize and organize your files with our full collection of free tools.

Explore all category tools