Robot.txt Generator

Create a custom robots.txt file for your website by toggling options below

Configuration Options

User-agent Settings

Allow all robots

Block all robots

Custom user agents

Add User-agent:

Directory Settings

Block common private directories

/admin/

/wp-admin/

/login/

/cart/

Custom disallow paths

Add disallow path:

Additional Settings

Add sitemap URL

Sitemap URL:

Add crawl-delay

Crawl-delay (seconds):

Preview & Download

This is how your robots.txt file will look:

# robots.txt generated by Robot.txt Generator User-agent: * Allow: /

Robot.txt Generator - A tool to help with your website's SEO and crawler management.