Robots.txt Generator

Generate robots.txt file to control search engine crawlers.

Configuration
Use * for all bots or specify: Googlebot, Bingbot, etc.
One path per line (leave empty to allow all)
Optional: Specific paths to allow
Optional: Seconds between requests
Generated robots.txt
Upload this file to your website root directory (https://yoursite.com/robots.txt)
Ads
Jupiter Optimus

A focused task manager for people who ship. GTD, focus mode, GitHub & GitLab issues and an AI assistant.

Get the app
Moyko

Your calm family organiser. Tasks, notes and reminders, with spaces you can share. Private, synced through your own iCloud.

Get the app
NerdCycle

A cycle tracker that includes your partner. Smart predictions, a daily partner briefing and your data kept in your private iCloud.

Get the app
Codeep

An open-source AI coding agent for your terminal, Mac, VS Code and Zed. Works with any model, hosted or local.

Try Codeep
Support & Stay Connected
Follow Updates Bluesky Vote on Product Hunt
10% OFF
10% Off IFTTT Pro Plans!

Power up your workflows with IFTTTs no-code magic, discounted 10% for new subscribers.

Subscribe Now
20% OFF
AI automation apps — live in 60 seconds, 20% off

A personal AI assistant that runs daily tasks around the clock. Telegram and WhatsApp, web search, built-in AI credits, full CLI, no server setup.

Get the deal