Robots.txt generator

Blocking the AI training bots will not. GPTBot, ClaudeBot and CCBot have nothing to do with Google search. Blocking Google-Extended is different: that controls AI Overviews and Gemini, not ordinary Google results, so your blue links stay either way, but you take yourself out of the AI answers. Blocking Googlebot itself would remove you from Google entirely, which is why this tool keeps it separate.

GPTBot collects text to train OpenAI models. OAI-SearchBot fetches pages so ChatGPT can cite them when answering a question. People routinely block both when they only meant the first, and then wonder why ChatGPT never mentions them. They are separate lines in robots.txt and this generator separates them.

It is a request, not a lock. The named bots above publish that they honour it and generally do. A crawler that ignores robots.txt will ignore this file too, and stopping that needs blocking at your server or CDN.

The root of your domain, so it answers at yoursite.co.uk/robots.txt. A file in a subfolder is never read. If you are on WordPress, most SEO plugins have a robots.txt editor that writes it for you.

It is a real decision with a real trade-off and there is no universally right answer. Blocking keeps your writing out of models. Allowing means the models know you exist, which increasingly matters for being recommended. Publishers and people whose words are the product often block. Most local service businesses have more to gain from being known.

No. The whole thing runs in your browser and the file is built locally. There is no upload and no log.