Is your hotel website blocking ChatGPT without you knowing?
We checked 360 independent hotel, guesthouse and hostel websites. Few block ChatGPT, and never on purpose. A quarter could not be read at all.
Short answer
Probably not. Of 360 independent hotel, guesthouse and hostel websites we checked in six countries, 2% blocked ChatGPT's search crawler, and none did it by name: it came from a general rule meant for other bots. The bigger problem was that 27% of the websites did not load at all, so no assistant, search engine or guest could read them.
Key takeaways
- In our test, 1.9% of independent accommodation websites blocked ChatGPT search, always through a rule written for every crawler.
- Blocking AI training bots, which 3.8% did, does not remove a site from ChatGPT search; those are separate crawlers.
- 19% of the websites listed for these properties no longer existed, and another 8% failed to load.
- A robots.txt check cannot see blocking done by a firewall such as Cloudflare, so look at both.
An assistant can only recommend a place it can read. So we wanted to know how many independent hotels, guesthouses and hostels close the door on ChatGPT without realising it.
We expected the answer to be “a lot”. It was not. The real problem turned up somewhere else.
What did we check?
We took 360 independent properties listed on OpenStreetMap, the open map that many apps and services draw on, in six countries: Ireland, Portugal, Spain, Greece, Thailand and Mexico. We left out chains, and listings that point to Facebook or a booking site instead of the property’s own website.
For each website we read its robots.txt, the file that tells crawlers what they may read, and applied the rules the way the official standard (RFC 9309) says crawlers should. Then we checked what each crawler was allowed to do on the home page.
How many hotel websites block ChatGPT?
Very few. Of the 263 websites that answered, five blocked ChatGPT’s search crawler.
| Crawler | What it is for | Blocked |
|---|---|---|
| OAI-SearchBot | ChatGPT search answers | 1.9% |
| PerplexityBot | Perplexity search answers | 1.9% |
| Claude-SearchBot | Claude search answers | 1.9% |
| GPTBot | Training OpenAI’s models | 3.8% |
| ClaudeBot | Training Anthropic’s models | 3.8% |
| Google-Extended | Training Google’s AI models | 3.8% |
| Googlebot | Google Search | 0.8% |
Not one of the five named ChatGPT. The block came from a general rule.
How does a website block ChatGPT by accident?
We found two ways, and both are easy to miss.
Everything blocked. Two sites had a rule that turns away every crawler, Google included. This is often left over from when a website was being built, and nobody removed it at launch:
User-agent: *
Disallow: /Only Google let in. Three sites blocked every crawler and then made an exception for Google. Ten years ago that looked sensible. Today it also turns away ChatGPT, Perplexity, Claude and Bing, which ChatGPT search draws on.
What was the bigger problem?
The websites that never answered. Of the 360 websites listed for these properties, 68 (19%) pointed to a domain that no longer exists. Another 29 (8%) failed for other reasons: the server did not respond, refused the connection, or had a broken or expired security certificate.
That is 27% of listed websites that nobody could read: not ChatGPT, not Google, and not a guest who tapped the link on a map.
If the website address on your map listings, booking profiles and social pages is out of date, fixing it matters far more than anything in robots.txt.
Does blocking AI training hurt you?
Not directly. OpenAI, Anthropic and Google each run separate crawlers for training and for answering questions. OpenAI’s documentation says sites that block OAI-SearchBot will not appear in ChatGPT search answers, but blocking GPTBot, the training crawler, is a different setting.
In our sample, 3.8% blocked the training crawlers by name, usually all of them together. That is a reasonable choice for some businesses. Just make sure the search crawlers are not caught in the same rule.
What about Cloudflare?
Our test only read robots.txt files, so it could not see blocking done by a firewall. That matters, because a growing number of small sites sit behind Cloudflare.
Cloudflare announced that from 15 September 2026, new domains and all free plans block AI training and agent crawlers by default on pages that show ads, while search crawlers stay allowed. It also warns that when training is blocked, crawlers used for more than one purpose, such as Googlebot and Bingbot, can be blocked too. If your site uses Cloudflare, check those settings in its dashboard.
What should you do this week?
- Open your website’s address followed by /robots.txt and look for “Disallow: /” under “User-agent: *”.
- If you only allow Google, add the same allowance for Bingbot, OAI-SearchBot, PerplexityBot and Claude-SearchBot, or remove the general block.
- If you use Cloudflare, open its AI crawler settings and check what is blocked.
- Search for your property on Google Maps, OpenStreetMap, Booking.com and your social profiles, and tap the website link on each. Every one should open your site.
Questions owners ask
What is robots.txt?
A small text file at yourdomain.com/robots.txt that tells crawlers which pages they may read. Search engines and AI companies say their crawlers follow it. Most website builders create one for you, which is why many owners have never seen theirs.
Should I block GPTBot, the OpenAI training crawler?
That is a choice about training, not about bookings. OpenAI uses a separate crawler, OAI-SearchBot, for ChatGPT search, and blocking GPTBot does not remove your site from those answers. If you have no strong view, allowing both is the simpler choice for a hotel that wants to be found.
My site is on Cloudflare. Am I blocking AI assistants?
Possibly, and robots.txt will not tell you. Since 15 September 2026, new Cloudflare domains and free plans block AI training and agent crawlers by default on pages with ads. Open the AI crawler settings in your Cloudflare dashboard and check what is blocked.
How do I check my own website in two minutes?
Type your domain followed by /robots.txt into a browser. If you see "User-agent:" followed by an asterisk and then "Disallow:" followed by a single slash, every crawler is turned away. If the page does not load at all, or your domain is not found, fix that first.
Sources
- OpenAI. Overview of OpenAI crawlers
- Perplexity. Perplexity crawlers
- Anthropic. Does Anthropic crawl data from the web, and how can site owners block the crawler?
- IETF. RFC 9309: Robots Exclusion Protocol
- Cloudflare. Your site, your rules: new AI traffic options for all customers
- © OpenStreetMap contributors. Map data used for the sample (ODbL)