Research Methodology

The first question was whether the traffic was genuine. A bot can claim any name it wants in a user agent, so 43,000 requests labeled GPTBot do not automatically become 43,000 OpenAI requests.

Across the retained measurement window, the network logged 46,829 GPTBot-classified requests. Of those, 46,727 came from 41 of 43 source IP addresses inside OpenAI's published GPTBot ranges.

That does not verify what OpenAI did with the pages. It verifies something narrower: nearly all of the traffic attributed to GPTBot came from published OpenAI crawler infrastructure.

The spike was real.

Then the more interesting question became: what did it spend all those requests doing?

The bars property had twelve venues.

It also had four filters that could be combined, plus sorting. Every selected combination produced another crawlable URL.

By the time GPTBot arrived, those twelve venues had become 26,863 discoverable URL combinations. Across the network, the filter system exposed 40,404.

Some combinations made sense. Plenty did not. The site was perfectly willing to assemble things like a kid-friendly nightclub with a DJ in one neighborhood, whether a venue matched the request or not.

Even an empty combination still returned a complete page. Two nonsense combinations we sampled produced the same venue list and empty-state message inside a full 22 KB response.

That is how a small directory turned into a very large maze.

It is tempting to look at absurd filter combinations and conclude the bot was stupid.

That is the least interesting explanation.

We created tens of thousands of discoverable URLs. GPTBot followed them. It behaved pretty rationally according to the structure it encountered.

The machine did not invent the pen.

We built the pen, opened the gate, and left 40,404 rooms on the other side.

• 46,829 GPTBot-classified requests across the retained period
• 46,727 requests from IP addresses inside OpenAI's published GPTBot ranges
• 41 of 43 source IP addresses matched those ranges
• 43,309 GPTBot requests on September 9 alone
• 38,971 distinct requested URL strings across the six live properties that day
• 40,404 filter combinations exposed across the network
• 26,863 combinations on one property containing twelve venues
• 46,504 requests to filter HTML across the retained period

While GPTBot was busy walking the filters, the machine-readable work received almost nothing.

The venue JSON was requested twice.

The AI foundation files were requested twice.

The AI instructions file was requested once.

That was not evidence that crawlers dislike structured data. The discovery layer was weak, incomplete, and in one place openly contradictory.

The website offered one enormous door to the filters and a few poorly marked doors to the resources we cared about. The request counts tell us which one the crawler found.

A request proves that a request reached a recorded URL.

It does not prove indexing. It does not prove an AI citation, training use, a ranking change, search visibility, or a human reader.

It also does not prove those 43,309 URLs had value. In this case, the request pattern is evidence of a crawl trap, not a trophy.

The restraint matters because the number is dramatic enough to invite a bigger claim than it can carry.