GPTBot Took 6.84 Gigabytes. The Bot That Could Cite Us Took 2.8 Kilobytes.

Take a sister site, the Jacksonville bars guide, because its numbers are small enough to hold in your head.

Its sitemap has 25 pages. Its listing page carries 85 filter links, and those links belong to six filters:

➤ 1. Sort order, 2 choices ➤ 2. Experience, 30 choices ➤ 3. Discovery, 22 choices ➤ 4. Neighborhood, 14 choices ➤ 5. Dietary, 4 choices ➤ 6. Venue type, 12 choices

Any filter can be combined with any other. Multiply the choices out and the site has roughly 2,085,525 reachable addresses.

Every one of them returns a real page with a real list on it. None of them is in the sitemap, because none of them is a page we wrote. They are views of the same 25 pages.

That gap between 25 and two million is not a bug. It is what a filter does. The question is only who walks it.

Same site. Same twelve days. Same server, same code, same 85 filter links sitting in the same place on the page.

GPTBot followed 220,049 of them. ClaudeBot followed zero.

This matters more than the size of the bill, because it kills the explanation most people reach for first. Nobody can say we built a trap, or that our filter grid is somehow malformed, or that a crawler got lost. Two crawlers read the identical page in the same week and made opposite decisions.

It is a policy difference at the vendor, not a property of our site.

And the vendors who do walk it are the big four. Over the 28 days to October 11, four crawlers took 4,055,700 of the 4,139,970 declared AI crawler requests across the whole portfolio:

➤ GPTBot ➤ meta-externalagent ➤ Amazonbot ➤ ClaudeBot

That is 97.96 percent of everything, up from 67.84 percent in the 28 days before that. The other seventeen crawler names we recognize moved from 89,017 requests to 84,270, which is slightly down.

Last week I published a piece called ClaudeBot Changed Its Mind About Filter URLs, and the honest version of the finding was narrower than the headline. ClaudeBot had started walking filter grids, but only on guide sites it had never requested before. The ones it already knew stayed clean.

So on October 11 we registered that as a prediction in the research ledger. Hosts ClaudeBot already knew would keep getting zero filter requests. Hosts it met for the first time would be walked on arrival.

Over the next five days the BellyUp! network rolled a sports guide out across the city network, and ClaudeBot met ten of those subdomains for the first time between October 12 and October 16.

It walked the filter grid on all ten, on the day it arrived or the day after.

Meanwhile, across the five cities named in that prediction, the six guide subdomains ClaudeBot already knew made 442 requests between them and not one carried a filter. Zero. Over seven days.

The new ones in those same five cities took 101,393 filter requests in the same week.

Here is the part that should change how you read your own numbers.

Across the whole portfolio, declared AI crawler requests went from 1,125,473 in the week of September 28 to 2,885,615 in the week of October 5. Both are complete weeks. That is 2.56 times in seven days.

If you stopped there you would conclude that AI systems suddenly got much more interested in Krisada's portfolio.

They did not. In the same two complete weeks, the seven editorial and research properties went from 18,786 requests to 14,071. They went down 25 percent while the total went up 156 percent.

Both of those figures come from our rolled-up crawler table. When I checked the same week against the raw request log instead, the seven properties came to 15,807 rather than 14,071. The two tables count slightly different things and I have not yet pinned down which filter causes the 11 percent gap. The direction of the week is the same either way, and the gap is logged.

Every one of the thirty largest site-and-crawler pairs in those twelve days is a BellyUp! guide subdomain being filter-walked at between 99.5 and 100 percent query-string share. The surge is not interest. It is arithmetic, and the arithmetic belongs to a filter grid we published.

The lesson is not that the number is useless. It is that an AI crawler count with filter views in it is measuring the size of your filter, and you cannot tell by looking at the chart.

Three things published in the last week point the same way, and they are the reason I keep separating the training crawlers from the retrieval ones.

On October 11, Yunxuan Jiang and four co-authors posted Understanding ChatGPT Citations, an observational study of four ChatGPT web settings from version 5.2 to 5.5. Two findings stand out. Google Search and the OpenAI Responses API, combined, recover only about one quarter of the URLs ChatGPT actually cites. And both of those proxies match ChatGPT better at the hostname and entity level than at the exact URL.

So if you are using Google rankings as a stand-in for AI citation, you are seeing about a quarter of the picture, and the part you do see is more about your website than your page.

On October 15, Owen Walters, Lihu Chen and Francesca Toni posted Retrieved but Not Read. They found that models often answer straight from search snippets without opening the source page, and that doing so goes with more outdated answers and more citations that do not hold up.

That is the uncomfortable one. If a model can cite you without fetching you, then your fetch log will never show the citation.

And on October 15 Cloudflare published a post with OpenAI about a pilot that began in July 2026. Cloudflare's network gives OpenAI signals about which pages are worth fetching, and on about one percent of Cloudflare's network it has already helped OpenAI discover and index hundreds of millions of new pages. OpenAI still ranks them independently.

I checked all seven of our research properties today. Every one serves straight from our own Apache with no Cloudflare in front of it. So none of this reached us, which is itself a useful thing to know: the discovery channel being built here is one we are not standing in.

This one is unrelated to the crawlers and I am leaving it in because it bit us twice.

Our research extract reports weekly numbers with a field called week_start. I went and read the query that builds it. The field is the earliest date that has any data inside that week, not the day the week begins.

So a week labelled October 10 is really the week beginning October 12 in calendar terms, with nothing recorded before the 10th. Divide by seven and you get a per-day rate that is wrong, in a direction that depends on which site you are looking at.

Two of last week's numbers were off for exactly that reason, and the corrections are in the ledger beside the originals. Every figure in this article is taken from whole weeks or named date ranges instead.