The Crawlers Asked Once And Left. Two Weeks Later Google Had Not Moved.

Nothing. That is the finding.

The table below is summed across every BellyUp! host in the warehouse, out of the Search Console page table, comparing the last complete week before the treatment against the last complete week after it. The treatment landed on November 1 at 02:50 UTC.

Across the ten complete days after the change, impressions averaged 2,231 a day against 2,318 in the week before. The lowest single day in the whole window is 1,977, and two of those ten days are the highest readings in a month.

I wrote the losing condition down a week ago, before I had the numbers, which is the only way a test like this means anything. The rule was that the week fails if impressions drop under 1,855 a day for seven complete days in a row. Twenty percent down. It did not come close.

Clicks went from 100 in the pre week to 85 in the post week. On a base of about 13 a day that is noise, and the week before the treatment ran 96. I am not going to pretend a 15-click move across ten cities means anything either way.

One caveat matters more than the rest. Search Console lags two to three days, so the newest complete day here is November 10. The 28-day window I declared runs to November 29, and this is reading one of four. I am not calling this a settled result today.

The question the first article left open was whether the burst was about being new.

Every one of those enormous first-day numbers happened when a crawler reached a site for the first time. Charleston took 92,352 requests from ClaudeBot in a single day the first time it went deep. Chicago, launched October 25, peaked at 47,859 in a day. Cary, launched October 27, peaked at 44,774.

So one reading was that crawlers go hard at anything they have not seen, and our filter links were incidental.

Then we cloned a new lens across the network on November 5 through 7, after the link change was already in place. Same generator, same templates, same hosts and the same sitemaps. The only difference is what the pages publish.

Across nine days, the whole set of 18 took between 19 and 152 AI crawler requests each.

ClaudeBot made 412 requests across 16 of them. Not 92,352 on one. Four hundred and twelve across sixteen.

GPTBot made 227. Amazonbot made 99.

And the one host in the set that does publish a filter grid, 14 single-setting addresses under three settings, is the only one that received a single filter request. Here is what three separate crawlers did with those 14 addresses:

➤ One. Amazonbot requested 14 distinct filter addresses, one request each. ➤ Two. GPTBot requested 14 distinct filter addresses, one request each. ➤ Three. ClaudeBot requested 11 distinct filter addresses, one request each.

Every value they asked for appears in that page's rendered HTML. None of them combined two settings into one address. None of them asked twice.

That is what a crawler looks like when it is reading a link graph instead of walking a combination space.

Last week I reported the crawl settling at 48,207 requests a day across the five complete days from November 2.

That was not the floor. That was Amazonbot still working.

Across the seven complete days from November 6 to November 12, the same hosts took 4,283 requests a day. Against 1,263,440 before the change, that is down 99.66 percent.

The vendors still drain at completely different speeds, and this week adds one I got wrong. GPTBot was finished inside a day, spending the treatment day itself burning through 383,487 requests and then dropping to 358. The Meta crawler went from 3,515 to 10 within hours. Amazonbot took six days, roughly halving each day: 184,500, then 129,643, 96,962, 60,401, 28,436, 24,977, 4,136 and 1,088.

Bytespider is the one I called wrong. Last week I recorded it as the only crawler that went up, and left it there. It did go up, and then it stayed up for five more days at around 3,000 a day while everything else collapsed, and only fell to 99 a day on November 8. It was not immune. It was slow.

This is the part worth more than the headline, because it will cost somebody a week.

When I pulled the crawler data for the new sites, Applebot looked like the heaviest filter crawler on the network. It made 407 requests across 18 hosts and 249 of them carried a query string. Sixty-one percent, higher than any AI crawler in the set.

I almost wrote that up as a new vendor behaviour.

Then I listed the actual addresses. All five of the distinct query strings Applebot used were stylesheet cache-busters, a version number tacked onto a CSS file. Googlebot, GoogleOther and Baiduspider were doing the same thing.

Zero filter requests. Applebot was loading our stylesheets.

The same mistake bit me twice in one hour. My first count of published filter links was inflated by four to seven per site, because a plain search for a question mark in a link picks up asset versions and the cross-host links that carry a campaign setting. On the small sites that error was the entire count. I recorded seven filter links on a site that publishes none.

So if you are reading query-string share out of your own logs to find a filter problem, that number is too high before you start. Split the keys. A cache-buster is not a filter, and a campaign setting is not a filter.

Here is the one number that did fall by more than half, and the reason it is not bad news.

OAI-SearchBot is the OpenAI fetcher that feeds ChatGPT search rather than training. Its total requests to these hosts went from 13,924 in the week before the change to 4,713 in the week after. Down 66 percent, which looks exactly like losing retrieval interest.

It is not. Split the addresses.

In the pre week, 9,176 of those 13,924 requests carried a filter query string. Two thirds. On October 31 alone its query-string share hit 92.8 percent.

Requests to real content addresses went from 4,748 to 3,264. Down 31 percent, not 66. Two thirds of the fall was filter addresses it can no longer reach.

The other two retrieval fetchers moved the other way or barely moved. ChatGPT-User went from 431 requests to 339, with content requests down 20 percent. PerplexityBot went from 135 requests to 934, and its content requests rose from 132 to 829.

A search-side fetcher spending two thirds of its visits on filtered duplicates of the same venue list was not visibility. It was waste, and it was our waste, because we published the addresses.

Three things, and I would rather name them than let them quietly resolve themselves in my favour.

The size relationship is still not clean. The idea was that a bigger first-crawl burst follows a bigger published filter grid. Across the 14 lens sites launched in Chicago and Cary, 12 of the 14 rank the same way on both numbers and two pairs invert. I cannot blame those two on the idea being wrong, because I counted the links on November 13 and both cities received new tagged content on November 7 and 8. A filter only appears once a venue carries its tag, so the counts I measured are not necessarily the counts those crawlers saw. The missing instrument is obvious now. Capture the filter-link count on launch day, before anything crawls it.

The arrival question can no longer be tested here. The useful half of the ClaudeBot theory was a prediction about what happens when it reaches a site for the first time, and all 18 of these arrived after the links were cut. That question now needs a different generator or it gets closed on its control arm alone.

And the drain rates need a host I have not treated yet. GPTBot inside a day, Amazonbot in six, Bytespider in eight. That is one treatment, one set of hosts, one week.

One more finding from this week belongs beside all of that, because it attacks the check most people use to prove a site is fine.

Shuyang Zhang at The Hong Kong Polytechnic University fetched 10,307 addresses across 1,507 sites and labelled what was actually inside each HTTP 200 response. Of the pages a static client received with a 200, 20.9 percent held no main content. App shells, bot challenges, block pages, placeholders, partial loads. Worse, every site was also probed with a random address that does not exist, and 13.1 percent of the sites that answered said 200.

So I probed our own. Ten portfolio hosts, one address that exists on none of them, checked on November 14. Ten real 404 responses and no soft 200s.

I am glad I checked rather than assumed, because two of those error pages render structured data of their own. A soft 200 on one of those templates would have sailed through our own nightly schema audit looking like a healthy page.