I Checked the Same Data Three Times. One Answer Was Wrong.

Every week I take a copy of our own data. It watches AI bots visit about 130 websites we run. It counts who showed up.

In early September it told me something big. Three AI bots had grown their visits about ten times in one month.

Ten times is a great story. So I wrote it up. I marked it at our highest level of proof. That level is for things we measured ourselves.

My own logs felt like hard proof.

That was my mistake. But not in the way you might think. I did not write the query wrong. I did not read the answer wrong.

I did what almost everyone does. I measured something one time. Then I called it a fact.

I took another copy a few days after that. Same question. Same weeks. Almost all the same days.

The bot that showed nearly 11,000 visits now showed about 1,100.

That is not a small change. Nine out of ten visits were gone. And they were gone from weeks that had already ended.

That is the part that bothered me.

Google Search Console data shows up late. So its newest numbers move for a day or two, then settle down. Server logs do not work like that. The server either wrote the line down at the time, or it did not. Last month cannot get better overnight.

So something changed its mind. Not the logs. Something between the logs and the summary table.

I wrote that up last week and took the loss. I left the first claim up, with the correction sitting next to it. Deleting it would have looked cleaner. It would have been worth a lot less.

The third copy came seven days later. I went in expecting more of a mess.

Nine of the eleven bots I track came back close to where they were. Not exact. But close.

Those weeks had rolled over about a quarter of their days. So close is what a steady measurement should look like.

And the bot that lost nine out of ten visits? It came back at almost the same number as the week before. Four visits apart.

Two bots came in much lower than before. One was Apple's. One was Anthropic's. For about an hour I thought the whole thing was falling apart again.

Then I looked at the older weeks in the same file. Both bots had been busy all summer. Both went quiet in September.

So they did not go missing. They left. And they left right at the edge between two time windows.

This part matters more than it sounds. A bot that slows down looks exactly like a bot the tool stopped recognizing. Both draw the same falling line on a chart.

The only way to tell them apart is to look at the weeks on either side. Almost no tool lets you do that.

It would be easy to call this settled now. It is not.

All three copies come from the same system. They share every assumption. If the system is wrong, all three are wrong together. I have not checked any of them against the raw server logs.

The test I wrote down last week has not happened yet either. I said I would compare two copies taken close together in time. These were a full week apart. That is further apart than I asked for. So some of the agreement I like might just be that.

I also asked for three things to be added to the data. None of them exist yet.

The big one is a label. It would say which version of the bot list made the numbers. Without that label I cannot tell two things apart: a bot that stopped visiting, and a bot the tool stopped naming.

That is the whole question. I still cannot answer it.

So nothing moves up a level this week. The claim from two weeks ago stays right where it is. Two copies now disagree with it instead of one.

I think it is probably wrong. But probably wrong is not a level on our ladder. Making up a new level so I feel better would ruin the point of having one.

While I was busy doubting my numbers, Google misplaced some of its own.

Barry Schwartz wrote about it at Search Engine Roundtable on 2026-09-20. The Search Console Crawl Stats report was missing a whole day, for every account. He put it plainly: "This is not an issue with your site, it is an issue with Google's reporting in Google Search Console."

That day came back two days later.

A three week gap in the June indexing report did not come back. It never will. Google does not fill in old indexing data.

That same week, John Mueller answered a question about how position gets counted in AI Mode. He said: "The goal is not a written-in-stone absolute truth for position counting (that's impossible)."

I do not read that as an apology. I read it as Google being more honest about its own limits than most tools built on top of it are.

My data lost a week. Google's crawl report lost a day. Google's indexing report lost three weeks and kept them.

Everyone's tools are shakier than the charts make them look.

One part of every copy has matched perfectly all three times. That is the Search Console side. Same weeks. Same impressions. Same average positions.

So here is a clean number to finish on. It is not a happy one.

Over the last four weeks this site got about five and a half times more impressions than it got in the four weeks before.

Clicks went from three to two.

Five and a half times more views bought one less click.

That means more pages became eligible for more searches, at about the same depth in the results. It does not mean the site ranks better for anything it already ranked for.

Calling that a ranking win would be the easiest lie in this whole set of numbers. It is also the one I hear most often.