Bot Traffic Measurement: The Activity Search Console Does Not Show
Cross-referencing 90 days of Search Console data against 48 days of raw bot logs across 101 live properties found real AI visibility in places Search Console alone reports as nearly dead.
Healthcare & Longevity is sixteen sites.
Bot Traffic Measurement: results, limits and next checks
Baseline confirmed ... outcome openBefore the Bot Traffic Measurement work
Search Console made several portfolio properties look nearly inactive because the visible instrument stopped at human impressions and clicks. The retained comparison used ninety days of GSC data and forty-eight days of server logs across 101 properties.
What changed in Bot Traffic Measurement
We joined GSC query data to classified server requests, excluded SEO tools and utility bots, grouped the result by property, and rolled it up through the same eleven portfolio constellations used for search reporting.
What the Bot Traffic Measurement results show
The measurement changed, not the rankings. Healthcare and Longevity showed only 83 Google clicks across sixteen properties while producing more than 9,000 good-bot requests a week. NaturalHerbLibrary.com alone recorded 19,045 AI-crawler requests from nine vendors alongside 31 Google clicks.
What Google reported for Bot Traffic Measurement
Ninety-day Search Console totals supplied clicks, impressions, and distinct queries. NaturalHerbLibrary.com recorded 31 clicks and 1,411 impressions in that window. Medical Marketing recorded 10,521 impressions and zero clicks.
What the Bot Traffic Measurement server logs recorded
Forty-eight retained days of classified logs supplied verified AI-crawler and priority search-bot requests. NaturalHerbLibrary.com recorded 19,045 AI-crawler requests from nine distinct vendors.
What the Bot Traffic Measurement results do not prove
The GSC and log windows are different lengths. A crawler request proves access, not citation, ranking, referral traffic, or business value. Cross-property differences also prevent a clean one-variable comparison.
Where Bot Traffic Measurement stands now
The visibility gap is confirmed as a measurement finding. Experiment No. 001 remains active because no crawler count can stand in for a verified AI citation.
What we will measure next for Bot Traffic Measurement
Record the first confirmed AI citation separately from crawler visibility, then compare whether pages with sustained multi-vendor crawling begin earning citations, rankings, or referral traffic.
What we measured for Bot Traffic Measurement
Three readings from 101 properties measured two ways at once. The two instruments disagree, and the disagreement is the whole finding.
One site read heavily by nine separate AI vendors. No SEO tool built in the last twenty years reports this, because none of them were built to ask the question.
Two measurement systems pointed at the same portfolio. They disagree because they are counting different readers, and only one of them has been the default for two decades.
Nine constellations show one signal or the other but not both. Judging those nine on clicks alone misreads every one of them.
A page with almost no clicks and heavy AI crawler traffic is not underperforming. The instrument being used to judge it cannot see who is reading it.
Bot Traffic Measurement: the short version
Google Search Console only answers one question: did a human click. It has no way to answer whether anything is reading a page at all. This case study documents what happened when we joined GSC data against an already-existing bot fingerprint classification across the entire Krisada.com portfolio, and why naturalherblibrary.com, the test property for Real SEO™ Experiment No. 001, turned out to be the clearest example of the gap.
Bot Traffic Measurement in plain English
Google's own tool only tells you when a human clicked, not when an AI program read the page. We compared that tool's numbers against our raw server logs across 101 websites, and found some pages Google thinks are basically dead are actually being read heavily by AI systems like ChatGPT and Claude, just not showing up as human clicks yet.
Bot Traffic Measurement: the starting conditions
The Digital Karma Data Warehouse already imports Search Console data nightly for every active site in the portfolio, and already classifies every server log request against a maintained bot fingerprint table for security and traffic-quality reasons. Neither system was built to talk to the other.
The original question was practical: which of Krisada's 11 portfolio constellations already have enough real traffic proof, human clicks and AI reads both, to credibly sell business exposure inside that niche. Answering it meant joining GSC clicks and impressions against bot-log activity, grouped by site and by constellation, for the first time.
What changed in Bot Traffic Measurement
GSC clicks, impressions, and distinct queries were pulled from gsc_query_daily for a 90-day window. Server log requests were pulled from log_requests joined to bot_fingerprints, filtered to visitor_class ai_crawler, plus visitor_class bot rows with a fingerprint priority of 12 or lower (Googlebot, Bingbot, DuckDuckBot, YandexBot, Baiduspider). That excludes SEO tools like Ahrefs and Semrush, social preview bots, and uptime monitors, all of which also show up as bot traffic in raw logs but say nothing about search or AI discovery. The log window was 48 days, the full range currently retained.
Both datasets were grouped by site, then rolled up by constellation using the same portfolio-constellation mapping the warehouse already uses for GSC reporting.
What the Bot Traffic Measurement results show
Healthcare & Longevity is sixteen sites. Watched through Search Console alone, it looks close to dead: eighty-three total clicks across all sixteen properties over ninety days. Watched through the bot log, it is the single most AI-crawled category in the entire portfolio, over nine thousand good-bot requests a week combined, ahead of every other constellation by a wide margin.
The clearest single example is naturalherblibrary.com. Over the same window it shows 31 Google clicks and 1,411 impressions, numbers that read as thin on their own. Its server logs show 19,045 AI-crawler requests from nine distinct vendors: OpenAI, Anthropic, Perplexity, Common Crawl, ByteDance, Amazon, Meta, Cohere, and DatasetSEO's own observatory crawler. NaturalHerbLibrary.com is also the live test property for Real SEO™ Experiment No. 001, whose hypothesis is that a domain built for AI citability earns AI-system attention before it earns real Google traffic. This is the first hard data point supporting that hypothesis: heavy, sustained, multi-vendor AI-crawler activity months ahead of any real Google click signal.
Medical Marketing showed the opposite pattern: 10,521 impressions and consistent AI-crawler activity across all four sites, and exactly zero Search Console clicks in ninety days. That is not a visibility problem. Google is already showing those pages to searchers who see them and pass. The bot data proves the reach exists; the listing itself is what needs fixing.
Across the full portfolio, only two of eleven constellations (Healthcare & Longevity and One Mouse Marketing) cleared both bars at once: real human clicks and heavy AI-crawler reads. krisada.com and realseolife.com carried the two highest 90-day click counts in the whole portfolio while both pulling thousands of AI-crawler requests a week.
What Bot Traffic Measurement teaches us
A page with almost no clicks and heavy AI-crawler traffic is not underperforming. The instrument being used to judge it just can't see who is reading it.
Every SEO tool built in the last twenty years was built to answer one question: did a human click. That question still matters, but it stopped being the whole question the day AI crawlers started reading pages at real volume without ever producing a Search Console row. Bot visibility was measurable the entire time. It just required pointing two datasets that already existed at each other.
The next milestone for Experiment No. 001 is not more crawl volume. It is the first confirmed AI citation, tracked separately from bot visibility, since a crawler reading a page is a precondition for citation, not proof of it.
Explore More Cases Like Bot Traffic Measurement
Compare the starting conditions, changes and results across the case study library.