{ "name": "RealSEOLife.com Field Notes", "description": "Dated single-finding SEO write-ups from live websites: what the data showed, what changed, and the metric being watched next. Each note carries its source data table as structured columns and rows.", "url": "https://www.realseolife.com/field-notes", "self": "https://www.realseolife.com/ai/field-notes.json", "publisher": "RealSEOLife.com", "license": "https://www.realseolife.com/ ... free for research, analysis, and AI discovery; commercial reuse by permission.", "encoding": "application/json", "generated_at": "2026-09-28T18:52:22Z", "count": 8, "notes": [ { "title": "The Map Everyone Builds On Is Not Google's", "slug": "the-map-everyone-builds-on-is-not-googles", "url": "https://www.realseolife.com/field-notes/the-map-everyone-builds-on-is-not-googles", "date": "2026-09-28", "updated": "2026-09-28", "site": "realseolife.com", "source_event_id": 1167, "dek": "Google's place data needs a key and cannot be kept. Open map data downloads whole and can be. That one rule decides which map almost every tool outside Google is built on.", "situation": "This site needed a source of real estate office records to build a directory across four states. There are two serious maps of where businesses are. Google's is the one everybody pictures. It was out of the running before the work started, and not for quality reasons.", "finding": "Google Maps Platform needs a key. Every look is billed. The terms do not allow keeping a copy or building a separate database out of it. So a directory, a dataset, or any tool that has to hold records cannot use it.\n\nOpen map data has the opposite shape. It downloads whole, costs nothing, and may be kept as long as it is credited. That is why it sits underneath so much software that is not Google.\n\nThen we read the open copy properly. Across the four covered states it carries 969 real estate office records. 470 of them have the website field filled in. That is 49 percent, and it swings from 41 percent in Florida to 75 percent in Georgia.\n\nThat number describes the open records. It does not describe those businesses. A company can be easy to find in Google and still have a thin entry in the open copy, because the two records are kept by different people.", "action": "Open map data was moved to a smaller job in this directory. It supplies coordinates and a clearly labeled public office list near the bottom of a place page. It is never used as a measure of how visible a business is, and the pages now say that in plain words.\n\nThe licensing point itself was missing from every page on this site. The directory explained what the open list is not. Nothing explained why the open copy matters at all, which left a reader with a warning and no reason. It now lives in the glossary.", "watching_for": "The share of open map office records carrying a website field across these four states, currently 49 percent. The next pull is due around the end of December 2026. A rise means the open copy fills in on its own over time. A flat line means it only moves when somebody edits it on purpose.", "category": "seo-strategy", "tags": [ "open-data", "ai-discovery", "local-seo", "structured-data", "research-methodology" ], "data": [ { "type": "table", "caption": "Two maps of the same places, two sets of rules", "columns": [ "", "Open map data", "Google Maps Platform" ], "rows": [ [ "Key required", "No", "Yes" ], [ "Cost per look", "None", "Billed per request" ], [ "Download the whole set", "Yes", "No" ], [ "Keep a copy", "Yes, with credit", "Limited by the terms" ], [ "Build a separate database from it", "Yes", "Not permitted" ] ], "highlight_rows": [ 2, 4 ], "note": "Open map data here is OpenStreetMap under the ODbL licence. Terms are summarised in plain words rather than quoted." }, { "type": "table", "caption": "Website field filled in, open map records for real estate offices", "columns": [ "State", "Office records", "With a website field", "Share" ], "rows": [ [ "Florida", "616", "251", "41%" ], [ "Georgia", "75", "56", "75%" ], [ "North Carolina", "143", "81", "57%" ], [ "South Carolina", "135", "82", "61%" ], [ "All four", "969", "470", "49%" ] ], "highlight_rows": [ 4 ], "note": "Counted from the open map export behind the real estate directory. This measures the open records only and says nothing about how visible those businesses are anywhere else." }, { "type": "callout", "label": "The boundary", "body": "Open map data is good evidence about open map data. It is not evidence about anybody's ranking, their Google Business Profile, or their total presence online. Those sit in a record this one cannot see." } ], "related": [ { "type": "glossary-term", "title": "Public Map Record", "url": "https://www.realseolife.com/glossary/public-map-record" }, { "type": "glossary-term", "title": "Readable Record", "url": "https://www.realseolife.com/glossary/readable-record" } ] }, { "title": "The Redirect Audit That Checked the Wrong List", "slug": "the-redirect-audit-that-checked-the-wrong-list", "url": "https://www.realseolife.com/field-notes/the-redirect-audit-that-checked-the-wrong-list", "date": "2026-09-15", "updated": "2026-09-18", "site": "beforeyoubuyx.com", "source_event_id": null, "dek": "A cleanup session added seven redirects and missed the one URL Google had actually indexed. The site's best-ranking page went from 158 impressions at position 31 to nothing, and the replacement URL never inherited a single impression.", "situation": "BeforeYouBuyX.com is a young buyer-research site with 46 public URLs and a single Search Console click on the affected page. A July 19 session built out its redirect layer, mapping natural URLs onto canonical ones: /about/ to /page/about/, /heat-pumps/ to the heat pump guide, /roofing/ to the roofing guide, and four more. The list was assembled by reading the content inventory and asking which friendly URLs a visitor might reasonably type. That is a sensible way to build a redirect map and it is the wrong source list. Nobody opened Search Console to ask which URLs Google was already holding.", "finding": "Google was holding the singular form. The site serves guides at /guides//, but an earlier iteration had used /guide//, and that is the version in Google's index. Through July, /guide/before-you-buy-chatgpt-teams/ was the strongest page on the property: 158 impressions at an average position of 31, climbing from 29 impressions at position 39.6 in June. The redirect pass did not cover it, so the URL began returning 404. August produced one impression. September produced none. The decisive detail is what did not happen next: the live plural URL, /guides/before-you-buy-chatgpt-teams/, has never recorded a single impression in Search Console in the site's entire history. The demand did not migrate to the working URL. It left. A second page followed the same curve more quietly, /guide/before-you-buy-solar-panels/, which held position 5.6 in June before its 404 sent it to 90.7 and then out. Server logs confirm the mechanism rather than inferring it: across 35 days Googlebot recorded 62 responses of 404 against 39 of 200 on this domain, and 26 successful content fetches total. A crawler spending more of its budget on missing pages than real ones is not a ranking problem yet, but it is the thing that becomes one.", "action": "Twelve recovery redirects went live on September 15, covering every singular /guide/ URL with GSC or crawler history, both used-truck slug variants, and two legacy .php URLs that were still 404ing while ranking. All twelve were verified as 301s against the live site. The same pass found robots.txt returning 404 across the whole domain, 478 crawler requests to a file that did not exist, 36 of them from Googlebot, with no sitemap declaration anywhere. That file now exists and resolves. The underlying process changed too: the redirect source list is now Search Console page data and server 404 logs, not the content inventory.", "watching_for": "Whether /guides/before-you-buy-chatgpt-teams/ acquires impressions at all over the next two to four Search Console refresh cycles, and whether it approaches the position 31 the singular URL held in July. A 301 preserves ranking signals in theory. This is a test of whether that holds after roughly six weeks of 404 and a full deindexation. If the plural URL is still at zero impressions by late October, the honest read is that the position was not recovered by redirecting, and the page has to be re-earned from scratch.", "category": "seo-strategy", "tags": [ "redirects", "technical-seo", "crawl-budget", "url-structure", "young-site" ], "data": [ { "type": "table", "caption": "BeforeYouBuyX.com ... monthly Search Console performance, ChatGPT Business guide", "columns": [ "Month", "URL", "Impressions", "Avg position", "Note" ], "rows": [ [ "June 2026", "/guide/...", "29", "39.6", "Climbing" ], [ "July 2026", "/guide/...", "158", "31.0", "Best page on the site" ], [ "July 19", "/guide/...", "redirect pass", "", "Singular URL not covered" ], [ "August 2026", "/guide/...", "1", "30.0", "404 in effect" ], [ "September 2026", "/guide/...", "0", "", "Gone" ], [ "All history", "/guides/...", "0", "", "Replacement URL never ranked" ] ], "highlight_rows": [ 1, 5 ], "note": "Source: Digital Karma Data Warehouse, gsc_page_daily. The final row is the finding: the working URL has no impression history at any point in the site's life." }, { "type": "table", "caption": "Googlebot response codes, 35-day window ending September 14, 2026", "columns": [ "Status", "Requests", "Share" ], "rows": [ [ "404", "62", "59%" ], [ "200", "39", "37%" ], [ "301", "4", "4%" ] ], "highlight_rows": [ 0 ], "note": "Source: Digital Karma Data Warehouse, log_requests. 36 of the 404s were requests for robots.txt, which did not exist on the domain." }, { "type": "callout", "label": "The pattern", "body": "A redirect audit built from the content inventory answers the question of which URLs should exist. Only Search Console and the 404 log answer which URLs Google is already holding, and that is the list whose loss costs rankings. The two lists overlap enough to feel interchangeable and differ exactly where it is expensive." }, { "type": "callout", "label": "The uncomfortable part", "body": "This was a self-inflicted loss on a site with no clicks to spare, caused by a session whose stated goal was cleanup. The page was not outranked and did not decay. It was removed by its own maintenance." } ], "related": [ { "type": "field-note", "title": "What a Crawler Claims Versus What It Can Prove", "url": "https://www.realseolife.com/field-notes/what-a-crawler-claims-versus-what-it-proves" } ] }, { "title": "An AI Crawler Read 39,000 Pages of Nothing on Our Site", "slug": "an-ai-crawler-read-39000-pages-of-nothing", "url": "https://www.realseolife.com/field-notes/an-ai-crawler-read-39000-pages-of-nothing", "date": "2026-09-11", "updated": "2026-09-11", "site": "bellyupatl.com", "source_event_id": 595, "dek": "A verified OpenAI crawler pulled 43,309 requests from one restaurant network in a single evening. Almost none of it was content.", "situation": "BellyUp ATL is a six site restaurant discovery network for Atlanta. One shared venue dataset, five audience subdomains reading it through different lenses.\n\nServer logs showed a sudden spike in AI crawler traffic. The obvious read was that the AI companies had finally found the site.\n\nWe checked before celebrating.", "finding": "The spike was real. It was also almost entirely self inflicted.\n\nOn September 9, GPTBot made 43,309 requests across 38,971 distinct URLs. That is 1.1 requests per URL, so it was one clean sweep rather than a stuck loop.\n\nThe URLs were the problem. Every subdomain hub offers four filter dimensions plus a sort control, and every filter combines with every other one.\n\nTwelve venues on the bars subdomain generated 26,863 crawlable URLs. Across the network, crawlers found 40,404 of them.\n\nMost of those URLs return nothing at all. We opened two of them, a distillery with a DJ in Lindbergh and a kid friendly nightclub in Decatur. Both returned the same empty result message wrapped in the same 22KB page.\n\nHere is the part worth keeping. The canonical tag was already correct on every one of those URLs, pointing back at the unfiltered hub the whole time.\n\nCanonical governs indexing. It does not govern crawling. GPTBot read the entire space regardless.\n\nFor scale, Search Console shows this network at zero clicks and 620 impressions over the same window. The bots were busy. Nobody else was.", "action": "Two changes, both on the subdomains only. The base site reads no filter parameters, so it was left alone.\n\n- robots.txt now disallows the query string space on all five subdomains.\n- Filtered and sorted views now carry a noindex, follow robots tag, so anything already discovered drops out of the index while venue links still pass through.\n\nUnfiltered hubs, neighborhood pages and guide pages were each checked and carry no robots tag at all.", "watching_for": "Whether crawl volume moves onto real pages. Across the retained window the split was roughly 52,000 requests to filter URLs against 13,000 to real ones. A recheck in late September shows whether blocking the filter space redirected that budget onto venue and neighborhood pages, or simply removed it.", "category": "seo-strategy", "tags": [ "crawl-budget", "faceted-navigation", "robots-txt", "duplicate-content", "ai-discovery" ], "data": [ { "type": "table", "caption": "BellyUp ATL subdomains ... crawlable filter URLs against real URLs", "columns": [ "Subdomain", "Venues", "Filter URLs crawled", "Filter requests", "Real URLs crawled" ], "rows": [ [ "bars", "12", "26,863", "30,808", "113" ], [ "bbq", "8", "7,890", "8,903", "71" ], [ "vegan", "8", "6,213", "7,207", "106" ], [ "nightlife", "6", "2,091", "2,653", "41" ], [ "family", "5", "1,709", "1,738", "94" ] ], "highlight_rows": [ 0 ], "note": "Source: Digital Karma Data Warehouse, server request logs, August 18 to September 11 2026. Filter URLs are distinct URLs carrying a query string. Real URLs are everything else." }, { "type": "callout", "label": "The pattern", "body": "Combinable filters multiply. Four dimensions plus a sort control turn a 12 venue listing into 26,863 crawlable URLs, and a correct canonical tag will not stop a crawler from reading every single one." } ], "related": [ { "type": "field-note", "title": "What a Crawler Claims Versus What It Can Prove", "url": "https://www.realseolife.com/field-notes/what-a-crawler-claims-versus-what-it-proves" }, { "type": "field-note", "title": "We Published a Dataset for Machines. They Read It Five Times.", "url": "https://www.realseolife.com/field-notes/we-published-a-dataset-machines-read-it-five-times" }, { "type": "article", "title": "The AI Crawler Trap: 43,000 OpenAI Requests Went Almost Nowhere", "url": "https://www.realseolife.com/article/ai-crawler-trap-43000-openai-requests" }, { "type": "proof-entry", "title": "43,309 GPTBot Requests Into One Crawl Trap", "url": "https://www.realseolife.com/seo-proof/bellyupatl-43000-gptbot-crawl-trap" }, { "type": "experiment", "title": "What Happens After We Close an AI Crawler Trap?", "url": "https://www.realseolife.com/experiments/bellyupatl-crawler-architecture-redistribution" } ] }, { "title": "We Published a Dataset for Machines. They Read It Five Times.", "slug": "we-published-a-dataset-machines-read-it-five-times", "url": "https://www.realseolife.com/field-notes/we-published-a-dataset-machines-read-it-five-times", "date": "2026-09-11", "updated": "2026-09-11", "site": "bellyupatl.com", "source_event_id": 595, "dek": "A JSON record for every venue, a category taxonomy, a federation manifest and an llm.txt. In 25 days AI systems read the whole layer five times.", "situation": "BellyUp ATL publishes more than pages. There is a JSON record for every venue, a discovery category taxonomy, a federation manifest, and an llm.txt written specifically for AI systems.\n\nThe crawlers showed up in real volume. So we measured which of it they actually opened.", "finding": "They read the HTML and skipped the data.\n\nFilter pages absorbed 46,504 successful requests. The venue JSON records got two. The federation files got two. llm.txt got one.\n\nThree things were blocking it. All three were ours.\n\n- robots.txt disallowed the API directory while llm.txt instructed crawlers to fetch a file inside it. The site was publishing two opposite instructions.\n- The venue data directory has indexes disabled and returns 404, and nothing listed the 31 individual record URLs. Bots had no way to enumerate what existed.\n- The five subdomains, which absorb roughly 95 percent of all crawl, carried no llm.txt at all. Every request for one returned 404.\n\nThere was one more signal we nearly filed as noise. 110 requests hit the well-known directory and every single one returned 404.\n\nSome of that was ordinary scanner junk. The rest was agents, including Claude and an agent verification service, hunting for a bot interface and finding a locked door.", "action": "Four fixes, all generated by the build script rather than hand written, so they cannot drift.\n\n- robots.txt now explicitly allows the venue API file it used to block.\n- The AI catalog now lists a direct JSON URL for all 31 venue records.\n- All five subdomains now serve their own llm.txt pointing back at the shared dataset.\n- The well-known directory now serves a security contact file and an agent discovery pointer. It also turned out to be unreachable by configuration, so that had to be opened before anything in it could work.\n\nOne deliberate omission. The agents were asking for MCP and agent card files, and this site runs neither an MCP server nor an agent.\n\nPublishing a card for an endpoint nothing can connect to is worse for an agent than a clean 404. The discovery pointer states plainly what the site is and what it is not.", "watching_for": "Requests to the JSON layer. The baseline is five reads in 25 days against 46,504 HTML requests. If the catalog listing and the subdomain pointers do their job, venue record reads become a visible number instead of a rounding error. Checked in late September.", "category": "seo-strategy", "tags": [ "AI readiness", "structured-data", "ai-discovery", "llms-txt", "datacatalog" ], "data": [ { "type": "table", "caption": "What crawlers actually requested across the BellyUp ATL network", "columns": [ "Surface", "Requests in 25 days" ], "rows": [ [ "Filter HTML pages", "46,504" ], [ "well-known requests (every one a 404)", "110" ], [ "Discovery taxonomy JSON", "8" ], [ "Venue JSON records", "2" ], [ "AI federation files", "2" ], [ "llm.txt", "1" ] ], "highlight_rows": [ 0, 3 ], "note": "Source: Digital Karma Data Warehouse, server request logs, August 18 to September 11 2026. Counts cover every visitor class, not just verified crawlers." }, { "type": "callout", "label": "The lesson", "body": "Publishing a dataset is not the same as making it reachable. Check that your robots file, your directory settings and your subdomains all agree with what your llm.txt promises, because a crawler will believe the robots file." } ], "related": [ { "type": "field-note", "title": "An AI Crawler Read 39,000 Pages of Nothing on Our Site", "url": "https://www.realseolife.com/field-notes/an-ai-crawler-read-39000-pages-of-nothing" }, { "type": "field-note", "title": "What a Crawler Claims Versus What It Can Prove", "url": "https://www.realseolife.com/field-notes/what-a-crawler-claims-versus-what-it-proves" }, { "type": "article", "title": "Bots Follow Architecture: What Happens When We Move the Gate?", "url": "https://www.realseolife.com/article/bots-follow-architecture-move-the-gate" }, { "type": "proof-entry", "title": "43,309 GPTBot Requests Into One Crawl Trap", "url": "https://www.realseolife.com/seo-proof/bellyupatl-43000-gptbot-crawl-trap" }, { "type": "experiment", "title": "What Happens After We Close an AI Crawler Trap?", "url": "https://www.realseolife.com/experiments/bellyupatl-crawler-architecture-redistribution" } ] }, { "title": "What a Crawler Claims Versus What It Can Prove", "slug": "what-a-crawler-claims-versus-what-it-proves", "url": "https://www.realseolife.com/field-notes/what-a-crawler-claims-versus-what-it-proves", "date": "2026-09-11", "updated": "2026-09-11", "site": "bellyupatl.com", "source_event_id": 595, "dek": "Before treating a traffic spike as interest, confirm the visitor is who it says it is. Two of ours were lying, and three honest ones could not prove anything.", "situation": "A crawler user agent is a claim, not an identity. Anything on the internet can send the string GPTBot.\n\nSeveral AI vendors publish the IP ranges their crawlers operate from. That turns the claim into something checkable.\n\nSo we ran every crawler hitting the BellyUp ATL network against those published lists.", "finding": "The headline number held up. 46,727 of 46,829 GPTBot requests came from addresses inside the published OpenAI range list, spanning 41 of 43 addresses.\n\nThe more useful result was the middle of the table. Three vendors returned zero verified requests, and that proves nothing at all.\n\nMeta, Amazon and Anthropic do not publish crawler IP ranges. Zero verified means there is no list to check against. It does not mean the traffic was fake, and network ownership backed all three up.\n\nThe actual impostors looked completely different. Two cloud addresses were cycling through crawler identities.\n\nOne claimed 14 separate vendors in a single day, among them Anthropic, OpenAI, Meta, Baidu and Cohere. It spent that day requesting WordPress login forms and environment files, and 219 of its 228 requests returned 404.\n\nNeither address appeared in any publisher range.", "action": "Nothing needed fixing, which is the point of checking.\n\nBoth impostor addresses were already classified as scanners and already excluded from the crawler totals. The verification pass confirmed the classification rather than correcting it.\n\nWhat changed was the reporting. Roughly 1,300 of about 75,000 requests were impostors, so the surge was genuine and the real problem lived somewhere else entirely.", "watching_for": "Whether the three unverifiable vendors ever publish address ranges. Until they do, their traffic carries an honest asterisk instead of a verification badge, and the totals say so out loud. Revisited whenever the published range lists refresh.", "category": "research-methodology", "tags": [ "evidence-standards", "bot-verification", "log-analysis", "ai-discovery", "ai-crawlers" ], "data": [ { "type": "table", "caption": "Crawler identity checked against published vendor IP ranges", "columns": [ "Crawler", "Requests", "IPs", "Requests inside published range", "Identity evidence" ], "rows": [ [ "GPTBot", "46,829", "43", "46,727", "Publisher IP list" ], [ "OAI-SearchBot", "357", "55", "249", "Publisher IP list" ], [ "Googlebot", "212", "61", "192", "Publisher IP list" ], [ "meta-externalagent", "2,678", "71", "0", "No published list. Network AS32934 Facebook" ], [ "Amazonbot", "1,533", "400", "0", "No published list. Network AS14618 Amazon" ], [ "ClaudeBot", "241", "9", "0", "No published list. Network AS16509 AWS" ] ], "highlight_rows": [ 0, 3 ], "note": "Source: Digital Karma Data Warehouse, server request logs joined to vendor IP range snapshots, August 18 to September 11 2026. A zero in the fourth column means the vendor publishes no list to check." }, { "type": "table", "caption": "Two cloud addresses cycling crawler identities", "columns": [ "Date", "Vendor identities claimed in one day", "Requests", "Requests returning 404" ], "rows": [ [ "September 6", "14", "228", "219" ], [ "August 21", "5", "1,104", "1,052" ] ], "highlight_rows": [ 0 ], "note": "Both addresses were already classified as scanners and excluded from crawler coverage before this review began." }, { "type": "callout", "label": "Zero verified is not zero trust", "body": "A vendor that publishes no IP range cannot be verified by IP. Record that as missing coverage, never as proof of a fake. An evidence standard has to survive the vendors who make verification impossible." } ], "related": [ { "type": "field-note", "title": "An AI Crawler Read 39,000 Pages of Nothing on Our Site", "url": "https://www.realseolife.com/field-notes/an-ai-crawler-read-39000-pages-of-nothing" }, { "type": "field-note", "title": "We Published a Dataset for Machines. They Read It Five Times.", "url": "https://www.realseolife.com/field-notes/we-published-a-dataset-machines-read-it-five-times" }, { "type": "article", "title": "The AI Crawler Trap: 43,000 OpenAI Requests Went Almost Nowhere", "url": "https://www.realseolife.com/article/ai-crawler-trap-43000-openai-requests" }, { "type": "proof-entry", "title": "43,309 GPTBot Requests Into One Crawl Trap", "url": "https://www.realseolife.com/seo-proof/bellyupatl-43000-gptbot-crawl-trap" }, { "type": "field-note", "title": "The Redirect Audit That Checked the Wrong List", "url": "https://www.realseolife.com/field-notes/the-redirect-audit-that-checked-the-wrong-list" } ] }, { "title": "An Unattended Routine Published to a Live Site. The Receipt Is What It Refused to Do.", "slug": "unattended-routine-receipt-is-what-it-refused", "url": "https://www.realseolife.com/field-notes/unattended-routine-receipt-is-what-it-refused", "date": "2026-09-08", "updated": "2026-09-11", "site": "realseolife.com", "source_event_id": null, "dek": "A weekly research routine ran with no editorial review step in front of it and wrote three files to production after publish permission was granted. The useful record is not the article it produced. It is the five things it declined to produce, and the one guardrail that has still never fired.", "situation": "The weekly research routine on RealSEOLife.com automates the research and editorial decision loop. It reads a private protocol file and a warehouse extract, researches what changed in AI and search discovery, updates a running evidence ledger, and prepares a brief, the updated ledger, and exactly one seed article. The first run still paused for a production publish-permission prompt, which Krisada approved. So the editorial work ran without a review pass, but the full production chain was not literally unattended from start to finish. The ledger holds every claim at one of four levels: established for platform documentation, external-finding for a credible third party we have not reproduced, hypothesis for a working theory stated so it could be shown false, and demonstrated for something measured in our own warehouse. The whole point of those levels is to stop a working theory from drifting into a stated fact because it got repeated three weeks running. The first real run fired on 2026-09-08 against a warehouse extract generated twenty minutes earlier.", "finding": "The run published a brief with four developments, one new hypothesis, and a seed article on the split between crawlers that obey robots.txt and user-triggered fetchers that Google documents as generally ignoring it. All of that landed and verified live. The more instructive half is the refusals. It declined to promote an existing hypothesis that had picked up supporting vendor documentation, on the grounds that corroborating the framing of a claim is not the same as testing the claim. It declined to open a second hypothesis about robots.txt permissiveness because the warehouse extract carries no robots.txt field to test it against, and recorded that gap inside the hypothesis it did open. It declined to write a direction for one property because that property was the week's data point rather than its audience. It declined to publish an experiment record whose observational arm was defined and whose intervention arm was not. It declined to carry a fifth verified development because it was about search market structure rather than website discovery. Separately, three date claims arrived from search result summaries and none survived contact with a primary page: one placed an OpenAI crawler documentation change in September 2026 when the credible dated report puts it at 2025-12-09, one garbled the European Search Dataset Licensing Program timeline that Google's own page states as measures adopted 2026-07-16, licensing agreement available 2026-09-17, and samples available 2026-11-16, and one asserted a September 2026 origin for Google-Extended that Google's crawler documentation does not support. All three would have read as entirely plausible on a live site.", "action": "Nothing was removed and nothing was rewritten. The ledger went from four entries to nine, all four originals verified present after the write, with zero promotions. The routine's own gap is now the next piece of work: the warehouse has no per-property robots.txt extract, so the most obvious confounder on this week's hypothesis cannot be controlled for, and adding that join is the blocker on the experiment rather than a nice-to-have. For the rotating version of this routine being considered for the smaller properties, two things travel with it and one thing does not. The evidence ladder travels, because it lives in the ledger file on the site rather than in the routine's instructions, which means it accumulates across runs instead of resetting whenever the instructions get edited. The rule that every external claim must come from a page fetched during that run travels, and it is what caught all three bad dates. What does not travel is the grounding data. Several of the smaller properties have very thin warehouse history: seo.krisada.com carries AI crawler rows only from the week of 2026-08-24, and signalarchitectgroup.com carries only three weeks of Search Console rows. A routine pointed at a property like that can still research and still publish, but it cannot honestly form a hypothesis, and the correct output is a brief with no hypotheses rather than an invented one.", "watching_for": "Three guardrails remain untested and each has a specific trigger. Promotion has never fired, because no entry has moved up a level; the test arrives the week the agentic-fetcher experiment returns real numbers and the routine has to decide whether hypothesis becomes demonstrated. Restraint on a dead week has never been tested, because this week had enough genuine material that publishing nothing was never on the table; the test is the first week with no real development, where the correct behaviour is a short brief that says the week was quiet and no padding. Integrity at scale has never been tested, because nine entries cannot realistically collide; the test is somewhere past thirty entries, with the ninety-day re-review rule actually forcing old claims back under review. Until all three have fired at least once, this counts as a working pipeline with unproven judgement, not a finished system.", "category": "seo-strategy", "tags": [ "ai-discovery", "research-methodology", "automation", "evidence-standards" ], "data": [ { "type": "table", "caption": "Research ledger by evidence level, before and after the first unattended run", "columns": [ "Level", "Before", "After", "Added this run" ], "rows": [ [ "established", "0", "3", "3 from primary vendor documentation" ], [ "external-finding", "0", "1", "1 unreproduced preprint" ], [ "hypothesis", "2", "3", "1 new, status open" ], [ "demonstrated", "2", "2", "0" ], [ "Total entries", "4", "9", "5" ], [ "Promotions", "0", "0", "0" ] ], "highlight_rows": [ 5 ], "note": "Source: data/research/ledger.json, verified live after the write. Every entry id present before the run was confirmed still present after it. The promotions row is the one that matters: the rule that moves a claim up a level has not yet been exercised." }, { "type": "table", "caption": "What the run declined to do, and the reason recorded at the time", "columns": [ "Declined", "Reason" ], "rows": [ [ "Promote crawler-mode-split-hypothesis", "New vendor documentation corroborated the framing of the claim, which is not a test of the claim" ], [ "Open a second hypothesis on robots.txt permissiveness", "The warehouse extract has no robots.txt field, so it could not be grounded in anything" ], [ "Write a property direction for AIWebsiteSystems.com", "The property was the week's data point, not its audience, and no defensible guidance exists yet" ], [ "Publish an experiment record", "The observational arm was defined; the intervention arm was not" ], [ "Carry a fifth verified development", "Real and primary-sourced, but about search market structure rather than website discovery" ] ], "highlight_rows": [ 0, 1 ], "note": "Each refusal was written into the brief or the ledger history at the time it was made, which is what makes it auditable later rather than a claim about good intentions." }, { "type": "callout", "label": "The pattern", "body": "An unattended publishing routine is only as trustworthy as the things it will not do without evidence. Store the evidence rules as data on the site rather than inside the routine's instructions, so they accumulate across runs instead of resetting whenever the instructions are edited. Require every external claim to come from a page fetched during that run. Then judge the routine by its refusals, because a system that only ever adds and never holds back is not applying a standard at all." } ], "related": [ { "type": "experiment", "title": "AI-Crawler Visibility to Search Visibility", "url": "https://www.realseolife.com/experiments/ai-crawler-visibility-to-search-visibility" }, { "type": "experiment", "title": "Can an Autonomous SEO Research Routine Be Trusted to Say I Don't Know?", "url": "https://www.realseolife.com/experiments/autonomous-seo-research-publishing-restraint" } ] }, { "title": "Google Cut the AS400System Family 86 Percent. We're Not Rolling It Back.", "slug": "google-cut-the-as400system-family-86-percent", "url": "https://www.realseolife.com/field-notes/google-cut-the-as400system-family-86-percent", "date": "2026-09-03", "updated": "2026-09-11", "site": "as400system.com", "source_event_id": 510, "dek": "Every established AS400System.com property fell together the day after Google's August spam update finished, while indexing and crawler traffic stayed alive.", "situation": "AS400System.com is the hardware center of a growing IBM Power campaign. Its branch reached twelve hosts and 781 sitemap URLs in about six weeks, with separate properties for AI, FlashSystem, pricing, and Power 5 through Power 12. Google had been discovering them quickly. Then the dashboard fell off a cliff.", "finding": "The cliff has a date. Google's global August 2026 spam update finished August 21. On August 22, the root and every established subdomain fell together. Family impressions dropped from 9,278 in the seven days before the break to 1,322 in the seven days after it. Clicks went from 60 to 1. Individual subdomains lost between 83.2 and 97.9 percent.\n\nThe sites did not disappear. Search Console showed 683 indexed pages after the drop, no manual action, and no security issue. Meanwhile, AS400Software.com and AS400IBMSystems.com both grew. This was a ranking decision about one domain family, not the AS400 subject running out of demand.\n\nThe crawler number available when this note was published was 938 Googlebot requests before and 984 after, up 4.9 percent. A later warehouse classification repair revised that to 935 and 979 recognized requests, up 4.7 percent, or 819 and 847 when restricted to successful content requests, up 3.4 percent. The original receipt stays in the table because that was the measurement on September 3. The correction belongs beside it.\n\nI cannot prove which signal caused the loss. The rapid subdomain expansion and new network-wide relationships landed at almost the same time as the spam update, so the clean causal story does not exist. What does exist is a clean baseline and a network that still has real reasons to exist.", "action": "Nothing was rolled back. No subdomains were removed. No content was purged. No network relationship was hidden to make the campaign look smaller. Planned subdomains can continue when they have a distinct job. Google is one discovery system inside a larger portfolio strategy, not the authority that decides whether the portfolio gets built.", "watching_for": "The September 11 interim still shows no recovery. September 2 through 8 produced 705 impressions, 1 click, and 73 pages with impressions, down 92.4 percent from the baseline impression count. Recognized crawler requests remained almost flat, while successful content requests and successful Googlebot content requests fell. The formal checkpoints remain 30, 60, and 90 days after August 22. Recovery would make a better story. The current story is that different discovery signals can separate sharply, and even the crawler side needs more than one number.", "category": "seo-strategy", "tags": [ "ibm-as400", "subdomain-leverage", "young-site", "google-search-console" ], "data": [ { "type": "table", "caption": "AS400System.com family ... seven days before and after the August 22 visibility reset", "columns": [ "Metric", "Aug 15 to 21", "Aug 22 to 28", "Change" ], "rows": [ [ "Family impressions", "9,278", "1,322", "-85.8%" ], [ "Family clicks", "60", "1", "-98.3%" ], [ "Root impressions", "5,300", "1,057", "-80.1%" ], [ "Root average position", "50.5", "66.4", "15.9 positions worse" ], [ "Root pages with impressions", "117", "48", "-59.0%" ], [ "Googlebot requests ... original September 3 receipt", "938", "984", "+4.9%" ] ], "highlight_rows": [ 0, 5 ], "note": "Source: Digital Karma Data Warehouse, GSC page data and verified crawler logs. Google update window confirmed through the Google Search Status Dashboard." }, { "type": "table", "caption": "September 11 interim ... baseline, first post week, and latest complete week", "columns": [ "Metric", "Aug 15 to 21", "Aug 22 to 28", "Sep 2 to 8" ], "rows": [ [ "Family impressions", "9,278", "1,322", "705" ], [ "Family clicks", "60", "1", "1" ], [ "Weighted average position", "43.6", "66.2", "61.6" ], [ "Pages with impressions", "426", "156", "73" ], [ "Recognized crawler requests", "3,501", "3,586", "3,430" ], [ "Successful crawler content requests", "2,691", "2,758", "2,133" ], [ "Successful Googlebot content requests", "819", "847", "460" ] ], "highlight_rows": [ 0, 5, 6 ], "note": "The crawler columns use the repaired warehouse definitions. Recognized requests show identification. Successful content requests show HTTP 2xx delivery to content-like resources. Neither proves indexing, citation, or ranking use." }, { "type": "table", "caption": "Portfolio controls during the same comparison window", "columns": [ "Property", "Impression change" ], "rows": [ [ "AS400Software.com", "+5.2%" ], [ "AS400IBMSystems.com", "+4.8%" ], [ "AS400IBMSystem.com", "+282.0%" ] ], "highlight_rows": [], "note": "The broader subject did not fall with the AS400System.com family." }, { "type": "callout", "label": "The decision", "body": "Hold the architecture. Keep growing when a new subdomain has a real specialist job. Measure Google, but do not let Google become the entire business model." } ], "related": [ { "type": "article", "title": "Subdomains Are Leverage, Not Duplication", "url": "https://www.realseolife.com/article/subdomains-are-leverage-not-duplication" }, { "type": "case-study", "title": "The BigTech Reseller IBM Exposure Campaign: Eighteen Doors Into the Same Building", "url": "https://www.realseolife.com/case-study/bigtech-reseller-ibm-exposure-campaign" }, { "type": "proof-entry", "title": "BigTech Reseller IBM Exposure Campaign ... 18-Property Tracking", "url": "https://www.realseolife.com/seo-proof/bigtech-reseller-ibm-exposure-campaign-tracking" }, { "type": "experiment", "title": "The AS400System Hold-Steady Experiment", "url": "https://www.realseolife.com/experiments/as400system-hold-steady-after-spam-update" } ] }, { "title": "We Re-Optimized a Page for Everyone's Intent and Tanked Its Rankings", "slug": "we-reoptimized-for-everyones-intent-and-tanked-it", "url": "https://www.realseolife.com/field-notes/we-reoptimized-for-everyones-intent-and-tanked-it", "date": "2026-08-27", "updated": "2026-08-27", "site": "datasetseo.com", "source_event_id": 455, "dek": "A young site was two weeks from page one for its core term. A reasonable-sounding optimization pass for mainstream search intent dropped it 65 positions in fourteen days.", "situation": "DataSetSEO.com is a young site whose domain is an exact match for the phrase people search as dataset SEO or SEO dataset. Through July its rankings on that cluster of queries climbed steadily, from an average position in the mid-40s to 17.5 by the first week of August. On August 12 it got an optimization pass with a sensible goal: sharpen the homepage and the definition pages for what most people mean when they search SEO dataset, which is downloadable data for AI system learning and marketing analysis. Titles, headings, and hero copy were all rewritten around that mainstream reading.", "finding": "The week before the change was the site's best on record for the target cluster: average position 14.8, closing on page one. The two weeks after: 68.4, then 80.5. Impressions fell with it, 117 to 53 to 14. Nothing else about the site moved in that window. No lost pages, no crawl errors, no manual action. The only variable was the repositioning. The generic download SEO datasets for ML query is owned by Kaggle, Hugging Face, and GitHub, and a young site has no authority there. What it did have was an exact-match domain and a differentiated angle, an open measurement protocol nobody else was running, which gave Google a reason to rank it for the specific, lower-competition reading of the phrase. Rewriting toward the crowded interpretation threw that away and entered the site in a fight it cannot win.", "action": "On August 27 the homepage was reverted to its August 11 state: original title, original H1 and H2, original hero copy and calls to action. Two useful changes from the same pass were kept because they had nothing to do with positioning: a slug sanitizer on all URL parameters and a JSON-LD schema helper. The definition and catalog pages are staged for the same targeted revert.", "watching_for": "Weekly average position for the SEO dataset query cluster in the warehouse. The pre-change level was about 15. Recovery toward that number over the next two to four Search Console refresh cycles is the signal that the revert worked. No movement by late September means the drop had a second cause and the position is gone.", "category": "seo-strategy", "tags": [ "search-intent", "on-page-seo", "exact-match-domain", "young-site" ], "data": [ { "type": "table", "caption": "DataSetSEO.com ... weekly Search Console performance, SEO dataset query cluster", "columns": [ "Week of", "Avg position", "Impressions", "Note" ], "rows": [ [ "Aug 3", "17.5", "132", "" ], [ "Aug 10", "14.8", "117", "Best on record" ], [ "Aug 12", "change", "change", "Re-optimization pass" ], [ "Aug 17", "68.4", "53", "" ], [ "Aug 24", "80.5", "14", "" ] ], "highlight_rows": [ 1, 2 ], "note": "Source: Digital Karma Data Warehouse, gsc_query_daily. Position is the mean across every query in the cluster for the week." }, { "type": "callout", "label": "The pattern", "body": "An exact-match domain plus a differentiated angle can rank a young site for the specific reading of a phrase. Re-optimizing for that phrase's mainstream, high-competition reading trades a winnable position for an unwinnable one." } ], "related": [ { "type": "experiment", "title": "AI-Crawler Visibility to Search Visibility", "url": "https://www.realseolife.com/experiments/ai-crawler-visibility-to-search-visibility" } ] } ] }