Google-GeminiNotebook and the Crawl Budget Rewrite (July 2026): What Michigan Catalog Sites Should Read in Crawl Stats Before Googlebot Capacity Drops

On July 17, 2026, Google’s crawling documentation changelog renamed the user-triggered fetcher Google-NotebookLM to Google-GeminiNotebook, with the old string supported only through a transition window reporting placed at roughly August 2026. Five days later, on July 22, Google rewrote its “Optimize your crawl budget” page and added a sentence that changes how you should read every crawl anomaly in your logs: the crawl capacity limit is shared across all crawlers.

Neither item is an algorithm update. Together they describe a resource ceiling most Michigan catalog and supplier sites have never measured — and a bot rename that will quietly break firewall rules somebody wrote in October 2025 and forgot about.

What exactly changed in Google’s crawl documentation in July 2026?

Two things: a user agent string and a doc rewrite. The string change is cosmetic; the doc rewrite is not.

Google first added the Google-NotebookLM fetcher in October 2025. When NotebookLM was rebranded to Gemini Notebook, the agent followed. The July 17 changelog entry read: “Updated the NotebookLM user agent to be Google-GeminiNotebook. If you hardcoded the old value in your code, update the string to avoid potential bugs. We will continue to support the old value to allow for a smooth transition.” Shauvik Kumar flagged it on X the same day, noting the former agent is supported until around August 2026.

The July 22 crawl budget rewrite was, in Google’s framing, about clarity and terminology consistency. Most of it is editorial cleanup. Three additions are not, as Search Engine Roundtable and Glenn Gabe both documented:

• “Every site starts with the same default, conservative crawl capacity limit. If there is demand to crawl more and the site remains healthy, Google’s systems will automatically adjust this limit over time.”
• “While each crawler has a different crawl demand, the crawl capacity limit is shared across all crawlers. This means that high demand from one crawler can reduce the capacity available for others.”
• New guidance to support HTTP 304 (Not Modified) responses, so Google reuses its cached copy instead of re-downloading a page that hasn’t changed.

Why does a renamed AI fetcher matter to a site that never blocked it?

Because firewall rules and log dashboards are string matches, and string matches stop matching when the string changes.

Two failure modes, both quiet. A WAF rule naming Google-NotebookLM silently lapses when the transition window closes, and traffic you deliberately excluded resumes with no alert firing. Or your bot report shows a “new” unrecognized agent hitting your PDF library and somebody blocks it in a panic. Neither costs a ranking on Tuesday. Both get discovered in November, while trying to explain a number.

Does Google-GeminiNotebook actually obey robots.txt?

No. Google’s own documentation states that user-triggered fetchers generally ignore robots.txt rules, because the fetch was requested by a person rather than initiated by Google.

This is documented design, not misbehavior, and it predates the rename. The practical consequence: a Disallow line will not stop it. Server-level or edge blocking is the only control that works. Google publishes the address ranges for these agents in user-triggered-fetchers.json and user-triggered-agents.json. Verify against those before blocking anything — a user agent header is a claim, not a credential, and spoofed Google strings are routine.

One dead end worth naming: Cloudflare’s Content Signals robots.txt directive is not a substitute. On July 6, 2026, John Mueller said on Reddit it has “no effects whatsoever for any crawler or llm” and “just adds bloat and future maintenance to your robots.txt file.”

How does a shared crawl capacity limit actually work?

Crawl capacity is a ceiling on how many simultaneous fetches Google will make and how fast. Google sets it from your server’s observed health under load — not from how important your content is.

Google raises the ceiling when responses stay fast and error-free as volume climbs, and lowers it when response times stretch or 5xx rates rise. That’s the whole feedback loop. Demand — how much Google wants from you — varies by crawler and by how fresh your pages look. Capacity — how much your origin will bear — is one shared pool. A demand spike from one Google agent consumes headroom another agent would otherwise have used.

Work that through. A 40,000-SKU aftermarket catalog in Warren publishes 3,000 new year/make/model fitment URLs in September. The same weeks, a Gemini Notebook fetch surge hits its installation-guide PDF library. New-product discovery in Search Console slows. Nothing about the content changed, no manual action exists, and the coverage report will never name the cause — because the cause is a latency signal, not a quality signal. The same mechanism predicts a case nobody warns you about: a heavy Shopping feed refresh hammering product pages will slow the crawl of your blog.

Should you read Google’s “shared across all crawlers” line literally?

Read strictly, probably not — and it doesn’t matter, because the strict reading points you toward the wrong fix.

Google’s sentence says “crawlers,” and Google’s own taxonomy files common crawlers, special-case crawlers, and user-triggered fetchers into three separate documents. Read literally, the shared-pool line most defensibly covers Googlebot, Google-Extended, and GoogleOther, and says nothing about Gemini Notebook. Don’t lean on that. What sets the ceiling is server health, and your origin does not consult Google’s taxonomy before it slows down. Whichever bucket a fetcher sits in, its requests hit the same machines, occupy the same connection pool, and generate the same latency Google reads. Budget total bot load as one number.

How do you diagnose a crawl capacity squeeze in one afternoon?

Start in Search Console’s Crawl Stats report to establish whether you’re capacity-limited or demand-limited, then confirm in raw server logs.

1. Chart requests against response time. In Search Console, open Settings → Crawl stats and compare crawl requests with average response time over 90 days. Requests falling while response time rises is the signature of a capacity ceiling.
2. Open the “By response” breakdown. A rising share of 5xx codes confirms Google is backing off deliberately.
3. Grep logs for both agent strings. Count daily requests for Google-NotebookLM and Google-GeminiNotebook across 60 days. Overlap is expected; a hard cutover tells you the window closed.
4. Verify before blocking. Match source IPs against Google’s published fetcher ranges, or run a reverse DNS lookup. Never block on the user agent header alone.
5. Hunt down every hardcoded old string. Check .htaccess, nginx config, Cloudflare WAF rules, log dashboards, and any Python or PHP parser your team wrote. Update each to test both values.
6. Turn on real 304 handling. Confirm your server honors If-Modified-Since and If-None-Match on product and category pages, not just static assets. Highest-leverage item here: it raises effective capacity for every crawler at once rather than trading one against another.
7. Cut crawl surface you never wanted indexed. Faceted filter combinations, session-ID parameters, internal search results, print views. On a fitment catalog, that is where the majority of crawlable URLs usually live.
8. Re-check Crawl Stats in 14 days. Google’s doc says the limit adjusts “over time,” not overnight. Judge the fix on two weeks, not two days.

If nobody on your team reads server logs, this is the kind of technical groundwork our Detroit SEO team runs before touching a word of content.

What should Michigan catalog and supplier sites put on the calendar?

Four dates, and only one of them is technically a crawl issue.

• Now through August 2026 — the Google-NotebookLM string remains supported during the transition. Update every hardcoded reference before that window closes.
• August 17, 2026 — Google’s target-based bidding change for budget-limited campaigns takes effect. Unrelated to crawling, but it competes for the same engineering hours, which is why it belongs here.
• September 15, 2026 — Cloudflare’s new defaults apply to domains onboarding to Cloudflare: Training and Agent blocked by default on pages that display ads, Search left allowed. If you migrate a site this fall, choose that setting rather than inheriting it.
• October through December — the worst time to discover a capacity ceiling. Winter service parts, powersports, and snow equipment inventory all land in the weeks Googlebot needs to re-crawl it.

The regional shape of this is specific. Aftermarket distributors in Warren and Livonia run parametric catalogs where one part multiplies into thousands of fitment URLs, so crawl capacity — not content quality — is the binding constraint on how fast new inventory becomes findable. Tier 1 and Tier 2 suppliers around Auburn Hills and Sterling Heights carry spec-sheet PDF libraries that are ideal fodder for a user-triggered AI fetcher and return no referral traffic in exchange.

Frequently asked questions

Will blocking Google-GeminiNotebook hurt my Google Search rankings?

No. Gemini Notebook’s fetcher is separate from Googlebot, and blocking it at the server or edge does not remove you from the search index. What you give up is inclusion when a person pastes your URL into Gemini Notebook. Weigh that against the server load and the absence of referral traffic, then decide per directory rather than sitewide.

Do I need to do any of this if my site has only a few hundred URLs?

Mostly no. Google has said for years that crawl budget is not a practical concern for small sites, and the July rewrite didn’t change that. Do the user agent string audit anyway if you ever wrote a block rule — that takes ten minutes. Skip the crawl surface reduction work unless your URL count is in the tens of thousands.

How do I tell a capacity problem from a demand problem?

Look at response time. If crawl requests are falling while average response time climbs, Google is throttling itself because your server is struggling — a capacity problem, fixable with engineering. If requests fall while response time stays flat and healthy, Google simply wants less from you — a demand problem, fixable only with better content, freshness, and links.

The through-line across both July changes is that visibility now rests on infrastructure decisions that never used to be marketing decisions. Response time, cache headers, and firewall rules determine how much of your catalog Google can see at all, before ranking enters the conversation. Pair the crawl fix with a deliberate Google Ads and AI search strategy so the capacity you free up has somewhere to send people.