What server log analysis steps should my developer take today to figure out why Googlebot is ignoring our most important newly published category pages?
Download run
Fan-out diagram
The searches the AI actually ran for this prompt.
SEED QUERY
What server log analysis steps should my developer take today to figure out why Googlebot is ignoring our most important newly published category pages?
2 real searches · 1 real-search pass
Google Search Central verify Googlebot requests reverse DNS forward DNS crawler IP ranges server logs crawl stats documentationGoogle ↗
OBSERVEDConsequence
Google Search Central crawling indexing troubleshooting newly published pages internal links sitemap canonical robots.txt server logsGoogle ↗
OBSERVEDReason
Badge and edge weight = share of passes that produced the query. Hover a query for its intent and routing format.
Query classes
Google's semantic query class for every sub-query: what kind of answer each search is after, simulated vs real.
ChatGPT real searches (2)
Classes come from the query-classifier (a local Laya model). 1 of 2 calls are low-confidence (under 50%) — hover a query in the table to see its confidence.
Searches and sources
ChatGPT2 real searches · gpt-6
Google Search Central verify Googlebot requests reverse DNS forward DNS crawler IP ranges server logs crawl stats documentation Google ↗
Google Search Central crawling indexing troubleshooting newly published pages internal links sitemap canonical robots.txt server logs Google ↗
- Troubleshoot Google Search Crawling Errors | Google Search Central | Documentation | Google for Developers · developers.google.com
- What Is Googlebot | Google Search Central | Documentation | Google for Developers · developers.google.com
- Why is my page missing from Google Search? · support.google.com
- Page indexing report · support.google.com
- Verify Requests from Google Crawlers and Fetchers | Google Crawling Infrastructure | Crawling infrastructure | Google for Developers · developers.google.com
- Debug Your Website Pages | Google Search Central | Support | Google for Developers · developers.google.com
Queries
2 queries · click a column to sort
| Query / type | Search | Intent | Query class | Routing format | Cluster | Freq ↓ | Weight |
|---|---|---|---|---|---|---|---|
Google Search Central verify Googlebot requests reverse DNS forward DNS crawler IP ranges server logs crawl stats documentation OBSERVED | Google ↗ AI Mode ↗ | — | Consequence | — | — | 1x of 1 | |
Google Search Central crawling indexing troubleshooting newly published pages internal links sitemap canonical robots.txt server logs OBSERVED | Google ↗ AI Mode ↗ | — | Reason | — | — | 1x of 1 |