Crawl Stats Report: Reading Host Status and Crawl Responses

In this article
- Where to find Google Search Console crawl stats and when a small site needs them
- The three summary charts: total crawl requests, total download size, average response time
- How to read host status in Search Console
- What do crawl requests by response tell you?
- File type, crawl purpose and Googlebot type
- Checking crawl stats after a server change: a practical routine
- Reading the crawl stats report with the change date in mind
- FAQ
The crawl stats report in Google Search Console tells you three things: how often Googlebot requested your pages, how your server answered, and whether Google hit any availability problems along the way. After a hosting, IP or proxy change, it is the fastest way to confirm that crawling carried on as normal.
Where to find Google Search Console crawl stats and when a small site needs them
Go to Settings (property settings), then Crawl stats, as described in Google’s Crawl Stats report help page. The report is available only for root-level properties: a Domain property or a root URL-prefix property. Google is direct about who needs it: “If you have a site with fewer than a thousand pages, you should not need to use this report or worry about this level of crawling detail.” The same page notes that crawl data includes both http and https requests, even for URL-prefix properties, and that a parent domain also shows requests to its subdomains. A small site owner still has good reasons to open it:
- after a server, IP or DNS change,
- when pages stop updating in search results.
The three summary charts: total crawl requests, total download size, average response time
Total crawl requests is how many requests Google made, and on which days. Total download size is how much data Googlebot pulled from your server.
After a move, the chart to watch is average response time. Googlebot is designed not to overload a site, so slow answers and fewer requests often go together. If requests drop right after the switch and response time climbs at the same moment, look at the new setup first.
How to read host status in Search Console
Host status answers one question: did Google run into availability issues while crawling your site? It covers three areas: robots.txt fetching, DNS resolution and server connectivity. If it shows a problem, open the details for each area.
Every detail chart has a dotted red line, and a day when the metric goes above that line counts as an issue for that area. In the days after you change an A record or an IP address, DNS resolution and server connectivity are the two to watch.
Robots.txt fetching is simpler. If the file returns neither a valid file nor a 404, Google slows or stops crawling until it gets an acceptable response; fixing those errors is a topic of its own. One side effect is worth knowing now: when robots.txt stays unavailable for too long, Google counts crawls it might have made, so the total in the report can be higher than the Googlebot hits in your server logs.
What do crawl requests by response tell you?
This breakdown shows how your server answered Googlebot. A fresh cluster of errors right after a move usually points at the change itself. In the days after the switch, watch 5xx server errors and the overall share of non-OK responses. Redirects and 404s can be expected if URLs changed at the same time, so judge them against what the move was meant to do. Click any response type to see example URLs.
File type, crawl purpose and Googlebot type
File type lists the kinds of resources Google requested: HTML, images, scripts and other files. Crawl purpose splits requests into two groups: Discovery means “The URL requested was never crawled by Google before”, and Refresh means “A recrawl of a known page”. If Refresh holds steady after a move, Google is still revisiting your known pages. Googlebot type shows which Google crawler made the requests.
Checking crawl stats after a server change: a practical routine
Compare the same metrics before and after the switch, day by day:
- Write down the exact date and time of the change for each site.
- Compare the weeks before and after that date in the summary charts.
- Watch 5xx responses and average response time for any new spike.
- Open host status and check DNS resolution and server connectivity.
Then match Googlebot requests in your server or proxy logs for the same days against the report. Small gaps are normal, but a long stretch with no Googlebot hits in the logs needs a closer look.
The same check applies when you move sites behind a reverse proxy with own IP such as Jalvo, which gives each site its own IP address and location while your hosting stays where it is. You point the A record to the new address and run the same before-and-after comparison. If you are still working out how many IP addresses you need across your sites, with options in the USA and Europe, the proxy plans and pricing list the available setups.
Reading the crawl stats report with the change date in mind
For small sites, the crawl stats report is a post-change check, not a daily dashboard. Read every chart against the date you changed hosting, IP or DNS. If host status shows no problems, response times are stable and there is no new 5xx spike, Googlebot is crawling your sites normally after the move.
FAQ
Why don’t I see Crawl stats in my Search Console property?
The report only exists for root-level properties: a Domain property or a URL-prefix property at the root of the site.
Does a small site need to check the crawl stats report regularly?
Usually not. Google says sites with fewer than a thousand pages shouldn’t need this level of crawling detail. Open it after a server, IP or DNS change, or when pages stop updating in search.
Why is the crawl count higher than the Googlebot hits in my server logs?
This can happen when your robots.txt file is unavailable for too long. Google then counts crawls it might have made but doesn’t send those requests. Check robots.txt fetching in host status to confirm the cause.


