djbaxter

Administrator
Administrator
Moderator
Joined
Jun 28, 2012
Messages
3,453
Reaction score
1,587

New and improved crawl stats for your site

by Hillel Maoz, Engineering lead for Search Console
Nov 24, 2020

To help website owners better understand how Googlebot crawls their sites, we’re launching a brand new version of the Crawl stats report in Search Console.

The new Crawl Stats report brings the following exciting new features:
  1. Total number of requests grouped by response code, crawled file type, crawl purpose, and Googlebot type.
  2. Detailed information on host status
  3. URL examples to show where in your site requests occurred
  4. Comprehensive summary for properties with multiple hosts and support for domain properties

Over-time charts​

The Crawl Stats report enables website owners to see Google crawl data totals and overtime charts for: total requests, total download size and average response time.

over-time-charts.png

Grouped crawl data​


The new version of the report also provides data on crawl requests broken down by response, file type of the fetched URL, purpose of the crawl request, and Googlebot agent. See example URLs of each type by clicking on a row in the grouping table:

grouped-crawl-data.png

High level & detailed information on host status issues​


Host status details in the report let you check your site’s general availability to Google in the last 90 days.

host-status-details.png

For domain properties with multiple hosts, you can check the host status for each of the top hosts presented in the report summary view. This can help you evaluate performance of all hosts under your domain in one place.
multi-hosts-domain.png

In summary, the new Crawl stats report will help you understand how Googlebot crawls your site:
  • See Google's crawling history in the overtime charts
  • See the file types and file sizes returned by your site
  • See crawl requests details in the example lists
  • Track your site’s availability issues in the host status view

Read more...

 

Login / Register

Already a member?   LOG IN
Not a member yet?   REGISTER

Most UpVoted Answers

Trending: Most Viewed

LocalU Podcasts

  Promoted Posts

New advertising option: A review of your product or service posted by a Sterling Sky employee. This will also be shared on the Sterling Sky & LSF Twitter accounts, our Facebook group, LinkedIn, and both newsletters. More...
Google Product Exert


Top Bottom