djbaxter

Administrator
Administrator
Moderator
Joined
Jun 28, 2012
Messages
3,515
Reaction score
1,662

New and improved crawl stats for your site

by Hillel Maoz, Engineering lead for Search Console
Nov 24, 2020

To help website owners better understand how Googlebot crawls their sites, we’re launching a brand new version of the Crawl stats report in Search Console.

The new Crawl Stats report brings the following exciting new features:
  1. Total number of requests grouped by response code, crawled file type, crawl purpose, and Googlebot type.
  2. Detailed information on host status
  3. URL examples to show where in your site requests occurred
  4. Comprehensive summary for properties with multiple hosts and support for domain properties

Over-time charts​

The Crawl Stats report enables website owners to see Google crawl data totals and overtime charts for: total requests, total download size and average response time.

over-time-charts.png

Grouped crawl data​


The new version of the report also provides data on crawl requests broken down by response, file type of the fetched URL, purpose of the crawl request, and Googlebot agent. See example URLs of each type by clicking on a row in the grouping table:

grouped-crawl-data.png

High level & detailed information on host status issues​


Host status details in the report let you check your site’s general availability to Google in the last 90 days.

host-status-details.png

For domain properties with multiple hosts, you can check the host status for each of the top hosts presented in the report summary view. This can help you evaluate performance of all hosts under your domain in one place.
multi-hosts-domain.png

In summary, the new Crawl stats report will help you understand how Googlebot crawls your site:
  • See Google's crawling history in the overtime charts
  • See the file types and file sizes returned by your site
  • See crawl requests details in the example lists
  • Track your site’s availability issues in the host status view

Read more...

 
Similar threads
Thread starter Title Forum Replies Date
M Google Search Console Errors "Coverage and Breadcrumbs"? Google My Business & Google Maps 4
djbaxter Google services down Dec 14, 2020: Google Ads, Google Analytics, Search Console, more Google News: Important Changes & Features 1
pbarnhart Google Search Console Merging UTM'd URLs into Canonical URLs - Suggestions? Local SEO Tools & Software 5
JoyHawkins Search Console vs Google My Business (GMB) Insights: A Large Case Study Google My Business & Google Maps 4
Barb Davids Google Search Console Bug? Local SEO Tools & Software 2
B Google Search Console Security Issue Local Search 10
Digitaldar Google Search Console Date Screen Issue Organic SEO 0
djbaxter Google's Search Console Training Video Series Local Search 0
M Google Search Console error? Local Search 4
djbaxter Google Search Console: False positive "mobility issues" frequent Mobile & Social 14
Nathan_NZ Adding New Users in Google Search Console - Help Needed Organic SEO 2
djbaxter Preferred Domain in Google Search Console deprecated Google News: Important Changes & Features 4
H Site Change of Address issue in Google Search Console Organic SEO 2
S How to get a DNS-verified Search Console domain to connect with Google Analytics? Organic SEO 3
Meguinness Google search console url format Organic SEO 4
djbaxter Google WordPress Plugin to Integrate Analytics, Search Console, AdSense, PageSpeed Websites, Software, and Security 4
djbaxter Google Search Console bug? Suddenly multiple messages about Mobility Usability Mobile & Social 10
J Blocked Resources on Google Search Console Local Search 3
Dan Foland Starting to see Google Search Console snippet in search results. Anyone else? Google News: Important Changes & Features 4
djbaxter On Google Search Console Organic SEO 0
djbaxter Google: New URL inspection tool & more in Search Console Organic SEO 0
F Google Search Console Issue Local Search 4
Rich Owings Hacked URLs showing up as 404s in Google Search Console Organic SEO 18
H Does Google Search Console Show impressions and clicks from GMB? Local Search 4
DanLeibson Google Search Consoles Search Analytics Report is Not To Be Trusted Local SEO Tools & Software 3
djbaxter Brightlocal Tool shows Large Google Local Search Algorithm Update March 10, 2021 Google Algorithm Discussions 1
djbaxter Webinar Get together with Google Search at Search Central Live - February 23, 2021 Events 0
Lluc Google launches Years in Business in Search Google My Business & Google Maps 3
djbaxter A (home) movie about how Google Search works Organic SEO 0
djbaxter Google Updates Search Quality Raters Guidelines Google News: Important Changes & Features 0
djbaxter SEO myth-busting: What is NOT a Google search ranking factor Organic SEO 3
jimmydoubleu Google business not listed/found on Search/Map after merge Google Duplicates & Merges 6
D SERPRiver - Google Search API in Real-Time to Scrape Search Results Recycle Bin 0
diptimayee Google Updated Its "How Search Works" Page Google News: Important Changes & Features 2
A Updating sitelinks in Google search for 404 pages Organic SEO 6
djbaxter New SEO Tool: Was There A Google Update Search Site Organic SEO 0
P Sudden near disappearance from Google My Business search results Help & Support for Google Local 2
J Google Business not showing up in search. Business not suspended and is verified Google My Business & Google Maps 5
PaulSteinbrueck GMB info in right sidebar for Google search? Local Search 5
A Google Maps Showing Different Map When Searching Name vs. Address Google My Business & Google Maps 1
Colan Nielsen Google Really Pushing Competitor Listings on Knowledge Panels and Branded Searches Google News: Important Changes & Features 4
djbaxter Google's Zero-Click Searches: How to get your traffic back Organic SEO 0
mattrigg Branded Paid Search - Impacts Google My Business from showing Paid Search and Local Service Ads 6
Ian Hammond "Your business is verified and is currently being connected to Google Search and Maps" (30+ days) Help & Support for Google Local 3
L Why is Google showing different categories for this business in the same search/view? NAP, Categories, Dashboard Data 3
L Google now allows Site Owners to Customize the Search Results Snippets, but what can be customized? Google News: Important Changes & Features 1
Yan Gilbert Icons in Google Desktop Search Results Local Search 2
Cordell Crowley Google Showing "Businesses You Manage" in search results? Google My Business & Google Maps 1
Srinath Dynamic matching of Google My Business categories to search queries? Google My Business & Google Maps 1
L How to Remove Non-GMB Location from Google Search? Local Search 2

Similar threads

Login / Register

Already a member?   LOG IN
Not a member yet?   REGISTER

Most UpVoted Answers

Trending: Most Viewed

LocalU Podcasts

  Promoted Posts

New advertising option: A review of your product or service posted by a Sterling Sky employee. This will also be shared on the Sterling Sky & LSF Twitter accounts, our Facebook group, LinkedIn, and both newsletters. More...
Google Product Exert


Top Bottom