Google Will Slow Crawling When Page Load Is Two-Seconds Or Longer

[djbaxter]

Google Will Slow Crawling Your Site When Page Load Is Around Two-Seconds Or Longer
by Barry Schwartz, Search Engine Roundtable
March 30, 2015

Google's John Mueller called out a specific load time as being too slow for GoogleBot to crawl the site at its normal rate. John Mueller said in a Google Webmaster Help thread:

We're seeing an extremely high response-time for requests made to your site (at times, over 2 seconds to fetch a single URL). This has resulted in us severely limiting the number of URLs we'll crawl from your site, and you're seeing that in Fetch as Google as well. My recommendation would be to make sure that your server is fast & responsive across the board. As our systems see a reduced response-time, they'll automatically ramp crawling back up (which gives you more room to use Fetch as Google too).

He specifically called out "over 2 seconds" to load a single URL on this site, which is resulting in GoogleBot "severely limiting the number of URLs" it will crawl on that site.

Read more...

 

[Linda Buquet]

Re: Google Will Slow When Page Load Is Two-Seconds Or Longer

Thanks for sharing David! So darn many things to keep on top of.

I can't imagine being a design or SEO firm having 20 clients to worry about, much less 2,000 customers.
It's overwhelming!

 

[Phil Rozek]

Re: Google Will Slow When Page Load Is Two-Seconds Or Longer

Thanks, David. Nice find. Between this and the upcoming mobile-friendliness diktat, Google sure is getting picky.