Why did google stop indexing only my Blog pages?

[mikepcservice]

Hey guys, a local plumber here has a site which was ranking very well for the past 5 years until last year when his Stats started slowly dropping each month. We were baffled by this as we were consistently still doing one monthly blog and monthly backlinking without missing not even month.

I only recently noticed that only his blog pages had issues all this time not being indexed so my fault in not seeing this sooner. GSC shows 59 blog pages not indexed so I reached out to a WP Coder who works on getting those pages indexed. Here's what he says when I asked 'why the pages were not being indexed any more'?

Issue is crawling. And also a crawl budget. Crawl budget is the number of pages Googlebot crawls and indexes on a website within a given timeframe.
So if your number of pages exceeds your site’s crawl budget, you’re going to have pages on your site that aren’t indexed. and they won't get indexed until the crawl budget is reset.

So I then asked what needs to be done to 'Reset the crawl budget' and also if that will solve the problem so no more indexing issues? His response:

Nothing.. we can't do anything to it..They pile up over time, we can try to index them. Most of These urls have not been crawled properly so i will re-crawl them which will result in Indexing. My work only affect current pages. So there won't be any fix for future post or pages.
But if you have a lot of urls already indexed those will help new one get indexed and also crawler won't have to worry about old urls..

So guys, any word on all of this please, is he 100% accurate in every he has said that nothing can be done to make sure all 'future' pages will be indexed

 

[CarrieHill]

I doubt "crawl budget" is the issue - there are million page sites that get plenty of "budget" and I doubt this plumber's site is "too big." More than likely - the blog posts are not unique enough from other content Google has found locally or on the web in general, so Google is crawling them, but not indexing. You can see crawled not indexed in GSC as well - I'd look at that under the pages report.

I'd do one of 2 things here -

1. If the content is not outdated and is good content - I'd write more unique content (NOT AI) and use things we recommend on local landing pages - like testimonials, project before and after photos, unique things about doing their work in that area (humidity/harsh weather/winters/hurricanes), etc. Also, ensure there are enough internal links from static crawled pages pointing at the post.

2. If the content is old and not very useful to someone needing a plumber these days, I'd prune it and redirect to something helpful or to the homepage.

Hope this helps!

 

[mikepcservice]

I doubt "crawl budget" is the issue - there are million page sites that get plenty of "budget" and I doubt this plumber's site is "too big." More than likely - the blog posts are not unique enough from other content Google has found locally or on the web in general, so Google is crawling them, but not indexing.

Thanks, I am thinking too that you are correct. There are many blogs on the site with the same old re-written content for the different keywords and topics which is what I am thinking had caused this problem and not crawling budget.

So first, before following your suggestions [thank you]. I am now thinking to remove all re-written blogs and leave only the original blog for each topic, might that be a good idea please?

 

[CarrieHill]

So first, before following your suggestions [thank you]. I am now thinking to remove all re-written blogs and leave only the original blog for each topic, might that be a good idea please?

If the "rewritten" urls are "crawled, not indexed" - but have better content - I'd merge the good stuff into the indexed url, redirect the "not indexed" url to the new url and see how that does for a few URLS before I went at this with a chainsaw when you might need a scissor.

It may be that the rewritten content is better - but not THAT much better - so eliminating the OLD url and redirecting to the new url may be better - you could try both tactics on a few pages and see which one gets you the better result.

 

[mikepcservice]

If the "rewritten" urls are "crawled, not indexed" - but have better content - I'd merge the good stuff into the indexed url, redirect the "not indexed" url to the new url and see how that does for a few URLS before I went at this with a chainsaw when you might need a scissor.

It may be that the rewritten content is better - but not THAT much better - so eliminating the OLD url and redirecting to the new url may be better - you could try both tactics on a few pages and see which one gets you the better result.

Hi Carrie. to update, google tech has just confirmed that it is indeed crawl budget issues and says nothing 'they' can do. So they have linked me to this article and have recommended we do all that's mentioned.

Crawl Budget: What it is and How to Optimize Yours

What is crawl budget in SEO? A crawl budget refers to the allowance Google gives its crawlers. Learn why it matters and how you can maximize it for SEO success

aioseo.com

Over the next few days I will try to find and remove all duplicate posts doing as you have suggested above. Will report back when all is said and done, totally appreciate your help and advice with this situation!

 

[CarrieHill]

Hi Mike,

What is "Google tech"? Google themselves will rarely tell you that X is the specific problem. I do think pruning your blogs, ensuring you have content that adds value to a subject and is unique will help - but I don't really consider that a "crawl budget" issue - more of a "duplicate and repeated content isn't needed and won't be included" issue. The issue is content, not "space"

 

[noahlearner]

I think asking the question, “Why does my content on THIS PAGE deserve to rank over all the content available on the web for this topic, ” is THE way to look at it.

If a site has lots of pages with meh answers for a topic, search engines have to decide which of them to index and rank.

And if meh is the best that a site offers, then that sites rankings will be...

 

[mikepcservice]

Hi Mike,

What is "Google tech"? Google themselves will rarely tell you that X is the specific problem. I do think pruning your blogs, ensuring you have content that adds value to a subject and is unique will help - but I don't really consider that a "crawl budget" issue - more of a "duplicate and repeated content isn't needed and won't be included" issue. The issue is content, not "space"

Oops, so sorry that was a typo, should have been Godaddy tech.

 

[mikepcservice]

Hi Mike,

but I don't really consider that a "crawl budget" issue - more of a "duplicate and repeated content isn't needed and won't be included" issue. The issue is content, not "space"

Totally agree! If it was a space issue then I am thinking the GD Tech would have said to upgrade the Hosting to like a dedicated server.

 

[noahlearner]

And Mike, I didn't mean that comment in a mean way; I just think that is the best question to frame all the others when doing an analysis like this.

 

[mikepcservice]

And Mike, I didn't mean that comment in a mean way; I just think that is the best question to frame all the others when doing an analysis like this.

I never thought your answer 'mean', exactly the opposite, it hits the nail on the head, that is perhaps the best question put in the best way, which I myself will ask myself when writing blogs from now on so THANK YOU!! You know the proverb 'teach a man to fish...'

 

[noahlearner]

Crawl budget problems don't really impact sites unless they have ~1 million pages or more. Almost all Local websites or service area business sites won't face that problem.

All (or nearly all) of the algorithm updates that we've seen over the past 16 months have focused on promoting helpful, useful content. Along the way lots of real content producers have been collateral damage, but for the most part the sites that have been hit the hardest have been low quality content sites.

 

[bdlowery]

There's already hundreds of thousands of 'blog' posts on how to unclog a toilet, how to fix a leaky faucet, etc etc. Your content needs to be unique and add something of value to be indexed / have backlinks pointing to it so Google knows it's a good piece of content.

Blog post #45 explaining how to clean your kitchen sink drain the same way the other 100,000 blog posts have written is never going to rank.

I'd take a look at what this website is doing - a1garage.com - and try to emulate that.

 

[mikepcservice]

There's already hundreds of thousands of 'blog' posts on how to unclog a toilet, how to fix a leaky faucet, etc etc. Your content needs to be unique and add something of value to be indexed / have backlinks pointing to it so Google knows it's a good piece of content.

Blog post #45 explaining how to clean your kitchen sink drain the same way the other 100,000 blog posts have written is never going to rank.

I'd take a look at what this website is doing - a1garage.com - and try to emulate that.

Wow, thank you very much for that link, I will indeed scrutinize that entire site and especially their ton of blogs to get ideas!

Yes, knowing what to blog about was always an issue since I myself am not a plumber. The plumber himself doesn't believe in showing people how to do stuff otherwise his question is 'then why would they come to me'?So 'how to' blogs was out for me.

When i advised him to write about any trending Industry news like the latest shower system or which might be the best washing machine to get and why e.t.c, his question is 'well how will that help me get jobs?'. Those were the issues I have always faced with him.

 

[mikepcservice]

I'd take a look at what this website is doing - a1garage.com - and try to emulate that.

Looking at the blogs there which will be great inspiration for me [thanks again!], I am noticing that they all have no internal and external links and mainly no CTA's, any reason for this you think?

 

[bdlowery]

Looking at the blogs there which will be great inspiration for me [thanks again!], I am noticing that they all have no internal and external links and mainly no CTA's, any reason for this you think?

Not sure. This is a website managed by @JoyHawkins and her team at Sterling Sky