KieranThomas

Member
Joined
Sep 30, 2021
Messages
35
Reaction score
11
Hi all.

We've never had a problem getting site content indexed, but for one of our clients we've noticed that none of their blog content has been getting indexed since July 2021 (over 8 months ago).

The current status for each is: "Discovered – currently not indexed"
The default explanation text says often because a site can't be reached. However, that doesn't seem to be the problem here as our sites have around 99% uptime, so even if it couldn't access to the site for a few seconds on one visit, it certainly should have been able to access it over 8 months (In addition, GSC confirms other pages & the sitemap have been accessed OK regularly)

The site also doesn't have a vast number of pages, so crawl budget shouldn't be an issue.

No matter what we try, we just can't seem to get it or any subsequent blog content to be indexed. Initially we wondered if they may have had a manual action before we took them on, but there seems to be no evidence of that and GSC confirms everything is clean.

We've checked everything we can think of to ensure there's not a problem our end and haven't been successful, so just curious if anyone can spot anything we've missed?

Here are 2 example URLS:
GSC confirms that:
  • There are no manual actions
  • There are no security issues
  • That each of the URLs have been "discovered"
  • That each of the URLs are able to be crawled (They just haven't been crawled)
To try to encourage Google to index it, we've checked that:
  • Each article is internally linked to from several other articles which are indexed, and the sitemap
  • We've linked to the article from various off-site sources including:
    • Facebook
    • LinkedIn
    • Google Business Profile Posts
  • We've even fired off pings using Pingomatic
  • We've checked the sitemap is valid, and update it to include <lastmod> dates (Albeit, these currently default to the date on which the sitemap was regenerated) http://www.thinking-software.com/sitemap.xml
  • http://www.thinking-software.com/sitemap.xml'] The pages aren't blocked by either the robots.txt file or robots on-page meta data
We've also checked:
  • The content vs other ranked content to ensure it adds value (We know indexing isn't guaranteed, and wanted to ensure it wasn't getting classed as "thin" content)
  • That there are no technical issues which could be preventing them from being indexed, or being desirable to include in the index (i.e. ensuring page speed isn't slow, etc)

    (Note: To help keep this post short, the above isn't a definitive list of everything we've tried, but gives the overall gist for the most obvious areas)
We've requested a manual re-indexing a couple of times over the 8 months (given the timeframe, we felt an additional request was acceptable)

We've even resorted to reporting it as a possible indexing bug, but have received no reply from Google: Sign in - Google Accounts

As an aside, we checked Bing Webmaster tools and that confirmed no issues, and "what is buddy punching" currently ranks in position 3 in Bing, so there doesn't seem to be a problem there.

We're fresh out of ideas. Can anyone else spot anything we may have missed?

Thanks!
 

KieranThomas

Member
Joined
Sep 30, 2021
Messages
35
Reaction score
11
Just a quick update to say that our developers have checked the system resources.

Disc storage is at 68% but as it's storage I don't expect that's the issue
Physical memory usage is typically around 3.62%
Bandwidth is 0.36%

So on the whole, that seems to indicate (to me) it's not a resources issue.

Admittedly page load times (on mobile) aren't great, but the site is due to undergo a rebuild shortly so this should be significantly improved.... plus, we've seen much slower times rank, so feel that alone wouldn't be the issue.

Any other ideas from anyone?
 

carolyn2

Member
Joined
Aug 10, 2015
Messages
42
Reaction score
36
I've been working on indexation testing since August 28, 2021 - I have discovered some essential information about how to ask Google to index - I shared all my testing and what is working in an SEO Fight Club show. Feel free to watch it and I'll answer any questions you have once you have seen all the testing -
 

KieranThomas

Member
Joined
Sep 30, 2021
Messages
35
Reaction score
11
Thanks for the video Carolyn. I'll have to try to watch it in my spare time, as there no chance I'll get away with watching a 1hr video at work :)

In the mean time, would you be able to quickly summarise what aspect of the video may help resolve the above issue / share a timestamp so I can jump to the relevant section?

(Note: We've already submitted a manual indexing request via GSC for each of the effected pages)

Thanks
 

carolyn2

Member
Joined
Aug 10, 2015
Messages
42
Reaction score
36
It's a full test review so that everything I say is backed up with repeated results. Synopsis - for new content do NOT submit via GSC indexation request - connect site into the Google Indexing API and IndexNow - only submit new content via indexing api at least for now. Initial questions are 90% likely to be answered by watching video.
 

JeffClevelandTN

LocalU Member
Joined
Apr 1, 2021
Messages
137
Reaction score
87
FYI - next week I'm doing a live presentation to share the results and some updates and take questions - if you'd like to get a zoom link heads up on date and time (I'm not sure right now but it should be next week once we're into May) go here - How To Get Pages Indexed into Google

Hey Carolyn, unfortunately I missed the live zoom presentation. Will you be posting this for offline viewing? Thank you!
 

Login / Register

Already a member?   LOG IN
Not a member yet?   REGISTER

LocalU

Most UpVoted Answers

  Promoted Posts

New advertising option: A review of your product or service posted by a Sterling Sky employee. This will also be shared on the Sterling Sky & LSF Twitter accounts, our Facebook group, LinkedIn, and both newsletters. More...
Google Product Exert


Top Bottom