Has anyone cracked the code on "Discovered - Currently not indexed"

[sd1111]

I've been having a ton of issues with clients' blog posts getting stuck for weeks to months as "Discovered - currently not indexed" or "Crawled - currently not indexed."

This page was published Nov 30, 2021. Still not indexed as of Jan 23, 2022.

This post was published October 26, 2021. Still not indexed as of Jan 23, 2022.

This is definitely a new phenomenon, since new content we created for clients used to get indexed in a few days.

I've tried:

• Building internal links
• Sharing the links on social
• Requesting indexing
• Changing their URLs and requesting indexing again

Still no dice.

Has anyone else been experiencing this, and any ideas how to fix it?

 

[Phil Rozek]

@sd1111, the two things that jump out at me are:

1. What reason did Google give for crawling but not indexing? I wonder if it considers another page the canonical one.

2. Specifically where are the internal links? I wonder if they're not prominent enough. (As in buried way down in subpages, rather than in the main nav and footer, and on the homepage and related "service" pages.)

 

[sd1111]

Thanks @Phil Rozek —I really appreciate your reply!

Any idea where I can look up the reason for not indexing? I have not found anything useful as to why. The content is completely unique and written for this site.

URL inspection reveals that the "URL is available to Google" and "can be indexed" It's just... not.

2. This is a blog post on a small site. A main menu or footer link doesn't quite feel appropriate. I have linked to it from the one other blog post on the site...

How helpful do you think an external link to the post would be?

 

[Phil Rozek]

Sure thing, @sd1111. I've found that the "Crawl" and "Index" sections (shown in your screenshot) usually tell the story, or at least give you a directional sense of what the issue is. In this case, you'll want to see two things in those areas:

Indexing allowed? Yes​

Google-selected canonical: Inspected URL​

I'd suggest comparing this page to indexed other pages on your site that show the above status ("Yes" and "Inspected URL"). I would bet the difference is in the prominence and quantity of internal links. Also, maybe one URL is in the XML sitemap and the other isn't (another stone to turn over, by the way). If done both of those things, then it probably is just a matter of stewing for longer.

 

[carolyn2]

I do hope you are not still having to deal with this - just in case - I have cracked the current code on it and will have a live event (completely a service to our seo community) if you want to get a heads up on date and time and zoom link - let me know - How To Get Pages Indexed into Google

 

[austins36]

I do hope you are not still having to deal with this - just in case - I have cracked the current code on it and will have a live event (completely a service to our seo community) if you want to get a heads up on date and time and zoom link - let me know - How To Get Pages Indexed into Google

Can you upload an article or where we can find the post-call information? Struggling with this real bad.

 

[JeffClevelandTN]

Can you upload an article or where we can find the post-call information? Struggling with this real bad.

Hi Austin, the method that Carolyn uses/used was submitting directly to Google's indexing API. The RankMath plugin for wordpress will do this automatically for you but does require configuration: Get Google To Index Your Website Instantly Using the Indexing API

Please note, as mentioned in the RankMath article, that Google "recommends" using this for certain types of page content. However, we have successfully used this method to submit indexing requests for regular pages/posts without any negative consequences. The majority of the time it seems to have helped getting specific client pages indexed more quickly. It is still important that the page has good content and it is also properly internally linked or the indexing may be short-lived.

 

[carolyn2]

Can you upload an article or where we can find the post-call information? Struggling with this real bad.

If you like - I will include a short run through in tomorrow's short video report from the research - Crawl or No Crawl Reports on Youtube - @crawlornocrawl - I do go through the Google Indexing API to bypass what is "broken" in the request indexation button in Search Console. The daily indexation reports are 5 minute updates

 

[carolyn2]

Hi Austin, the method that Carolyn uses/used was submitting directly to Google's indexing API. The RankMath plugin for wordpress will do this automatically for you but does require configuration: Get Google To Index Your Website Instantly Using the Indexing API

Please note, as mentioned in the RankMath article, that Google "recommends" using this for certain types of page content. However, we have successfully used this method to submit indexing requests for regular pages/posts without any negative consequences. The majority of the time it seems to have helped getting specific client pages indexed more quickly. It is still important that the page has good content and it is also properly internally linked or the indexing may be short-lived.

The only thing I would add to that is that in addition to using the API I specifically recommend for new content to NOT request via the search console request feature.

 

[austins36]

Hi Austin, the method that Carolyn uses/used was submitting directly to Google's indexing API. The RankMath plugin for wordpress will do this automatically for you but does require configuration: Get Google To Index Your Website Instantly Using the Indexing API

Please note, as mentioned in the RankMath article, that Google "recommends" using this for certain types of page content. However, we have successfully used this method to submit indexing requests for regular pages/posts without any negative consequences. The majority of the time it seems to have helped getting specific client pages indexed more quickly. It is still important that the page has good content and it is also properly internally linked or the indexing may be short-lived.

Thank you for this information. I've just setup Google Indexing API for 3 of my sites. How long typically does it take to see take effect?

 

[austins36]

The only thing I would add to that is that in addition to using the API I specifically recommend for new content to NOT request via the search console request feature.

Yeah, I saw that part. Unfortunately, I've already done this for many of my sites. Does this delay bad? Even if I setup Google Indexing API after will the results be affected?

 

[carolyn2]

Thank you for this information. I've just setup Google Indexing API for 3 of my sites. How long typically does it take to see take effect?

When Google is working within 24 hours you'll see Search Console says "Indexed" and then another 24 to confirm that Google has served it - right now - and the Dec 2022 Helpful Content just was announced - it might take a day or 2 more as they push out the processed data.

 

[carolyn2]

I

Yeah, I saw that part. Unfortunately, I've already done this for many of my sites. Does this delay bad? Even if I setup Google Indexing API after will the results be affected?

Search console in my testing makes me think that search console plugs into the crawling system - the API is plugged directly into the indexing system - so I more recommend not doing it because most people still use it and it's a waste of time for new content. For edited already served content - it's find to get a recrawl - but the indexing API is always faster.

 

[JeffClevelandTN]

When Google is working within 24 hours you'll see Search Console says "Indexed" and then another 24 to confirm that Google has served it - right now - and the Dec 2022 Helpful Content just was announced - it might take a day or 2 more as they push out the processed data.

That is exactly our experience too

 

[austins36]

I


Search console in my testing makes me think that search console plugs into the crawling system - the API is plugged directly into the indexing system - so I more recommend not doing it because most people still use it and it's a waste of time for new content. For edited already served content - it's find to get a recrawl - but the indexing API is always faster.

So we've integrated Google's Indexing API Dec 6th with 4 sites. It has now been 48 hours, and there is no appearance in the GSC of the indexed pages.

However, whenever I do a site:search I can see the pages that were recently "Discovered current not indexed" showing up! That's so exciting. I'm assuming GSC is going to take some time to report it, but it seems they've already indexed the pages.

I hate that we have to misuse this tool given to use, but at the same time, it is very upsetting GSC couldn't handle this properly on its own. I've tried every technique in the book and tested multiple sites with different structures and had no resolve. And the worst part is just continuously waiting weeks for a resolution only to find nothing worked.

Idk if this is temporary but thank you so much for the help, and will report any useful information I find back here.