"SEO company" Cleanup (~400 duplicate content / no-index pages)

[cfazio]

So ... I just took control of a website that is hosted on "imatrix" they do digital marketing for chiropractors and other medical professionals.

Screaming Frog shows nearly 400 blog posts that are all no-index.

Further research shows that this company sells a "seo content" package as part of their offerings which posts an insane amount of blog posts on the website. Problem is, they sell this "seo content" package to hundreds of chiropractors, and it is word for word duplicate content.

Look at the SS below, 2 different chiro websites, ~400 blog posts each, word for word duplicate content (and there are hundreds more of these copy / paste websites).

All of the category pages (prob like 60) are no index and canonicalized to the main /articles/general category page. None of the posts have canonical tags, also no index.

I would like to just remove this content package and delete all of these no index duplicate pages ... BUT are they having some round about effect on rankings that removal would diminish?

Never imagined running into this sort of issue ... anyone have advice?

*can post URL if anyone is curious to check out what is going on

 

[Justin Mosebach]

@cfazio I ran into a similar issue for a financial planner, where the CMS was specifically for that industry and they had a library of blog posts that were syndicated, but those were indexable.

Because they're noindex, it shouldn't matter. The 2 questions I'd ask are:

• 1. Does Google have any of them indexed?
  1. If all the blog URLs have something similar (like /articles/), do a Google site search for that URL. Example:

     site:example.com/articles/

     . If there's no results, then Google doesn't have anything indexed.
  2. You could also check Google Search Console: Go to Coverage > Valid, and export the lists of URLs listed there (in the screenshot below, that'd be "Submitted and indexed" and "Indexed, not submitted in sitemap"). Note that some of them might not have been crawled in a long time, so if the URLs are currently noindex, but showing up in either list, resubmit them to Google to be recrawled (put the URL in the "Inspect any URL in the current resource" box at the top of the page and then click "Request Indexing".

• 2. When did they become noindex?
  1. Use the Wayback Machine to see if you can figure this out - because they're noindex, the Wayback Machine might not even archive the pages (not sure), so this might not be the best option, but it's worth a shot.
  2. Ask iMatrix if they've always been noindex.

Assuming Google doesn't have them indexed, they should have no impact on Google SEO. You could do a similar check on Bing.

 

[cfazio]

@cfazio I ran into a similar issue for a financial planner, where the CMS was specifically for that industry and they had a library of blog posts that were syndicated, but those were indexable.

Because they're noindex, it shouldn't matter. The 2 questions I'd ask are:

• 1. Does Google have any of them indexed?
  1. If all the blog URLs have something similar (like /articles/), do a Google site search for that URL. Example:

     site:example.com/articles/

     . If there's no results, then Google doesn't have anything indexed.
  2. You could also check Google Search Console: Go to Coverage > Valid, and export the lists of URLs listed there (in the screenshot below, that'd be "Submitted and indexed" and "Indexed, not submitted in sitemap"). Note that some of them might not have been crawled in a long time, so if the URLs are currently noindex, but showing up in either list, resubmit them to Google to be recrawled (put the URL in the "Inspect any URL in the current resource" box at the top of the page and then click "Request Indexing".

• 2. When did they become noindex?
  1. Use the Wayback Machine to see if you can figure this out - because they're noindex, the Wayback Machine might not even archive the pages (not sure), so this might not be the best option, but it's worth a shot.
  2. Ask iMatrix if they've always been noindex.

Assuming Google doesn't have them indexed, they should have no impact on Google SEO. You could do a similar check on Bing.

Yea, site search they do not show up. Have not gotten this site into GSC yet.

Thanks for the wayback tip... the way this company operates it seems like they have been no index since the time they are published on the site. A new one is published every 5-10 days.. even the one from a few days ago is no index.

 

[Conor Treacy]

As @Justin Mosebach said, if they're no-indexed already, then there shouldn't be any effect.

We ran into a similar situation a few years ago with a Chiropractor also. Their posts were No-Index, and they stated on the page that the article was syndicated with permission. They actually used the blog post as part of their Monthly Newsletter to have some information to share, and then point users back to the site.

So while they were no-indexed, they actually did serve a purpose as part of their other marketing. We ended up leaving the items on the site since they were no-indexed and they clearly stated that the article was published with permission.

May want to double-check on your end that they're not using those articles to support other marketing efforts.

 

[Eric Rohrback]

I think Conor might be right about the email marketing/other marketing efforts. Everything I see in the screenshots are tagged as "Newsletter Library" so there must be an email play to some extent with these. Definitely check with the client before making any big moves with these.

May be useless for SEO, but could be good to get someone back to the site via email... or maybe these are for existing customers to remind them to come back in and get other services.

 

[Joe Mink]

I have extensive experience in taking over iMatrix websites for the veterinary services industry and can confirm what Justin, Conor and Eric have said. Those articles libraries and/or individual articles have no SEO value. In fact, we typically create new websites on WordPress, totally disconnect from the iMatirx articles, and publish a site with fresh relevant content along with proper local SEO that begins to outrank the former site in very little time. Not to mention out convert the old site!

 

[GeekMade]

They probably spinned an article and sold to thousand of people, even tho is not indexed google will penalise duplicate content, I think what the guys told you is the best option, just get rid off that content!

 

[djbaxter]

They probably spinned an article and sold to thousand of people, even tho is not indexed google will penalise duplicate content, I think what the guys told you is the best option, just get rid off that content!

This is a persistent myth that has never been true. Google does not "penalize" duplicate content. There is no "duplicate content penalty".

Google just ignores duplicate content.

 

[GeekMade]

This is a persistent myth that has never been true. Google does not "penalize" duplicate content. There is no "duplicate content penalty".

Google just ignores duplicate content.

I wish it was true, but I've done over 200 websites in all different ways, expired content from webarchive, autotranslatated content from other languages, etc.,... google does penalise duplicate content HARD

How to Specify a Canonical with rel=

When a site has duplicate content, Google chooses the canonical URL. Learn more about canonical URLs and how to consolidate duplicate URLs.

developers.google.com

 

[djbaxter]

I wish it was true, but I've done over 200 websites in all different ways, expired content from webarchive, autotranslatated content from other languages, etc.,... google does penalise duplicate content HARD

How to Specify a Canonical with rel=

When a site has duplicate content, Google chooses the canonical URL. Learn more about canonical URLs and how to consolidate duplicate URLs.

developers.google.com

Please show me where that document talks about a duplicate content penalty.

Here's what it does say:

Duplicate content on a site is not grounds for action on that site unless it appears that the intent of the duplicate content is to be deceptive and manipulate search engine results. If your site suffers from duplicate content issues, and you don't follow the advice listed in this document, we do a good job of choosing a version of the content to show in our search results.

and

However, if our review indicated that you engaged in deceptive practices and your site has been removed from our search results, review your site carefully. If your site has been removed from our search results, review our Webmaster Guidelines for more information.

That is not about duplicate content per se but rather about general attempts to deceive visitors and Google, e.g., trying to claim scraped content as your own.

 

[msilvija75]

That is not about duplicate content per se but rather about general attempts to deceive visitors and Google, e.g., trying to claim scraped content as your own.

that's true, but duplicating content is just going against effective SEO practices. In this case writing unique content would be ideal, but you also can leave some duplicate content, just make sure that you have something unique to add to that in order to 'dilute' it.

 

[cfazio]

I have extensive experience in taking over iMatrix websites for the veterinary services industry and can confirm what Justin, Conor and Eric have said. Those articles libraries and/or individual articles have no SEO value. In fact, we typically create new websites on WordPress, totally disconnect from the iMatirx articles, and publish a site with fresh relevant content along with proper local SEO that begins to outrank the former site in very little time. Not to mention out convert the old site!

Nice! This is exactly what I did with it.