Odd Rankings: Tools vs Human results

[SEO_Ron]

Anyone seeing odd ranking results lately when using tracking tools? We have a set of locally focused keywords that we track weekly and have been tracking for years. These keywords include the city name within the query and have historically shown consistent results no matter the user's location. Also historically when checking ranking positions for these keywords any automated checks would match human checks in private windows. Within the last month or so that changed. Now our automated tools whether it's SerpAPI, ScaleSerp or one-off checks using tools like https://technicalseo.com/tools/local-search/ frequently show different results than what humans see.

Again, these are keywords we have tracked manually and with tools for many years and until recently automated checks matched human checks. And I should add when I say "human checks" these are people unaffiliated with our company/business checking in private windows in various parts of the country. The human results are consistent. But now it almost seems as though Google is showing a different set of results for APIs/tools than it is for human/browser based checks.

Also, I should add these aren't minor discrepancies. We will see for example across all human checks our site ranking 1 or 2 for a specific keyword whereas the automated check won't have the site in the top 10.

Has anyone else seen anything like this? Any theories?

 

[fisicx]

Ignore the tracking tools. At best they will indicate trends. And you can mostly ignore your human checking as well.

The only thing you need to check is the number of new leads you get each day. They will tell you far more about how well your website performs than worrying about ranking.

 

[SchielerDeland]

Hey there, I would treat those tool results as directional right now, not absolute

The main reason is that most rank-tracking tools, APIs, and local search simulators are checking from sterile environments: clean IPs, no real user history, no Gmail account, no behavioral profile, and no normal search journey attached to the request

That matters more now because Google does not seem to be returning one fixed SERP for every user. It can shift the #1 result, the local pack, organic ordering, and even the overall SERP layout based on the profile, location context, device, prior behavior, and what Google thinks that user is most likely to click

One question I’d ask: when your people did the human checks, were they signed into their Gmail accounts or searching in private/incognito windows?

The common SEO habit is to validate rankings through incognito, but that may no longer reflect how real users actually search. Most real users are logged into Google, and that logged-in context can enrich or alter the SERP in ways an API or sterile rank tracker will not reproduce

I think what you’re seeing is very plausible. The tools may not be “wrong” in the sense that they are broken, they may simply be measuring a different version of Google than the one real users are seeing

I hope this helps, and I look forward to hearing back!

 

[JeffClevelandTN]

Hi Ron,

We have not seen any major ranking fluctuations across the tools we use, but we do occasionally see one-off increases or drops for certain keywords. As you know, Google algorithm updates can also create short-term volatility in some industries, although I would be careful about assuming rankings will automatically recover. In my experience, if the SEO work is solid and above board, the results often settle back down, but it is still worth watching closely.

You did not mention whether you are seeing this with local map pack rankings or organic search engine results page (SERP) rankings. I am assuming organic SERP results based on your description, but I will comment on both.

For local pack tracking, I would strongly recommend using a well-established subscription tool as part of your main toolbox, if you are not already. There are a lot of options, but we have had strong long-term success with BrightLocal (BL), Whitespark (WS) and Places Scout (PS). BL and WS are both very affordable, and Places Scout is definitely more expensive, especially if you are tracking multiple keyword phrases or multiple locations. That said, PS has been very reliable for deeper location-based tracking. Other than PS, I believe BL and WS offer free limited trials.

For local pack rank checking, one important thing to keep in mind is business hours. Whether you are checking manually or running scheduled reports, rankings can be affected by whether the business is open or closed at the time of the search. Sterling Sky covered this here: https://www.sterlingsky.ca/google-added-a-new-ranking-factor/

For organic SERP tracking, BrightLocal and Whitespark have good directional rank tracking tools, but the results are usually based on a single geographic coordinate. In even a small or midsized city, rankings can change noticeably based on a very small location shift. That is where Places Scout can be useful because it allows you to check multiple points around an address, almost like a grid, which gives a better picture of whether rankings are trending up or down.

Places Scout also now includes reporting for AI Overviews mentions for the same keywords, and if you manage pay-per-click (PPC), it can show whether it saw your ad during the latest run.

We still manually spot-check significant drops or sudden increases. For that, we typically use valentin.app, which lets you spoof your location and search as if you were at a specific address. The same developer also has the GS Location Changer plugin for Chrome and Firefox. There is a good thread about that here: https://localsearchforum.com/threads/location-spoofing-with-gs-location-changer.62274/#post-197529

For both manual checks and tool-based checks, I would use a private or incognito window whenever possible.

As for your specific question about APIs and tools showing different results from human/browser-based checks, I do think that is possible. @SchielerDeland already touch on this in his reply. Google may be treating some automated checks differently based on several factors: location, browser environment, personalization signals, device type, language settings and other parameters. Even small differences in those factors can produce a very different result.

That is one of the reasons I like having a paid subscription tracking tool. It may not perfectly match every human check, but if the methodology is consistent, it gives you a reliable directional view over time. It also makes client reporting much easier. We want clients to see the improvements we are making, and internally, our team needs a consistent way to measure progress over the last few months, the last 12 months or longer.

Tagging @whitespark and @Jenny_BrightLocal

Hope this is helpful!

 

[whitespark]

Can confirm that rankings are increasingly volatile these days. There is no single set of rankings. Pretty much every person and every tool gets different results.

 

[SchielerDeland]

Hi Ron,

We have not seen any major ranking fluctuations across the tools we use, but we do occasionally see one-off increases or drops for certain keywords. As you know, Google algorithm updates can also create short-term volatility in some industries, although I would be careful about assuming rankings will automatically recover. In my experience, if the SEO work is solid and above board, the results often settle back down, but it is still worth watching closely.

You did not mention whether you are seeing this with local map pack rankings or organic search engine results page (SERP) rankings. I am assuming organic SERP results based on your description, but I will comment on both.

For local pack tracking, I would strongly recommend using a well-established subscription tool as part of your main toolbox, if you are not already. There are a lot of options, but we have had strong long-term success with BrightLocal (BL), Whitespark (WS) and Places Scout (PS). BL and WS are both very affordable, and Places Scout is definitely more expensive, especially if you are tracking multiple keyword phrases or multiple locations. That said, PS has been very reliable for deeper location-based tracking. Other than PS, I believe BL and WS offer free limited trials.

For local pack rank checking, one important thing to keep in mind is business hours. Whether you are checking manually or running scheduled reports, rankings can be affected by whether the business is open or closed at the time of the search. Sterling Sky covered this here: https://www.sterlingsky.ca/google-added-a-new-ranking-factor/

For organic SERP tracking, BrightLocal and Whitespark have good directional rank tracking tools, but the results are usually based on a single geographic coordinate. In even a small or midsized city, rankings can change noticeably based on a very small location shift. That is where Places Scout can be useful because it allows you to check multiple points around an address, almost like a grid, which gives a better picture of whether rankings are trending up or down.

Places Scout also now includes reporting for AI Overviews mentions for the same keywords, and if you manage pay-per-click (PPC), it can show whether it saw your ad during the latest run.

We still manually spot-check significant drops or sudden increases. For that, we typically use valentin.app, which lets you spoof your location and search as if you were at a specific address. The same developer also has the GS Location Changer plugin for Chrome and Firefox. There is a good thread about that here: https://localsearchforum.com/threads/location-spoofing-with-gs-location-changer.62274/#post-197529

For both manual checks and tool-based checks, I would use a private or incognito window whenever possible.

As for your specific question about APIs and tools showing different results from human/browser-based checks, I do think that is possible. @SchielerDeland already touch on this in his reply. Google may be treating some automated checks differently based on several factors: location, browser environment, personalization signals, device type, language settings and other parameters. Even small differences in those factors can produce a very different result.

That is one of the reasons I like having a paid subscription tracking tool. It may not perfectly match every human check, but if the methodology is consistent, it gives you a reliable directional view over time. It also makes client reporting much easier. We want clients to see the improvements we are making, and internally, our team needs a consistent way to measure progress over the last few months, the last 12 months or longer.

Tagging @whitespark and @Jenny_BrightLocal

Hope this is helpful!

Wanted to follow up on your point specifically since I was tagged. Hopefully I’m not overstepping by sharing a screenshot here

At my agency, we use an internal Chrome extension to track SERP layouts across different logged-in Google accounts. What we’re seeing is pretty clear

When different logged-in users search the exact same query from the same city of origin, they are not always seeing the same SERP layout, and they are not always seeing the same positional rankings

In one small test of only 10 searches, we saw:

• 5 different websites appear in the #1 organic position
• 9–10 different ordered SERP layout variations
• Meaningful differences in how Maps, LSAs, organic results, and other SERP elements were arranged

So to your point, and as @whitespark mentioned, volatility is really the only consistent thing we’re seeing right now

Hope this helps someone out there!

 

[SEO_Ron]

Can confirm that rankings are increasingly volatile these days. There is no single set of rankings. Pretty much every person and every tool gets different results.

Thank you for responding. When you say "rankings are increasingly volatile these days." Are you seeing more volatility within the last month or so? Are you seeing discrepancies between api checks and human checks?

 

[SEO_Ron]

Hi Ron,

We have not seen any major ranking fluctuations across the tools we use, but we do occasionally see one-off increases or drops for certain keywords. As you know, Google algorithm updates can also create short-term volatility in some industries, although I would be careful about assuming rankings will automatically recover. In my experience, if the SEO work is solid and above board, the results often settle back down, but it is still worth watching closely.

You did not mention whether you are seeing this with local map pack rankings or organic search engine results page (SERP) rankings. I am assuming organic SERP results based on your description, but I will comment on both.

For local pack tracking, I would strongly recommend using a well-established subscription tool as part of your main toolbox, if you are not already. There are a lot of options, but we have had strong long-term success with BrightLocal (BL), Whitespark (WS) and Places Scout (PS). BL and WS are both very affordable, and Places Scout is definitely more expensive, especially if you are tracking multiple keyword phrases or multiple locations. That said, PS has been very reliable for deeper location-based tracking. Other than PS, I believe BL and WS offer free limited trials.

For local pack rank checking, one important thing to keep in mind is business hours. Whether you are checking manually or running scheduled reports, rankings can be affected by whether the business is open or closed at the time of the search. Sterling Sky covered this here: https://www.sterlingsky.ca/google-added-a-new-ranking-factor/

For organic SERP tracking, BrightLocal and Whitespark have good directional rank tracking tools, but the results are usually based on a single geographic coordinate. In even a small or midsized city, rankings can change noticeably based on a very small location shift. That is where Places Scout can be useful because it allows you to check multiple points around an address, almost like a grid, which gives a better picture of whether rankings are trending up or down.

Places Scout also now includes reporting for AI Overviews mentions for the same keywords, and if you manage pay-per-click (PPC), it can show whether it saw your ad during the latest run.

We still manually spot-check significant drops or sudden increases. For that, we typically use valentin.app, which lets you spoof your location and search as if you were at a specific address. The same developer also has the GS Location Changer plugin for Chrome and Firefox. There is a good thread about that here: https://localsearchforum.com/threads/location-spoofing-with-gs-location-changer.62274/#post-197529

For both manual checks and tool-based checks, I would use a private or incognito window whenever possible.

As for your specific question about APIs and tools showing different results from human/browser-based checks, I do think that is possible. @SchielerDeland already touch on this in his reply. Google may be treating some automated checks differently based on several factors: location, browser environment, personalization signals, device type, language settings and other parameters. Even small differences in those factors can produce a very different result.

That is one of the reasons I like having a paid subscription tracking tool. It may not perfectly match every human check, but if the methodology is consistent, it gives you a reliable directional view over time. It also makes client reporting much easier. We want clients to see the improvements we are making, and internally, our team needs a consistent way to measure progress over the last few months, the last 12 months or longer.

Tagging @whitespark and @Jenny_BrightLocal

Hope this is helpful!

Hi, thank you for your thoughtful response. We use ScaleSerp and SerpAPI for tracking. We found that these give us the control we need and flexibility to expand that we didn't get from subscription tools. Depending on the keyword we will track results across dozens of locations across the country since our terms skew towards local businesses. We have separate tools and reports for local pack tracking.

As mentioned earlier, these are serps we have watched very closely for many years. Up until the last month for these sets of keywords we have seen consistent results in api tools, paid subscriptions and manual checks. It's just within the last month we've seen a divergence.

 

[SEO_Ron]

One question I’d ask: when your people did the human checks, were they signed into their Gmail accounts or searching in private/incognito windows?

The common SEO habit is to validate rankings through incognito, but that may no longer reflect how real users actually search. Most real users are logged into Google, and that logged-in context can enrich or alter the SERP in ways an API or sterile rank tracker will not reproduce

I think what you’re seeing is very plausible. The tools may not be “wrong” in the sense that they are broken, they may simply be measuring a different version of Google than the one real users are seeing

Thank you for responding. Yes, the human checks have been done in private browsers. I understand your point about a logged in experience being different than incognito.

What is odd is that we aren't talking about minor fluctuations. We will have a site/keyword combination that has ranked in the top 2 for years, and still does across all manual checks in various locations, but the api pull will show it not within the top 20. If we rerun the rank check a few times we can get the top 2 result but obviously that is time consuming and inefficient when dealing with thousands of checks.

And as mentioned earlier, these are serps we have watched very closely for many years and it's only within the last month that we've run into these issues.

 

[SchielerDeland]

Thank you for responding. Yes, the human checks have been done in private browsers. I understand your point about a logged in experience being different than incognito.

What is odd is that we aren't talking about minor fluctuations. We will have a site/keyword combination that has ranked in the top 2 for years, and still does across all manual checks in various locations, but the api pull will show it not within the top 20. If we rerun the rank check a few times we can get the top 2 result but obviously that is time consuming and inefficient when dealing with thousands of checks.

And as mentioned earlier, these are serps we have watched very closely for many years and it's only within the last month that we've run into these issues.

Absolutely! So yes, few things here

1. People don't search incognito. Google notes that 41% of people are in active sessions out of 75% of a total US population that uses Gmail (can confirm this with a Google search). This means that Incognito is a sterile lab experiment, not true to search

2. I posted a screen shot above. For this keyword "Carpet Cleaning, Phoenix, AZ" we recorded 10 searches from Google (from logged in users), and across those 10 searches - six different domains surfaced as #1. This means that Google is switching in and out positional rank for users. We can only assume they are trying to sculpt it to the users specific journey and what company Google thinks they are more likely to identify with and click.

All in all - a lot of SEOs believe we are moving into a fully dynamic SERP, which means positional rank will change for the user depending on past behaviors, past clicks, and interacted company types

 

[whitespark]

Thank you for responding. When you say "rankings are increasingly volatile these days." Are you seeing more volatility within the last month or so? Are you seeing discrepancies between api checks and human checks?

It’s just that everyone seems to get different results. Doesn’t matter if it’s a tool, or your neighbor, or even your wife in your own house. But each person’s results appear to stay more or less stable.

Thus, you can’t compare what a tool sees against what a human sees and claim one is more accurate than the other. They’re both perfectly accurate and legitimate results.

The point is that there isn’t just one set of results. There are millions of potential variations based on each person’s setup and history.