Local Ranking reports that show AI Overviews?

[JeffClevelandTN]

I was curious, does anybody use a ranking tool that shows whether or not there is a mention of a business in Google's "AI Overview" for a keyword phrase? As an example, with Brightlocal, their Local Ranking report indicates if the monitored business shows up as a Knowledge Panel for the search phrase, and you can also track third party website/webpage mentions (FB, BBB, etc). But with the AI Overview essentially now sharing the "Position Zero" space, it would be very helpful to see that tracked too. Does the ranking tool you utilize track AI Overviews or Snippets?

 

[keyserholiday]

@Jenny_BrightLocal is this something BL can work on and incorporate?

 

[WilliamGMBE]

@JeffClevelandTN
As far as I know google does not differentiate the clicks from AI Overview/AI Mode and others in the data they provide. Neither is it in the API yet. So, it is something difficult to build product like this since solid underlying data is not there.

Another major issue with this is that with AI the response is not always repeatable. This is especially true when it comes to GMB, where the results are very dependent on location. There are product out there that pings AI searches and shows if you are a website or product is cited, but not sure sure how good they work. I havent used any yet. Also not sure if they will work for GMB.

There are many other factors like memory, context, specific model and other things which can also affect the specific mention of a business, product or website on AI Overview/AI Mode. Token completion is much more abstract concept that traditional algorithmic ranking. So that makes it tricky too. We also dont think Google has finished building it AI suite of tools yet for Search. It will keep on evolving.

We(at GMBEverywhere) had explored this idea that you mentioned, and did some POC's but decided not to launch it. Didnt feel it was ready. Hopefully this space will mature soon and there are more reliable way to measure this.

We are still watching this

 

[JeffClevelandTN]

@JeffClevelandTN
As far as I know google does not differentiate the clicks from AI Overview/AI Mode and others in the data they provide. Neither is it in the API yet. So, it is something difficult to build product like this since solid underlying data is not there.

Another major issue with this is that with AI the response is not always repeatable. This is especially true when it comes to GMB, where the results are very dependent on location. There are product out there that pings AI searches and shows if you are a website or product is cited, but not sure sure how good they work. I havent used any yet. Also not sure if they will work for GMB.

There are many other factors like memory, context, specific model and other things which can also affect the specific mention of a business, product or website on AI Overview/AI Mode. Token completion is much more abstract concept that traditional algorithmic ranking. So that makes it tricky too. We also dont think Google has finished building it AI suite of tools yet for Search. It will keep on evolving.

We(at GMBEverywhere) had explored this idea that you mentioned, and did some POC's but decided not to launch it. Didnt feel it was ready. Hopefully this space will mature soon and there are more reliable way to measure this.

We are still watching this

William, thank you as always for a thorough and thoughtful response. You bring up some very valid points I hadn't thought about, like repeatability, similar to tracking Google Ads placements. However, I have noticed that it is fairly easy if a query returns an AI Overview, you can replicate the same or very similar result the next day or later in the week. We do a lot of medical digital marketing and seeing a increasing number of AI Overviews where our client's name (or competitor's name) and/or information is getting cited at the local level for service specific queries, especially for very local intent searches like "flu testing city name". So it is becoming increasingly important to monitor these occurrences and unfortunately way too time consuming to do manually at any frequency. Please keep us posted if you decide to pursue this further, or have any updates to the landscape. Thanks again!

 

[WilliamGMBE]

@

William, thank you as always for a thorough and thoughtful response. You bring up some very valid points I hadn't thought about, like repeatability, similar to tracking Google Ads placements. However, I have noticed that it is fairly easy if a query returns an AI Overview, you can replicate the same or very similar result the next day or later in the week. We do a lot of medical digital marketing and seeing a increasing number of AI Overviews where our client's name (or competitor's name) and/or information is getting cited at the local level for service specific queries, especially for very local intent searches like "flu testing city name". So it is becoming increasingly important to monitor these occurrences and unfortunately way too time consuming to do manually at any frequency. Please keep us posted if you decide to pursue this further, or have any updates to the landscape. Thanks again!

Thank you @JeffClevelandTN for the kind words.

Yes, micro intent is a good way to think about it. More like an answer engines than a search engines.

Yes, if we are asking about AI Overview alone, there is some repeatability. Like, this is the best paper we have seen about this and this is what we tried to test when we tried to build the product mentioned before: https://patentimages.storage.googleapis.com/d3/2e/de/63defc51bec99a/US11769017.pdf

According to this paper this is how AI Overview approximately work:
Gets top search result data -> Creates AI Overview with LLM-> Tags the best supporting content(not always top ranked) from the initial data that was used during generation.



In practice though, it was a little nuanced when we tested:
- In our test the answer was not always repeatable. See example of same search I did just now. It is the exact same keywords and it was from the same account one after the other. With slight difference in keyword it can be even more different.

We can though say with some probability, but not enough certainty like the traditional ranking tools.
Anyway, hopefully google will come out with more data sources for this.


Some interesting things we did notice though in these experiments with AI Overview and other AI searches:
1. Reviews are super important. Even 'micro intents' mentioned in the review are top ranking factor. Maybe review are one of the most important factors here. This is more true for other AI search tools too.
2. Very important to provide more detailed data to the AI. Like attributes, services, and other things help as 'micro intent' or data for AI to work with.
3. AI backs up response with actual data, especially from website. Like RAG. So, that is another source to target micro intents. Some pages are more important than other.

Yes, we do plan to do some things to help with this landscape, though not a pure ranking feedback tool yet. Some of the things we plan to come out include:
- Comparison tool: This is covering the point 2 mention above. Like how you can fine tune the 'micro intents' of a business with others in the area. We are coming out with an intereting approach to analyze this. We hope to have the first version of it out within a month. This is not just for AI searches.
- More advanced review analytics: Our Review Audit already provides keywords counts now already, but would get upgraded to get more details about the 'micro intents' in reviews. Mentioned in point 1 above. It is in the backlog.

We do hope to come out with helpful tool in this area.

 

[Jenny_BrightLocal]

@Jenny_BrightLocal is this something BL can work on and incorporate?

We have had lots of conversations about doing something like this. I can't say exactly what's in the pipeline right now, but hopefully, we'll be able to meet these needs in the future.