Expert Opinion on the Webmaster Tools Data Mystery

Stuart Shaw 5 years ago

It's not news that Google has been clamping down on keyword data, but this was only present in Analytics due to the introduction of keyword "not provided".

To get around this, many of us, including ourselves, were using Webmaster Tools (WMT) search queries (alongside PPC data) to see what people were searching Google for to return particular pages. This helps us with a number of different aspects of our jobs and monitoring organic website activity.

However, in the midst of our weekly reporting, all Webmaster Tools search query data had disappeared! Many of us believed that this was an error with our Webmaster Tools accounts, but it soon became apparent that it wasn't - it was an error at Google's end.

This was widely reported by many of the industry news outlets, such as here:

Search Marketing news publications then chased Google for answers on this, and it does turn out to be an error. Google have reportedly fixed this issue, but we're seeing data being withheld for the latest week, which in essence keeps us a week behind. Is this the beginning of Google syphoning out keyword data entirely?

This is already apparent with other Webmaster Tools features, such as the International Targeting reports. These reports are being held back by almost 30 days. This isn't useful to webmasters as fixes or new methods may be applied to their website and they won't be able to see how Google digests this information until 30 days later.

There are some serious questions being asked in regards to the future of how Google reports keyword data and when it may disappear entirely:

  • Are Google removing all keyword data?
  • How else are we going to get this data?
  • Why are Google keeping us a week behind?
  • Shouldn't we be able to have this data?
  • Does it move more people into Google Adwords?

To gain better insight into future proofing ourselves from the removal of this data, I asked a few members of the Search & Data team at Zazzle to give me their opinions on this very topic. You can see these below:

Richard Petersen, Deputy Search & Data Manager

I believe it's Google's long term goal to remove all keyword data from its free tools in order to obtain greater revenues. This will be a gradual process bit by bit so websites don't notice it so much. This was confirmed with the introduction of them not being provided in Google Analytics and I can see these kind of changes happening to WMT.

If you still want to see the keyword data then I would recommend using competitor webmaster tools platforms like Bing or Yandex. They still provide some level of data. If you take, for example, the knowledge that 15% of your traffic comes from the keyword blue widgets in Bing then try to apply that to a percentage of the entire amount of organic traffic.

You can then use the same method with Yandex and see if you are coming back with the same figures.

If you want to still see the keyword data I would recommend using Bing Webmaster Tools and Yandex Webmaster Tools. There is still information in there and using analytics work out what percentage of traffic comes from Bing and then multiply the numbers by the entire visit amount and it will give you a better indication.

Google have always followed a plan: market domination, then they work out how they are going to make it profitable. They are currently focused on pulling in as much capital to allow them to diversify their business portfolio.

They also need to maintain sales figures to meet market expectations so it doesn't have an effect on their share price.

Tani Kopliku, Senior Search & Data Consultant

I think Google is definitely moving towards no keywords being provided in analytics. What keyword data we get through analytics these days is not worth looking at. It's not solid enough to make any decisions unless combined with landing pages and making some assumptions based on the keyword history to a page - then appending these keywords in Excel and trying to track rankings that way.

However, Search Metrics is a good substitute for me. I think Analytics will soon be its main competitor by making keyword data available for a monthly subscription. These are just my thoughts, I must admit: I don't have the inside track into Google (yet!)

The move with the week's delay in WMT is yet another blow in a series of data limitations introduced from Google in the last few years. There has yet to be an official statement by Google. Nothing has been reported as yet and in addition we can't see the reason why they would choose to limit the data refresh.

I don't think that this move on its own will move people to AdWords but there have been several other changes introduced. A combination of all these might have an effect:

  • No provision of organic keywords
  • Recent push of non-commercial website at the first page of results in competitive financial sectors (my observation)
  • Delay of WMT data
  • Google Local update (pushing non-local competitors out of page 1)
  • Numerous algorithmic and penalty updates (Now waiting a rumoured over-optimisation update with more negative results - soon to come)

If you look at all the above together than you can say, yes of course, Google's objective is to ultimately personalise and completely fragment search results so that bigger AdWords spenders spend even more and they can push more businesses down the AdWords route.

Rhys Davies, Search & Data Executive

It's all part of Google's plan, slowly removing exact keyword data to encourage users to use services like AdWords to see if they are really worth competing for within organic search. They claim it is protect their signed in users' privacy, but why would they give you a snippet, not the full picture? It's surprising they don't offer full keyword data in the paid version of Analytics.

The way most people are working around this is by using Webmaster Tools, but from my experience this still doesn't account for every organic click. The best you can do at the moment is sorting the keyword by destination page in Analytics. Then you can see where the not provided keywords are going to and you can put two and two together by using the page's title tag to see what keywords it is targeting. However this is a very long winded method.

I guess it takes a while for them to process the secured data and if you need this data urgently you will turn to other methods such as AdWords.

We should be able to have total control to see how users are visiting our websites, but Google are really clamping down on this and it's beginning to hinder our ability to produce detailed strategies.

It is definitely about moving people towards Google AdWords; you have to pay to play to get all the data you need. I suspect in the future Google Analytics Premium will come out with a feature to see the not provided keywords. I guess this is Google's way of getting back at SEOs as we are trying to manipulate rankings in our favour - this makes it a lot harder.

Gareth Torrance, Search & Data Consultant

Personally I think it is unlikely that Google will be removing all keyword data. Whilst the growth of Term Not Provided has made life difficult for many, it is unlikely that the recent hiccup with Webmaster Tools is something to be overly concerned with.

Whilst the data did stall on the 7th February, causing a great deal of worry, it has started to return. Currently it is still a few days behind, but gradually the data should catch up and return to the normal two day delay.

As with Analytics, many are already reporting that their data has returned, although some are still having issues with the real time view. Again, this does not impact the keyword data, so there is nothing that suggests Google will remove all the keyword data in the foreseeable future.

After Term Not Provided first came around there was a great deal of panic. Everyone was wondering where they could get their keyword data from going forward. Search Metrics is useful and gives a reasonable example of the keywords, but is there anything else that can be done?

One of the methods I have employed, whilst simplistic, did help ease the situation. By looking at past data and cross referencing it with Search Metrics to find where certain pages of a website rank, and for what terms, I then created a report based entirely on the performance of those pages. This allowed me to see trend in the performance of the page. If it changed, then I could cross reference again, manually checking the keywords it ranked for from my list to see whether it had moved.

I, like a number of others in the industry, believe that the data will slowly catch up and return to the two day behind period it usually has run at. At the moment, it is likely that Webmaster Tools is trying to recollect the data that was not given, and due to the sheer volume of sites that have to be looked at, it may take a little while.

We should absolutely be able to have this data. Not just for SEO, but for e-commerce as well. Many e-commerce sites use keyword data to determine which products are desired the most. This allows them to give the user, in this case the consumer, the products they want. Therefore, letting us have the keyword data actually allows us to give the users what they want, which is precisely what Google wants us to do.

I suspect more people will move into AdWords after this. Whether this is because of the keyword data or simply a sense of security is debatable, but in the end it will push people in that direction.

From my experience it is usually for a sense of security. Whilst AdWords is great in its own right, a lot of people move from being heavily focused on SEO to having some of that focus pointed to AdWords as a sort of safety net. They want to know they can still get the data they need, the traffic they want or just general visibility in one shape or form.


After listening to the opinions of several people in the industry, it's apparent that there are mixed feelings on this latest move by Google. Ultimately it holds more data back from us and continues to move towards their end goal of not allowing webmasters to tailor their websites for 'SEO', but for the user instead. Create a customer focused brand with great content and a great user journey and the organic rewards will follow.

Stay in touch with the Zazzle Media family

Sign up for our monthly newsletter and follow us on social media for the latest news.

Our website uses cookies for various purposes and to enhance the site’s functionality. This helps us understand how you use and interact with the website.

Settings Accept Cookies