Hey Google, why did you turn the lights out?
Posted by Hal MacDermot on 25 September 2013
The week’s big news is that Google are extending the https secure search to 100% of organic searches. This means that if you were tracking the keywords sending traffic to your web pages in Google Analytics, then they effectively turned out the lights. The good news is that if you’re using Wordtracker for keyword research your work is safe. Wordtracker’s keyword database is totally independent of Google.
Google encrypts 100% of organic keyword search data
Google’s move to encrypted search began in October 2011. If a user was signed onto their Google accounts, their searches were directed Google Secure Search (https). Their search queries were encrypted, with the effect that when clicks to a site were generated from an organic search, the site did not receive the referrer data – including the keyword data. The publisher no longer knew the keywords used to reach their pages from an organic search, where the user was signed-in.
In Google Analytics, under organic search sources, these searches now appeared as the somewhat unhelpful title of “(not provided)".
The result was a big and growing headache for marketers trying to understand how people search for their brand, or use keywords to improve the ranking and performance of a web page, or segment site visitors by keywords, or even prove to a client the link between traffic improvements and seo efforts.
Since 2011, the percentage of encrypted searches, or as Danny Sullivan memorably called it “Dark Google” has been growing steadily
Secure search was extended to the Firefox default search in July 2012, to Safari iOS 6 in September 2012 and of course, Chrome. The result has been a growth of Dark Google, documented by a site dedicated to documenting “not provided”
Why is Google making your lives difficult?
That’s an excellent question. Google’s explained the change in terms of protecting personal privacy and security. So that unwanted parties could not spy on your search activity/our lives.
Seems reasonable. But eyebrows were raised when it was clear that the keyword data hidden from web site owners, was openly available if they signed up for a Google Adwords account. Ad search traffic is not encrypted, so that when someone arrives on your site after searching Google and clicking on an Ad, the search data is available. Was Google really being that cynical, or was it a conflict between the Adwords dept and another dept. more concerned about personal security? I guess that’s for you to decide.
In the meantime, I do want to mention that another way to get limited search data, is through Webmaster tools. Using the menu on the left, just click on “Search Traffic,” then on “Search Queries” (Search Queries are the keywords that people type into Google’s search box).
It’s limited to the top 2000 terms and data only goes back 90 days (so you need to download regularly to keep an historic record, but it can be useful. But honestly, the search data in Webmaster is much harder to get to, and not as complete. I’ll follow-up in future posts with more on this.
In my moments of Google doubt, I have to ask myself about the new imbalance. Google has the right to crawl publishers’ websites so that it can provide a great search experience for its customers. In return, it seems reasonable that publishers should be allowed the information on how their own customers arrived at their sites. Making that information available only to people signed up for an Google Adwords account seems just a little bit strange.
Who knows, maybe the current big push to encrypt searches is a reaction from Google in the context of the recent NSA spying scandal. There has been huge criticism of big Internet players handing over their customers personal and US National Security Agency. Maybe Google wanted to show they were willing to protect people’s data. Whatever the reasons, we have to live with the effects.
Whatever the case, I would like to highlight that Microsoft are keeping Bing’s search data unencrypted. Thank you. Marketers really do use this data to improve people’s Internet experience, and it is very useful. Search data and keyword data helps us design websites that deliver what potential customers are looking for.
What do the changes mean for Wordtracker customers?
If you are using Wordtracker for keyword research, you will not be affected by the changes imposed by Google. Wordtracker’s database of 3.5 billion searches per month, containing 350 million unique keywords and rising, is completely independent of Google.
Wordtracker relies on data from SEMRush, and a major search engine advertising network. We can continue to enable your keyword research with our Keyword Tool. Our API will continue to give developers direct access to our independent database, and provide everything they need to help drive their applications and tools.
What the future holds
Even when Google goes completely dark for keywords, Wordtracker will still be providing the keyword data you need.
Wordtracker’s data is larger and fresher than ever. Our regular Keyword Tool offers access to data from 15 countries, and our Free Keyword Tool offers global, US and UK data. The Wordtracker API covers more than 190 countries and also offers you time segmentation.
Whatever Google do with their search data, your Wordtracker keyword research is safe.
About Hal MacDermot
Hal MacDermot is Content and Community Manager at Wordtracker. He has also worked in digital advertising, content strategy and social media. In former lives, Hal spent time in TV production in LA, and as a film journalist. Hal enjoys cooking and camping trips to the Wilderness. You'll find a fuller profile on Google+ or LinkedIn