You are here: Home / Featured Posts / Why Do You Care So Much About (not provided)?

Why Do You Care So Much About (not provided)?

December 27, 2012 by 18 Comments

I know what you are thinking – not another post about keywords being (not provided) in Google Analytics. Been there, done that. Or have you? Could it be that you don’t really understand what you are arguing?

Inspired by a conversation with a dear friend of mine, the Content Scientist, I have come to the realization that there is a lot of misinformation going around about how (not provided) works, why it exists and other conspiracy theories.  My goal is to provide some more perspective about what is actually happening and why you should care a lot less about (not provided) than you currently do. 

What is (not provided)? 

Not Provided Jeffalytics

First, a primer on what not (not provided) means. In October of 2011, Google made it so that when you search on their site while logged into your account, these pages are served as a secure (https) page and not a standard (http) page. This change resulted in a side effect that prevented referring data from being passed along to your website when visiting from Google.com.

What does that even mean? Well, the majority of the web traffic data you see in your web analytics tool is identified by information passed along to your website through your web browsers HTTP referrer property. Your web analytics tool is smart enough to know when a referrer is from a search engine domain like google.com, and can parse query terms out of the referrer string in order to determine the keyword that was entered into the search engine. 

Query String Example

It’s actually an elegant solution that we take entirely for granted. It’s really just a hack that analytics tools have accepted and implemented to make your referring data more useful. We take it for granted because it works out of the box in modern analytics tools. 

When an HTTPS page hyperlinks to a non HTTP page, the referrer data is not transmitted as part of the security protocol. With no referrer data, your web analytics tool can’t “hack” its way into determining the search query that was typed into the previous page, so the tool can only say that the keyword was not provided to them. Hence the term (not provided).

So far there should be no reason to don your tinfoil hat or think that this is a conspiracy. For myriad reasons, Google made a choice to make their core product (search) more secure and it resulted in a loss of functionality for a secondary product.

But You Can Get Keywords From AdWords if you PAY Them! How is That Not Evil?

The most common argument that is drummed up to claim that (not provided) is specifically slighting organic search is that you can still receive your keyword data from your paid search programs. Surely, this is meant to push people to pay for search results, right? Wrong.

You are comparing Apples to Apes. Paid search keywords are not determined in the same method as organic search keywords. They are determined in one of two ways:

  • Automatically tagging your Google AdWords URL’s with a unique gclid that allows two Google products to talk to each other. By enabling auto-tagging and opting into data sharing between products, you are overriding the referrer data from a search results page and setting the keyword for your search engine reports automatically. The overriding piece is important to understand. Auto tagging automatically overrides the default tracking method for your campaigns with exact values that come out of AdWords.
  • Manually tagging your paid search campaign (Google or Bing or others) destination URLs with campaign variables (aka utm variables in Google Analytics). By tagging your URL’s with these variables, you are overriding the referring source data with values that you are setting manually.
AutoTaggingManualTaggingGA
There is a tremendous difference between how keyword data is collected for organic search and paid search. In one instance, you are using a “hack” to infer the keyword that sent data to your site. In the other case, you are either manually or automatically setting the value to be the correct keyword that you paid to send to your site.
 
I can’t stress enough how big of a difference there is between these two methods of data collection. In fact, you would be surprised how many companies are sending paid traffic to their websites without doing any tagging to properly attribute their traffic to paid search. When paid traffic is not tagged, it shows up in web analytics reports as organic searches.
 
This results in overinflated traffic numbers for organic search. You would be surprised at how many of those complaining about (not provided) is not giving them enough credit for their efforts, yet are turning around and taking credit for organic traffic that was generated from paid ads.

Why Can’t We Override Keywords for Organic Search?

Because Paid Search is a Product, Organic Search is Not

Now that you understand how paid search data is showing up in reports as a result actions taken to override the default data transmission method, you are likely asking why you can’t do this with organic search data as well?
There are countless reasons why this would not work, and many more that I haven’t even thought of. Here are several reasons why this won’t work:
  • It would undermine the purity of algorithmic search results gathering
  • You cannot assume that everyone uses the same web analytics tool and tracking methods
  • If you use more than one web analytics tool, would you expect that both of these are tagged?
  • It is compromising organic search for the benefit of organic search
  • The potential for abuse of adding your tracking parameters to a search result is tremendous
  • Overriding URLs to have tracking variables in them would ultimately eat away at the simplicity of tracking websites using Google Analytics “out of the box”
The only way that this may be possible is by automatically “spoofing” the referrer information that is sent to your website from a Google search page to reflect what the page would look like without HTTPS, but this would likely cause them more privacy issues than it is worth.

Why is Google so mean to Organic Search Marketers?

Dawson Crying over Google Not Provided
Was Google being mean to SEOs when they gave us an industry-redefining web analytics tool for free? Before Google Analytics, you had to pay $100k+ for a robust web analytics tool that could do this type of analysis. By nature, very few organic search marketers would have the means to invest in such an expensive tool – or else they would have gone into paid search in the first place ;) .
Do you know what free web analytics tools looked like before Google Analytics? They looked like this:
Awstats Keyword Report.png
That’s so useful, isn’t it?
Before GA came out, the word “Analytics” was not in our collective vernacular. We called our website traffic “Stats” and we were happy with receiving a lot of “hits”. Things have changed for the better.
[Sarcasm alert]Yes, Google is being super mean to you by giving such amazing tools for free. Stop crying Dawson. [/Sarcasm alert]
The only people they were being mean to is the number of web analytics tool providers that they put out of business.
And let’s not even get started with the free keyword tool Google provides that has gotten significantly more useful over the years.
If Google wanted to make your life difficult, they certainly could. Whether you admit it or not, they have made certain aspects of an SEOs job easier over the years (and yes, I realize that SEO is much harder now than ever, so there is irony in my statement).

Why You Shouldn’t Care About (not provided)?

Honey Badger Don't Care
All of this is to say that at this point I don’t think you should care much about (not provided). Let’s go through some of the reasons.

If your #1 search term before October 2011 was your brand name, it still is

The reality is that for most well branded sites, the traffic distribution at the top doesn’t move a lot. It’s often brand terms and (not provided) hasn’t changed that.

Are you worried about incomplete data? Guess what, it’s always been incomplete

Have you ever tried to reconcile e-commerce transactions between your analytics tool and your cart software? It’s never 100% accurate.
What about the clicks you pay for from search engines and display networks? Have 100% of those clicks ever aligned with the visits in your tool? You are looking at an 85-95% alignment at best.
The point is that even if keywords were provided, they wouldn’t be nearly as accurate as you want them to be. We need to make decisions off of relative numbers, not absolutes.

Are you upset because everything is not perfect anymore?

When were things ever perfect? Please tell me a time when SEO was perfect. Was it during the days of meta keyword stuffing?
Find me a 2 year period where one pundit didn’t declare SEO dead while another subsequently declared that SEO is more difficult than ever and I will listen to that argument. 

Are you just lazy?

If your job is to target a niche to determine a keyword phrase to target, are you really going to rely on your web analytics data to determine if you were successful or not? Or will you be searching for your term to see how you can do better? Searching for your term is a way to get an answer in real-time.

Stop Being Reactive

By nature, using Web analytics tools are reactive. You are only reporting on what happened in the past, not what is happening currently. If you wait until you have a week, month or year of data in order to make decisions, the world will likely pass you by.
Live in the now dude!
Live in the Now

Is Keyword Data your Right?

Is it that you believe that you have an unalienable right to keyword data?
What about when you didn’t have this data available to you before Google Analytics, which they are giving away for FREE

We all have a short memory

Google Analytics made modern SEO possible. Sure things have changed over time and some features made our lives easier while others were a thorn in our side. That is the nature of this business though, isn’t it?
I think that we should stop being upset over what went away and focus our time more on what we can do to make ourselves better. For example, if you have not checked out the activity information available in Google Analytics from members of the Social Data hub, then you are really missing out. These are some of the most useful reports that I have ever used in my time as an analyst.
On a long enough timeline, the survival rate for any feature drops to zero. You can either embrace this change and get better or cry yourself to sleep like Dawson did.  Your choice.
So what did I get wrong? Please sound off in the comments!

Related posts:

Filed Under: Featured Posts, Google Analytics, SEO Tagged With: , ,
About jeffalytics

Jeff Sauer is a Digital Marketing Consultant, Speaker and Teacher based out of Minneapolis, MN in the USA. Jeff has been instrumental in growing Three Deep Marketing, an Online Marketing agency in St. Paul, MN by 500% since he joined the firm in 2008.

Jeff is an expert in WordPress, Google Analytics, Google AdWords and SEO and has spoken to groups as large as 600 people 25 times over the past two years.

  • http://www.facebook.com/profile.php?id=100001538626581 Jessica Parker

    Well if you ask me, social media marketing no doubt requires a great deal of attention and care, hence never go for and market a website without it…. Online Marketing Tactics

  • Webbingyourway

    Hi,

    I have to disagree with one thing – yes Google Analytics brought analytics to the forefront, but there were tools being used to do analytics well before GA came along. And we can argue ig GA was a good thing for the industry, but that is another story.

    But you only mentioned GA impact to the changes, how about the paid for tools where we cannot get this data for anymore (Adobe, Webtrends, premium GA…). It is one thing if you have a small site and can get clues what the search term may be, but when you need a more robust tool do do true analysis, you need to pay for it. The change Google did impacted these tools as well.

    The second issue, which I just read yesterday and was brilliant, you get GA free because you are giving your site usage for free to them. Now they are asking you to keep giving your site usage for free while not receiving your own keyword searches…. We are not asking for everyone, just the ones that drove traffic to our site.

    Google claims it is for the \”privacy of their users\” but historically, they have shown they do not care about our privacy. This would have gone down better If they were honest about why they did it.

    http://www.webbingyourway.com

    • http://www.jeffalytics.com/ Jeffsauer

      Thanks for the comment and thoughts. I was very clear other tools existed before GA in the post, and that the \”good\” ones were paid tools.

      I mentioned GA as a generalization in some instances, but for the most part I said web analytics tools.

      The site usage information you refer to is simply incorrect. By default, your data is not shared outside of the GA product. You must opt in to sharing with other Google products.

      You seem to have an opinion formed without understanding the facts. You are making the exact same argument that this post was meant to set straight. In other words, you either didn\’t read the post or you are so set in your opinion that you chose to ignore facts.

      Either way, I appreciate the comments and contribution to the debate.

      • http://twitter.com/aschottmuller Angie Schottmuller

        Yipes. Ease up on the attack. Your honor, can we have the record read back to us?
        \”Google Analytics made modern SEO possible.\”
        \”Before GA came out, the word \’Analytics\’ was not in our collective vernacular.\”
        \”Before Google Analytics, you had to pay $100k+ for a robust web analytics tool that could do this type of analysis.\”

        What in the world are you talking about??? Stop smoking the GA peace pipe, bro.

        1) GA, or any website analytics tool for that matter, has nothing to do with SEO. Referrers and other metrics are merely analytics data. It\’s completely separate from relevant and authoritative content — the driver of SEO.

        2) GA was not the the beginning of analytics for everyone. I was using Webtrends and WebSideStory/Hitbox back in 2000, and I likely began tracking search referrers around 2005. Google Analytics has never been my primary analytics tool, and it certainly wasn\’t a driving factor in me understanding analytics vernacular, web traffic behavior, or SEO.

        3) Whether a web analytics tool is free or paid is irrelevant. The \”(not provided)\” change impacted all web analytics tools the same. BTW, in the early 2000\’s, I probably paid ~$4k/yr for Hitbox. In 2010 (before Adobe jacked-up the package and price tag), I paid ~$12k/yr for Omniture HBX with ~20 sites/accounts. Today it\’s possible to get equivalent functionality from Adobe SiteCatalyst for ~$15k/yr w/ some persistent negotiation. Well worth it! These prices are a fraction of your $100k/yr reference for paid tools.

        Since I know you\’re familiar with SEO and life beyond GA, I\’m quite surprised by several of your remarks. I\’ll chalk it up to over-generalizations and comment-bait. I\’m more engaged by solid statements backed by data. [Go figure.] ;-)

        • http://www.jeffalytics.com/ Jeffsauer

          Please note that I tried to say \”Web Analytics tool\” whenever possible, because there are other tools. At times I did use GA as the example, because they are on over 50% of the top 1MM websites. That makes the tool the majority, and when writing I find it acceptable to cite the majority stakeholder in the example vs. explaining the nuanced market for simplicity sake.

          Much like how \”Google\” is used interchangeably with \”search engine\” in the US market due to dominant marketshare, GA is the de-facto web analytics tool whether you use it or not.

          And the rest of your comments are made ad hominem against a web analytics tool/my writing and not against the entire point of the post, which is that (Not Provided) is not the end of the world.

          You are nit-picking specific points of contention that really are inconsequential to the overall point. In rebuttal to your specific points:

          1) I agree that there are bigger things to focus on for SEO than your analytics tool/tool capabilities, so you are actually proving my point.

          2) I said majority of those who could not afford paid tools. I know that corporations used HBX to track analytics, but this was a handful of a percentage of website owners at corporations that could afford it. I used HBX myself and while it was a POS, it did have a few features that were similar to the first version of GA.

          3) Who cares about the exact amount paid? In my opinion, an SEO without a corporate backer would have just as much trouble paying for a $15k a year tool than they would for a $100k a year tool… They probably would have quite a difficult time paying for a $1k a year tool actually. I view this as the majority of people doing SEO, not those with the means to purchase paid tools.

          As for the specific quotes, they are highly opinion based. Again, I will say that my statements were aimed at the majority of people who are into SEO/looking to get into SEO. You are not in that majority.

          As for specifics:

          \”Google Analytics made modern SEO possible.\”

          Depends on your definition of Modern SEO, but I actually still stand by that statement, no matter how ridiculous it may sound now. I stand by it because it fits my definition of modern SEO. That could be a post in itself, so I will hold off on that for now.

          \”Before GA came out, the word \’Analytics\’ was not in our collective vernacular.\”

          I stand by this one simply because the word analytics is still barely used in a business setting. In fact, spellcheck programs still often believe that Analytics is a misspelled word! I\’m sure that you have used this term for years, but you are also not the average user.

          The last part of your comment irks me most – you say that I should be making \”solid statements backed by data.\” The point of this post was not to be a research paper. I don\’t have footnotes or citations, because this is an opinion piece and not an academic paper.

          I find that comment to be patronizing and a complete cop-out, because it is indefensible as a writer. If everyone had to back their opinion pieces with firm data, the Internet would not exist.

          I could likely comment on everything that has been written about SEO on the Internet and say the same thing.

          Either way, this is a fun debate and I can\’t wait for Josh Braaten to chime in his misguided opinion as well :) .

          • http://twitter.com/aschottmuller Angie Schottmuller

            I was not commenting to irk you, Jeff. My apologies if you took it that way. I just shared how some of your statements came across to me as being inaccurate or over-generalized. You presented a very inflated price for paid tools, and then responded to my actual prices with \”who cares about price?\” =( So confusing. On the \”modern SEO\” note, we clearly have very different definitions. Perhaps that\’s a future post you could do with crowdsourced responses?

            When I said, \”I\’m more engaged by solid statements backed by data.\” it was reference to my personal preference and simply informational. As a blogger, I appreciate constructive feedback from readers telling me what they like and don\’t like, so I can plan content accordingly. If I\’m not your target reader audience persona, just disregard it. Kindly ease up, and view the feedback as a possible opportunity… not a threat.

            BTW, for me a \”solid statement\” is one that doesn\’t require a follow-up explanation. If it\’s an opinion, it\’s helpful to reference that versus stating it as fact. Supportive data can be something as simple as infographics, case studies, or inline links to more in-depth articles. You did provide this is several parts of your article.

            It seems you\’re very defensive and eager to fight in comments. I\’m just interested in intellectual, fun conversation. Your remarks are a bit intimidating. If I comment again, are you just going to beat me up? ;-)

          • http://www.jeffalytics.com/ Jeffsauer

            You have provided some things to consider. Happy New Year!

  • http://twitter.com/NickLeRoy Nick LeRoy

    Jeff – my primary concern with \”not provided\” is that I can\’t track the profitability of specific keywords to the extent I use to. For instance if I had a keyword that was converting extremely well but I was only ranking #9 I would take the time to bump that ranking up and hopefully make some more cash. It\’s a lot harder to identify these opportunities when the data falls in \”not provided\” category because the individual is logged in or using the secure version of google search.

    • http://www.jeffalytics.com/ Jeffsauer

      It\’s definitely harder… but my overall point was that it was MUCH harder before Google Analytics came around. There was not even a concept of a visitor or a conversion before GA outside of paid tools.

      So they made your life easier 90% of the way with the tool and yet we complain when a change is made that affected our easier lives. It\’s like being pissed off that you have to pay for your soft drinks when you get free admission + food at a sporting event (although not entirely, because they are not offering the opportunity to pay for the items you want).

      How have you found that Google Webmaster tools data suffices as a proxy for what you are missing? It integrates nicely with GA, but I have found that I can\’t rely on the numbers provided.

      If GWT becomes more accurate, does that ease the burden a little bit?

      I do hear your point about converting keywords and I can see how that can be a huge loss as a result. That\’s not easily replaced.

      • http://twitter.com/NickLeRoy Nick LeRoy

        Absolutely agreed Jeff. While i\’m not exactly \”happy\” I don\’t complain because half a valuable GA tool is better then none!

  • http://twitter.com/ItsHogg Jon

    Have you guys seen NotProvidedCount.com?

    • http://www.jeffalytics.com/ Jeffsauer

      Yeah, but I take it with a grain of salt. For this site the count is more like 75%, so I think the number is awfully low.

      • http://twitter.com/ItsHogg Jon

        Is that 75% of all traffic or just Google organic traffic? 75% is the highest % I\’ve heard of any site so wondering if you\’re using the \’right\’ rules?

        • http://www.jeffalytics.com/ Jeffsauer

          Hey Jon,

          For 2012 looking at the organic search report, I see 1,476 visits from search engines and 1,040 of those have a keyword of (not provided). I am sure this has to do with G+ authorship and the fact it\’s a new blog, so most of my traffic comes from known people. By that calculation, my (not provided) count for all traffic is 70.4% and it\’s been rising even more lately.

          • http://twitter.com/ItsHogg Jon

            Wow that’s high. Good luck with the blog, didn’t realise it was new. Cool logo btw.

          • http://www.jeffalytics.com/ Jeffsauer

            Glad you like the logo, I’ll tell Mark the designer and he’ll be thrilled. Yes, still a very new site and trying to build an audience. Appreciate your patronage!

  • http://twitter.com/aschottmuller Angie Schottmuller

    Interesting discussion, Jeff! A controversial topic is always fun! Classifying the opposition as lazy, reactive, forgetful, perfectionists… Oooh. Harsh.

    My beef with “not provided” has basically nothing to do with SEO, since I can still get an adequate list of keywords driving search traffic from Webmaster Tools and Compete. My concern is on the impaired ability to audit and optimize the user experience to best fit search queries.

    Two areas impacted by “not provided”:

    1) Optimizing Paths Based on User Queries.
    Understanding what a user searched for brings significant light to evaluating web traffic paths. If a user searched for XYZ, did they find what they were seeking? How many clicks did it take? Was the path intuitive or scattered? If they gave up, at what point did they abandon the site? Evaluating keyword referrers and common traffic paths is extremely useful for auditing page usability, optimizing paths, choosing featured content, and even planning call-to-action text.

    2) Optimizing Relevance of Landing Page Content.
    If the keyword referrer is known, content on the page can be tailored to feature respective content or use that vernacular. For example, on e-commerce home and category pages I’ve used JavaScript on-load to default a slideshow to displaying the slide content that best matches the keyword referrer. This helps to feature the content most relevant to the query and helps to avoid the user having to look around for it. (It works awesome for reducing bounce rates!) This concept of “maintaining scent” from link/ad to landing page is common practice for PPC conversion optimization, so it makes sense to leverage it for organic searches as well.

    Both of these examples point out how blocking the keyword referrer impairs the ability for marketers to continually improve upon and optimize the user experience. The change is in contradiction to Google’s philosophy of providing the most relevant results for users, which is one of the reasons so many marketers object. If the user doesn’t find what they’re seeking, both Google and the website have essentially failed.

    I’m grateful the blocked keyword referrer change hasn’t carried over to mobile devices [yet], so as mobile visits continue to increase that’s some hope of getting more of that data back for analyzing.

    Keyword referrer data aids click path analysis and landing page optimization. It’s not lazy if there’s no data to analyze. It’s good to evaluate the data and leverage it for decision making. Although this could be considered “reactive”, I think of it more as being proactive for improving the experience for the next web visitor.

    On the perfectionist note, I don’t think marketers think of any analytics tool as “perfect”. I am an idealist and a conversion optimist though, so the more data available to help improve a process the better. I don’t believe it’s my right to have access to keyword referrer data, but I do believe that blocking it impairs the ability for marketers to optimize the user experience. If a battle’s in order, I fight for the user. =)

    What do you think? Were you using keyword referrers the way I described above?

  • http://www.facebook.com/rob.walker.58173 Rob Walker

    Google Analytics is a tool that businesses should look at that is not only free but also easy to use! Businesses can use this for tracking results for SEO such as keywords.