Go to Top

Google Analytics Bummers

I’ll be the first to admit that Google Analytics is, like many Google tools, well-designed, free, and puts a great deal of functionality in the hands of users who might otherwise have no access to such data. But while some great posts have been written about Google Analytic’s strengths–(check out this comprehensive list of high points)–it’s much rarer to find web analysts complaining about some of its weaknesses. The few posts I have read on this subject often cite two in particular: the problems with data accuracy, and the lack of real time data.

These weaknesses are actually less of a bother to me, in comparison to a couple other “Analytics bummers” that have always bugged me a bit. So I’m finally calling them out:

1. The Danger of Averages
With the exception of a few reports, most of the numbers Analytics provides are averages over the Date range. This poses a risk whenever a user is trying to get an accurate picture of what a user’s experience on the site is like. Some sites may have a minority of dedicated visitors (a blog, for example, think the bloggers friends and family) as well as a majority of very dissatisfied users with high abandonment rates. Google Analytics in its current state obscures situations like this by pushing averages and not providing the same easy access to indicators like median and mode. Often in analysis its the high-end and low-end performers that should be zeroed in on for real results.

Also, some reports can’t be segmented (Visitor Loyalty, Depth of Visit, etc) so you’re given only averages over all visitor segments. What if the average is lumping one segment, with a very high depth of visit for example, with another segment with a very low depth? The picture you’re given via the number hides this completely.

2. No Emphasis on Significance

My next gripe has to do with what seems to me a complete lack of effort on Google’s part, in designing the tool, to add functionality that addresses the importance of calculating statistical significance before getting excited about a trend. Think of any market with seasonal fluctuations. If my pageviews and length of visits, etc are way up in the last month for a certain online channel, should I necessarily assume that I did something right? A split test is an easy way to find out if a trend is really worth watching, but I wouldn’t count on the fact that most users of Analytics are bothering to do it before they start acting in response. How hard would it have been to make a little applet for users to enter their numbers into, along with an alpha rate, in order to find out the confidence level?

3. My Data = Your Data?

My final concern has to do with the fact that Google stands to profit quite a bit from having access to extensive data on a large portion of the indexed web, especially given that they do profit considerably by pitting website owners against one another via the paid search auction. Between their index and the pile of data Analytic’s users are handing over, they have every opportunity to continue monopolizing the web and steering it in whichever direction they please. Do you trust Google? I do. But I can’t help but be aware that in the end, Analytics equals a data for data exchange, within a very unbalanced relationship.

facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

3 Responses to "Google Analytics Bummers"

  • Daniel O'Neil
    June 20, 2008 - 3:01 pm Reply

    Amen on Median. I’m not so sure about mode; in my experience the data from an analytics dataset is too granular and variable to infer anything about a single value.

    If you were to recommend that Google put one statistics measure into Analytics, what would it be?

  • jhullman
    June 20, 2008 - 3:24 pm Reply

    Good question. I find correlation coefficients useful anywhere you’re dealing with multiple variables, so they seem like they’d be really helpful on analytics, like anytime you’re segmenting data. Yourself?

  • Daniel O'Neil
    June 20, 2008 - 5:36 pm Reply

    Hi Jessica,

    Corellation coefficients are basically comparisons of “fit” between samples. The only problem with that approach is that you don’t always have a “fit” to match something to. In the context of Search Engine Marketing what you are usually doing is trying to determine variance across two or more data sets with no implication about historical trends, which suggests a better test would be ANOVA or chi-square tests.

    However, if you had enough historical data and could compare event “A” to event “B” correlation coefficients would be perfect.

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>