Google Analytics multi-channel funnel – four-item Venn diagram is misleading

So, we’re thrilled with Google Analytics’ addition of multi-channel funnel analysis. Google Analytics has a great price – free! – and the addition of advanced features is wonderful for us and our clients.

Why? Multi-channel funnel analysis can show how marketing campaigns interact with each other to influence behavior and the all-important leads and sales that power our clients’ businesses. Among other things, we can assess which campaigns initiate conversations (say non-brand organic or non-brand paid search) against those that close leads and sales, helping us invest our clients’ funds in the strategies that win across the customer journey from awareness through consideration, conversion, and loyalty.

Last week, we prepared for a client meeting involving new higher-ups who required a visual summary instead of a long and nuanced explanation of interactive influences. To help the visual folks in the room, we looked to the Venn diagram on the overview page in Google Analytics’ Multi-Channel Funnel.

It’s a great illustration, easily showing how each medium contributes and overlaps to support the end goal – a web form submission in this case. You can see the legend here on the right – paid search, organic search, direct (people who type the web URL into their browser or use a bookmark), and then referral, social network, and email were options for analysis.

Well, one of us was looking at the three-item Venn diagram (paid search, organic search, and direct) and another of us had checked the top four items in the legend (paid search, organic search, direct, and display). You wouldn’t think we should get different visualizations that would lead to different interpretations, right?

And….sad trombone…we got ourselves all confused by the Venn diagram. So, this post is a word to the wise to be careful when using that visualization.

You can see a difference between the three-item and four-item Venn diagrams: In the four-circle version, there is almost no overlap between paid search (blue) and direct (yellow). The underlying data is the same, so what gives?

After some fussing around and hovering over the overlap areas and looking at the numbers of visitors in each (which is a nice real-time way to interact with the Venn in Google Analytics) we realized that it’s just a problem with the visualization. Four-way Venn diagrams are wonky.

A quick Google search reveals that this is a known issue with Venn diagrams. If you use circles, you can only represent the overlap of three things. To add the fourth item, you actually lose fidelity in the data. Yikes!

As a cool-looking but inaccurate diagram is unacceptable, we opted for the three-item Venn for our meeting. We learned something important – don’t use the Multi-Channel Visualization Venn diagram with more than three items checked.

Don’t believe this quick visual? See Amy N. Myers’ paper Are Venn Diagrams Limited to Three or Fewer Sets?, available from the Bryn Mawr website. She writes

Are we missing something, or is a Venn diagram using circles for four (or more) sets simply impossible?

Well, here’s to happy analysis! (Just make sure you use your Venn diagrams responsibly.)