Google recently discontinued Google Website Optimizer (GWO) and replaced it with Analytics Content Experiments. Unlike GWO, Content Experiments does not have an option for multi-variate experiments, only allowing A/B tests. Multivariate experiments usually did not collect enough data to finish in a reasonable time period, unless you defined a conversion as an intermediate step to a sale.
Our clients are interested in increasing sales and leads directly, so we did not usually implement multivariate tests. But there did happen to be a case where using a multivariate setup to execute an A/B test came in handy.
Given a template used for different categories of pages that had static URLs, it was convenient to setup a multivariate test as detailed here. For example, let’s say I sell different tropical fruit with separate pages for their country of origin like this:
http://www.stevesproduce.com/phillipines/fruit/
http://www.stevesproduce.com/indonesia/fruit/
http://www.stevesproduce.com/brazil/fruit/
etc.
These pages all share the same template and I want to test a different template with URLs like this:
http://www.stevesproduce.com/phillipines/fruit/?variation=b
http://www.stevesproduce.com/indonesia/fruit/?variation=b
http://www.stevesproduce.com/brazil/fruit/?variation=b
etc.
I don’t want to set up an A/B experiment for every location I have fruit for, because the experiment wouldn’t get enough traffic to finish in a timely manner for every page and because it would take too long to setup an experiment for every location. The logical thing to do is to run a single A/B test across every location and group together the results to see what page template does better: the original or variation b.
Translating the GWO code
We can no longer use the method outlined in GWO Tricks, but there seems to be a way to use a similar method using an A/B test in Analytics Content Experiments. Here’s how I’d do it:
1. Setup a new A/B experiment in Google Analytics.
2. Enter my experiment page like this: http://www.stevesproduce.com/LOCATION-TO-SUBSTITUTE/fruit/
3. Enter my variation page like this: http://www.stevesproduce.com/LOCATION-TO-SUBSTITUTE/fruit/?variation=b
4. Copy the script it gives you and click to validate the code, but select to skip validation.
5. Modify the script and run the test.
Modifying the script
The script you get from the Analytics interface will look like this:
<script>
<!-- Google Analytics Content Experiment code -->
function utmx_section(){}function utmx(){}(function(){var
k='EXPERIMENT_ID',d=document,l=d.location,c=d.cookie;
if(l.search.indexOf('utm_expid='+k)>0)return;
function f(n){if(c){var i=c.indexOf(n+'=');if(i>-1){var j=c.
indexOf(';',i);return escape(c.substring(i+n.length+1,j<0?c.
length:j))}}}var x=f('__utmx'),xx=f('__utmxx'),h=l.hash;d.write(
'')})();</script>
<script>utmx('url','A/B');</script>
<!-- End of Google Analytics Content Experiment code -->
For my fruit experiment, I would modify it to look like this:
<script>
function getUrlVars(url) {
var vars = {};
if(url){
var parts = url.replace(/[?&]+([^=&]+)=([^&]*)/gi, function(m,key,value) {
vars[key] = value;
});
}else{
var parts = window.location.href.replace(/[?&]+([^=&]+)=([^&]*)/gi, function(m,key,value) {
vars[key] = value;
});
}
return vars;
}
</script>
<script>
<!-- Google Analytics Content Experiment code -->
function utmx_section(){}function utmx(){}(function(){var
k='EXPERIMENT_ID',d=document,l=d.location,c=d.cookie;
if(l.search.indexOf('utm_expid='+k)>0)return;
function f(n){if(c){var i=c.indexOf(n+'=');if(i>-1){var j=c.
indexOf(';',i);return escape(c.substring(i+n.length+1,j<0?c.
length:j))}}}var x=f('__utmx'),xx=f('__utmxx'),h=l.hash;d.write(
'')})();
</script>
<!-- End of Google Analytics Content Experiment code -->
<script>
var b = utmx('variation_content', 'A/B');
function filter(v) {
var u = v;
if (b && u.substr(0,7) == 'http://' && b.substr(0, 7) != 'http://') {
u = u.substr(7);
}
var l = document.location.href;
var prefix = 'stevesproduce.com/';
var i = l.indexOf(prefix);
var j = l.indexOf('/', i + prefix.length);
u = u.replace('LOCATION_TO_SUBSTITUTE', l.substring(i + prefix.length, j));
if(u.indexOf('?') == -1){
u = u + '?utm_expid=EXPERIMENT_ID';
}else{
u = u + '&utm_expid=EXPERIMENT_ID';
}
var bVars = getUrlVars(u);
var aVars = getUrlVars();
for(param in aVars){
if(!bVars[param]){
u = u + '&' + param + '=' + aVars[param];
}
}
if(document.referrer){
u = u + '&utm_referrer=' + encodeURIComponent(document.referrer);
}
return u;
}
if(b){
window.location = filter(b);
}else{
utmx('url','A/B');
}
</script>
The script “manually” redirects a visitor to the variation page, when Google Content Experiments indicates it’s time for the variation page to be shown through this statement: utmx(‘variation_content’, ‘A/B’). That function returns the literal URL “http://www.stevesproduce.com/LOCATION-TO-SUBSTITUTE/fruit/?variation=b” when it’s time for the variation to show.
We have to edit that URL to substitute the actual location (put “brazil” in where LOCATION-TO-SUBSTITUTE is, for example). We also have to append to the redirect URL the experiment ID, referrer, and any other tracking parameters that were in the original URL.
In the script “EXPERIMENT_ID” isn’t literal the way “LOCATION-TO-SUBSTITUTE” is; it represents an actual experiment ID you get when you generate the code in Google Analytics. The experiment number should be substituted anywhere EXPERIMENT_ID is located.
If it’s not time for the variation page to show, the page just uses utmx(‘url’,’A/B’) to handle the redirect. This just appends the experiment id and possibly a referrer to the URL — no substitution or “manual” redirect is necessary.
That’s it. This code has not been completely tested, but works in preliminary scenarios. It may require a few tweaks but it seems to function correctly.
