R is a programming environment that implements the S language. It is typically used for statistical analysis. When I was in college, quite a while back now, we used to use MATLAB a lot and R kind of reminded me of MATLAB (I have since used MATLAB all of zero times).
Unlike MATLAB, R is free! Anyone can download and get started with documentation — and the price is right. If you manage a Google AdWords account, or are generally interested in analyzing statistics about your website, R has plenty of built in functionality to help U analyze experiment data.
For example, let’s say you want to analyze the conversion rates of two different landing pages to see the probability that one conversion rate is equal to the other; i.e. if one landing page is outperforming the other. One way to do this is to open up R and use “prop.test.”
This uses a chi-squared test. The differences between two proportions are significant when p < 0.05 (95% confidence interval). Here’s an example in R, for two landing pages: one with 76 conversions and 2,742 visits; another with 61 conversions and 2,723 visits. > conversions visits prop.test(conversions, visits)
2-sample test for equality of proportions with continuity
correction
data: conversions out of visits
X-squared = 1.3693, df = 1, p-value = 0.2419
alternative hypothesis: two.sided
95 percent confidence interval:
-0.003336243 0.013966708
sample estimates:
prop 1 prop 2
0.02771699 0.02240176
The p-value is looking pretty big. So we probably want to change our alternative hypothesis to specify that one page is a test page that we’re testing to have a greater conversion rate than that of the original page. Using R’s help function, we can find the option to specify this and get a new p-value:
> help(prop.test)
starting httpd help server … done
> prop.test(conversions, visits, alternative = c(“greater”))
2-sample test for equality of proportions with continuity
correction
data: conversions out of visits
X-squared = 1.3693, df = 1, p-value = 0.121
alternative hypothesis: greater
95 percent confidence interval:
-0.002004153 1.000000000
sample estimates:
prop 1 prop 2
0.02771699 0.02240176
P is still greater than 0.05, so we can’t be very certain that the pages perform differently. That’s fun and all, but there are tools all over the web that can do this analysis.
Here’s one that uses a T-test. If we were using Google’s Website Optimizer, the values it’d give would probably be closer to those given by the one-sided T-test than the chi-squared test (Although Google hasn’t published what type of statistical model it uses with Website Optimizer, despite a lot of asking).
There are different factors for deciding which kind of analysis is ideal, and it can get complicated, so I’m not going to get into that here.
As it turns out, R has a bunch of other stuff that could potentially be useful for analyzing AdWords data. Such as a function for fitting linear models. In my last post, I mentioned using a test across different geo’s to see whether bidding on trademarks is effective.
For a test like that, you could run your experiment, export data from Google Analytics, read it into R, and use R’s lm function to fit it to a linear model. This is a less trivial example of one of R’s benefits. If I get some time to try it out myself, I’ll write a post about it.
