What you can achieve with Google Analytics Content Experiments (and other multivariate testing software)

If you’re serious about online profits, you need to be carrying out split tests on your webpages. The results from split testing can be amazing; increases in revenue of over 100% are not uncommon.

What is a split test?

In a simple split test, different visitors see different versions of one of your webpages. For example, some visitors may see the existing version of a page, whereas others see a new version. Special software “keeps score” of which version generates the most conversions. (A conversion can be whatever you want it to be—orders placed, leads submitted, etc.)

Once you have enough data, the software announces which page was the winner…at which point you promote that page to be the new “champion”.

The test described above would be called an A/B test, because two page versions were tested: Version A and Version B. A test with three page versions would be called an A/B/C test. “A/B/n” can be used as shorthand for a test that has n different page versions.

What is multivariate testing? Is it scary?

Depending on how frequently you get conversions, it can take weeks or months—or longer—to collect enough data to declare a winner. You may get frustrated waiting for one test to finish before you run the next one. If so, multivariate testing allows you to carry out several A/B/n split tests on different page elements simultaneously. For example, you may choose to test two different headlines whilst testing two different calls to action.

So, no, it’s not scary.

If you haven’t heard of multivariate testing, maybe you know it by another name, such as

• MVT
• Taguchi method (which is a type of multivariate testing)
• The name of one of the software tools, such as Google Analytics Content Experiments, Optimost or Omniture Test&Target

Most of the software tools allow you to carry out both multivariate tests and A/B/n split tests.

Why you need user reviews of the software

So, which software should you use? The answer’s not obvious.

Yes, it’s easy to compare the features of the different software tools, but there are many more important issues:

• How easy is it to set up?
• How helpful is the customer support?
• Will it be compatible with your website?
• Does it “do what it says it does”?

This website contains the results of millions of dollars worth of experience from real users who have been generous enough to give their feedback. All we ask is that, once you have begun using multivariate testing software, you return here to share your thoughts by submitting a review.

Recommended resources

Here are some great resources to help you get the best results from multivariate testing software:

• You can see the results of other companies’ split tests by visiting Anne Holland’s Which Test Won? site and ABTests.com.
• Avinash Kaushik’s book Web Analytics—An Hour A Day contains some solid advice on carrying out split testing. See pages 237–262. Here is one of Avinash’s blog posts about experimentation and testing.
• Marketing Sherpa’s Landing Page Handbook is an intelligently written overview of the subject.
• Google’s Conversion Room blog has some good articles.
• This article includes an overview of multivariate testing, and a list of 108 ways to use it.
• Jonathan Mendez created this good and thorough guide to multivariate testing.

What now?

By now, you should be ready to start comparing software tools.