Kinds of Split Testing Software
Dominic DeLong
You've probably heard that split testing elements of your website can improve your conversion rates. And that's true. But there are many kinds of split testing out there. And there are many ways to set up split tests on your website.
I can't discuss every feature of every testing tool out there, but I can give you a major overview of the different kinds of split testing software.
First, there's a difference between multivariate and single variable testing.
With single variable testing you just pick one element on your page to test. Take your headline, for example. If your current headline says "Are you having trouble with tooth decay?", you might want to see how "Do your kids have tooth decay?" works. In a single variable test, you just find a way to rotate each of those headlines on your page, and keep track of how many sales each generates.
Paul Hancox has a simple split testing tool. And Google has one that's free, though it takes a bit to set up all your accounts to use Google's tool. And the reporting is not real-time.
Single variable testing can be slow. How would you like to test 10 times as much with the same amount of traffic? Well, you can. You use a method called multi-variable, or multivariate testing. This allows you to test several elemens on your page simultaneously.
Among multiple variable software solutions, there are hosted plans and self-install plans. If you want to install software on your server (which can be faster and cheaper) you will have more control over your data. And you can usually find someone to install it for you (often the vendor provides this service for a nominal fee). If you want to go with a hosted plan expect to pay more.
Google Website Optimizer is a hosted plan. But it requires you to have an Adwords account and an analytics account. There is also a reporting delay. Other hosted plans include SiteSpect, and Offermatica. The Split Test Accelerator an MuVar are installed solutions.
Another 3-way distinction among multivariate split testing tools is between Taguchi-based systems, Full factorial systems, and Random independent rotation of factors. With a Taguchi system, you run traffic to only a small, carefully-selected subset of the possible combinations of your factor levels. This allows you to cycle through every combination every 12, 18, or 36 pages, whichever the case may be for the test design you're using. This allows you to find the best combination very quickly. Some care does have to be taken to avoid interaction effects among the factors. Taguchi-based tools include the Split Test Accelerator and Vertster.
If you want to get around interaction effects you can with a full factorial tool. It will take more traffic to avoid the interactions -- a lot more -- but with a FFT, like Google's Website Optimizer, you can just see how every combination plays out, and live with the interactions.
Some multivariate tools randomly serve the options for each factor independently from what is served for the other factors. This can lead to more flexible test design, but also to more interaction effects and other statistical noise, leading to longer tests.
That sums up some of the main difference among split testing tools. There are many other features to compare, but this should give you the big picture.
Dominic DeLong has used http://www.splittestaccelerator.com/index.php) split testing tools for years to improve web sites.
|
Categories 
