What the Heck IS Multivariate Testing???
Multivariate testing is a way you can test all manner of different permutations of layout and functionality on your site. Unlike A/B testing - which limits you to two variations (like two versions of the same page) - MVT allows you to manage individual elements like block treatments on a Drupal site or the styles on a header or location of graphical elements on a page. The options are pretty much limitless.

When utilising MVT, you need to have a way to track clicks on the different variations. Essentially you use your live Web audience as a test group carving segments to each of the variations and then observing through your traffic monitor what version gets the most clicks or the more desirable path through the site. It is common to use a traffic tracking suite like Omniture or Google Analytics to manage the analysis of the patterns. Often MVT is used by marketing departments to determine the best treatment of pages.

When a variation is determined to be the better, that treatment is adopted and the process starts again.

What Different MVT Methodologies are Out There?

There are basically two ways to do MVT - DNS highjacking and Tagging. Basically using DNS, you set up a proxy in the same physically location as your server. The proxy redirects certain amounts of traffic to an alternate variation of your page(s). All the variation logic on the server(s). Tagging involves adding javascript to the site itself. The javascript places the variations and tracks the activities of the user on the site.

MVT and Drupal

There are quite a few options to make use of multivariate testing with Drupal. Three jumped out when looking at different options.

http://drupal.org/project/multivariate (free)
http://drupal.org/project/google_website_optimizer (free)
http://visualwebsiteoptimizer.com/ (paid service)

Google Optimizer seemed to have the strongest combination of inexpensiveness (free) and great features. It is also pretty keen that there is a module out there that makes use of the Optimizer service.

The Magic of Exponential Numbers

Making many changes at once, can greatly increase the velocity of your optimization plans because you are measuring many more elements than you would in a simple split or A|B test. A|B testing can be better for base comparative tests - but MVT is better suited to large site optimization jobs. Something to consider though is the complexity of getting to larger and larger tests. In an A|B test, you only have two variations, but the larger number of elements you change along with the number of variations of those changes you create can become overwhelming. The math is simple.

You would need a giant sampling to figure out the efficacy of 4*4*4*4*4*4*4*4*4*4 variations to a page and the challenge of working out what was the best combination would be nearly impossible.

So, Why Do It?
Multivariate testing allows you to set many different variables on different elements. You might swap out the header on your site with a different header, change text, move buttons around, and so forth. The tests then swap out these elements and allow you to measure the effectiveness of the changes. You set what your goals will be for the tests and then analyse which variations gave you the best results. On an ecommerce site, you might looks for the greatest conversions that turn into sales. If you have a community site, you might be looking for the combination of elements that get a user to sign up on the site and make a first post.

If you run an ecommerce site - you are leaving dollars on the table if you don't use MVT. If you have a community site - you are not maximizing the number of members you have signing up. If you have a news or information site - you are losing that extra click to the next article. MVT can help you increase your conversions no matter what kind of site you manage and no matter how you define your success metrics. Finally, it allows you to sculpt an experience for your users that incrementally improves and will produce a happier user base.

Lemur Image Credit: Keven Law
Under a Creative Commons License