Multivariate Testing & Analysis

Multivariate testing as its name suggests tests multiple elements of a web page simultaneously rather than individual elements as in an A/B test .

It can test multiple options of each element and using sophisticated statistical analysis provides information on the most successful options and combinations.

A multivariate test could test 4 variations of a headline, 4 variations of the text content, 4 variations of a product image and 4 variations of a [call to action]. A [call to action] can be any outcome for the purpose of the page eg in the case of an ecommerce site it could on an Add to Cart or Buy Now button.

The main disadvantage is the almost exponential increase in the pages needed to test all possible combinations (256 above example).

Multivariate testing is an experimental framework

In the above example each element:-

• main headline
• page content
• product or other image
• Add to Cart buttom

would have 4 different versions. In the case of the headline it could be its colour, size or wording. The content could vary in its description of the product or service, the length of copy and its format ie all text or bulleted list of features/benefits with the image varying in size, position or picture content. Similarly the [Add to Cart] button could test variations in size, colour or wording.

Google Website Optimizer

Google's Website Optimizer provides both the method and means to set up multivariate testing enabling each one of the possible 256 combinations to be run as an individual experiment and be shown randomly to your site visitors.

Each visit is tracked to measure its effectiveness with each visitor only ever seeing the one experimental page even if they returned multiple times.

These change to your web site can be tested to monitor the effect on visitor behaviour and show successfully converting combinations. If the change has a positive effect, keep it. If it doesn't, drop it and test another element.