Search code examples
htmlcssresponsive-designab-testing

a/b testing a major html/css redesign


At my company we are redesiging our e-commerce website. HTML and CSS is re-written from the ground up to make the website responsive / mobile friendly.

Since it concerns one of our biggest websites which is responsible for generating of over 80% of our revenue it is very important that nothing goes "wrong".

Our application is running on a LAMP stack.

What are the best practices for testing a major redesign?

Some issues i am thinking of:

  • When a/b testing a whole design (if possible) i guess you definitaly dont want Google to come by and index youre new design (since its still in test phase). How to handle this?
  • Should you redirect a percentage of the users to a new url (or perhaps subdomain)? Or is it better to serve the new content from the existing indexed urls based on session?
  • How to compare statistics from a Google Analytics point of view? How to hint Google about a new design? Should i e.g. create a new UA code?

Solution

  • Solution might be to set a cookie only for customers who enter the website via the homepage. Doing so, you're excluding adwords traffic and returning visitors, who might be expecting an other webdesign, serve them the original website and leave their experience untouched.

    Start the test with home traffic only, set cookie and redirect a percentage to a subdomain. Measure conversion rate by a dimension in Google analytics, within same analytics account. Set a 'disallow subdomain' in your robots.txt to exclude the subdomain from crawling by SE's.