If you’re having an e-commerce site or you’re planning on building one, there are many important factors to consider. One of those is the usability of your website. And as you probably don’t want to leave nothing to chance, using a proven method to provide a great user experience can’t be a bad idea.
A/B testing, or split testing, can help you with exactly that. It involves creating two or more versions of your website and comparing the results of each to identify the best way of driving users directly to your product or service.
Unlike other forms of usability testing, the great thing about A/B testing is that it provides an opportunity to test your site with live markets. A well-done test will not only provide valuable future marketing information, but also may pay for itself in the process. It also can have a beneficial side effect in that it can help keep people engaged in your website or product, quite simply because most people like new content.
You may find a lot of conflicting information about how long you should wait before redesigning your website, with ranges claiming between 6 months and 3 years. However, realistically you should be redesigning your site when and if it stops being effective. How can you tell? Obvious indicators can include lagging sales, or evidence that people visit your site and leave before purchasing, or it could be they are just getting bored. A/B or split testing can help with this, keep your business on its toes, and at the same time keep your customers or users interested.
Change for the sake of change is not necessarily a good thing, however change to keep people engaged is, especially if it increases your bottom line.
DID YOU KNOW? A/B Testing has been shown to generate 20-25% more leads for eCommerce sites, and 30-40% more for B2B sites.
Methodology – Tips for Effective A/B Testing
Tip 1. Do Some Preliminary Testing First
It’s a good idea to base your research on some opinions of others before you start. Sometimes we get too close to our websites to notice problems that others will spot right away. I described a few of these methods previously that you can take a look at. While an exercise such as card-sorting can provide some valuable information, at the very least you should do a little hallway testing to get some basic ideas about areas which could be improved.
Tip 2. Sample Sizes
Unlike other forms of usability testing which suggest you can test fewer than 10 people, with A/B testing you will need to work with a much larger sample.
While I don’t want to bore you with a detailed lecture about statistics, you should still be aware of a few concepts, and particularly statistical significance. This may sound complicated, however it is simply a test of whether the results you get are likely to be real, or if they are simply due to chance. To be able to have good confidence in your results largely depends on the size of your sample, or the number of people you need to test to decrease the chance that your results are not random.
Typically if you can get yourself to a 95% certainty that your results are accurate, this is something you can take as fairly reliable. In other words, with this level, there would only be a 5% likelihood that your results are due to chance.
It’s important to note that just because something is statistically significant does not necessarily mean that the effect is large. For example, say you can get one out of 10,000 people to purchase your product on one campaign, but on another, you get two out of the same sample. Though the results might be statistically significant, and you could claim that your results doubled, realistically you only got an increase of 0.01 percent. This is rarely what most would consider “success”. However if the sample is only 100, then that level of increase would mean a lot more. For that reason, you will want to set a baseline of what results you wish to achieve from your tests.
The length of the time you want to keep your test up depends largely on the sample size you feel you need to determine whether your results are valid.
Here’s a helpful calculator for determining your sample size.
Tip 3. Start With a Few Hypotheses
Select a few areas that your subjects chose in #1 above, and set these aside as different areas to test. Hypotheses are not theories; they are a set of educated “guesses” that you, the scientist that you now are, will make before you start testing. Make a list of these, with clearly defined outcomes of what the results you expect to find.
Here are a couple of example questions:
- Will a “purchase now” button at the top right of the page will have more effect than one at the bottom left?
- Will having links to external sites open in a new window keep people on your site? Will they cause people to leave because of too much information?
Tip 4. Test Your Theories
Make detailed notes on your results. Compare the effects against your site usage statistics. Identify information that differs. It’s a good idea to look at how many pages people click on before leaving, what is the ratio between number of pages clicked through to the shopping cart or contact information, and the ratio of visits to actual sales or contacts. Google Analytics provides some excellent detail regarding usage on your site, including what devices customers are using, and where they are coming from. One of my favorite features is “path through site” or “users flow” which tells you how users are actually using your site:
Tip 5. Test Individual Theories One at a Time
One common mistake that people make with A/B testing is the attempt to test everything at once. The problem with this approach is that you may be misled by the results you get. You don’t know which changes you made had which effect. This may lead you to make changes to the wrong thing and could hurt your bottom line.
Tip 6. Use Incremental Testing
Once you have determined that one method works better than the other, continue to test additional hypotheses based on the results you have found. You may find that one method works at getting people further along the way, but not to where you want them to be (such as actually purchasing your product!) Continue searching to find better ways of getting the user to the endpoint. In other words, to borrow a term from sales, find an approach which will turn your website into the best “closer.”
Tip 7. Test Different Versions of Your Site at the Same Time
There are many factors than can affect and possibly lead to misleading test results. For example, you may be running a different ad in different locations, which could change the types of people you might be reaching. Your results could be thrown off due to a holiday season, variation on demand, or if it’s simply a slower time of the year. For this reason, you should be running both samples of your test simultaneously, so as to reduce the likelihood that external factors will affect the validity of your results.
Tip 8. Accept That You Might Be Wrong.
It’s very common for people to search for and find information that confirms what they already believe. This is called “confirmation bias” and is a fallacy, and if you don’t avoid it, it can have serious implications on your bottom line. Be willing to change your mind based on your results. Sometimes you may need to start over from the beginning.
Tip 9. Understanding the Results
An extremely common mistake, particularly with new researchers, is to mistake correlation with causality.
In other words, just because two things happen at the same time does not mean that one thing caused the other. Your positive results might not be because of your different designs; it could be coincidental. You could have seen a surge of popularity at a specific time, due to a sudden demand in the market, or the fact that your advertisement started hitting the right people. The danger of these types of mistakes is that you can make changes that may be ineffective, or worse, harmful. For this reason:
Tip 10. Test and Test Again
This is especially true if the results of #6 above are not what you expected, or to make sure you don’t make the mistakes in #8. Take the results of your previous test and try and develop new hypotheses. Test those. Make changes, and test again. Keep trying until you get positive results. Test your positive results in different circumstances. Also, never assume that because your business is in the black that it can’t be doing better!
What Should I Test?
Great, you say, “now I have a handle on the methodology, but now I need to know which parts of my site I should test.” A/B Testing can be done on a lot of different sections of your website.
Home Page – Typically the first thing you want to test is your website’s home page. Does the design of the main page of your site make a difference? Do images make a difference? How about detailed explanations of your product?
Landing Pages – What about landing pages? In other words, test the pages where people entering your site. This is typically driven by your marketing campaigns, be they email marketing, advertising, or SEO pages. Landing pages are one of the best areas to experiment with getting your feet wet with A/B testing because they can be changed easily. You will likely see more traffic on these pages. Use them to find ways to drive traffic further into your site.
DID YOU KNOW? Businesses with over 40 landing pages have been shown to create 12 times more leads than those with only 1-5.
Calls to Action – Are you providing people with direct offers to purchase your item, or contact you about your services? Are you driving people away by demanding that they provide personal information before learning more? Test different methods here.
User Confidence – Does including links to privacy statements help? How about links to third party confirmation/reliability sites, such as the Better Business Bureau. Do links to testimonials make any difference? How about customer reviews?
Individual Parts of Each Page – Does moving text around on the page bring better results? How about where you place the “purchase now” button? Does the navigation of your site work well for people, or do they give up and leave? How about ways of getting people to choose your mailing list? Look through each element and try to imagine them being presented in a different way.
A/B Test Beyond Your Website – Another thing that is easy to forget is that there are a lot of other places to use A/B testing beyond your website. You can conduct tests on pretty much any one of your online marketing activities or anything that drives people to your website. Try this on marketing emails and your advertisements. One of the advantages of this is it provides a great way of evaluating different landing pages. You can send out identical ads (remember the rule about testing one element at a time) with the only difference being that it brings the customer to a different landing site. Does one work better than others? Continue to tweak these.
Conclusion
A/B or Split-testing may seem like a lot of work, however the time spent can be well-worth it. If you follow the above instructions, you will save yourself a lot of headaches and you will stand a very good chance of improving the bottom line for your eCommerce site.
Also, considering the fact that it can be done with an active site, and that you can see results in real-time should make it more satisfying. If you consider this as part of your online business marketing as a whole, you will almost certainly see better results than not doing it at all. And don’t forget, the excitement in seeing your leads or sales increase make the entire process that much more enjoyable. If you are just starting out, consider yourself lucky as you will be beginning ahead of the curve. If you haven’t been doing it before, you may soon wonder why you hadn’t been doing it all along.
In case you have any more questions our you want to leave us some feedback, please drop us a comment below and we’ll get back to you as fast as possible!
The authors
Learn more about us
THE BEHIND THE SCENES OF THIS BLOG
This article has been written and researched following a precise methodology.
Our methodology