Earlier in the month I read this great article by Justin Rondeau, on what CRO is and what it encompasses. The below graphic couldn’t sum it up any better.
Whilst AB testing plays a huge role in our CRO strategy, there is significant work which takes place in other areas and help drive our testing hypotheses. The below tools are just a snapshot of some of them.
We use Formisimo to track our purchase page. It is a single tag added to the page, which then tracks all interactions with the form fields on the page. It helps provide data on drop-off points, problem fields and fields which need correcting often. For any form heavy site, it’s worth taking a look at.
GoSquared is the latest piece of tracking we’ve added to the site. It’s strengths lie in analysing data in real time and providing a week on week estimate. You can push through revenue data, and from an early stage in the day you can track how you think the site is performing compared to any other previous point in time. We originally looked at GoSquared for it’s people analytics, which allows you to tie events to users and report on that. We originally wanted to integrate this with Optimizely, but this specific aspect of GoSquared is still in it’s infancy – so we’re yet to make use of it.
GA needs no introduction, pretty much every site has used it at some point for the high-level analysis. We use GA heavily to track aggregated user data to see which features are performing, and how. The free version does have it’s limitations though, and the sampled data does have to be taken account particularly when dealing with specific events undertaken on the page.
If there’s one source of truth, then it’s this. Tracking through GA or other analytics can go wrong, so it’s good to have somewhere which is treated as the single correct source. Our internal reports measure visits, bookings and other user interactions. Reports are then emailed around on a daily basis to the company so we can measure how we are tracking on a Day on Day basis, and compared to last week
We use UserTesting.com for our user testing. You set up your page(s) that you want testing, select your questions for the test and then within approximately 30 minutes you have a full testing session completed at the cost of $50USD. We’ve found it particularly useful for competitor comparisons, where we send users to our site and a competitors and we look for struggle points. These tests drive a number of AB test hypotheses, as well as helping future design decisions.
Free 5 second tests, or comparison tests, are available through Usabilityhub.com – this is great for early on in the design process.
Qualaroo is our tool of choice for running on-site surveys. Generally speaking we run them on exit-intent and use them to drive our AB test hypotheses, as well as general business direction. There is also a useful integration with Optimizely, so you can push users into different segments depending on their survey results.
Data from within customer services
Customer services is the closest we get to a customer at Secret Escapes, and their feedback is essential to our optimisation program. Currently one individual from Product/Tech team spends 1 day each month shadowing our customer service agents at the call centre. The shadowing includes listening into phone calls, working through zendesk issues as well as discussing common issues with the supervisors up there. We are also provided with a weekly update of frequently asked questions from our members, so we can look to find simple tech solutions. This has driven some of our most useful feedback, so far.
Last but not least, we use Optimizely for running our AB test experiments. Originally we started out using Google Content Experiments which is a great, free, introduction to the world of split testing. As we looked to increase our testing velocity, and quality, we moved to a solution which didn’t require releases. You can read more about our switch here