A simple effort at personalisation

Personalisation is one of the things I really expect to blow up in 2015. With so much data lying around about our customers, and their behaviour, we now have a great opportunity to optimise their experience by providing more relevant sales and information. Personalisation can be as simple as adding a name to an email subject line, or it can be as complex as completely changing a user experience of a website.

In 2015 I’ve been keen to experiment with personalisation of the web, and how we at Secret Escapes present our offers to our users. Currently our ‘homepage’ for signed in users is a list of sales, often 200 long. Whilst we curate and hand-pick every offer that features on the site, we don’t do a great job at creating a personal list for our users. Using Optimizely, and a hack-day, I was keen to implement something that would prove the concept and be sufficient for a very rudimentary A/B test.

Our current list of hand-picked sales

Our current list of hand-picked sales



There were multiple ways I looked at using data about our users, as well as how we may be able to quickly create some. I explored 3 different types of personalisation:

  1. Search terms. Can we personalise the site and increase the prominence of sales which match previous searches made by the user?
  2. Sale type, based upon most visited sales. We group and tag sales by location and sale type (All-inclusive, Spa). If a user shows a certain propensity per type, for example 80% of sales visited are London based, should we promote those sales?
  3. PPC keyword registration. When we acquire our users via PPC, they are often acquired via a specific keyword. For example “Spa weekend breaks”. If they are looking for these sales, should we promote them for the next x days?

I used Optimizely and Jquery cookie to set a cookie with the above information for the user. We set the Optimizely experiment to run for 100% of users, on the appropriate URLs. For search terms, I stored the last 3 search terms made by a user. For sale type, we incremented a cookie for each sale type match they made. And for PPC keyword we simply set a cookie which stored the keyword that the user reached the site on (this was extracted from a query string).

The JS used to set the cookie, this was set in a separate experiment on sign-up page

The JS used to set the cookie, this was set in a separate experiment on sign-up page

I then set up multiple Optimizely experiments, and used their targeting to enter users into the appropriate experiment.  If the user matched the cookie, they were entered into an A/B test. The test promoted all relevant sales to the top of the listing page. For example, a visitor who signed up under the keyword of New York, would see the following page:

If you signed-up to the site with a keyword featuring NY, you would now see this

If you signed-up to the site with a keyword featuring NY, you would now see this

The complexity of this hack would lie in how we treat users who fall into multiple experiments. So for V1 of the experiment we opted to only run with a single cookie type, which was PPC keyword registration. This was the simplest of the implementations, as it wasn’t a dynamic or changing cookie. Once they had signed up it would remain the same until they expired.


Our initial results are incredibly promising, for the New York keyword we saw the following increases after running for 140 days:

Clicks to sale from listing page – +24.4% 
Clicks to purchase page – +30.4%
Completed purchases – +210%

Other keywords, including “All-Inclusive” saw very similar results. Seeing significant results for clicks to sale from the listing page.

Whilst the numbers on completed purchases are relatively low, all of the KPI’s are positive and it’s something we’re soon to be extending even further across other keywords.

Potential problems

Naturally this type of personalisation isn’t scalable, however it’s enough to validate an initial hypothesis and spend further time investigating more robust solutions.

It’s worth considering the amount of users who will fit into each bucket. The more granular you get with your personalisation, the harder it is to validate the result as significant. We initially chose our highest performing (in terms of signed-up users) keywords, to test good levels of traffic.

Cookie compliance. The EU has specific directives on what cookies can be set without opting in, this needs to be taken account and if running this experiment in the EU your users may need to be aware if you are using cookies to personalise their experience



One Comment

  1. Pingback: Case Study: Testing Personalized Experiences at Secret Escapes | The Optimizely Blog

Leave a Reply

Your email address will not be published. Required fields are marked *