Crowdvert

(604) 842-8553

Suite101 Finds a 4% Lift in Click Through Rate by Testing Different Fonts and Color Shades

Jun 18, 2012   //   by yosemsweet   //   case studies  //  2 Comments

Before starting Crowdvert I was the CTO at Suite101.com. In my time there we shifted from a gut-feel based organization to a company making data-driven decisions. There were a lot of great tests performed at Suite101 and over the coming months I’ll highlight a few of them.

This first one is a multivariate test we ran with AdSense in early 2011 attempting to find a color and font combination that maximized our revenue.

Test Setup

Suite101 had 2 major hypotheses:

  1. Changing the shade of blue used in AdSense titles to the Google standard blue would increase AdSense click-through-rates.
  2. Changing the font to differentiate ads from Suite101 content would reduce AdSense click-through-rates (the so-called “Trick people into clicking effect”).

In order to test these hypotheses we set up a multivariate test with 2 independent variables: AdSense title color and AdSense font. For our color variable we decided to test 4 variations:

  • #0000FF – the “blue-blue”. This was the baseline
  • #0044CC – Bing’s publicized outperforming blue
  • #0033CC – Bing’s actual blue
  • #2200C1the Google blue at the time

For fonts we tested two variations:

  • Arial – This was the baseline
  • Courier

The plan called for 50% of our traffic to be distributed equally across the 8 combinations until a statistically significant winner was found.

Here are screenshots of the 8 variations:

Test Results

After a few days Suite101 began to find some very interesting results!

One of our hypothesis held true, the other was in fact false. Both the AdSense title color and font choice had a statistically significant impact on AdSense click-through-rates. By varying the shade of blue used we could generate a variance in click through rates of 4.7% (our baseline color was the second most performant shade tested). Furthermore using a different font generated an average increase in click-through-rates of 4.2%.

Breaking it down into specific combinations we found the Google Blue combined with the Courier font generated the best overall lift with a 4.4% lift in click-through-rates over baseline.

Test Oddity

One interesting data point we found over the course of the test was that the Google Blue performed extremely well during business hours but under performed all other color shades after 3pm Pacific. We never dived into the reasons for this in greater detail or figured out if it was a statistical anomaly, but I can state no other colors exhibited this type of behavior.
If anyone has any ideas what may have caused the Google Blue to show this variance by time of day I’d love to hear from you.

2 Comments

  • Isn’t that around the time your company fell apart?

    • Actually Suite101 suffered from Google’s Panda update, which significantly changed traffic but not monetization. An argument can be made (and I have made it several times) that a key reason Suite101 was impacted by Panda was that we were targeting the wrong metric. Generally speaking there are far better KPIs to focus a company around than eCPM. For instance my favorite is lifetime value, another great one is retention rate, or for some business models what’s most important is revenue per employee.

Leave a comment

What is Crowdvert?

Crowdvert gives you a team of optimization experts who create A/B experiments to test on your site using our software. All the work is done on a pay for performance basis so there are no fees unless your conversion rate improves.

Find out more at crowdvert.com