How The Telegraph increased conversion rates while improving subscriber quality

  • Drawing meaningful analyses out of large quantities of data
  • Testing and implementing design changes in days, not weeks
  • Recommending high-stakes tests with confidence and
  • Boosting subscriptions toward an ambitious company goal

Chasing a North Star Metric

  • Varied audiences mean no one-size-fits-all solution. The Telegraph draws a large UK and US reader base, plus audiences throughout the rest of the world. Readers are as varied in intent as they are in geography, too. “If someone comes through [search engine results], we know those users are probably going to be much cooler because they’ve seen a link somewhere,” David explained, “They might not even know who the Telegraph is.” Those “cool” readers may simply want access to a piece of content, compared to readers from social media, who are “warmed up” with brand context and may crave more engagement.
  • Testing on a live site is expensive. Every live A/B test involves identifying a hypothesis, designing variants, coding the test, getting stakeholder signoff (on multiple levels), and more. These inputs mean each test risks time, resources, and revenue. So, running the wrong tests can incur extraordinary sunk costs.
  • Test result interpretation. The Telegraph consists of clever experts in cross-functional teams. It is rich in data but felt a gap in interpretation — how to question and formulate tests in order to tell a story in the results from all the rich data they collected.

Finding a partner who melds with the team

Improving paywall conversions and subscriber quality

  • Simplify their governance of paywalls: The Telegraph’s various overlays were complex to manage.
  • Improve user experience for readers: The user’s paywall journey wasn’t optimized by context (referral channel or number of visits) and could feel disjointed. “The user actually had to work quite hard to find all the information they needed,” David explained. For example, site visitors would sometimes see seemingly competing offers on the same screen.

Using better user experience to drive improved conversion rates

  • Behavioral differences on desktop vs. mobile
  • Conversion rate differences on old vs. new site content
  • Content preferences for registered vs. anonymous visitors

Making data-backed changes in days, not weeks

  • A full A/B testing campaign is impractical in terms of budget, time, or resources
  • A company needs to make fast changes during a seasonal time crunch
  • A team wants to make interim changes to a broken site/process to bide more time for a complete conversion program
  • Limiting politics of A/B testing (what to A/B test is informed by rapid test data vs. executive opinions)
  • Narrowing down many ideas to the best ones

Reducing same day cancellations by 30% while improving subscriber quality and increasing acquisition rates

Improving onboarding for customers

  • A paywall exists to encourage a website visitor to purchase a subscription or start a free trial. A paywall maximizes conversions.
  • Onboarding exists to familiarize a visitor with the brand and product immediately after they buy a subscription or start a free trial. Onboarding maximizes retention.

Diagnosing problems before prescribing solutions

  • What’s the best place for a new subscriber to onboard?
  • What’s the best language, design, and number of actions required for onboarding?
  • Why aren’t visitors completing this key action?
  • Click-click-clicking through onboarding patterns to skip to what they wanted
  • Overlooking much of the onboarding content
  • Missing important celebratory milestones

Ideal customer experience standards to guide testing

  • Asking, “Which of these variants will be most effective at communicating the value of downloading the mobile app?”
  • Gathering data from a small group of visitors, through various user testing methods (such as card sorting)
  • Bringing findings to The Telegraph team, who implemented a live site change in days

Shifting the internal testing mindset

Seeing the customer’s experience with third-party assistance

Increasing efficiency of internal teams

Winning testing bets with high confidence and fast approvals



Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
The Good

The Good

Our mission is to remove all the bad digital experiences until only the good remain, so ecommerce companies can realize their full potential.