Draft fting a Testing Strategy How a subscription-based business - PowerPoint PPT Presentation

Draft fting a Testing Strategy How a subscription-based business increased conversions by 121% Ben Eva Benjamin Huppertz Global Head of Conversion Management Senior Research Manager Euromoney Institutional Investor MECLABS

Session speaker Be Ben Eva @bevamail Glob lobal l He Head of of Con Conversion Managemen ent Euromoney In Insti titutional l In Inves estor Headshot Ben Eva is an experienced digital marketing professional with more than 17 years of experience in developing strategies to grow revenues for a wide variety of B2B information businesses. @BEvaMail

@BEvaMail About: • One of Europe’s largest business and financial media companies that has interests in financial publishing, training and event organization. • It is 70% owned by The Daily Mail and General Trust Group. • Publishes 200+ online information services plus global events and training courses. • 2013 turnover: £404.7/$679.8 million • 51% of revenue comes from subscription products, primarily online.

@BEvaMail History Graph: Conversion rates 2008-2012 Conversion to 8.5% Sale 6.8% 3% 1.4% .82% 2006 2007 2008 2009 2010

@BEvaMail History Graph: Conversion rates 2008-2012 Why? Conversion to 8.5% Sale 6.8% 3% 1.4% .82% 2006 2007 2008 2009 2010

@BEvaMail Why have conversions declined? External Factors Internal Factors • Higher product pricing • Poor development cycle • Significant increase in • Few UX resources traffic volumes • Poorer quality trials • Inability to run A/B testing

Initial sales funnel Anonymous Registered

Initial sales funnel 2.) 60% of traffic Anonymous Registered dropping out of the trial funnel 1.) Less than 1% of traffic converts to 3.) Less a lead than 1% of leads convert to a sale

Initial Sales Funnel: 6-month period Of these, 40K Anonymous Registered complete trial registration For every 10M visitors, 100K go through the Less than 1% trial funnel of trials convert to a sale – 300 sales worth $1M

Initial Sales Funnel: 6-month period Of these, 40K Anonymous Registered complete trial registration For every Time to launch a new strategy! 10M visitors, 100K go through the Less than 1% trial funnel of trials convert to a sale – 300 sales worth $1M

@BEvaMail Background: Channels and analytics Other 1% Bounce Rate Conversion Rate Referral 16% Organic 35% None 24% 57% CPC 25% 1% Traffic Source

@BEvaMail Background: Pre-analysis 53% proceed to user details 75% proceed to free trial part 2 40% conversion

@BEvaMail Background: Looking into the user experience

@BEvaMail Strategy in three phases Convert more traffic into the trial process Increase the completion rate for trial sign-up process Identify the best trials to convert to sales

@BEvaMail Step 1: : Strategy Convert more traffic into the trial process Increase the completion rate for trial sign-up process Identify the best trials to convert to sales

@BEvaMail Exp xperiment 1: : Background Exp xperiment ID ID: Metal Bu Bulle lletin in, paywall Loc Location: Loc Locked art rtic icle Tes est t De Desig ign: A/B /B Res esea earch Notes: Background: Only 1% of anonymous traffic entering the trial process. Bac Objective: To increase entry into the trial funnel. Ob Prim rimary ry Res esearch Qu Question: Can we reduce the bounce rate and take more people into the trial process? Tes est t El Elements: CTA, UX, design

@BEvaMail MetalBulletin: Control Instructions Subscription or free trial CTAs Login

@BEvaMail MetalBulletin: Variation 1 Banner with CTA Login

@BEvaMail MetalBulletin: Variation 2 Free trial or subscribe CTAs Login

@BEvaMail MetalBulletin: Variation 3 Free trial CTA Login

@BEvaMail Side by side Variation 1 Variation 2 Variation 3

@BEvaMail Results: MetalB lBulletin cumula lative performance 153% improvement in cumulative performance All variations outperformed the control Conversion MetalBulletin.com Improvement Rate Control 1.7% -- Variation 1 2% 19.1% Variation 2 2% 17.6% Var ariation 3 4.2% 4.2% 153% 153% What You ou Nee eed to to Understand: Variation 3 met and exceeded the target of ! 20% with a confidence level of 95%.

@BEvaMail Results: MetalB lBulletin cumula lative performance 153% improvement in cumulative performance All variations outperformed the control Shared Learning: Conversion Metal Bulletin Improvement Rate Control 1.7% -- Now rolling out business-wide Variation 1 2% 19.1% Variation 2 2% 17.6% Var ariation 3 4.2% 4.2% 153% 153% What You ou Nee eed to to Understand: Variation 3 met and exceeded the target of ! 20% with a confidence level of 95%.

@BEvaMail Step 2: : Strategy Convert more traffic into the trial process Increase the completion rate for trial sign-up process Identify the best trials to convert to sales

@BEvaMail Exp xperiment 2: : Background Exp xperiment ID ID: In Insti titutional l In Investor, stage 1, , tri trial l step 1 Location: Tria Loc rial Process Bac Background: 60% of traffic in the funnel is not completing the process. Ob Objective: To increase step 1 and 2 conversion. Prim rimary ry Res esearch Qu Question: Can we increase conversion in steps 1 and 2? Tes est t El Elements: CTA, UX, design

@BEvaMail Institutional Investor: Free trial Step 1: Variation 2 External Traffic Step 1: Step 2 Step 3 Control Internal Traffic Step 1: Variation 1

@BEvaMail Institutional Investor: Control

@BEvaMail Institutional Investor: Treatment 1 Rotating banners:

@BEvaMail Institutional Investor: Treatment 2

@BEvaMail Institutional Investor: Treatment 3

@BEvaMail Side by side Control Treatment 1 Treatment 2 Treatment 3

@BEvaMail Results: KPI I as conversion 36.4% improvement over the control The single-panel page had the highest conversion rate. Conversion InstitutionalInvestor.com Improvement Rate Control 32.3% -- Treatment 1 – Slideshow 38.2% 18.4% Treatment 2 – Form fields 41.1% 27.4% Treatment 3 3 – Sing Single pan panel 44% 44% 36 36.4% .4% What You ou Nee eed to to Understand: Variation 3 has a 95.4% chance of beating ! the baseline.

@BEvaMail Results: KPI I as conversion 36.4% improvement over the control The single panel page had the highest conversion rate. Shared Learning: Conversion Variation Improvement Rate Control 32.3% -- Now rolling out over 11 sites Treatment 1 – Slideshow 38.2% 18.4 Treatment 2 – Form fields 41.1% 27.4 Treatment 3 3 – Sing Single pan panel 44% 44% 36 36.4% .4% What You ou Nee eed to to Understand: Variation 3 has a 95.4% chance of beating ! the baseline.

@BEvaMail Exp xperiment 3: : Background Exp xperiment ID ID: EuroWeek , , shared le learning g roll ll ou out, stage 1 tria trial process Location: Tria Loc rial process Bac Background: 60% of traffic in the funnel not completing the process. Ob Objective: To increase step 1 and 2 conversion. Prim rimary ry Res esearch Qu Question: Can we increase conversion in steps 1 and 2? Tes est t El Elements: CTA, UX, design

@BEvaMail EuroWeek: Variation 1 Breadcrumbs Form fill and features

@BEvaMail EuroWeek: Variation 2 Features Form fill and CTA

@BEvaMail EuroWeek: Variation 3 Breadcrumbs Picture upper left Form fill and CTA Icons and features

@BEvaMail Variations: Side by side Variation 1 Variation 3 Variation 2

@BEvaMail Results: Prim imary KPI I as form submit itted 20% improvement in conversion rate None of the variations underperformed the control. EuroWeek.com Conversion Improvement Control 45% -- Var ariation 1 54.7% 54 .7% 20% 20% Variation 2 52.9% 16.8% Variation 3 45% 1% What You ou Nee eed to to Un Understand: The top performance tier target of 20% was ! reached and exceeded by Variation 1.

@BEvaMail Results: Prim imary KPI I as form submit itted 20% improvement in conversion rate None of the variations underperformed the control Shared Learning: Euroweek Conversion Improvement Control 45% -- Now rolling out over 11 sites Var ariation 1 54.7% 54 .7% 20% 20% Variation 2 52.9% 16.8% Variation 3 45% 1% What You ou Nee eed to to Un Understand: The top performance tier target of 20% was ! reached and exceeded by Variation 1.

@BEvaMail Exp xperiment 4: : Background Exp xperiment ID ID: He Hedge Fund Fund In Intell lligence, com ombin ine steps 1 and 2, , red educing step eps in n the e funnel Loc Location: Tria rial process Bac Background: Poor UX in stages 1, 2 and 3. Ob Objective: To increase steps 1-3 conversion. Prim rimary ry Res esearch Qu Question: Can we increase conversion in steps 1-3? Tes est t El Elements: CTA, UX, design

@BEvaMail Results analyzed 68% Proceeded to email address 75% Proceeded to password

@BEvaMail Free trial sign-up process Step 1: Step 2: Control Control Internal Step 3 Step 4 Traffic Step 1 and 2 of 4: Test Variation

@BEvaMail Control Step 1: Step 2: Control Control Internal Step 3 Step 4 Traffic Step 1 and 2 of 4: Test Variation Email page Options page