Draft fting a Testing Strategy How a subscription-based business - - PowerPoint PPT Presentation

▶
draft fting a testing strategy
SMART_READER_LITE
LIVE PREVIEW

Draft fting a Testing Strategy How a subscription-based business - - PowerPoint PPT Presentation

Nov 29, 2023 635 likes •1.22k views

Draft fting a Testing Strategy How a subscription-based business increased conversions by 121% Ben Eva Benjamin Huppertz Global Head of Conversion Management Senior Research Manager Euromoney Institutional Investor MECLABS Session speaker

slide-1
SLIDE 1

How a subscription-based business increased conversions by 121%

Draft fting a Testing Strategy

Ben Eva Global Head of Conversion Management Euromoney Institutional Investor Benjamin Huppertz Senior Research Manager MECLABS

slide-2
SLIDE 2

Session speaker

Be Ben Eva @bevamail Glob lobal l He Head of

  • f Con

Conversion Managemen ent Euromoney In Insti titutional l In Inves estor

Ben Eva is an experienced digital marketing professional with more than 17 years of experience in developing strategies to grow revenues for a wide variety of B2B information businesses.

Headshot @BEvaMail

slide-3
SLIDE 3

@BEvaMail

About:

  • One of Europe’s largest business and financial media companies that has interests in financial

publishing, training and event organization.

  • It is 70% owned by The Daily Mail and General Trust Group.
  • Publishes 200+ online information services plus global events and training courses.
  • 2013 turnover: £404.7/$679.8 million
  • 51% of revenue comes from subscription products, primarily online.
slide-4
SLIDE 4

@BEvaMail

History Graph: Conversion rates 2008-2012

8.5% 6.8% 3% 1.4% .82% Conversion to Sale 2006 2007 2008 2009 2010

slide-5
SLIDE 5

@BEvaMail

History Graph: Conversion rates 2008-2012

8.5% 6.8% 3% 1.4% .82% 2006 2007 2008 2009 2010 Conversion to Sale

Why?

slide-6
SLIDE 6

@BEvaMail

Why have conversions declined?

  • Higher product pricing
  • Significant increase in

traffic volumes

  • Poorer quality trials
  • Poor development

cycle

  • Few UX resources
  • Inability to run A/B

testing External Factors Internal Factors

slide-7
SLIDE 7

Anonymous Registered

Initial sales funnel

slide-8
SLIDE 8

Anonymous Registered

Initial sales funnel

1.) Less than 1% of traffic converts to a lead

3.) Less than 1% of leads convert to a sale

2.) 60% of traffic dropping

  • ut of the

trial funnel

slide-9
SLIDE 9

Anonymous Registered

Initial Sales Funnel: 6-month period

For every

10M visitors, 100K go through the trial funnel

Of these, 40K complete trial registration

Less than 1%

  • f trials

convert to a sale – 300 sales worth $1M

slide-10
SLIDE 10

Anonymous Registered

Initial Sales Funnel: 6-month period

For every

10M visitors, 100K go through the trial funnel

Of these, 40K complete trial registration

Less than 1%

  • f trials

convert to a sale – 300 sales worth $1M

Time to launch a new strategy!

slide-11
SLIDE 11

@BEvaMail

Background: Channels and analytics

None 24% Referral 16% Organic 35% CPC 25%

Other 1%

57% 1%

Bounce Rate Conversion Rate

Traffic Source

slide-12
SLIDE 12

@BEvaMail

Background: Pre-analysis

53% proceed to user details 75% proceed to free trial part 2 40% conversion

slide-13
SLIDE 13

@BEvaMail

Background: Looking into the user experience

slide-14
SLIDE 14

@BEvaMail

Strategy in three phases

Convert more traffic into the trial process Increase the completion rate for trial sign-up process Identify the best trials to convert to sales

slide-15
SLIDE 15

@BEvaMail

Step 1: : Strategy

Convert more traffic into the trial process Increase the completion rate for trial sign-up process Identify the best trials to convert to sales

slide-16
SLIDE 16

@BEvaMail

Exp xperiment 1: : Background

Exp xperiment ID ID: Metal Bu Bulle lletin in, paywall Loc Location: Loc Locked art rtic icle Tes est t De Desig ign: A/B /B

Bac Background: Only 1% of anonymous traffic entering the trial process. Ob Objective: To increase entry into the trial funnel. Prim rimary ry Res esearch Qu Question: Can we reduce the bounce rate and take more people into the trial process? Tes est t El Elements: CTA, UX, design

Res esea earch Notes:

slide-17
SLIDE 17

@BEvaMail

MetalBulletin: Control

Instructions Subscription or free trial CTAs Login

slide-18
SLIDE 18

@BEvaMail

MetalBulletin: Variation 1

Banner with CTA Login

slide-19
SLIDE 19

@BEvaMail

MetalBulletin: Variation 2

Free trial or subscribe CTAs Login

slide-20
SLIDE 20

@BEvaMail

MetalBulletin: Variation 3

Free trial CTA Login

slide-21
SLIDE 21

@BEvaMail

Side by side

Variation 1 Variation 2 Variation 3

slide-22
SLIDE 22

@BEvaMail

! Results: MetalB lBulletin cumula lative performance

What You

  • u Nee

eed to to Understand: Variation 3 met and exceeded the target of 20% with a confidence level of 95%.

MetalBulletin.com Conversion Rate Improvement Control 1.7%

  • Variation 1

2% 19.1% Variation 2 2% 17.6% Var ariation 3 4.2% 4.2% 153% 153%

153% improvement in cumulative performance

All variations outperformed the control

slide-23
SLIDE 23

@BEvaMail

! Results: MetalB lBulletin cumula lative performance

What You

  • u Nee

eed to to Understand: Variation 3 met and exceeded the target of 20% with a confidence level of 95%.

Metal Bulletin Conversion Rate Improvement Control 1.7%

  • Variation 1

2% 19.1% Variation 2 2% 17.6% Var ariation 3 4.2% 4.2% 153% 153%

153% improvement in cumulative performance

All variations outperformed the control

Shared Learning:

Now rolling out business-wide

slide-24
SLIDE 24

@BEvaMail

Step 2: : Strategy

Convert more traffic into the trial process Increase the completion rate for trial sign-up process Identify the best trials to convert to sales

slide-25
SLIDE 25

@BEvaMail

Exp xperiment 2: : Background

Exp xperiment ID ID: In Insti titutional l In Investor, stage 1, , tri trial l step 1 Loc Location: Tria rial Process

Bac Background: 60% of traffic in the funnel is not completing the process. Ob Objective: To increase step 1 and 2 conversion. Prim rimary ry Res esearch Qu Question: Can we increase conversion in steps 1 and 2? Tes est t El Elements: CTA, UX, design

slide-26
SLIDE 26

@BEvaMail

Institutional Investor: Free trial

External Traffic Internal Traffic Step 1: Variation 1 Step 1: Control Step 1: Variation 2 Step 2 Step 3

slide-27
SLIDE 27

@BEvaMail

Institutional Investor: Control

slide-28
SLIDE 28

@BEvaMail

Institutional Investor: Treatment 1

Rotating banners:

slide-29
SLIDE 29

@BEvaMail

Institutional Investor: Treatment 2

slide-30
SLIDE 30

@BEvaMail

Institutional Investor: Treatment 3

slide-31
SLIDE 31

@BEvaMail

Side by side

Control Treatment 1 Treatment 2 Treatment 3

slide-32
SLIDE 32

@BEvaMail

! Results: KPI I as conversion

What You

  • u Nee

eed to to Understand: Variation 3 has a 95.4% chance of beating the baseline.

InstitutionalInvestor.com Conversion Rate Improvement Control 32.3%

  • Treatment 1 – Slideshow

38.2% 18.4% Treatment 2 – Form fields 41.1% 27.4% Treatment 3 3 – Sing Single pan panel 44% 44% 36 36.4% .4%

36.4% improvement over the control

The single-panel page had the highest conversion rate.

slide-33
SLIDE 33

@BEvaMail

! Results: KPI I as conversion

What You

  • u Nee

eed to to Understand: Variation 3 has a 95.4% chance of beating the baseline.

Variation Conversion Rate Improvement Control 32.3%

  • Treatment 1 – Slideshow

38.2% 18.4 Treatment 2 – Form fields 41.1% 27.4 Treatment 3 3 – Sing Single pan panel 44% 44% 36 36.4% .4%

36.4% improvement over the control

The single panel page had the highest conversion rate.

Shared Learning:

Now rolling out over 11 sites

slide-34
SLIDE 34

@BEvaMail

Exp xperiment 3: : Background

Exp xperiment ID ID: EuroWeek, , shared le learning g roll ll ou

  • ut, stage 1 tria

trial process Loc Location: Tria rial process

Bac Background: 60% of traffic in the funnel not completing the process. Ob Objective: To increase step 1 and 2 conversion. Prim rimary ry Res esearch Qu Question: Can we increase conversion in steps 1 and 2? Tes est t El Elements: CTA, UX, design

slide-35
SLIDE 35

@BEvaMail

EuroWeek: Variation 1

Breadcrumbs Form fill and features

slide-36
SLIDE 36

@BEvaMail

EuroWeek: Variation 2

Features Form fill and CTA

slide-37
SLIDE 37

@BEvaMail

EuroWeek: Variation 3

Picture upper left Icons and features Form fill and CTA Breadcrumbs

slide-38
SLIDE 38

@BEvaMail

Variations: Side by side

Variation 1 Variation 2 Variation 3

slide-39
SLIDE 39

@BEvaMail

! Results: Prim imary KPI I as form submit itted

What You

  • u Nee

eed to to Un Understand: The top performance tier target of 20% was reached and exceeded by Variation 1.

EuroWeek.com Conversion Improvement Control 45%

  • Var

ariation 1 54 54.7% .7% 20% 20% Variation 2 52.9% 16.8% Variation 3 45% 1%

20% improvement in conversion rate

None of the variations underperformed the control.

slide-40
SLIDE 40

@BEvaMail

! Results: Prim imary KPI I as form submit itted

What You

  • u Nee

eed to to Un Understand: The top performance tier target of 20% was reached and exceeded by Variation 1.

Euroweek Conversion Improvement Control 45%

  • Var

ariation 1 54 54.7% .7% 20% 20% Variation 2 52.9% 16.8% Variation 3 45% 1%

20% improvement in conversion rate

None of the variations underperformed the control

Shared Learning:

Now rolling out over 11 sites

slide-41
SLIDE 41

@BEvaMail

Exp xperiment 4: : Background

Exp xperiment ID ID: He Hedge Fund Fund In Intell lligence, com

  • mbin

ine steps 1 and 2, , red educing step eps in n the e funnel Loc Location: Tria rial process

Bac Background: Poor UX in stages 1, 2 and 3. Ob Objective: To increase steps 1-3 conversion. Prim rimary ry Res esearch Qu Question: Can we increase conversion in steps 1-3? Tes est t El Elements: CTA, UX, design

slide-42
SLIDE 42

@BEvaMail

Results analyzed

68%

Proceeded to email address

75%

Proceeded to password

slide-43
SLIDE 43

@BEvaMail

Free trial sign-up process

Internal Traffic Step 1 and 2 of 4: Test Variation Step 1: Control Step 3 Step 4 Step 2: Control

slide-44
SLIDE 44

@BEvaMail

Control

Options page Email page

Internal Traffic Step 1 and 2 of 4: Test Variation Step 1: Control Step 3 Step 4 Step 2: Control

slide-45
SLIDE 45

@BEvaMail

Treatment

Header Breadcrumbs Features Contrast CTA

Internal Traffic Step 1 and 2 of 4: Test Variation Step 1: Control Step 3 Step 4 Step 2: Control

slide-46
SLIDE 46

@BEvaMail

! KPI: I: Conversion rate

What You

  • u Nee

eed to to Un Understand: Variation 1 beat the control with a 93% level

  • f confidence.

HedgeFundIntelligence.com Conversion Rate Improvement Control 40%

  • Variation 1

50.5% 26 26.1% .1%

26.1% improvement in cumulative performance

Combining steps 2 and 3 performed better than the control.

slide-47
SLIDE 47

@BEvaMail

! KPI: I: Conversion rate

What You

  • u Nee

eed to to Un Understand: Variation 1 beat the control with a 93% level

  • f confidence.

HedgeFund Intelligence Conversion Rate Improvement Control 40%

  • Variation 1

50.5% 26.1%

26.1% improvement in cumulative performance

Combining steps 2 and 3 performed better than the control

Shared Learning:

Now built into Web product development process

@bevamail

slide-48
SLIDE 48

@BEvaMail

Step 3: : Strategy

Convert more traffic into the trial process Increase the completion rate for trial sign-up process Identify the best trials to convert to sales

slide-49
SLIDE 49

@BEvaMail

Exp xperiment 4: : Background

Exp xperiment ID ID: He Hedge Fund Fund In Intell lligence, com

  • mbin

ine steps 1 and 2, , red educing step eps in n the e funnel Loc Location: Tria rial process

Bac Background: Poor UX in stages 1, 2 and 3. Ob Objective: To increase step 1-3 conversion. Prim rimary ry Res esearch Qu Question: Can we increase conversion in steps 1-3? Tes est t El Elements: CTA, UX, design

slide-50
SLIDE 50

@BEvaMail

Exp xperiment 5: : Background

Exp xperiment ID ID: Le Lead scorin ing /profili ling – op

  • pti

timizing lea leads sen ent t to

  • Sales

les Loc Location: All ll sit ites es

Bac Background: We don’t want to increase our sales team capacity or increase costs. Ob Objective: Increase the quality of leads without increasing the volume. Prim rimary ry Res esearch Qu Question: As we increase lead volumes, can we maintain sales team levels and identify the best leads? Tes est t Des Desig ign: A/B split test

slide-51
SLIDE 51

@BEvaMail

A/B testing lead scoring and engagement

Total leads converting to sales 0.82%

slide-52
SLIDE 52

@BEvaMail

A/B testing lead scoring and engagement

Group A Group B Control

Segments

0.96% Conversion 1.73% Conversion 0.96% Conversion

slide-53
SLIDE 53

@BEvaMail

A/B testing lead scoring and engagement

Group A Group B Control

Segments

0.96% Conversion 1.73% Conversion 0.96% Conversion

Shared Learning:

Now rolled out to all businesses

slide-54
SLIDE 54

@BEvaMail Anonymous Registered

New sales funnel*

For every 10M visitors, 123K go through the trial funnel

Of these, 60K complete trial registration *When all stages are rolled out over all products

38% of trials are sales- ready, converting at 1.73%, worth $1.22M

slide-55
SLIDE 55

@BEvaMail

Top takeaways

Internal development was our roadblock, we had to find alternatives. Find the “quick wins” that you can roll out as shared learning. Break the problem down into manageable chunks.

slide-56
SLIDE 56

@BEvaMail

Questions?

slide-57
SLIDE 57

@BEvaMail

Thanks to my team:

Il Ileana Lu Lund an and Dar Darren Her Herbert Conversion Rate Optimization Team Latitude Digital Marketing

ileana.lund@latitudegroup.com

slide-58
SLIDE 58

@BEvaMail

Thank you

Be Ben Eva Global Head of Conversion Management Euromoney Institutional Investor Plc.

beva@Euromoneyny.com @BEvaMail www.linkedin.com/in/beneva

Be Benja jamin Hu Huppertz Senior Research Manager MECLABS

Headshot Headshot