5 Optimization Discoveries from the SAP Website Test Lab Shawn Burns Brian Carroll Global Vice President Executive Director, Digital Marketing MECLABS SAP
Session Speaker Shawn Burns Global Vice President Digital Marketing, SAP Black & White Headshot Worldwide lead for SAP Digital Marketing, including online demand generation, search media @shawnpburns partnerships and our flagship website, SAP.com and its 72 country sites in 40 languages. Shawn is based in Paris, France with his wife and three sons. 2
“Web Analytics is a table stake and it doesn’t get you very far” 3
“New People, new skills, and new attitudes are required” 4
“Testing without a process is driving without roads – there is no point to even having a car” 5
SAP Test Lab: • 3 years in the making • 140 tests launched 6
Some of the results…
5 Optimization Discoveries from the SAP Test Lab Search is (even) more important than ever 1 2 Photographic imagery sharpens response 3 Great results don’t have to be complex 4 Make those CTAs easy to spot (and click) 5 Test surprises happen, which is why we test!
5 Optimization Discoveries from the SAP Test Lab Search is (even) more important than ever 1
Multi-touch attribution vs. last touch approach uncovers true drivers Last touch attribution: journey with 5 touch points – last one gets all the credit 100% Multi-touch attribution: journey with 5 touch points – each getting partial, proper credit 20% 10% 30% 20% 30%
Bonus Discovery: Multi-touch attribution vs. last touch approach uncovers true drivers Experiment ID: SAPXXX Location: US Test Protocol: TPXXXX Research Notes: Background: We tested a more detailed and balanced attribution approach versus the traditional last touch attribution model we’ve been using for years Objective: Understand what each media vehicle truly brings to the table, from start to finish of visitor journey Primary Research Question: How do media vehicles stack up when considering the end-to-end journey? Test Design: Deep-dive regression analysis for 5 months of historical data, spanning search, display and content syndication activity. 11
Multi-touch attribution test: Results • Showed the real contribution of each channel vs. last touch model Vehicle Last Touch Multi-touch Difference Banner 4,634 4,248 -8% SEO 3,732 3,839 3% SEM 1,792 2,035 14% Content Syndication 887 924 4% SEM gained 14% additional conversion credit, when moving from Last Touch to multi-touch attribution model What You Need to Understand : the content deluge = confusion for ! customers, pushing them to search more often as they educate themselves WELL BEFORE they have a selection set. This means search has moved well beyond lead gen into the heart of brand awareness.
5 Optimization Discoveries from the SAP Test Lab Search is (even) more important than ever 1 2 Photographic imagery sharpens response
Discovery #2: Photographic imagery sharpens response Experiment ID: SAPXXX Location: Globally Test Protocol: TPXXXX Research Notes: Background: With different types of imagery in its arsenal the natural question of which is best surfaced the need to test Objective: Increase lead form submissions Primary Research Question: Which image type – vector iconography or photographic imagery – drives more response and conversion rates? Test Design: Test replaces vector style imagery across entire landing page with photographic imagery to gauge differences in response triggered by image type 14
Photographic imagery sharpens response + 46% Testing vector images VS photographic images . A story of customer response VS marketing production costs
Experiment 2: Results 46% increase in conversion Sharper photographic imagery generates more visitor response and conversion rate (46% more) Form Page version Submits Control – hero, no CTA 93 Treatment – hero with CTA 134 Relative Difference (conversion rate) + 46% ! What You Need to Understand : If you have different image types, it’s worth testing them against each other – results can be very disparate 16
5 Optimization Discoveries from the SAP Test Lab Search is (even) more important than ever 1 2 Photographic imagery sharpens response 3 Great results don’t have to be complex
Discovery #3: Great results don’t have to be complex Experiment ID: SAPXXX Location: UK Test Protocol: TPXXXX Research Notes: Background: Testing button copy on key landing pages is always part of our refinement and optimization strategy Objective: Increase lead form submissions Primary Research Question: What button copy works best at driving more conversions? (Get free premium content OR Download for free) Test Design: Simple AB test pitted 2 button texts against each other 18
Great results don’t have to be complex Get Free Premium Content › Download For Free › + 47% Copy test – small changes make a BIG impact
Experiment 3: Results 47% increase in conversion Even with offers below hero, getting opportunity for action in prominent hero makes a substantial difference Form Page version Submits Control – hero, no CTA 127 Treatment – hero with CTA 160 Relative Difference (conversion rate) + 47% ! What You Need to Understand : For business critical pages, testing CTA copy can drive non-trivial results! 20
5 Optimization Discoveries from the SAP Test Lab Search is (even) more important than ever 1 2 Photographic imagery sharpens response 3 Great results don’t have to be complex 4 Make those CTAs easy to spot (and click)
Discovery #4: Make those CTAs easy to spot (and click) Experiment ID: SAPXXX Location: US Test Protocol: TPXXXX Research Notes: Background: SAP Test Lab wanted to see what the difference would be in click rates if we moved from text link to button in our hero areas of SAP.com Objective: Increase engagement rates with hero offers Primary Research Question: What CTA style garners more engagements (clicks) – text links or buttons? Test Design: Simple AB test pitted 2 CTA styles against each other 22
Make those CTAs easy to spot (and click) Smaller text link Button design approach + 7% Key question – can CTAs be made easier to find?
Experiment 4: Results 7% increase in conversion buttons garner more clicks – while lift was modest, when syndicating results globally, lift has strong impact on overall site engagement Page version Hero Clicks Control – hero, text link CTA 212 Treatment – button CTA 227 Relative Difference (conversion rate) + 7% ! What You Need to Understand : Make sure it is easy for visitors to spot those important CTAs 24
5 Optimization Discoveries from the SAP Test Lab Search is (even) more important than ever 1 2 Photographic imagery sharpens response 3 Great results don’t have to be complex 4 Make those CTAs easy to spot (and click) 5 Test surprises happen, which is why we test!
Discovery #5: Test surprises happen, which is why we test! Experiment ID: SAPXXX Location: US Test Protocol: TPXXXX Research Notes: Background: We set out to test different ways to distinguish premium (registration gated) and non-gated content to see which approach is optimal at driving registrations Objective: Increase lead form submissions Primary Research Question: What mix of icons and messaging that distinguishes gated and non-gated content drives the most registrations? Test Design: AB/n approach pitted 4 challengers against the control, which was no messaging at all 26
Test surprises happen, which is why we test! + 17% Testing assumptions about transparency with the reality of customer behavior
Experiment 5: Results 17% decrease increase in conversion despite nice gesture of distinguishing gated and non-gated content, the visual cues suppressed registration, the thinking that visitors were more aware of where to find the “free’ content Page version Hero Clicks Control – no messaging 293 Treatment – “login required” 235 Relative Difference (conversion rate) -17% ! What You Need to Understand : Giving visual cues as to what offers require registration can actually suppress overall conversion rates 28
5 Optimization Discoveries from the SAP Test Lab Search is (even) more important than ever 1 2 Putting content questions to the test 3 Great results don’t have to be complex 4 Make those CTAs easy to spot (and click) 5 Test surprises happen, which is why we test!
Holistic understanding where to focus efforts Up to 780% Up to 97% Up to 92% Registration and Design and Content and Form Pages Layout Offers *Potential Lift is based on the positive benefit seen in actual tests. 6
A word about change management What does SAP Test Lab do? SAP Test Lab is an internal team at SAP dedicated to testing the SAP web presence. Using data from real customers, SAP Test Lab finds what works best to drive more leads from the traffic you already have, right now. That way, you can optimize your marketing strategy by uncovering new insights about your visitors and how to serve them best. 3 It’s impossible to beat the “testing drum” too loud
Thank You & Questions Shawn Burns Global Vice President Digital Marketing SAP @shawnpburns
5 Optimization Discoveries from the SAP Test Lab Turn heroes into action heroes 6
Recommend
More recommend
