- Multivariate A/B Testing with Drupal MontenaSoft MD Systems - - PowerPoint PPT Presentation

▶

Feb 29, 2024 136 likes •488 views

- Multivariate A/B Testing with Drupal MontenaSoft MD Systems Miro Dietiker Ivica Puljic Zeljko Puljic Overview Basic of Drupal A/B Testing: Concepts & History (Zeljko Puljic, MontenaSoft) Case Study Acredis (Miro Dietiker,

SLIDE 1

Multivariate –

A/B Testing with Drupal

MontenaSoft

Ivica Puljic Zeljko Puljic Miro Dietiker

MD Systems

SLIDE 2

Overview

Basic of Drupal A/B Testing: Concepts & History

(Zeljko Puljic, MontenaSoft)

Case Study Acredis

(Miro Dietiker, MD-Systems)

Hands-On: distribution, usage, enchancement

LINK TO DISTRIBUTION DOWNLOAD

(Ivica Puljic, MontenaSoft)

SLIDE 3

Purpose of A/B Testing

Every website has a target audience (segmentation)
But perhaps you want to expand to new markets
No Website is ever ready (optimized & finished)
Even if marketing department has all answers ready,

there is no way around reality

so you should test & measure the impact of

different settings to find out what really works

But … you need to have clear idea about initiate & success criteria

SLIDE 4

Usual suspects... On diagram

SLIDE 5

Success, Conversion, Path

clear def of initiate paths & success

Success examples: (1) user has visited Page XYZ or (2) has bought Product XYZ Specific path example (for 2): Products list overview, product page, basket, checkout, credit card details,

rder confirmation

SLIDE 6

Basic Concepts of A/B Testing (1)

Product List P1 P2 PN Product P1 Description BASKET AD1 NE WS AD2 AD2 Menu

Point1
Point2
Point3

Product List P1 P2 PN Product P1 Description BASKET AD1 NE WS AD2 AD2 Menu

Point1
Point2
Point3

SLIDE 7

Do it with Drupal

Several external tools to integrate

(change something on your landing page, for instance)

http://drupal.org/project/multivariate

(D6 taken over by MontenaSoft & MD-Systems)

and today, for the first time ever ...

official announce of D7 Port

SLIDE 8

Multivariate Vocabulary & A/B (1)

1:N

Test

1:N

Set

1:N

Variant Mutation

AD2

Variant1

Layout 1
CSS 1

Variant2

Layout 2
CSS 2

A/B Test Set

Variant1, Variant2, Variant1, ....

SLIDE 9

Multivariate Vocabulary & A/B (2)

1:N

Test

1:N

Set

1:N

Variant Mutation Multivariate Test

AD2

Variant1

Layout 1

Variant2

Layout 2

Set 1

[Set1] x [Set2] (Layout1, CSS1), (Layout1, CSS2), (Layout2, CSS1), (Layout2, CSS2),

AD2

Variant1

CSS 1

Variant2

CSS 2

Set 2

SLIDE 10

Multivariate Measurements

Assign & Track mutation combinations from different Sets to user

Module responsability: specify activation conditions (cache)

Measure conversion

define success criteria: landing on a page, create user account... measure & present results

Plugable Architecture

Ctools Plugin

SLIDE 11

Testjng - Why

Solves your real business problems
Can make you rich! ;-)

SLIDE 12

Leads - Business

Cost per Lead

htup://en.wikipedia.org/wiki/Cost_per_Lead

SLIDE 13

Leads - Conversion

Lead marketjng platgorms
Convert a Visitor
… to a qualifjed Lead
Own leads
Trade leads

SLIDE 14

Conversion - Improvements

Small changes
+10 .. 30%
Difgerent approaches
+100 … 1000%

SLIDE 15

Conversion - Sequences

3 Iteratjons of +26%
+100% Conversion
1.26*1.26*1.26 = 2.00+
+100% Profjt ;-)
With virtual goods...

SLIDE 16

Cases - Landing Page

Visual difgerence

SLIDE 17

SLIDE 18

Cases - Registratjon

Big signup form
Small reduced form

SLIDE 19

Case 1 - Registratjon Page 1/2

SLIDE 20

Case 1 - Registratjon Page 2/2

SLIDE 21

Variatjons - A few Ideas

Images, Characters
Design, Color, Style, Efgects
Content, Length / Details,

Perspectjve

Personalizatjon
+++

SLIDE 22

Conversion - Aspects 1/2

Conversion = Trust
Expectatjons
Client needs
Frame / Emotjons

SLIDE 23

Focus / Distractjon
Service quality
Value
Right Price
Your reliability

Conversion - Aspects 2/2

SLIDE 24

Ask less
Reductjon, Simplifjcatjon
Expected more Conversion
Ask more
Provided more Conversion

Conversion - Unexpected Form

SLIDE 25

Trust
Sensitjve informatjon
Care about everything
Feel safe

Unexpected Form - Reasons

SLIDE 26

Truth
NO Philosophy
NO debate
Decisions
Let clients decide what's best

Testjng - It's all about

SLIDE 27

Risks - Cautjon!

Don’t break your site
Multjple states...
Know what you’re testjng
False conclusions possible

SLIDE 28

Alright! - But...

Start Testjng NOW!

SLIDE 29

Current state of Multivariate

What is currently done
Tests are now entity based
Mutations plugins for path mutation, path

redirection, variable change, node load, google analytics, internal css, form class and inline css

Success plugins for path and form submit
Basic table and chart reports
Override tests for specific user

SLIDE 30

Additional features needed for D7 stable release

Bring back simple plugin support
Add theme_change mutation plugin
Should other two tables be entities so we

have views support for them also?

Add Views support
More reports
Time based charts
Upgrade path from D6

SLIDE 31

Future

ctools export / import
Even more plugins out of the box
Provide documentation, howto, video, ...
Set A/B/X rates instead of equal distribution
Permanently switch to variant (disable testing)
Automatically switch to permanent variant on statistical (winner) condition
Improved continuous stats (add date-from, date-to, counters)
Introduce hooks (e.g. success, apply mutation, ...) / rules integration
Push AB test states into Drupal.settings JS so client side JS can build on it
And mooooore! ;) current list on

http://drupal.org/node/1136868#comment-5716584

SLIDE 32

As always... Help needed

SLIDE 33

Multivariate D7 TODO

Finish D7 port effort
Add simpletests testing
Fix all important problems/improvements
CTools cache problems
... and the rest of @todo's in code ;-)