Predictive Analytics Broken Down - - PowerPoint PPT Presentation

▶
predictive analytics broken down
SMART_READER_LITE
LIVE PREVIEW

Predictive Analytics Broken Down - - PowerPoint PPT Presentation

Feb 25, 2023 271 likes •993 views

Predictive Analytics Broken Down https://en.wikipedia.org/wiki/File:Second_Foundation_cover.jpg Who is this guy? CEO / Co-Founder Conductrics www.conductrics.com matt@conductrics.com Past: Database Marketing Education: Artificial Intelligence

slide-1
SLIDE 1 https://en.wikipedia.org/wiki/File:Second_Foundation_cover.jpg

Predictive Analytics Broken Down

slide-2
SLIDE 2

CEO / Co-Founder Conductrics www.conductrics.com matt@conductrics.com Past: Database Marketing Education: Artificial Intelligence & Economics

twitter:@mgershoff, @conductrics Email:matt@conductrics.com

www.conductrics.com/blog

Who is this guy?

slide-3
SLIDE 3

Confidential

  • 1. Cloud-based Adaptive Testing and Decision Engine
  • 2. API-Based Testing, Targeting and Optimization
  • REST API: Compatibility with CMS systems and other platforms
  • Native Programming Wrappers (iOS, PHP, jQuery, Node, etc.)
  • New JavaScript API for super fast decisions at scale
  • 3. “WAX” Framework for point-and-click style customers
  • Client-side, tag-based, “skip IT” style implementation
  • 4. Browser UI
  • Admin Console
  • Reporting

What is Conductrics?

twitter:@mgershoff, @conductrics Email:matt@conductrics.com

slide-4
SLIDE 4

Confidential

  • 1. Experimentation
  • AB and Multivariate Testing
  • Adaptive / Bandit Testing
  • 2. Personalization
  • Targeting with Business Logic
  • Targeting via machine learning

What does Conductrics do?

twitter:@mgershoff, @conductrics Email:matt@conductrics.com

slide-5
SLIDE 5

Confidential

Promise of Predictive Analytics

The Promises:

  • Help make predictions about the future
  • Predictions about customer:
  • Preferences
  • Intent
slide-6
SLIDE 6

Confidential

Benefits of Predictive Analytics

The Benefits:

  • Provide customers with right set of

experiences

  • Eliminate marketing waste
slide-7
SLIDE 7

Confidential

Why care how it works?

  • Better consumer of predictive analytics

tools

  • How to get the most out of it predictive

analytics

  • Help ensure you understand its

limitations

slide-8
SLIDE 8

Confidential

Scope of talk?

All Predictive Analytics Uses

Transactional System

slide-9
SLIDE 9

9

twitter: @mgershoff

1.Data

2.Logic

Two Requirements for Personalization

slide-10
SLIDE 10

Data: ‘Sensing’ the World

slide-11
SLIDE 11

Conductrics twitter: @mgershoff Blog: www.conductrics.com/blog

  • Return Customer
  • Weekend/Weekday
  • Mobile/DeskTop
  • Browser Type
  • User Age
  • Geo/Census
  • Weather
  • Tenure/RFM Score

Observable

  • Lottery Game
  • Price
  • Sales Offers
  • Shipping Type
  • Layout/UX
  • Which Products
  • Suggested

Quantities Intervention Types of Data

slide-12
SLIDE 12

12

twitter: @mgershoff

1.Data

2.Logic

Two Requirements for Personalization

slide-13
SLIDE 13

Confidential

Requirements for Personalization

Decision logic links Observations to Actions

slide-14
SLIDE 14

How to come up with the Logic?

IF [Customer] THEN [Experiences?]

slide-15
SLIDE 15

How to come up with the Logic?

IF [Customer] THEN [Experiences?]

…in way that Optimizes Performance

slide-16
SLIDE 16

twitter: @mgershoff

Example: Veikkaus

slide-17
SLIDE 17

twitter: @mgershoff

Example: Lottery Games Show high price games …

slide-18
SLIDE 18

Or show the low price games

twitter: @mgershoff

Example: Lottery Games

slide-19
SLIDE 19

To keep it simple just look at:

  • New or Repeat Player
  • Weekday or Weekend

twitter: @mgershoff

Example: Lottery Games

slide-20
SLIDE 20

How to come up with the Logic?

IF [Repeat and/or Weekend] THEN [High/Low Price?]

…in order to be most profitable

slide-21
SLIDE 21

How its Done

1 Learn how Repeat and Weekend customers predict low price games

twitter: @mgershoff

slide-22
SLIDE 22

How its Done

twitter: @mgershoff

2 Learn how Repeat and Weekend customers predict high price games 1 Learn how Repeat and Weekend customers predict low price games

slide-23
SLIDE 23

How its Done

1 Learn how Repeat and Weekend customers predict low price games

twitter: @mgershoff

2 Learn how Repeat and Weekend customers predict high price games 3 Then compare for each customer (Choose the one with the highest value)

slide-24
SLIDE 24

@mgershoff

  • Deep Learning Nets
  • Decision Trees
  • Gaussian Process (is a Bayesian method)
  • Support Vector Machines
  • KNN - actually kinda like segmentation
  • Naive Bayes (is NOT a Bayesian method)
  • Logistic Regression

Predictive Analytics Methods

http://conductrics.com/data-science-resources/ http://conductrics.com/data-science-resources-2

slide-25
SLIDE 25

We are going to use Linear Regression

@mgershoff

slide-26
SLIDE 26

twitter: @mgershoff

Why Linear Regression?

Benefits: 1.Has nice Statistical Properties 2.Easy(ish) to interpret 3.In practice, often all you need

slide-27
SLIDE 27

twitter: @mgershoff

What is Linear Regression

A model of relationships in this form:

slide-28
SLIDE 28

twitter: @mgershoff

What is Linear Regression

A model of relationships in this form: Prediction = Base + B1*Attribute1 … + Bj*Attributej

slide-29
SLIDE 29

twitter: @mgershoff

What is Linear Regression

A model of relationships in this form: Prediction = Base + B1*Attribute1 … + Bj*Attributej Just Add up all of the customer ‘attributes’ by the impact (B) of the Feature

slide-30
SLIDE 30

twitter: @mgershoff

What is Linear Regression We will learn two models, one for each game:

slide-31
SLIDE 31

twitter: @mgershoff

What is Linear Regression We will learn two models, one for each game: Game High = BaseH + WH*Weekend + RH*Return

slide-32
SLIDE 32

twitter: @mgershoff

What is Linear Regression We will learn two models, one for each game: Game High = BaseH + WH*Weekend + RH*Return Game Low = BaseL + WL*Weekend + RL*Return

slide-33
SLIDE 33

Linear Regression + Sequential Learning

@mgershoff

slide-34
SLIDE 34

twitter: @mgershoff

Benefits of Sequential Learning 1.Don’t have to wait to collect the data

slide-35
SLIDE 35

twitter: @mgershoff

Benefits of Sequential Learning 1.Don’t have to wait to collect the data 2.Constantly updating you can use it real time

slide-36
SLIDE 36

twitter: @mgershoff

Benefits of Sequential Learning 1.Don’t have to wait to collect the data 2.Constantly updating you can use it real time 3.Scalable–any real production PA is almost certainly going to use the method

slide-37
SLIDE 37

twitter: @mgershoff

Benefits of Sequential Learning 1.Don’t have to wait to collect the data 2.Constantly updating you can use it real time 3.Scalable–any real production PA is almost certainly going to use the method 4.The computations are simple to understand

slide-38
SLIDE 38

twitter: @mgershoff

slide-39
SLIDE 39

twitter: @mgershoff

The Sequential Algorithm in words

1) Observe the data for a single customer 2) Using the current parameter values to make a prediction 3) See how far off your predicted value was from the actual value 4) Use how far off you prediction was to update your parameter values 5) Adjust how much you update by something like O(1/n) – sort of like an average 6) Repeat

slide-40
SLIDE 40

twitter: @mgershoff

ParameterNew:= Parameterold- Adjustment The Sequential Algorithm

Current Value The Difference (Error) of the actual value and the predicted result

Adjustment = (Predicted - Actual) * 1/sqrt(n)

slide-41
SLIDE 41

twitter: @mgershoff

How it is done: No data yet, high cost game

Hidden

Base R W Return WkEnd Sales Predict Error

What We Know

slide-42
SLIDE 42

twitter: @mgershoff

Hidden

Base R W Return WkEnd Sales Predict Error

1 1.00 What We Know

Observe New Customer on Weekend

slide-43
SLIDE 43

twitter: @mgershoff

Hidden

Base R W Return WkEnd Sales Predict Error

1 1.00 What We Know

Observe New Customer on Weekend Prediction= BaseH + WH*Weekend + RH*Return

slide-44
SLIDE 44

twitter: @mgershoff

Hidden

Base R W Return WkEnd Sales Predict Error

1 1.00

  • 1.00

What We Know

Plug in values

0 = 0 + 0*0 + 0*1

slide-45
SLIDE 45

twitter: @mgershoff

Hidden

Base R W Return WkEnd Sales Predict Error

1 1.00

  • 1.00

What We Know Updated

Base R W

1 1

Update Base and Weekend Impact Score

slide-46
SLIDE 46

twitter: @mgershoff

Hidden

Base R W Return WkEnd Sales Predict Error

1 1 2.00 What We Know

Observe New Customer Weekday Prediction= BaseH + WH*Weekend + RH*Return

slide-47
SLIDE 47

twitter: @mgershoff

Hidden

Base R W Return WkEnd Sales Predict Error

1 1 2.00 1.00

  • 1.00

What We Know

Plug in values

1 = 1 + 0*0 + 1*0

slide-48
SLIDE 48

twitter: @mgershoff

Hidden

Base R W Return WkEnd Sales Predict Error

1 1 2.00 1

  • 1.00

What We Know Updated

Base R W

1.5 1

Update Just the Base Impact Score

slide-49
SLIDE 49

twitter: @mgershoff

Hidden

Base R W Return WkEnd Sales Predict Error

1.5 1 1 3.00 What We Know

Observe Return Customer on Weekday Prediction= BaseH + WH*Weekend + RH*Return

slide-50
SLIDE 50

twitter: @mgershoff

Hidden

Base R W Return WkEnd Sales Predict Error

1.5 1 1 3.00 1.50

  • 1.50

What We Know

Plug in values

3 = 1.5 + 0*1 + 1*0

slide-51
SLIDE 51

twitter: @mgershoff

Hidden

Base R W Return WkEnd Sales Predict Error

1.5 1 1 3.00 1.50

  • 1.50

What We Know Updated

Base R W

1.75 0.75 1

Update the Base and Return Impact Score

slide-52
SLIDE 52

twitter: @mgershoff

Online Regression

After 200 Iterations

Base R W

2.0 1.0

  • 1.0

High Price Model Results Sales = 2.0 + 1.0*Return -1.0*Weekend

slide-53
SLIDE 53

twitter: @mgershoff

Online Results: 200 Iterations

  • 1.50
  • 1.00
  • 0.50

0.00 0.50 1.00 1.50 2.00 2.50 50 100 150

Parameter Value Observations Base Level Returning Parameter Weekend Parameter

slide-54
SLIDE 54

54

matt@conductrics.com; www.conductrics.com

Back to Our Task

Model: High Price Game

High = 2.0 + 1.0*Return -1.0*Weekend

slide-55
SLIDE 55

55

matt@conductrics.com; www.conductrics.com

Back to Our Task

Model Low Price Game

Low = 1.0 + 1.0*Return + 0.5*Weekend

slide-56
SLIDE 56

twitter: @mgershoff

Tabular Targeting Logic

Returning Weekend High Price Low Price Selection

N N 2.0 1.0 High Y N 3.0 2.0 High N Y 1.0 1.5 Low Y Y 2.0 2.5 Low

slide-57
SLIDE 57

twitter: @mgershoff

Targeting Logic as Rule

IF [Weekend] THEN [Low] Else [High]

slide-58
SLIDE 58

twitter: @mgershoff

Targeting Logic as Rule

IF [Weekend] THEN [Low] Else [High]

Expressing the logic as a set of succinct rules is generally a hard problem

slide-59
SLIDE 59

How to evaluate our targeting logic?

www.conductrics.com/blog

slide-60
SLIDE 60

Conductrics twitter: @mgershoff Blog: www.conductrics.com/blog

A/B TEST!!!

How do we measure our Targeting?

slide-61
SLIDE 61

Need a Baseline

www.conductrics.com/blog

slide-62
SLIDE 62

www.conductrics.com/blog

Random Selection as Control

Good:

  • 1. A baseline we can always use
  • 2. Selecting randomly is often good policy

when we don’t have any additional information Bad: Often hard to beat (which is Good) Random ain’t too bad

slide-63
SLIDE 63

Users

50/50

Model Evaluation: AB Style Low

Model

Standard Control Random Play

www.conductrics.com/blog

slide-64
SLIDE 64

Users

50/50

Compare the Results of Each

Low

Model

Standard Control Random Play

www.conductrics.com/blog

slide-65
SLIDE 65

Random

Results: Model ROI/Lift

Target - Random

Lift =

www.conductrics.com/blog

slide-66
SLIDE 66

www.conductrics.com/blog

Beware the Faustian Bargain!

Description Michał Elwiro Andriolli Date 1895 Source http://www.pinakoteka.zascianek.pl/Andriolli/Index.htm
slide-67
SLIDE 67

www.conductrics.com/blog

Customer data needs to be ‘accurate’ AND available at decision time We need to create AND manage experiences and content Need to create AND mange our decision logic Difficult to know what state system is in. Before just

  • ne state, now many.

Targeting = Complexity

slide-68
SLIDE 68

𝑾𝒃𝒎𝒗𝒇 𝑫𝒑𝒏𝒒𝒎𝒇𝒚 𝑻𝒛𝒕𝒖𝒇𝒏 − 𝑾𝒃𝒎𝒗𝒇(𝑻𝒋𝒏𝒒𝒎𝒇) 𝑾𝒃𝒎𝒗𝒇(𝑻𝒋𝒏𝒒𝒎𝒇)

www.conductrics.com/blog

Targeting and the ROI of Complexity

slide-69
SLIDE 69

What about Segmentation?

@mgershoff

slide-70
SLIDE 70 Image Source: http://en.wikipedia.org/wiki/File:Carl_Sagan_-_1980.jpg;

Just 20 Customer Features

1 Million Combinations

@mgershoff power(2,10) assuming all non mutually exclusive

Segmentation Doesn't Scale

slide-71
SLIDE 71

Wake up. We are Done!

Twitter:mgershoff Email:matt.gershoff@conductrics.com