Mina Kwon - 2019.10.15 - Randomized Clinical Trial ? Randomized - - PowerPoint PPT Presentation

Mina Kwon

• 2019.10.15 -

Randomized Clinical Trial ?

Randomized Clinical Trial

https://www.statisticshowto.datasciencecentral.com/experimental-design/randomized-clinical-trial-rcts/

• Purpose: assess treatment’s efficacy or effectiveness
• “randomized”: random assignment to each study group


— implies balance in baseline characteristics between groups
 —allows eliminating many pre-analytical differences that might bias the entire study

Randomized Clinical Trial

Francesco, Stephane, Fabrizio, Pier (2017) 
 How to design a randomized clinical trial: tips and tricks for conduct a successful study in thoracic disease domain, Journal of Thoracic Disease;9(8)

Francesco et al (2017)

• Select intervention that could be correctly assessed
• Choose and report randomization methods correctly
• Balance groups using stratification technique
• Blinding
• Intention to treat analysis (ITT)

Randomized Clinical Trial

https://www.statisticshowto.datasciencecentral.com/experimental-design/randomized-clinical-trial-rcts/

Introduction

Introduction

• Physical inactivity, a major risk factor for disability and

death

• Challenge in motivating individuals to increase physical

activity

• Financial rewards, a useful tool for encouraging healthy

behavior!

Introduction

• “Financial incentives for physical activity in adults:

systematic review and meta-analysis” (Mitchell, Orstad, Biswas et al; 2019) Key principles of effective incentive design to promote physical activity

1) immediate incentives
 2) realistic daily goals
 3) longer interventions (>24 weeks) 
 4) frequent and personalized feedback
 + Size of incentives does not have to be large

Introduction

• HOWEVER, much still remains unknown
• 1. HOW to distribute the small incentives over time, for

maximal effect?

• 2. HOW to create behavior change that is sustained after

incentives are inevitably removed?

Introduction

Simple

CONSTANT incentive

Prevent tolerance

INCREASING incentive DECREASING incentive

Diminish reliance on financial reward 
 & Motivate initiation

Primary objective

Comparing 3 different 
 2-week incentive programs 
 with rewards for daily steps taken 
 to determine which was the most effective for increasing the number of steps both during and after the intervention

CONSTANT INCREASING DECREASING

METHOD

Study Design

• Experiment with online platform (“Achievement”) recording

daily steps of users through pedometers

Study Design

• Experiment with online platform (“Achievement”) recording

daily steps of users through pedometers

• Intervention: offering 20-fold incentives for 2 weeks.
• Intervention

2 weeks 3 weeks 3 weeks

Pre-intervention Follow-up

Study Design

CONSTANT INCREASING DECREASING CONTROL

10,000 steps = $0.01 10,000 steps = $2 10,000 steps = $0.05 $3.5 10,000 steps = $ 3.5 $0.05

Participants

CONTROL CONSTANT INCREASING DECREASING

Statistical Analysis

CONTROL CONSTANT INCREASING DECREASING

VS

• 1. Test the effect of 20-fold incentive increase on walking

behavior

• 2. Test the effect of incentive structure

CONSTANT INCREASING DECREASING

VS VS

Statistical Analysis

• Handling Missing data

Missing data: any day with fewer than 2000 recorded steps.

1) Intent-to-treat strategy (ITT) 
 : replace missing data with a mean of a given participant’s preintervention daily step counts greater than 2000 steps. 2) Sensitivity Analysis 
 : deleted all daily step data recording fewer than 2000 steps.

Statistical Analysis

• Ordinary least squares regression to determine the overall

and separate effects of 3 treatment groups on participants daily steps.

• Wald test to assess differences between treatment

conditions

• Cost-effectiveness analysis of additional steps taken per

$1 paid to each treatment condition participant relative to control participant

Statistical Analysis

• Ordinary least squares regression to determine the overall

and separate effects of 3 treatment groups on participants daily steps.

• Wald test to assess differences between treatment

conditions

• Cost-effectiveness analysis of additional steps taken per

$1 paid to each treatment condition participant relative to control participant

RESULTS

• 1. Effect of 20-fold increase in incentive
• Intervention period

• Postintervention period: No significant difference 


CONSTANT INCREASING DECREASING CONTROL 135.0 additional daily steps (95% CI, 41.0 - 228.9 steps)

>

• During the Intervention
• 2. Effect of Incentive Structure

6900 7000 7100 7200 7300

d P < .01. e P<.05. f P<.10.

control constant increasing decreasing

306.7 additional daily steps 
 (95% CI, 91.5 - 521.9 steps)
 p = .005 305.1 additional daily steps 
 (95% CI, 89 - 512.2 steps)
 p = .006

>

209.8 additional daily steps 
 (95% CI, -5.7 - 425.3 steps) 
 p = .06

INCREASING DECREASING CONSTANT CONTROL

96.9 additional daily steps 
 (95% CI, 15.3 - 178.5 steps) 
 p = .02 95.3 additional daily steps 
 (95% CI, 11.3 - 179.3 steps) 
 p = .03

>

1.5 additional daily steps 
 (95% CI, -81.6 - 84.7 steps)
 p = .97

INCREASING DECREASING CONTROL

>

DECREASING INCREASING CONTROL

~ ~

• 2. Effect of Incentive Structure
• After the Intervention

6900 7025 7150 7275 7400 1st week

329.5 additional daily steps 
 (95% CI, 20.6 - 638.4 steps) p = .04 397.8 additional daily steps 
 (95% CI, 89.2 - 706.4 steps) p = .01

>

308.6 additional daily steps 
 (95% CI, 0.1 - 617.1 steps) p = .05

INCREASING DECREASING CONSTANT CONTROL

Week 1 Week 2 Week 3

6900 7025 7150 7275 7400 2nd week

136.3 additional daily steps 
 (95% CI, 30.3 - 242.3 steps) p = .01

>

INCREASING DECREASING

6900 7025 7150 7275 7400 3rd week

315.2 additional daily steps 
 (95% CI, 6 - 624.4 steps) p = .046

>

INCREASING CONSTANT

• 3. Cost-effectiveness
• For additional $1 paid,

CONSTANT INCREASING DECREASING

+ 582.4 steps + 107 steps + 153.1 steps

CONSTANT INCREASING DECREASING

$15.48 $14.54 $14.67

• Amount of money earned

DISCUSSION

Summary

• incentives significantly increased physical

activity relative to all other conditions.


• held for 1 week after the intervention & dissipated after then.
• group performed equally as well as those

receiving a 20-fold greater incentive

CONSTANT

Not only incentive size, but also incentive structure matters!

CONTROL

Summary

: easier to remember —> more salient and effective at promoting physical activity

: confusing or unfair —> related ineffectiveness

WHY constant incentive?

CONSTANT Increasing Decreasing

Limitations

• Dependent on participants’ device-wearing behaviors
• Other physiological metrics might be more relevant to long-

term health outcomes

• Lack of demographic data for the population
• Short intervention period, small incentive size compared

with prior experiments.

• Unanswered question: how to promote long-term changes?

What else?

Could prospect theory, loss aversion, and regret used to design an effective weight loss intervention?

Study Design

• Main outcome: Weight loss after 16 weeks
• Participants: 57 healthy participants with BMI 30-40
• Compared 2 different incentive groups & control group

Intervention

4 months 3 months

Follow-up

Study Design

Weigh-in control group Deposit contract group Lottery incentive group

Goal: -16lb over 16 weeks

• opportunity to contribute $0.01 ~

$3.00 for each day (beginning of the month)

• refundable at the end of the month if

they reach the goal

• incentive: if deposit, +$3 day
• could earn $0 ~ $252
• Daily feedback “how much money

they earned that day

• eligible for a daily lottery prize with

an expected value of $3/day 
 ( 0.01% $100 / 0.05% 10$)

• Receive money at the end of the

month if they reach the goal

• Daily feedback about each day payoff

• No incentives.
• received incentives for monthly

weigh-in regardless of weight loss 


Results

3.5 7 10.5 14 Weight loss

13.1lb 14lb

3.9lb 2.5 5 7.5 10 Month 7

9.2lb 6.2lb 4.4lb

15 30 45 60 Goal achieved

52.6% 47% 10.5%

100 200 300 400 Money earned

$272.8 $378.5 $0.0

control Deposit Lottery control Deposit Lottery

Discussions

• Lost to follow-up rates were much lower than typical

weight-loss studies

• Weight loss improved immediate outcomes such as blood

pressure, glycerin control, serum lipid levels etc

• Still, substantial amounts of weight were regained.

Discussions

• Even small reward and

punishments can have great incentive value if they occur immediately

• Motivated by experience of past

rewards and the prospect of future reward, and emotionally attracted to small probabilities of large rewards

• Desire to avoid regret
• Rapid feedback about whether they

won money/ non-adherent participants: about whether they would have earned money

• Lottery provided frequent small

payoffs and infrequent large payoffs.


• Feedback about what they would

have won had they been adherent

• Enrollment of Self-selected patients
• Active Control conditions
• Timing of enrollment
• Imperfect Execution
• Peanut effects
• Immediate rewards & Incentive framed as ‘loss’ rather

than ‘gain’ are more effective

“Thank you”

• 2. Effect Incentive Structure
• After the Intervention

Study Design

Weigh-in control group Deposit contract group Lottery incentive group

Goal: -16lb over 16 weeks

• opportunity to contribute $0.01 ~

$3.00 for each day (beginning of the month)

• refundable at the end of the month if

they reach the goal

• incentive: if deposit, +$3 day
• could earn $0 ~ $252
• Daily feedback “how much money

they earned that day

• eligible for a daily lottery prize with

an expected value of $3/day 
 ( 0.01% $100 / 0.05% 10$)

• Receive money at the end of the

month if they reach the goal

• Daily feedback about each day payoff

• No incentives.
• received incentives for monthly

weigh-in regardless of weight loss 


Loss aversion regret

variable reinforcement

regret