Dynamic Measurement Scheduling for Event Forecasting Using Deep RL - - PowerPoint PPT Presentation

▶
dynamic measurement scheduling for event forecasting
SMART_READER_LITE
LIVE PREVIEW

Dynamic Measurement Scheduling for Event Forecasting Using Deep RL - - PowerPoint PPT Presentation

Oct 02, 2023 98 likes •389 views

Dynamic Measurement Scheduling for Event Forecasting Using Deep RL Chun-Hao Chang Mingjie Mai Anna Goldenberg (Kingsley) University of Toronto, Vector Institute The Hospital of Sickkids Motivation Motivation Outcomes Sepsis Mortality

slide-1
SLIDE 1

Dynamic Measurement Scheduling for Event Forecasting Using Deep RL

Chun-Hao Chang (Kingsley)

University of Toronto, Vector Institute The Hospital of Sickkids

Mingjie Mai Anna Goldenberg

slide-2
SLIDE 2

Motivation

slide-3
SLIDE 3

Motivation

Sepsis Mortality Treatments

Outcomes

slide-4
SLIDE 4

Motivation

Lactate Blood Test O2

Measurements

Sepsis Mortality Treatments

Outcomes

slide-5
SLIDE 5

Uniform policy

time Urine Blood Test O2 Waste lots of measurements

slide-6
SLIDE 6

Dynamic policy

time Urine Blood Test O2 Adaptive to patient’s condition

  • Cost-saving
  • Early detection

Healthy Critical

slide-7
SLIDE 7

Contributions

  • RL framework for cost-sensitive scheduling of

measurements in time-series

  • Scalable to large number of measurements
  • Promising results in a real-world ICU dataset (MIMIC3)
slide-8
SLIDE 8

System Pipeline

Event Probability

slide-9
SLIDE 9

System Pipeline

Event Probability

slide-10
SLIDE 10

Problem of large action space

  • Any combination of D measurements is a valid action

○ 2D possible actions

slide-11
SLIDE 11

Problem of large action space

  • Any combination of D measurements is a valid action

○ 2D possible actions

  • Solutions:

○ Independent Policy ○ Sequential Policy ■ Only D+1 actions

slide-12
SLIDE 12

Policy Illustration

slide-13
SLIDE 13

Policy Illustration

slide-14
SLIDE 14

Policy Illustration

Prob Gain from M2 Measurement cost

slide-15
SLIDE 15

Policy Illustration

slide-16
SLIDE 16

Policy Illustration

Prob Gain from M1 Measurement cost

slide-17
SLIDE 17

Policy Illustration

slide-18
SLIDE 18

Policy Illustration

No Measurement Cost

slide-19
SLIDE 19

Policy Illustration

slide-20
SLIDE 20

Policy Illustration

slide-21
SLIDE 21

Off-Policy Policy Evaluation

slide-22
SLIDE 22

Off-Policy Policy Evaluation

3X ↑

slide-23
SLIDE 23

Off-Policy Policy Evaluation

Cost 38% ↓

slide-24
SLIDE 24

Measurements frequency

Physician’s policy

slide-25
SLIDE 25

Measurements frequency

Hemoglobin Phosphate SBP FiO2

Physician’s policy

slide-26
SLIDE 26

Measurements frequency

Hemoglobin

Physician’s policy RL policy

slide-27
SLIDE 27

Dynamic Measurement Scheduling for Event Forecasting Using Deep RL

  • Code and data preprocessing are released at

https://github.com/zzzace2000/autodiagnosis

  • Poster # 247 @ Pacific Ballroom
  • Wed 06:30 -- 09:00 PM
slide-28
SLIDE 28