Comparing Cohorts of Event Sequences A VISUAL ANALYTICS APPROACH presented by Sana Malik with Fan Du, Catherine Plaisant, and Ben Shneiderman May 26, 2016 — HCIL 33 rd Annual Symposium, College Park
often, analysts compare cohorts within datasets
any groups of users, patients, or records often, analysts compare cohorts within datasets
?
FREQUENT PATTERNS
ABSENCE OF EVENTS
DURATION
Data Collection Statistics Cohort Selection
Data Collection Visual Analytics Statistics Cohort Selection
Data Collection Visual Analytics Statistics Cohort Selection
EVENTFLOW Monroe et al. Temporal event sequence simpli fi cation . IEEE Transactions on Visualization and Computer Graphics (TVCG 2013).
EVENTFLOW Monroe et al. Temporal event sequence simpli fi cation . IEEE Transactions on Visualization and Computer Graphics (TVCG 2013).
EVENTFLOW ? Monroe et al. Temporal event sequence simpli fi cation . IEEE Transactions on Visualization and Computer Graphics (TVCG 2013).
Data Collection Visual Analytics Statistics Cohort Selection
Aspirin -> Emergency -> Normal Floor Bed -> ICU -> Exit 38.37, 0.0, 4.11e-123 Emergency -> ICU -> Exit 24.61, 0.0, 2.11e-73 Emergency -> Normal Floor Bed -> Exit -> ICU 5.26, 0.0, 4.12e-15 Emergency -> Normal Floor Bed -> ICU -> Exit 5.26, 0.0, 4.12e-15 Aspirin -> Emergency -> ICU -> Normal Floor Bed -> Exit 7.89, 3.22, 1.20e-06 Aspirin -> Emergency -> ICU -> Intermediate Care -> Exit 4.15, 0.0, 4.22e-12 Emergency -> Normal Floor Bed -> ICU -> Intermediate Care -> Exit 2.97, 0.0, 7.02e-09 Aspirin -> Emergency -> ICU -> Exit 2.80, 0.0, 2.02e-08 Emergency -> Normal Floor Bed -> ICU -> Normal Floor Bed -> Exit 1.44, 0.0, 9.83e-05 Emergency -> Normal Floor Bed -> ICU -> Normal Floor Bed -> ICU -> Normal Floor Bed -> ICU -> Normal Floor Bed -> ICU -> Normal Floor Bed -> ICU -> Normal Floor Bed -> ICU -> Normal Floor Bed -> Exit 1.01, 0.0, 0.00 Aspirin -> Emergency -> Normal Floor Bed -> ICU -> Intermediate Care -> Exit 0.76, 0.0, 0.00 Aspirin -> Emergency -> Normal Floor Bed -> Exit 0.0, 1.61, 3.37e-05 Aspirin -> Emergency -> ICU -> Normal Floor Bed -> ICU -> Exit 0.0, 2.37, 2.84e-07 Aspirin -> Emergency -> Normal Floor Bed -> ICU -> Normal Floor Bed -> Exit 3.82, 6.45, 0.00 Emergency -> ICU -> Normal Floor Bed -> ICU -> Exit 0.0, 4.07, 7.17e-12 Emergency -> ICU -> Normal Floor Bed -> ICU -> Normal Floor Bed -> Exit 0.0, 4.24, 2.48e-12 Emergency -> Exit 0.0, 11.88, 9.00e-34 Emergency -> ICU -> Normal Floor Bed -> Exit 0.0, 16.12, 2.18e-46 Aspirin -> Emergency -> Exit 0.0, 47.79, 2.49e-162 Aspirin -> Emergency -> ICU -> Normal Floor Bed -> ICU -> Normal Floor Bed -> Exit 0.0, 0.59, 0.02 Aspirin -> Emergency -> Normal Floor Bed -> ICU -> Normal Floor Bed -> ICU -> Normal Floor Bed -> ICU -> Normal Floor Bed -> ICU -> Normal Floor Bed -> ICU -> Normal Floor Bed -> ICU -> Normal Floor Bed -> Exit Aspirin -> Emergency -> ICU -> Exit 2.80, 0.0, 2.02e-08 Emergency -> Normal Floor Bed -> ICU -> Normal Floor Bed -> Exit 1.44, 0.0, 9.83e-05 Emergency -> Normal Floor Bed -> ICU -> Normal Floor Bed -> ICU -> Normal Floor Bed -> ICU -> Normal Floor Bed -> ICU ->
5.26, 0.0, 4.12e-15 Emergency -> Normal Floor Bed -> ICU -> Exit 5.26, 0.0, 4.12e-15 Aspirin -> Emergency -> ICU -> Normal Floor Bed -> Exit 7.89, 3.22, 1.20e-06 Aspirin -> Emergency -> ICU -> Intermediate Care -> Exit 4.15, 0.0, 4.22e-12 Emergency -> Normal Floor Bed -> ICU -> Intermediate Care -> Exit 2.97, 0.0, 7.02e-09 Aspirin -> Emergency -> ICU -> Exit 2.80, 0.0, 2.02e-08 Emergency -> Normal Floor Bed -> ICU -> Normal Floor Bed -> Exit 1.44, 0.0, 9.83e-05 Emergency -> Normal Floor Bed -> ICU -> Normal Floor Bed -> ICU -> Normal Floor Bed -> ICU -> Normal Floor Bed -> ICU -> Normal Floor Bed -> ICU -> Normal Floor Bed -> ICU -> Normal Floor Bed -> Exit 1.01, 0.0, 0.00 Aspirin -> Emergency -> Normal Floor Bed -> ICU -> Intermediate Care -> Exit 0.76, 0.0, 0.00 Aspirin -> Emergency -> Normal Floor Bed -> Exit 0.0, 1.61, 3.37e-05 Aspirin -> Emergency -> ICU -> Normal Floor Bed -> ICU -> Exit 0.0, 2.37, 2.84e-07 Aspirin -> Emergency -> Normal Floor Bed -> ICU -> Normal Floor Bed -> Exit 3.82, 6.45, 0.00 Emergency -> ICU -> Normal Floor Bed -> ICU -> Exit 0.0, 4.07, 7.17e-12 Emergency -> ICU -> Normal Floor Bed -> ICU -> Normal Floor Bed -> Exit 0.0, 4.24, 2.48e-12 Emergency -> Exit 0.0, 11.88, 9.00e-34 Emergency -> ICU -> Normal Floor Bed -> Exit 0.0, 16.12, 2.18e-46 Aspirin -> Emergency -> Exit 0.0, 47.79, 2.49e-162 Aspirin -> Emergency -> ICU -> Normal Floor Bed -> ICU -> Normal Floor Bed -> Exit 0.0, 0.59, 0.02 Aspirin -> Emergency -> Normal Floor Bed -> ICU -> Normal Floor Bed -> ICU -> Normal Floor Bed -> ICU -> Normal Floor Bed -> ICU -> Normal Floor Bed -> ICU -> Normal Floor Bed -> ICU -> Normal Floor Bed -> Exit Aspirin -> Emergency -> ICU -> Exit 2.80, 0.0, 2.02e-08 Emergency -> Normal Floor Bed -> ICU -> Normal Floor Bed -> Exit 1.44, 0.0, 9.83e-05 Emergency -> Normal Floor Bed -> ICU -> Normal Floor Bed -> ICU -> Normal Floor Bed -> ICU -> Normal Floor Bed -> ICU -> Normal Floor Bed -> ICU -> Normal Floor Bed -> ICU -> Normal Floor Bed -> Exit 1.01, 0.0, 0.00 Aspirin -> Emergency -> Normal Floor Bed -> ICU -> Intermediate Care -> Exit 0.76, 0.0, 0.00 Aspirin -> Emergency -> Normal Floor Bed -> Exit 0.0, 1.61, 3.37e-05
SAS STATA SAS Business Analytics Software. Vers. 9.4. SAS StataCorp. 2015. Stata Statistical Software: Institute, 2014. Computer software. Release 14. College Station, TX: StataCorp LP.
Data Collection Visual Analytics Statistics Cohort Selection
Visual Analytics Data Collection Cohort Selection Statistics
HIGH-VOLUME Hypothesis Testing
HIGH-VOLUME Hypothesis Testing Systematic Exploration OF RESULTS
HIGH-VOLUME Hypothesis Testing Systematic Exploration OF RESULTS REAL-WORLD Case Study
HIGH-VOLUME Hypothesis Testing
Emergency Room
Normal Floor Bed
ICU
Discharged
start and end of record
non-consecutive (contains other events between)
1 SHORT SEQUENCE 14 UNIQUE PATTERNS non-consecutive (contains other events between)
RECORD COVERAGE Does this sequence occur in more records in one cohort than the other? DURATION On average, does this sequence take longer in one cohort than the other? FREQUENCY On average, does this sequence occur more frequently per record in one cohort than the other?
RECORD COVERAGE Does this sequence occur in more records 14 UNIQUE PATTERNS in one cohort than the other? DURATION X 3 METRICS On average, does this sequence take longer in one cohort than the other? 42 HYPOTHESES FREQUENCY On average, does this sequence occur more frequently per user in one cohort than the other?
HIGH-VOLUME Hypothesis Testing Systematic Exploration OF RESULTS REAL-WORLD Case Study
Systematic Exploration OF RESULTS
Demo
HIGH-VOLUME Hypothesis Testing Systematic Exploration OF RESULTS REAL-WORLD Case Study
REAL-WORLD Case Study
MULTI-DIMENSIONAL IN-DEPTH LONG-TERM CASE STUDIES (MILCS) Entry Interview & Training (1 session) Partners Use Tool Partners Provide Feedback (3 months) Researchers Re fi ne Tool Exit Interview (1 session) For Partners For Researchers Demonstrate utility, re fi ne tool Papers, insights, discoveries B. Shneiderman and C. Plaisant. Strategies for evaluating information visualization tools: Multi- dimensional in-depth long-term case studies. In BELIV ’06: Proceedings of the 2006 AVI workshop on BEyond time and errors, pages 1–7. ACM, 2006.
CASE STUDY PARTNERS
CASE STUDY PARTNERS
PARTICIPANTS & DATASET Three analysts at Adobe • One experienced user • Two novice users Users’ events on a product website viewing the display ads • signing up for promotions or free trials • purchasing products • Dataset Size • 6,999 users • 124 events types / 81,563 events
GOAL Compare users who purchased a product with using trials versus without using trials to understand ad-related behaviors
SYSTEM USE
SYSTEM USE
SYSTEM USE “Event filtering was the most helpful to focus the analysis”
SYSTEM USE “Reduced metric calculation time provided a much better user experience for data analysis”
Recommend
More recommend
