Vi Visual ualizing ng Data a for Anal Analysis and and Communi - - PowerPoint PPT Presentation
Machine Learning for Precision Public Health: Vi Visual ualizing ng Data a for Anal Analysis and and Communi unication Anamaria Crisan Vanier Canada Scholar & UBC Public Scholar PhD Candidate, Computer Science University of British
@amcrisan http://cs.ubc.ca/~acrisan acrisan@cs.ubc.ca
Anamaria Crisan
Vanier Canada Scholar & UBC Public Scholar
PhD Candidate, Computer Science University of British Columbia
Ma Master of Science
( ( Bioinformat atics )
Ph PhD
(C (Computer Science) Ge GenomeDX Bi Biosciences Br British Columbi bia Centre for for Disease Con
2010 2010 2013 2013 2015 2015 2008 2008
Ph PhD Candidate, Comp
Un University of British Columbia
http://bit.ly/1FxtT2z
It is not always easy to reason consistently with numbers
60%
Probability Frequency Visualization
6 in 10
< <
Whiting (2015) “How well do health professionals interpret diagnostic information? A systematic review”
Least Understandable Most Understandable
Do you have a research
Do all the
But eventually you’ll have a problem right?
the public!
https://www.ratbotcomics.com/comics/pgrc_2014/1/1.html
Do all the
Maybe data
the public!
right? Do you have a research But eventually you’ll have a problem
Do all the
Did it work? Maybe data
the public!
right? Do you have a research But eventually you’ll have a problem
Do all the
Did it work? Maybe data
the public!
right? Do you have a research But eventually you’ll have a problem
Do all the
the public!
Did it work? Maybe data
right? Do you have a research But eventually you’ll have a problem
Do all the
the public!
Missed Opportunity for Exploration
§ Exploration is looking at your data, trying different analysis methods, assessing if there are outliers or missing data etc.
Autodesk Research (2017). Same Stats, Different Graphs: https://www.autodeskresearch.com/publications/samestats
Same stats, different graphs
Autodesk Research (2017). Same Stats, Different Graphs: https://www.autodeskresearch.com/publications/samestats
Same stats, different graphs (Datasaurus)
Opening up the machine learning black box
Chi Chihua huahua hua or muf uffin? n? Mo Mop or sheep dog?
Goodfellow (2014). “Explaining and Harnessing Adversarial Examples”
Olah (2018). “Building blocks of interpretability” (https://distill.pub/2018/building-blocks/)
Ma Made with : JavaScript
Selecting the appropriate data visualization is challenging
§ True for exploration & communication applications
Baseline Visualization Alternative 1 Alternative 2
Zikmund-Fisher (2013). A demonstration of ''less can be more'' in risk graphics.
Borkin (2011). “Evaluation of Artery Visualizations for Heart Disease Diagnosis”
Ma Made with : Processing
EX EXISTI TING STANDARD RD Ac Accuracy : 39% 39% REV REVISED ED VISUALIZATI TION Ac Accuracy: 91% 91%
Borkin (2011). “Evaluation of Artery Visualizations for Heart Disease Diagnosis”
Ma Made with : Processing
Ho How w do you ma make a a visual alizat ation? What datavis tools are available? Is Is it the appr appropr priat ate vi visualization?
How should we visualize data?
Human Perception & Cognition Computer Graphics Data Analysis
Visualization Design & Analysis
§ Non-trivial to condense knowledge across all these areas § Still an ongoing area of research § I will try convey a simpler intuition about design & analysis
Image Source: Valentin Antonucci via Pexels
Why do you need to visualize data? How will you, or others, use the visualization?
34 34
Why do you need to visualize data? How will you, or others, use the visualization?
What kind of data is being visualized? What tasks are performed with the data?
35 35
People tend to jump to this level and ignore why and what
What kind of data is being visualized? What tasks are performed with the data?
How do you make the visualization? Is it the right visualization?
Why do you need to visualize data? How will you, or others, use the visualization?
36 36
Does the visualization address the the intended need? Are you using the right data, or deriving the right data? Are the visual & interactive choices appropriate for the data and tasks? Does the visualization support the tasks using that data? If interactive / computer based, is the visualization easy to use and reliable (i.e doesn’t crash all the time)
37 37
Image Source: Valentin Antonucci via Pexels
Do Domain Pr Problem* Da Data + + Task Vi Visual + Interaction De Design Ch Choices Al Algorithm
Infovis (Information Visualization) research advocates an ite itera rativ tive process
*Domain Problem = Motivation
An iterative approach to development allows us to get feedback before committing to ineffective design choices
probl blem that effects you or a group
Do Domain Pr Problem Da Data + + Task Vi Visual + Interaction De Design Ch Choices Al Algorithm
Nu Nurses Cl Clinicians
Me Medical He Health Of Officers
Re Researchers Co Community Le Lead aders
§ Mu Multidisciplinary decision making teams
§ More data & diverse data types = more informed decision making § BUT – different stakeholder abilities to interpret data & different needs
Pol Policy Makers Pa Patients
Do Domain Pr Problem Da Data + + Task Vi Visual + Interaction De Design Ch Choices Al Algorithm
Original (Raw) Data Derived Data Example Example when this advice is ignored
XKCD
Source : Atlanta CDC
Geographic Overview of Prostate Cancer
§ Useful for epidemiologists and policy makers § Supports surveillance tasks
Individual Prostate Cancer Risk
§ Good for patients and doctors § Supports treatment decision making tasks
Source : http://riskcalc.org/PCPTRC/ (UT San Antonio)
Standard Map Cartogram
Standard Map Snakey Diagram
Di Discovery De Design Im Implement
Information Gathering Design & Evaluation Finalize Design
Expert Consults Task & Data Questionnaire Design Sprint Design Choice Questionnaire TB Workflow Map
Data Gathered Qualitative Quantitative Study Design
Exploratory Sequential Model Embedded Model
MYCOBACTERIUM TUBERCULOSIS GENOME SEQUENCING REPORT NOT FOR DIAGNOSTIC USE Paent Name JOHN DOE Barcode Birth Date 2000-01-01 Paent ID 12345678910 Locaon SOMEPLACE Sample Type SPUTUM Sample Source PULMONARY Sample Date 2016-12-25 Sample ID A12345678 Sequenced From MGIT CULTURED ISOLATE Reporng Lab LAB NAME Report Date/Time 2017-01-01, 15:36 Requested By REQUESTER NAME Requester Contact REQUESTER@EMAIL.COM Summary The specimen was posive for Mycobacterium tuberculosis. It is resistant to isoniaizd and ri- WGS equivalent DIAGNOSIS TASKS TREATMENT TASKS SURVEILLANCE TASKS TOTAL SCORE Diagnose Latent TB Diagnose Active TB Reactive vs New Infection Characterize Transmission Risk Choose Meds Choose Tx Duration Assess Response to Tx Guide Contact Tracing Report to Public Health Define a Cluster Connect Case to Existing Cluster Guide Public Health Response Patient Identifier Same 3 3 3 3 3 3 3 2 1 1 1 1 26 Sample Collection Date Same 3 3 2 3 3 3 3 1 1 1 1 1 24 Patient Prior TB Results Same 3 2 3 3 3 3 3 1 1 1 1 23 Speciation Speciation 1 3 2 3 3 3 3 2 1 1 1 1 23 Sample Type (sputum, fine needle aspirate etc.) Same 2 3 2 3 3 3 3 1 1 1 1 22 Culture results NA 1 3 2 3 3 3 3 2 1 1 1 22 Sample Collection Site (lymph node, lung etc..) Same 2 3 2 3 3 3 3 1 1 1 21 Acid Fast Bacilli Smear Speciation 2 3 2 3 2 3 3 1 1 1 1 21 Resistotype Predicted DST 2 3 1 3 3 2 2 1 1 1 1 19 Phenotypic DST Predicted DST 2 3 2 3 3 2 1 1 1 1 18 Chest x-ray NA 3 3 2 3 2 3 1 17 Report Release Date Same 2 2 1 2 2 2 2 1 1 1 15 Requester IDs Same 2 2 2 2 2 2 2 1 15 Interpretation or comments from reviewer Same 2 2 1 2 2 2 3 1 15 Predicted DST Predicted DST 2 2 1 3 3 2 1 1 15 MIRU-VNTR SNPs 2 3 1 1 1 1 1 1 1 1 1 13 Cluster Assignment Same 2 2 1 1 1 1 1 1 1 1 11 SNP/variant distance SNPs 1 2 1 1 1 1 1 1 1 1 10 Phylogenetic Tree Same 2 1 1 1 1 1 1 1 1 9 Reviewer ID Same 1 1 1 1 1 1 1 1 8 TST results Speciation* 3 1 1 1 1 7 IGRA results Speciation* 3 1 1 1 1 7 Lab QC WGS Specific 1 2 1 1 1 1 7 Spoligotype SNPs 1 1 1 3 RFLP SNPs 1 1 1 3
Data
3 (>75%) 2 (50% - 25%) 1 (25% -50%) 0 (<25%)
Consensus among participants
% agree cat.
MYCOBACTERIUM TUBERCULOSIS GENOME SEQUENCING REPORT
NOT FOR DIAGNOSTIC USE Paent Name JOHN DOE Barcode Birth Date 2000-01-01 Paent ID 12345678910 Locaon SOMEPLACE Sample Type SPUTUM Sample Source PULMONARY Sample Date 2016-12-25 Sample ID A12345678 Sequenced From MGIT CULTURED ISOLATE Reporng Lab LAB NAME Report Date/Time 2017-01-01, 15:36 Requested By REQUESTER NAME Requester Contact REQUESTER@EMAIL.COM
Summary
The specimen was posive for Mycobacterium tuberculosis. It is resistant to isoniaizd and ri-
Organism
The specimen was posive for Mycobacterium tuberculosis, lineage 2.2.1 (East-Asian Beijing).
Drug Suscepbility
Resistance is reported when a high-confidence resistance-conferring mutaon is detected. “No mutaon detected” does not exclude the possi- bility of resistance.No drug resistance predicted Mono-resistance predicted
Extensive drug resistance predicted Drug class Interpretaon Drug Resistance Gene (Amino Acid Mutaon) Ethambutol No mutaon detected Suscepble Pyrazinimide No mutaon detected Isoniazid katG (S315T) First Line Resistant Rifampin rpoB (S531L) Streptomycin No mutaon detected Ciprofloxacin No mutaon detected Ofloxacin No mutaon detected Moxifloxacin No mutaon detected Amikacin No mutaon detected Kanamycin No mutaon detected Second Line Suscepble Capreomycin No mutaon detected Page 1 of 2 Paent ID: 12345678910 | Date: 2017-01-01 | Locaon: Someplace
problem and set of tasks & data
interaction!)
Do Domain Pr Problem Da Data + + Task Vi Visual + Interaction De Design Ch Choices Al Algorithm
Mark rk:
Basic Graphical Element (basic building block)
Ch Channel:
Controls the appearance of marks
49 49
Example
Ba Bar Cha Chart
Position Common Scale
Pi Pie Chart
Angle & Area
50 50
Colour Blind Simulator: http://www.color-blindness.com/coblis-color-blindness-simulator/
Original Visualization Visualization as seen by color blind person
(color blindness (deuteranopia) impacts men more often))
Colour scales also impact interpretation!
Perceptual research from Liu et al (2018)
Liu et al. (2018) - Somewhere Over the Rainbow: An Empirical Assessment of Quantitative Colormaps
ggplot (data = mpg, ae aes( x= x= display, y y = ct cty, co colour = cl class)) + ge geom
point( )
Channel: Position Channel: Colour Mark: Point
No Note: Generally in ggplot2 aesthetics refer to channels and geoms refer to marks, but there are complex geoms that aren’t simple marks but chart types (i.e. geom_density) and there are aesthetics that have little to do with the visual channels directly (i.e. group)
https://rpubs.com/hadley/ggplot-intro 51 51
51 51
Marks Channels
Chart Chooser
https://bit.ly/2P9zLEW
Data to viz
https://www.data-to-viz.com/
http://gevit.net
Crisan et. al (2018) “A systematic method for surveying data visualizations and a resulting genomic epidemiology visualization typology: GEViT”
OXFORD BIOINFORMATICS
#specify individual charts phyloTree_chart<-specify_base(chart_type = "phylogenetic tree",data="tree_dat") epicurve<-specify_base(chart_type = "histogram",data="tab_dat",x = "month") map_chart<-specify_base("geographic map",data="tab_dat",lat = "latitude",long = "longitude") #specify a combination colour_ combo<-specify_combination(combo_type = "color_linked", base_charts = c("phyloTree_chart","map_chart","epicurve"),link_by="country") #plot the result plot(color_combo)
# Analyze different # data types automatically harmon_obj<-data_harmonization(tab_dat, tree_dat,genomic_dat,all_spatial) # Create specifications # that compile to minCombinr component_specs<-get_spec_list(harmon_obj) #plot the result one view at a time plot_view(component_specs,view_num=1)
Preliminary Result
problem and set of tasks
solution could be a new algorithm)
Do Domain Pr Problem Da Data + + Task Vi Visual + Interaction De Design Ch Choices Al Algorithm
develop) with stakeholders
Do Domain Pr Problem* Da Data + + Task Vi Visual + Interaction De Design Ch Choices Al Algorithm
probl blem that effects you or a group
t data ta stakeholders use (is it available)?
t sta takeholders rs do with the data [ta tasks]
pr probl blem and set of ta tasks & data ta
r own soluti tion (vis and/or algorithm)
tiple alte ternati tives (including new ones you develop) with stakeholders
tati tive & quanti tita tati tive evaluation data
Design Evaluation
Lisa Charlotte Rost has an excellent blog post about this: http://bit.ly/2gRGx1J I am presenting her figures here
Lisa Charlotte Rost has an excellent blog post about this: http://bit.ly/2gRGx1J
Analysis vs Presentation
Lisa Charlotte Rost has an excellent blog post about this: http://bit.ly/2gRGx1J
Extent of Flexibility
How easy/hard it is to make data visualizations (including custom/novel visualizations)
Lisa Charlotte Rost has an excellent blog post about this: http://bit.ly/2gRGx1J
Static vs Interactive
Lisa Charlotte Rost has an excellent blog post about this: http://bit.ly/2gRGx1J “There are no perfect tools, just good tools for people with certain goals”
See a detailed table here: http://bit.ly/2DeWPwV
Another take with commonly used tools : https://bit.ly/2SgrOzS
Dear Data Project (Lupi & Posavec)
§ Visualizati tions of data ta are useful
§ Helpful in instance of low numeracy § Can used in communication and and exploration
§ But. t.. visualizati tion design also matte tters rs
§ Many different alternatives, important to test
§ It’ t’s possible to to th think syste temati tically about t visualizati tions
§ Many disciplines cross cut information visualization research § At the minimum think “Why”, “What”, “How”
§ Encode data ta well so th that t oth thers rs can decode it t late ter § Da Data ta visualizati tion is a re researc rch pro rocess with th open and inte teresti ting problems
§ Bo Books to consider:
§ Interpretable Machine Learning: https://christophm.github.io/interpretable-ml-book/ § Making Data Visual: A Practical Guide to Using Visualization for Insight by Danyel Fisher and Miriah Meyer § Visualization Design and Analysis by Tamara Munzner (more technical )
§ On Online resources:
§ Distill Publication : https://distill.pub/ § UBC Infovis Resource Page : http://www.cs.ubc.ca/group/infovis/resources.shtml § UW Interactive Data Lab : https://medium.com/@uwdata § Data stories podcast : http://datastori.es/
§ In Inspiration :
§ Information is Beautiful : https://informationisbeautiful.net/ § Visualization WTF (examples of what not to do) : http://viz.wtf/
§ I’l I’ll be presenting more on my own research on June 18th
th!
@amcrisan http://cs.ubc.ca/~acrisan acrisan@cs.ubc.ca
Anamaria Crisan
Vanier Canada Scholar & UBC Public Scholar
PhD Candidate, Computer Science University of British Columbia