Deconstructing Visualizations Ma Maneesh Agrawala CS 448B: - PDF document

Deconstructing Visualizations Ma Maneesh Agrawala CS 448B: Visualization Winter 2020 1 Announcements 2 1

Final project New visualization research or data analysis project I Research : Pose problem, Implement creative solution I Data analysis : Analyze dataset in depth & make a visual explainer Deliverables I Research : Implementation of solution I Data analysis/explainer : Article with multiple interactive visualizations I 6-8 page paper Schedule I Project proposal: Wed 2/19 I Design review and feedback: 3/9 and 3/11 I Final presentation: 3/16 (7-9pm) Location: TBD I Final code and writeup: 3/18 11:59pm Grading I Groups of up to 3 people, graded individually I Clearly report responsibilities of each member 3 Design Feedback (Week 10) Signup for a 10 min slot https://docs.google.com/spreadsheets/d/1BtXmbQHrC3-chPT6kKS51Q-2p9XhbiM3Qct0N847yPM/edit?usp=sharing I M 3/9 4-6pm I T 3/10 7-8pm (SCPD only) I W 3/11 4-6pm Plan to give a 5 min presentation (mostly demo) of work so far. We will give oral feedback. 4 2

Final Presentation M Mar 16 7-10pm, Location TBD I Short presentation (5 min, mostly demo) I Make sure there is time for questions 5 Deconstructing Visualizations 6 3

7 Pixels are poor representation Hard for machines to retrieve data 8 4

9 10 5

Pixels are poor representation Hard for machines to retrieve data Hard for people to manipulate 11 Pixels are a poor representation of charts and graphs Cannot index, search, manipulate or interact with the data Goal: Re Reconstruct higher-le level l representation of charts and graphs that lets machines and people re redesign, re reuse and revitalize them re 12 6

What is a good representation? 13 Year Exports Imports mark: lines Mappings 1700 170,000 300,000 Marks Year à x-pos (Q) Data 1701 171,000 302,000 Exports à y-pos (Q) 1702 176,000 303,000 Imports à y-pos (Q) 1703 180,000 312,000 Exports à color (N) 1704 187,000 319,000 Imports à color (N) … … … 14 7

Disease Budget Mappings mark: areas Aids 70.0% Marks Budget à angle (Q) Alzheimer’s 5.0% Data Cardiovascular 1.1% Disease à color (N) Diabetes 4.8% Hepatitus B 4.1% Hepatitus C 3.8% Parkinson’ 6.0% Prostate 5.2% 15 Disease Budget Mappings mark: lines Aids 70.0% Marks Budget à length (Q) Alzheimer’s 5.0% Data Cardiovascular 1.1% Disease à color (N) Diabetes 4.8% Hepatitus B 4.1% Hepatitus C 3.8% Parkinson’ 6.0% Prostate 5.2% 16 8

Approach Classification: Determine chart type Mark extraction: Retrieve graphical marks Data extraction: Retrieve underlying data table 17 17 Classification 18 9

Training the Classifier 19 Training the Classifier 20 10

Training the Classifier Bar Charts Pie Charts Scatter Plots 21 Classifying an Input Image 22 11

Classifying an Input Image 23 Classifying an Input Image 24 12

Classifying an Input Image 25 Classifying an Input Image 26 13

Classifying an Input Image 27 Classifying an Input Image SVM Pie Chart Classifier Corpus: 667 charts, 5 chart types [ Prasad 2007] Average Accuracy [Prasad 2007] Multi-class SVM 84% ReVision: Multi-class SVM 88% ReVision: Binary SVM (yes/no for each chart type) 96% 28 14

Our Corpus Over 2500 labeled images and 10 chart types ReVision binary SVMs give 96% classification accuracy http://vis.berkeley.edu/papers/revision 29 Mark and Data Extraction 30 15

Assumptions Bar charts and pie charts only No shading or texture, 3D, stacked bars, or exploded pies 31 Bar Charts marks: lines y-value x-value 50 A 25 B 4 C 75 D 32 16

Bar Charts marks: lines y-value x-value 50 A 35 B 4 C 75 D Extract Marks Extract Data Identify Find Associate Recover Bar Orientation Foreground Labels with Values Rectangles and Baseline Bars Scale: 2 pixels/unit 43 Pie Charts marks: areas percentage category 22.3 A 22.4 B 10.8 C 5.6 D 5.6 E 33.3 F Extract Marks Extract Data Unroll Pie Fit Ellipse Compute Associate and Find Area Labels with Using Transitions RNASAC Percentages Areas Scale: 50 pixels/percent 44 17

Extraction Results 60 53 52 50 41 Number of Charts 40 33 29 30 21 20 79% 62% 40% 56% 10 0 Bar Pie Total charts Mark extractions Data extractions 45 Data Extraction Error Bar Charts Pie Charts 4.6% 7.7% Average chart size: 342 x 452 pixels [Prasad 2007] 46 18

Redesign 47 Original Redesign 48 19

Original Redesign 49 Redesign #1 Original 50 20

Redesign #1 Original Redesign #2 51 Limitations Additional Chart Types Handling Legends 53 21

Graphical Overlays Visual elements that are layered onto a chart to facilitate the perceptual and cognitive processes involved in chart reading 54 Taxonomy 55 22

Demo 56 Reference Structures Help by breaking marks into regular segments and aid reading axis values 57 23

Highlights Draws viewers’ attention to specific marks 58 Redundant Encodings Emphasize data values or trends 59 24

Summary Statistics Enables comparison with statistics based on the data 60 Annotation Provide context and support collaboration 61 25

mark: lines year money 2000 85 2001 78 2002 87 2003 90 2004 98 … … Most overlays only require access to marks Reference structures ( marks ) Highlights ( marks ) Redundant encodings ( marks and data ) Summary statistics ( marks ) Annotations ( marks ) 62 Interactive Documents How can we facilitate reading text and charts together? 63 26

Goal: Extract references between text and chart 64 Problem: Diversity of writing styles 65 27

Example 1: Pew Research Skepticism for capitalism is lowest in Brazil (22%), China (19%), Germany (29%) (although East Germans are less supportive than West Germans) and the U.S. (24%). Skepticism for free markets is highest in Mexico (60%) and Japan (60%). 66 Example 1: Pew Research Skepticism for capitalism is lowest in Brazil (22%), China (19%), Germany (29%) (although East Germans are less supportive than West Germans) and the U.S. (24%) . Skepticism for free markets is highest in Mexico (60%) and Japan (60%). 67 28

Example 2: Economist Top earners have attracted more opprobrium as their salaries and the performance of the economy have headed in opposite directions. Europeans and Latin Americans tend to have similar attitudes to the rich; the Anglo-Saxon world is a bit more forgiving. 68 Example 2: Economist Top earners have attracted more opprobrium as their salaries and the performance of the economy have headed in opposite directions. Europeans and Latin Americans tend to have similar attitudes to the rich; the Anglo-Saxon world is a bit more forgiving. 69 29

Preprocessing Crowdsourcing Clustering and Merging Document Reference Merge segmentation extraction Mark and data Split extraction Cluster Select representative 70 Demo 71 30

Evaluation 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 All workers Passed Clustered gold and merged Avg. F 1 distance: expert specified references vs. crowd specified references 72 Deconstructing D3 Charts Data deconID name type cost 2 apple fruit 1.00 3 pear fruit 2.00 4 beef meat 5.00 Marks fill xPosition height green 35 px 20 px green 60 px 40 px red 85 px 100 px Mappings C type fi ll L height cost L yPos cost L cost area L deconID xPos D3 Code D3 Chart Our Deconstruction Automatically convert D3 code into mapping based representation to enable redesign and style reuse Deconstructing and Restyling D3 Visualizations. Jonathan Harper and Maneesh Agrawala. User Interface Software Technology (UIST) 2014. 74 31

Deconstructing D3 Charts country rate deconID Namibia 37.6 17 Macedonia, FYR 32.0 21 Armenia 28.6 25 Bosnia and Herzegovina 27.2 29 Lesotho 25.3 33 South Africa 24.7 37 Spain 20.1 41 Latvia 18.7 45 … … … Deconstructing and Restyling D3 Visualizations. Jonathan Harper and Maneesh Agrawala. User Interface Software Technology (UIST) 2014. 75 Deconstructing D3 Charts country rate deconID Namibia 37.6 17 Macedonia, FYR 32.0 21 Armenia 28.6 25 Bosnia and Herzegovina 27.2 29 Lesotho 25.3 33 South Africa 24.7 37 Spain 20.1 41 Latvia 18.7 45 … … … Deconstructing and Restyling D3 Visualizations. Jonathan Harper and Maneesh Agrawala. User Interface Software Technology (UIST) 2014. 76 32

Automatic Redesign Can we automatically redesign charts to improve Perceptual effectiveness? Visual aesthetics? Accessibility for vision impaired users? Data Source Style Target Result 77 Data Source Style Target Converting Basic D3 Charts into Reusable Style Templates. Jonathan Harper and Maneesh Agrawala. IEEE TVCG. 2018. 78 33

Reusable Style Templates Converting Basic D3 Charts into Reusable Style Templates. Jonathan Harper and Maneesh Agrawala. IEEE TVCG. 2018. 79 Document Collections Many specialized collections Scientific: PLOS, JSTOR, ACM DL, … Web visualizations: D3, Processing, … News: New York Times, Pew research, … How can deconstruction aid search? Search by chart type, data type, marks, data, … Similarity search with inexact matching Query expansion 80 34