How to evaluate a visualization: So many methods, how to pick? Nested model: Four levels of visualization design Information Visualization • Computational benchmarks? • domain situation – who are the target users? – quant: system performance, memory Data Abstraction • abstraction • User study in lab setting? domain – translate from specifics of domain to vocabulary of visualization – quant: (human) time and error rates, preferences abstraction • what is shown? data abstraction – qual: behavior/strategy observations Nested Model • why is the user looking at it? task abstraction • Field study of deployed system? idiom Tamara Munzner – often must transform data, guided by task – quant: usage logs algorithm Department of Computer Science • idiom – qual: interviews with users, case studies, observations University of British Columbia – how is it shown? • Analysis of results? • visual encoding idiom: how to draw Lect 2, 9 Jan 2020 – quant: metrics computed on result images • interaction idiom: how to manipulate [A Nested Model of Visualization Design and Validation. – qual: consider what structure is visible in result images Munzner. IEEE TVCG 15(6):921-928, 2009 • algorithm (Proc. InfoVis 2009). ] • Justification of choices? http://www.cs.ubc.ca/~tmm/courses/436V-20 [A Multi-Level Typology of Abstract Visualization Tasks – efficient computation Brehmer and Munzner. IEEE TVCG 19(12):2376-2385, 2013 (Proc. InfoVis 2013). ] – qual: perceptual principles, best practices 2 3 4 Different threats to validity at each level Interdisciplinary: need methods from different fields at each level Mismatches: Common problem • cascading effects downstream • mix of qual and quant approaches (typically) Domain situation problem-driven Domain situation qual anthropology/ Observe target users using existing tools Observe target users using existing tools work Domain situation ethnography You misunderstood their needs Data/task abstraction Data/task abstraction What: Data Abstraction Data/task abstraction Visual encoding/interaction idiom Visual encoding/interaction idiom qual You’re showing them the wrong thing design Justify design with respect to alternatives Justify design with respect to alternatives benchmarks can't computer Algorithm quant Algorithm Visual encoding/interaction idiom technique-driven confirm design Measure system time/memory Measure system time/memory science The way you show it doesn’t work work Analyze computational complexity Analyze computational complexity qual Analyze results qualitatively Analyze results qualitatively Algorithm psychology lab studies can't quant Your code is too slow Measure human time with lab experiment ( lab study ) Measure human time with lab experiment ( lab study ) confirm task qual abstraction anthropology/ Observe target users after deployment ( ) Observe target users after deployment ( ) ethnography quant Measure adoption Measure adoption 5 [A Nested Model of Visualization Design and Validation. Munzner. IEEE TVCG 15(6):921-928, 2009 (Proc. InfoVis 2009). ] [A Nested Model of Visualization Design and Validation. Munzner. IEEE TVCG 15(6):921-928, 2009 (Proc. InfoVis 2009). ] 8 6 7 What does data mean? Now what? Now what? Now what? 14, 2.6, 30, 30, 15, 100001 • semantics: real-world meaning • semantics: real-world meaning • semantics: real-world meaning • What does this sequence of six numbers mean? • data types: structural or mathematical interpretation of data – two points far from each other in 3D space? –item, link, attribute, position, (grid) – two points close to each other in 2D space, with 15 links between them, and a weight of –different from data types in 100001 for the link? programming! – something else?? Basil, 7, S, Pear • What about this data? – food shipment of produce (basil & pear) arrived in satisfactory condition on 7th day of month – Basil Point neighborhood of city had 7 inches of snow cleared by the Pear Creek Limited snow removal service – lab rat Basil made 7 attempts to find way through south section of maze, these trials used pear as reward food 9 10 11 12 • flat table • flat table Items & Attributes Other data types Dataset types Dataset types –one item per row –one item per row • item: individual entity, discrete • links Tables –each column is attribute Tables –each column is attribute attributes: name, age, shirt size, fave fruit –eg patient, car, stock, city –express relationship between two items –cell holds value –cell holds value for item-attribute pair Items Items –"independent variable" –eg friendship on facebook, interaction between proteins –unique key (could be implicit) Attributes Attributes attributes: name, age, shirt size, fave fruit attributes: name, age, shirt size, fave fruit • attribute: property that is • positions measured, observed, logged... –spatial data: location in 2D or 3D –eg height, blood pressure for patient –pixels in photo, voxels in MRI scan, latitude/longitude –eg horsepower, make for car Tables Tables • (grids) –"dependent variable" –sampling strategy for continuous data Attributes (columns) Attributes (columns) Items Items (rows) (rows) Cell containing value Cell containing value item: person item: person item: person 13 14 15 16
Table Table Dataset types Visualizing tables • multidimensional tables Tables –indexing based on multiple keys item cell •eg genes, patients Items Attributes attribute Tables Multidimensional Table Attributes (columns) Items (rows) Value in cell Cell containing value https://bl.ocks.org/jasondavies/1341281 17 18 19 20 Dataset types Visualizing networks Dataset types Spatial fields • network/graph • attribute values associated with cells Tables Networks & Tables Networks & Fields Trees Trees • cell contains value from continuous –nodes (vertices) connected by links (edges) domain Items Items (nodes) Items Items (nodes) Grids –tree is special case: no cycles Positions –eg temperature, pressure, wind velocity Attributes Links Attributes Links •often have roots and are directed Attributes Attributes Attributes • measured or simulated Spatial Net Spatial Networks Networks Net Tables Tables Fields (Continuous) Fields (Continuous) Attributes (columns) Attributes (columns) Grid of positions Link Link Grid of positions Items Items (rows) (rows) Cell Node Node (item) (item) Cell Node Cell containing value Cell containing value Node em) Trees https://observablehq.com/@d3/force-directed-graph https://bost.ocks.org/mike/miserables/ Trees Attributes (columns) em) Attributes (columns) http://atlas.cid.harvard.edu/explore/? tradeDirection=import&year=2012&product=726&country=undefined&red irected=true Value in cell Value in cell 21 22 23 24 Spatial fields Spatial fields Dataset types Geometry • attribute values associated with • attribute values associated with • shape of items Tables Networks & Fields Geometry scalar cells Trees cells • explicit spatial positions Items Items (nodes) Grids Items • cell contains value from • cell contains value from • points, lines, curves, surfaces, regions Positions continuous domain continuous domain Attributes Links Positions –(volumes outside scope of class) Attributes Attributes – eg temperature, pressure, wind – eg temperature, pressure, wind • boundary between computer graphics velocity velocity and visualization • measured or simulated Spatial • measured or simulated vector Networks Net Tables –graphics: geometry taken as given • beyond the scope of this class • beyond the scope of this class Fields (Continuous) Geometry (Spatial) Attributes (columns) –vis: geometry is result of a design decision – sampling – sampling Link Grid of positions Items where attributes are measured (rows) where attributes are measured Node (item) Cell – interpolation Position – interpolation Cell containing value Node tensor Trees how to model attributes elsewhere em) how to model attributes elsewhere Attributes (columns) – grid types – grid types, tensors Value in cell 25 26 27 28 Dataset types Collections Dataset and data types Attribute types • how we group items • which classes of values & Tables Networks & Fields Geometry Clusters, Data and Dataset Types Trees Sets, Lists measurements? • sets Attribute Types Tables Networks & Fields Geometry Clusters, Items Items (nodes) Grids Items Items Categorical Ordered • categorical (nominal) –unique items, unordered Trees Sets, Lists Positions Attributes Links Positions Ordinal Quantitative –compare equality • lists Items Items (nodes) Grids Items Items Attributes Attributes –no implicit ordering –ordered, duplicates possible Positions Attributes Links Positions • ordered Attributes Attributes • clusters Spatial Net Networks Tables –ordinal –groups of similar items Data Types Fields (Continuous) Geometry (Spatial) • less/greater than defined Attributes (columns) Items Attributes Links Positions Grids Link Grid of positions –quantitative Items (rows) Node • meaningful magnitude (item) Cell Position Cell containing value Node • arithmetic possible Trees em) Attributes (columns) Value in cell 29 30 31 32
Recommend
More recommend
