Overview InfoVis Validation Approaches Paper Types: Technique - PowerPoint PPT Presentation

Overview InfoVis Validation Approaches Paper Types: Technique ◮ algorithm complexity analysis Lecture 16: Writing InfoVis Papers ◮ Initial Stage: Paper Types ◮ implementation performance (speed, memory) ◮ paper types as guide through validation choices Information Visualization ◮ quantitative metrics CPSC 533C, Fall 2007 ◮ Middle Pitfalls: Visual Encoding ◮ qualitative discussion of result pictures ◮ technique/algorithm ◮ user anecdotes (insights found) ◮ most common: here’s new algorithm to do X ◮ Late Pitfalls: Paper Strategy, Tactics, Results Tamara Munzner ◮ user community size (adoption) ◮ do first, or do better ◮ validation ◮ informal usability study ◮ Final Pitfalls: Style and Submission UBC Computer Science ◮ complexity, performance ◮ laboratory user study ◮ quant metrics, qual discussion of pix 28 November 2007 ◮ Generality ◮ field study with target user population ◮ design justification from task analysis ◮ visual encoding justification from theoretical principles Paper Types: Design Study Paper Types Type Pitfalls Type Pitfalls ◮ design study ◮ Design in Technique’s Clothing ◮ systems ◮ justify visual encoding choices ◮ design study for library/toolkit architectural choices ◮ if no major algorithm contrib, probably design study ◮ what is mapping from domain problem to visual encoding ◮ not for application-level visual encoding ◮ why does it solve problem ◮ lessons learned: why does anybody else care? ◮ abstraction and justification is critical ◮ summative evaluation / user studies ◮ not just apply technique X to domain Y ◮ formative evaluation: ethnographic analysis, iterative design ◮ lab studies of abstracted tasks ◮ field studies with target users ◮ validation ◮ model ◮ anecdotes, adoption ◮ taxonomies: aid to thinking, finding gaps ◮ design justification from task analysis ◮ formalism: new models/definitions (ex: space-scale) ◮ visual encoding justification from theoretical principles ◮ secondary: user studies ◮ commentary: advocate (ex: fisheye followup) Type Pitfalls Type Pitfalls Type Pitfalls Middle Stage: Visual Encoding ◮ Design in Technique’s Clothing ◮ Design in Technique’s Clothing ◮ Design in Technique’s Clothing ◮ if no major algorithm contrib, probably design study ◮ if no major algorithm contrib, probably design study ◮ if no major algorithm contrib, probably design study ◮ Application Bingo ◮ Application Bingo ◮ Application Bingo ◮ don’t just pick random technique-problem combinations ◮ don’t just pick random technique-problem combinations ◮ don’t just pick random technique-problem combinations ◮ must justify why technique solves problem ◮ must justify why technique solves problem ◮ must justify why technique solves problem ◮ All That Coding Means I Deserve A Systems Paper ◮ All That Coding Means I Deserve A Systems Paper ◮ only if you have architectural lessons to share ◮ only if you have architectural lessons to share ◮ Neither Fish Nor Fowl ◮ hard to straddle boundaries ◮ pick one primary contrib, vs. others as secondary Middle Stage: Visual Encoding Middle Stage: Visual Encoding Middle Stage: Visual Encoding Middle Stage: Visual Encoding 2 ◮ Unjustified Visual Encoding ◮ Unjustified Visual Encoding ◮ Unjustified Visual Encoding ◮ Color Cacophony ◮ should justify why visual encoding design choices ◮ should justify why visual encoding design choices ◮ should justify why visual encoding design choices ◮ blatant disregard for basic color perception facts appropriate for problem appropriate for problem appropriate for problem ◮ huge areas of highly saturated color ◮ requires clear statement of problem and encoding, of ◮ requires clear statement of problem and encoding, of ◮ requires clear statement of problem and encoding, of ◮ color coding intended for regions too small for course course course distinguishability ◮ nominal color coding for too many (15+) categories ◮ Hammer In Search Of Nail ◮ Hammer In Search Of Nail ◮ red/green with no luminance difference ◮ characterize capabilities of new technique before submitting ◮ characterize capabilities of new technique before submitting ◮ encode 3 separate variables with RGB paper paper ◮ even if start from technique-driven place ◮ even if start from technique-driven place ◮ 2D Good, 3D Better ◮ must justify when benefits 3D outweigh cost of occlusion ◮ abstract visual encoding allows choice over mapping variables to spatial position

Middle Stage: Visual Encoding 2 Later Stage Later Pitfalls: Strategy Later Pitfalls: Strategy ◮ What I Did Over My Summer Vacation ◮ Color Cacophony ◮ focus on effort not contribution ◮ blatant disregard for basic color perception facts ◮ too low-level ◮ huge areas of highly saturated color ◮ after bulk of work done ◮ color coding intended for regions too small for ◮ before begin writing draft distinguishability ◮ nominal color coding for too many (15+) categories ◮ red/green with no luminance difference ◮ strategy: paper-level structure ◮ encode 3 separate variables with RGB ◮ tactics: section-level problems ◮ Rainbows Just Like In The Sky ◮ results: results section in specific ◮ unjustified use of continuous rainbow colormap ◮ hue does not have implicit perceptual ordering ◮ standard rainbow colormap is perceptually nonlinear ◮ for many nameable regions, quantize into segmented colormap Later Pitfalls: Strategy Later Pitfalls: Strategy Later Pitfalls: Strategy Later Pitfalls: Tactics ◮ What I Did Over My Summer Vacation ◮ What I Did Over My Summer Vacation ◮ What I Did Over My Summer Vacation ◮ focus on effort not contribution ◮ focus on effort not contribution ◮ focus on effort not contribution ◮ too low-level ◮ too low-level ◮ too low-level ◮ Least Publishable Unit ◮ Least Publishable Unit ◮ Least Publishable Unit ◮ tiny increment beyond (your) previous work ◮ tiny increment beyond (your) previous work ◮ tiny increment beyond (your) previous work ◮ bonus points: new name for old technique ◮ bonus points: new name for old technique ◮ bonus points: new name for old technique ◮ Dense As Plutonium ◮ Dense As Plutonium ◮ so much content that no room to explain why/what/how ◮ so much content that no room to explain why/what/how ◮ fails reproducability test ◮ fails reproducability test ◮ Bad Slice and Dice ◮ two papers split up wrong ◮ neither is standalone, yet both repeat Later Pitfalls: Tactics Paper Writing: Contributions Later Pitfalls: Tactics Later Pitfalls: Tactics ◮ Stealth Contributions ◮ Stealth Contributions ◮ what are your research contributions? ◮ it’s your job to tell reader explicitly ◮ it’s your job to tell reader explicitly ◮ what can we do that wasn’t possible before? ◮ consider carefully, often different from original goals ◮ consider carefully, often different from original goals ◮ how can we do something better than before? ◮ what do we know that was unknown or unclear before? ◮ I Am So Unique ◮ don’t ignore previous work ◮ determines everything ◮ both on similar problems and with similar solutions ◮ Stealth Contributions ◮ from high-level message to which details ◮ it’s your job to tell reader explicitly ◮ often not obvious ◮ consider carefully, often different from original goals ◮ diverged from original goals, in retrospect ◮ state them explicitly and clearly in introduction ◮ don’t hope that reviewer or reader will fill in for you ◮ don’t leave unsaid what should be obvious after close reading of previous work ◮ pw very important - but many readers skip ◮ goal is clarity, not overselling ◮ do include limitations: often later, in discussion subsection Later Pitfalls: Tactics Later Pitfalls: Tactics Later Pitfalls: Tactics Later Pitfalls: Results ◮ Stealth Contributions ◮ Stealth Contributions ◮ Stealth Contributions ◮ it’s your job to tell reader explicitly ◮ it’s your job to tell reader explicitly ◮ it’s your job to tell reader explicitly ◮ consider carefully, often different from original goals ◮ consider carefully, often different from original goals ◮ consider carefully, often different from original goals ◮ I Am So Unique ◮ I Am So Unique ◮ I Am So Unique ◮ don’t ignore previous work ◮ don’t ignore previous work ◮ don’t ignore previous work ◮ both on similar problems and with similar solutions ◮ both on similar problems and with similar solutions ◮ both on similar problems and with similar solutions ◮ Enumeration Without Justification ◮ Enumeration Without Justification ◮ Enumeration Without Justification ◮ “X did Y” not enough ◮ “X did Y” not enough ◮ “X did Y” not enough ◮ must say why previous work doesn’t solve your problem! ◮ must say why previous work doesn’t solve your problem! ◮ must say why previous work doesn’t solve your problem! ◮ what limitations of theirs does your approach fix? ◮ what limitations of theirs does your approach fix? ◮ what limitations of theirs does your approach fix? ◮ Sweeping Assertions ◮ Sweeping Assertions ◮ cite source or delete assertion or flag as contrib ◮ cite source or delete assertion or flag as contrib ◮ check what “everybody knows” ◮ check what “everybody knows” ◮ I Am Utterly Perfect ◮ discussion of limitations makes paper stronger