I590 Interactive Visual Analytics Week 11 | Nov 2, 2016 Task - - PowerPoint PPT Presentation

Khairi Reda | redak@iu.edu School of Informa5cs & Compu5ng, IUPUI

Week 11 | Nov 2, 2016 Task abstraction Evaluation

I590 Interactive Visual Analytics

Administra/via…

• Project 2
• 10 minutes presenta5on per team
• 2 minutes for Q & A
• Rehearse!
• Please send me contribu5ons of individual

team members

• Project 3
• Proposals due next week in class (1 page)
• Teams of 3 or less. Individual project is OK

Raw data Processed data pre- processing Visual structure visual encoding Visualization (multiple views

• f visual things)

interaction

user

vis designer

Visualiza/on design process

Study Design Build Evaluate

What is shown? Why is the user looking at it? How is it shown?

data abstrac/on

visual encoding

• Visualiza5on helps people carry out a task on a dataset
• To design an effec5ve visualiza5on, we need to understand those tasks
• Tasks come from the domain and the background of the users
• Despite apparent differences, many tasks can be similar
• “Contrast the prognosis of pa5ents who were admiXed to hospital to

pa5ents receiving home care/rest” [epidemiologists studying flu]

• “See if the results for 5ssue samples treated with LL-37 match up with

the ones without pep5de” [biologists studying immune system response]

• Both tasks are essen5ally about “comparing values between two

groups”

• Task abstrac5on allow us to iden5fy common visualiza5on designs

despite apparent domain differences

Task abstrac/on

Task abstrac/on

{action, target} pairs

discover distribu5on compare trends locate outliers browse topology

Based on a slide by Miriah Meyer

Task abstrac/on

{action, target} pairs

Analyze Search Query

Discover vs Present

Explore vs. Explain

Discover

Present

Enjoy

Task abstrac/on

{action, target} pairs

Analyze Search Query

Annotate

Mid level ac/ons

{action, target} pairs

Analyze Search Query

{action, target} pairs

Analyze Search Query

Mid level ac/ons

Example: Compare & Derive

Via Alex Lex

Task abstrac/on

{action, target} pairs

Target

Example

Trends: How did the job market develop since the recession overall? Outliers: Looking at real estate related jobs

Alex Lex

Real-world example of Domain task analysis and design

Inferring Grevy’s social interac5ons

Mayank Lahiri

Domain Tasks

• Find communi5es in zebra society, and

influen5al individuals who play a role in shaping the social structure

• Understand how the social structure of

Grevy’s zebra evolve over-5me

• Understand how Grevy’s zebra society

responds to environmental variables

Domain Tasks

• Find communi5es in zebra society, and

influen5al individuals who play a role in shaping the social structure

• Understand how the social structure of

Grevy’s zebra evolve over-5me

• Understand how Grevy’s zebra society

responds to environmental variables

Rubenstein et al., 2015

Ac/on: Explore (unknown target, unknown loca5on) Target: communi5es (groups of zebras that hang out together)

Domain Tasks

• Find communi5es in zebra society, and

influen5al individuals who play a role in shaping the social structure

• Understand how the social structure of

Grevy’s zebra evolve over-5me

• Understand how Grevy’s zebra society

responds to environmental variables

Ac/on: Explore & Compare Target: All communi5es over 5me

A B C Q R X Y A B X Y C Q R A B C Q R X Y

T1 T2 T3

A B C Q R X Y A B X Y C Q R A B C Q R X Y

T1 T2 T3

Time-changing groups

A B C Q R X Y A B X Y C Q R A B C Q R X Y

T1 T2 T3

Q R X Y A B C

time

group switch

Time-changing groups

Time-changing groups

individuals community

Domain Tasks

• Find communi5es in zebra society, and

influen5al individuals who play a role in shaping the social structure

• Understand how the social structure of

Grevy’s zebra evolve over-5me

• Understand how Grevy’s zebra society

responds to environmental variables

Ac/on: Relate Target: Communi5es and environment

Social structure Group movement

• ver space and 5me

+

Social structure + geography

Social structure + geography

Visualiza/on design process

Study Design Build Evaluate

Why evaluate?

• Evalua/on / valida/on is “about whether you have

built the right product”

• Does the visualiza5on serve its intended purpose? Does

it enable users to perform their intended analysis tasks?

• Is it “easy” to use? Are there any usability issues in the

interface?

• Does the visualiza5on enable accurate percep5on of

values, distribu5ons, and/or trends in the data?

• Does it provide new insights about the data?
• Is the visualiza5on memorable and/or engaging?
• How can we improve the visualiza5on?

What to evaluate?

• Think about your contribu5on. That is, the new

idea in your visualiza5on

• The user interface?
• The visual encoding?
• The interac5on technique?
• The abstract tasks you iden5fied from

interviewing domain experts?

• Or bits of the above?

Four nested levels of vis design

Munzner, 2014

Four nested levels of vis design

Munzner, 2014

Study domain, interview users, iden/fy needs

Four nested levels of vis design

Munzner, 2014

Iden/fy tasks and

• data. Translate from

domain-dependent to abstract tasks and data types

Four nested levels of vis design

Munzner, 2014

Sketch/design visual encoding and interac/on techniques

Four nested levels of vis design

Munzner, 2014

Implement visualiza/on using code

Threats to validity

Munzner, 2014

Threats to validity

Munzner, 2014

Evalua/on methods

• Evaluate algorithm speed / memory usage
• Controlled [lab] studies with any user
• Qualita/ve studies
• Insight-based evalua/on
• Evalua/ng the data analysis process
• Field deployment

Controlled [lab] studies

• Usually done with any poten5al user
• Goal is to control usage condi5on is much as possible
• Allows for comparison between different techniques or

representa5ons

• Generally provides accurate conclusion, but results may not

generalize beyond lab condi5ons or tested tasks

• Typically quan/ta/ve in nature
• Finely-scoped tasks
• Measure accuracy and performance 5me

Qualita/ve studies

• Usually open-ended usage scenarios
• Smaller number of par5cipants compared to

quan5ta5ve lab studies

• More in-depth analysis
• Analyze videos, audio, or comments from users
• Can ask par5cipants to fill surveys, or provide

subjec5ve feedback on the visualiza5on

• Usually involves domain experts and target audience
• f the visualiza5on

Insight-based evalua/on

• The goal of visualiza5on is to generally generate new

insight

• Evalua5on should therefore include insight-genera/on
• Think-aloud protocols: have the users say what they are

thinking

• Transcribe and code:
• Observa5on
• Hypothesis
• Ques5on
• Exploratory goal

Evalua/ng the data analysis / explora/on process

• The focus is on the visual analysis process of users, as opposed

to the outcome of the analysis

• Want to understand the analy5c dialogue between the user and

the data

• Want to capture and analyze mul5ple aspects:
• Interac/ons with the visualiza/on
• Interac5on logs, videos
• Reasoning process
• Think aloud protocol: have the par5cipant say what they are

thinking

• Eye gaze behavior

Filter Change view Hypothesis Change view Observe outliers Hypothesis Query Filter Decision making … … …

Evalua/ng visualiza/ons is s/ll tricky!