Learning Perceptual Kernels for Visualization Design a atay - - PowerPoint PPT Presentation

▶
learning perceptual kernels for visualization design
SMART_READER_LITE
LIVE PREVIEW

Learning Perceptual Kernels for Visualization Design a atay - - PowerPoint PPT Presentation

Feb 05, 2023 135 likes •1.06k views

Learning Perceptual Kernels for Visualization Design a atay Demiralp Michael Bernstein Jeffrey Heer Stanford University Stanford University University of Washington Interactive Data Lab @ UW 15 11.25 7.5 3.75 0 13 9.75 6.5 3.25

slide-1
SLIDE 1

Learning Perceptual Kernels for Visualization Design

Interactive Data Lab @ UW

Stanford University

Çağatay Demiralp

Stanford University

Michael Bernstein

University of Washington

Jeffrey Heer

slide-2
SLIDE 2

Visualizations Leverage Perception

3.25 6.5 9.75 13 4 7 11 14 3.75 7.5 11.25 15 5 10 15 20

balance against balance in favor England

2

slide-3
SLIDE 3

Engineering Perception Into Visualization Design?

slide-4
SLIDE 4

A Measure of Perceptual Reality

Perceptual Kernel 2D Projection

4

slide-5
SLIDE 5

What are Perceptual Kernels Useful For?

5

slide-6
SLIDE 6

Automating Visualizations

slide-7
SLIDE 7

Palette Design

l

n

7

slide-8
SLIDE 8

Palette Design

l ed

n

2 3 4 5 6 7 8 9 10

8

  • riginal

reordered

slide-9
SLIDE 9

Palette Design

l ed

n

2 3 4 5 6 7 8 9 10

2D Projection

9

reordered

slide-10
SLIDE 10

Palette Design

l ed

n

2 3 4 5 6 7 8 9 10

10

  • riginal

reordered

slide-11
SLIDE 11

l ed

n

2 3 4 5 6 7 8 9 10

Palette Design

Palettes re-ordered to maximize perceptual discriminability

11

  • riginal

reordered

slide-12
SLIDE 12

Visual Embedding: A Model for Visualization

12

slide-13
SLIDE 13

Visualizations as Functions

Data Points

f : X →Y

X

Visual Primitives

Y

quantitative

  • rdinal

nominal … color size shape

  • rientation

texture …

13

slide-14
SLIDE 14

Visual Embedding

Data Points

f : X →Y

X

x3 x4 x

1

x2

Visual Primitives

Y

y3 y4 y

1

y2

large small large small

X

d

Y

d

quantitative

  • rdinal

nominal … color size shape

  • rientation

texture …

14

slide-15
SLIDE 15

Visual Embedding

Data Points

f : X →Y

X

x3 x4 x

1

x2

Visual Primitives

Y

y3 y4 y

1

y2

large small large small

X

d

Y

d

quantitative

  • rdinal

nominal … color size shape

  • rientation

texture …

15

slide-16
SLIDE 16

Visual Embedding

Data Points

f : X →Y

X

x3 x4 x

1

x2

Visual Primitives

Y

y3 y4 y

1

y2

large small large small

X

d

Y

d

quantitative

  • rdinal

nominal … color size shape

  • rientation

texture …

16

NOT NEED TO BE METRIC

slide-17
SLIDE 17
  • i

f l s

  • e-
  • Color Names

CIELAB CIEDE2000 Kernel (Tm)

Rank Correlations

A B C D

A B C D A 1.00 0.75 0.67 0.59 B 1.00 0.81 0.77 C 1.00 0.87 D 1.00

17

slide-18
SLIDE 18
  • i

f l s

  • e-
  • Color Names

CIELAB CIEDE2000 Kernel (Tm)

Rank Correlations

A B C D

A B C D A 1.00 0.75 0.67 0.59 B 1.00 0.81 0.77 C 1.00 0.87 D 1.00

18

slide-19
SLIDE 19
  • i

f l s

  • e-
  • Color Names

CIELAB CIEDE2000 Kernel (Tm)

Rank Correlations

A B C D

A B C D A 1.00 0.75 0.67 0.59 B 1.00 0.81 0.77 C 1.00 0.87 D 1.00

19

slide-20
SLIDE 20
  • i

f l s

  • e-
  • Color Names

CIELAB CIEDE2000 Kernel (Tm)

Rank Correlations

A B C D

A B C D A 1.00 0.75 0.67 0.59 B 1.00 0.81 0.77 C 1.00 0.87 D 1.00

20

slide-21
SLIDE 21
  • i

f l s

  • e-
  • Color Names

CIELAB CIEDE2000 Kernel (Tm)

Rank Correlations

A B C D

A B C D A 1.00 0.75 0.67 0.59 B 1.00 0.81 0.77 C 1.00 0.87 D 1.00

21

slide-22
SLIDE 22
  • i

f l s

  • e-
  • Color Names

CIELAB CIEDE2000 Kernel (Tm)

Rank Correlations

A B C D

A B C D A 1.00 0.75 0.67 0.59 B 1.00 0.81 0.77 C 1.00 0.87 D 1.00

22

slide-23
SLIDE 23

Cluster Connectivity

Encode community clusters in a character co-occurrence graph.

23

slide-24
SLIDE 24

24

slide-25
SLIDE 25

CONTRIBUTIONS

25

slide-26
SLIDE 26

CONTRIBUTIONS

1) Estimate perceptual kernels

shape size size-color shape-size shape-color color

26

slide-27
SLIDE 27

CONTRIBUTIONS

2) Compare alternative judgment types

pairwise-5 pairwise-9 triplet matching triplet discrimination manual

27

slide-28
SLIDE 28

3) Assess using existing models

CIELAB CIEDE2000 Color Names I ∼ M β Stevens’ Power Law

CONTRIBUTIONS

vs.

Garner’s Integrality

28

slide-29
SLIDE 29

CONTRIBUTIONS

4) Demonstrate in visualization automation

designing palettes visual embedding

29

slide-30
SLIDE 30

Crowd-sourcing Perceptual Kernels

30

slide-31
SLIDE 31

Study Overview

Variables

size-color shape size shape-size shape-color color

Platform

31

Tableau Tableau

Subjects

600 Turkers based in the US 95% approval rate minimum 100 approved HITs

Tasks

pairwise-5 pairwise-9 triplet matching triplet discrimination manual spatial arrangement

L5 L9 SA Tm Td

reference a b a b c
slide-32
SLIDE 32

Univariate Perceptual Kernels

shape color size

L5 L9 SA Tm Td

32

slide-33
SLIDE 33

Bivariate Perceptual Kernels

shape-color shape-size size-color

L5 L9 SA Tm Td

33

slide-34
SLIDE 34

Judgment Tasks

1.Pairwise rating on 5-point scale (L5) 2.Pairwise rating on 9-point scale (L9) 3.Triplet ranking with matching (Tm) 4.Triplet ranking with discrimination (Td) 5.Spatial arrangement (SA)

34

slide-35
SLIDE 35

Judgment Tasks

  • 1. Pairwise rating on 5-point scale (L5)

35

slide-36
SLIDE 36
  • 1. Pairwise rating on 5-point scale (L5)

Judgment Tasks

36

slide-37
SLIDE 37
  • 2. Pairwise rating on 9-point scale (L9)

Judgment Tasks

37

slide-38
SLIDE 38
  • 3. Triplet ranking with matching (Tm)

Judgment Tasks

38

slide-39
SLIDE 39
  • 3. Triplet ranking with matching (Tm)

Judgment Tasks

39

slide-40
SLIDE 40
  • 4. Triplet ranking with discrimination (Td)

Judgment Tasks

40

slide-41
SLIDE 41
  • 5. Spatial arrangement (SA)

Judgment Tasks

41

slide-42
SLIDE 42
  • 5. Spatial arrangement (SA)

Judgment Tasks

42

slide-43
SLIDE 43

Perceptual Kernels & Models of Perception

43

slide-44
SLIDE 44

Size (Tm)

Consistent with Stevens’ Power Law!

perceptual kernel 2D projection

44

slide-45
SLIDE 45

Stevens’ Power Law

I ∼ M β

True Magnitude (M) Perceived Intensity (I) length brightness electric shock

stimulus dependent exponent

(β=3.5) (β=1.1) (β=0.5)

45

slide-46
SLIDE 46

Stevens’ Power Law Fit

46

slide-47
SLIDE 47

Stevens’ Power Law Fit

47

slide-48
SLIDE 48

Stevens’ Power Law Fit

48

slide-49
SLIDE 49

3) Assess using existing models

I ∼ M β Stevens’ Power Law

CONTRIBUTIONS

49

CIELAB CIEDE2000 Color Names

vs.

Garner’s Integrality

details are in the paper

slide-50
SLIDE 50

Which Judgment Task to Use?

50

slide-51
SLIDE 51

51

Pairwise Likert ratings (L5 & L9) faster & cheaper than triplet comparisons Manual spatial arrangement (SA) fastest, cheapest high variance, high sensitivity Triplet matching (Tm) lowest variance, most robust, shortest unit

reference a b

Triplet comparisons (Tm & Td) longest experiment time, highest cost

reference a b a b c
slide-52
SLIDE 52

52

Pairwise Likert ratings (L5 & L9) faster & cheaper than triplet comparisons Manual spatial arrangement (SA) fastest, cheapest high variance, high sensitivity Triplet matching (Tm) lowest variance, most robust, shortest unit

reference a b

Triplet comparisons (Tm & Td) longest experiment time, highest cost

reference a b a b c

best

slide-53
SLIDE 53

53

Pairwise Likert ratings (L5 & L9) faster & cheaper than triplet comparisons Manual spatial arrangement (SA) fastest, cheapest high variance, high sensitivity Triplet matching (Tm) lowest variance, most robust, shortest unit

reference a b

Triplet comparisons (Tm & Td) longest experiment time, highest cost

reference a b a b c

best worst

slide-54
SLIDE 54

CONCLUSIONS

Perceptual Kernels

  • perational model

Use ordinal triplet matching unless prohibited by time & cost Avoid manual spatial arrangement Read the paper

54

slide-55
SLIDE 55

Acknowledgments

data & source code

https://github.com/uwdata/perceptual-kernels https://github.com/uwdata/visual-embedding IDL Group Members

55

slide-56
SLIDE 56

Data Processing

Pairwise judgments Produce a distance matrix directly Identical pairs to detect spammers Triplet judgments Generalized non-metric multidimensional scaling Use triplets with two identical elements to detect spammers Spatial arrangements Align to a reference and filter-out the outliers Planar Euclidean distances produce a distance matrix

56

slide-57
SLIDE 57

l ed

n

2 3 4 5 6 7 8 9 10

Palette Design

57

slide-58
SLIDE 58

What About Context?

58

slide-59
SLIDE 59

What About Context?

What about it?

59

slide-60
SLIDE 60

What About Context?

60

slide-61
SLIDE 61

What About Context?

61

early results suggest no significant effect

slide-62
SLIDE 62

Why Tableau?

62

slide-63
SLIDE 63

Why Tableau?

I have Tableau stocks

63

slide-64
SLIDE 64

Why Tableau?

I have Tableau stocks?

64

slide-65
SLIDE 65

Why Tableau?

I have Tableau stocks?

65

slide-66
SLIDE 66

Why Tableau?

66

Manually designed with perceptual considerations in mind discriminability, saliency and naming

  • f colors, robustness to spatial
  • verlap of shapes

Provides ecological validity and good baseline

slide-67
SLIDE 67

What About Individual Differences?

67

slide-68
SLIDE 68

Per-subject SAs: size

The layout with gray background is the medoid of the layouts in affine space.

68

slide-69
SLIDE 69

Sensitivity

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.5 0.6 0.7 0.8 0.9 1 1.1

shape

Td L5 L9 SA Tm 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.5 0.6 0.7 0.8 0.9 1 1.1

color

L5 L9 SA Tm Td 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.5 0.6 0.7 0.8 0.9 1 1.1

size

L5 L9 SA Tm Td

corr

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.5 0.6 0.7 0.8 0.9 1 1.1

corr shape-color

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.5 0.6 0.7 0.8 0.9 1 1.1

shape-size

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.5 0.6 0.7 0.8 0.9 1 1.1

size-color

L5 L9 SA Tm Td L5 L9 SA Tm Td L5 Td L9 SA Tm

69

slide-70
SLIDE 70

Why SA Performs Poorly?

70

slide-71
SLIDE 71

Unstructured nature, leading to higher variance across subjects Expressivity limited to two dimensions expression of perceptual structures.

Why SA Performs Poorly?

71

slide-72
SLIDE 72

Why Tm Outperforms Td?

72

slide-73
SLIDE 73

It involves a binary decision (vs. trinary) Detects more fine-grained similarities

Why Tm Outperforms Td?

73

slide-74
SLIDE 74

It involves a binary decision (vs. trinary) Detects more fine-grained similarities

C A B

3 6 8

Why Tm Outperforms Td?

74

slide-75
SLIDE 75

It involves a binary decision (vs. trinary) Detects more fine-grained similarities

C A B

3 6 8 d(A,B)<d(A,C) d(A,B)<d(B,C)

Why Tm Outperforms Td?

task type=Td

75

slide-76
SLIDE 76

It involves a binary decision (vs. trinary) Detects more fine-grained similarities

C A B

3 6 8 task type=Tm

Why Tm Outperforms Td?

76

slide-77
SLIDE 77

It involves a binary decision (vs. trinary) Detects more fine-grained similarities

C A B

3 6 8 task type=Tm d(A,B)<d(A,C)

Why Tm Outperforms Td?

77

slide-78
SLIDE 78

It involves a binary decision (vs. trinary) Detects more fine-grained similarities

task type=Tm d(A,B)<d(A,C)

C A B

3 6 8

Why Tm Outperforms Td?

78

slide-79
SLIDE 79

It involves a binary decision (vs. trinary) Detects more fine-grained similarities

task type=Tm d(A,B)<d(A,C) d(B,C)<d(A,C)

C A B

3 6 8

Why Tm Outperforms Td?

79

slide-80
SLIDE 80

It involves a binary decision (vs. trinary) Detects more fine-grained similarities

task type=Tm d(A,B)<d(A,C)

C A B

3 6 8 Td cannot elicit d(B,C)<d(A,C)

Why Tm Outperforms Td?

80

slide-81
SLIDE 81

Univariate Perceptual Kernels with MDS Projections*

L5 L9 SA Tm Td *For each visual variable, projections are aligned to the projection of the L5 kernel

81

slide-82
SLIDE 82

Bivariate Perceptual Kernels with MDS Projections

L5 L9 SA Tm Td

82

slide-83
SLIDE 83

Bivariate Perceptual Kernels with 3D MDS Projections

L5 L9 SA Tm Td

83

slide-84
SLIDE 84

kernel ¡ (L5) CIELAB CIEDE2000 Color ¡ Names kernel ¡ (L9) CIELAB CIEDE2000 Color ¡ Names kernel ¡ (SA) CIELAB CIEDE2000 Color ¡ Names kernel ¡(L5)

1.00 0.67 0.59 0.76

kernel ¡(L9)

1.00 0.77 0.66 0.79

kernel ¡(SA)

1.00 0.23 0.09 0.45

CIELAB

0.67 1.00 0.88 0.82

CIELAB

0.77 1.00 0.88 0.82

CIELAB

0.23 1.00 0.88 0.82

CIEDE2000

0.59 0.88 1.00 0.77

CIEDE2000

0.66 0.88 1.00 0.77

CIEDE2000

0.09 0.88 1.00 0.77

Color ¡ Names

0.76 0.82 0.77 1.00

Color ¡ Names

0.79 0.82 0.77 1.00

Color ¡ Names

0.45 0.82 0.77 1.00

¡ kernel ¡ (Tm) CIELAB CIEDE2000 Color ¡ Names kernel ¡ (Td) CIELAB CIEDE2000 Color ¡ Names kernel ¡ (Tm)

1.00 0.68 0.60 0.76

kernel ¡(Td)

1.00 0.69 0.51 0.72

CIELAB

0.68 1.00 0.88 0.82

CIELAB

0.69 1.00 0.88 0.82

CIEDE2000

0.60 0.88 1.00 0.77

CIEDE2000

0.51 0.88 1.00 0.77

Color ¡ Names

0.76 0.82 0.77 1.00

Color ¡ Names

0.72 0.82 0.77 1.00

Comparison of Perceptual Kernels with Color Models: Rank Correlation Matrices

L5 L9 SA Tm Td corr

kernel CIELAB CIEDE2000 Color Names

84

slide-85
SLIDE 85

Comparison of Perceptual Kernels with Color Models

L5 L9 SA Tm Td corr

kernel CIELAB CIEDE2000 Color Names kernel CIELAB CIEDE2000 Color Names

Rank correlation matrices displayed as gray-scale images (brighter entries indicate higher correlations)

85

slide-86
SLIDE 86

Comparison of Perceptual Color Kernels with Color Models

The palette shapes representing the models are chosen automatically with visual embedding (using the triplet matching kernel). They reflect the correlations between the

  • variables. For example the correlation between the CIELAB and CIEDE2000 is higher than

the correlation between the perceptual kernels and color names and the assigned shapes reflect this relationship perceptually. All projections are aligned to the CIELAB projection in the plane using similarity transformations

86

slide-87
SLIDE 87

Per-subject SAs: size

The layout with gray background is the medoid of the layouts in affine space.

87

slide-88
SLIDE 88

Per-subject SAs: shape

88

slide-89
SLIDE 89

Per-subject SAs: color

89

slide-90
SLIDE 90

Per-subject SAs: shape-color

90

slide-91
SLIDE 91

Per-subject SAs: shape-size

91

slide-92
SLIDE 92

Per-subject SAs: size-color

92