ColorBless: Augmenting Visual Information for Colorblind People - - PowerPoint PPT Presentation

▶
colorbless augmenting visual information for colorblind
SMART_READER_LITE
LIVE PREVIEW

ColorBless: Augmenting Visual Information for Colorblind People - - PowerPoint PPT Presentation

Nov 22, 2022 160 likes •844 views

National University of Singapore NUS-HCI Lab ColorBless: Augmenting Visual Information for Colorblind People with Binocular Luster Effect Soon Hau Chua, Haimo Zhang,

slide-1
SLIDE 1

ColorBless: Augmenting Visual Information for Colorblind People with Binocular Luster Effect

Soon Hau Chua, Haimo Zhang, Muhammad Hammad, Shengdong Zhao, Sahil Goyal, Karan Singh

National University of Singapore NUS-HCI Lab National University of Singapore, University of Toronto

1

slide-2
SLIDE 2

ColorBless is a digital colorblind aid that applies binocular luster effect to differentiate confusing colors for colorblind people

National University of Singapore NUS-HCI Lab

2

slide-3
SLIDE 3

Binocular Luster Effect

National University of Singapore NUS-HCI Lab

3

Visual Illustration of Binocular Luster Effect

slide-4
SLIDE 4

Binocular Luster Effect

National University of Singapore NUS-HCI Lab

4

Video Illustration of Binocular Luster Effect (60fps)

slide-5
SLIDE 5

Binocular Luster Effect

  • Perception of metallic shininess (shimmering) in

binocular vision

  • Caused by the difference in brightness perceived by

left and right eye

  • Brightness differences > Binocular Fusion Limit
  • Binocular rivalry effect

National University of Singapore NUS-HCI Lab

5

slide-6
SLIDE 6

ColorBless & PatternBless

  • Introduce colorbless and patternbless
slide-7
SLIDE 7

National University of Singapore NUS-HCI Lab

7

slide-8
SLIDE 8

The Problem for Colorblind

  • afflicts ~200 million people

worldwide (mostly men)

  • difficulty in differentiating

certain confusing colors

  • Red-green (protanope,

deuteranope) and Blue- yellow (tritanope) colourblind

National University of Singapore NUS-HCI Lab

8

slide-9
SLIDE 9

The Problem for Colorblind

  • Significance: ~200 million

people

  • Type: 


Red-green 
 (protanope, deuteranope) 
 Blue-yellow 
 (tritanope)

National University of Singapore NUS-HCI Lab

9

slide-10
SLIDE 10

The Problem for Colorblind

  • afflicts ~200 million people

worldwide (mostly men)

  • difficulty in differentiating

certain confusing colors

  • Red-green (protanope,

deuteranope) and Blue- yellow (tritanope) colourblind

National University of Singapore NUS-HCI Lab

10

slide-11
SLIDE 11

Colorblind Techniques

  • (i) Contextual Inferences
  • (ii) Substituting Colors
  • (iii) Augmenting Visual Information

National University of Singapore NUS-HCI Lab

11

slide-12
SLIDE 12

Contextual Inferences

National University of Singapore NUS-HCI Lab

12

Normal color vision Red-green colourblind

slide-13
SLIDE 13

Contextual Inferences

National University of Singapore NUS-HCI Lab

13

Normal color vision Red-green colourblind

slide-14
SLIDE 14

Substituting Colors

National University of Singapore NUS-HCI Lab

14

Physical Lenses

EnChroma Lenses Before After Simulated Effects of X-Chrome Lens

slide-15
SLIDE 15

Substituting Colors

National University of Singapore NUS-HCI Lab

15

Physical Lenses

Before After Simulated Effects of X-Chrome Lens

slide-16
SLIDE 16

Substituting Colors

National University of Singapore NUS-HCI Lab

16

Digital Recoloring Aids

slide-17
SLIDE 17

Substituting Colors

National University of Singapore NUS-HCI Lab

17

Digital Recoloring Aids

Original Image Colorblind Image Daltonized Colorblind Image Daltonized Original Image Isn’t the color a bit off? Are these still tomatoes?

slide-18
SLIDE 18

Substituting Colors

National University of Singapore NUS-HCI Lab

18

Digital Recoloring Aids

Original Image Colorblind Image Daltonized Colorblind Image Daltonized Original Image Isn’t the color a bit off? Are these still tomatoes?

slide-19
SLIDE 19

Substituting Colors

National University of Singapore NUS-HCI Lab

19

Digital Recoloring Aids Original image

slide-20
SLIDE 20

Substituting Colors

National University of Singapore NUS-HCI Lab

20

Digital Recoloring Aids Daltonized Colorblind Image

slide-21
SLIDE 21

Substituting Colors

National University of Singapore NUS-HCI Lab

21

Digital Recoloring Aids Daltonized Colorblind Image

slide-22
SLIDE 22

Augmenting Visual Information

National University of Singapore NUS-HCI Lab Pattern technique by Sajadi et al. (2012)

22

Original Image Patterned Colorblind Image

slide-23
SLIDE 23

Augmenting Visual Information

National University of Singapore NUS-HCI Lab Pattern technique by Sajadi et al. (2012)

23

Original Image Patterned Colorblind Image

slide-24
SLIDE 24

Augmenting Visual Information

National University of Singapore NUS-HCI Lab ColorADD System

24

Transportation Map in Portugal

slide-25
SLIDE 25

Augmenting Visual Information

National University of Singapore NUS-HCI Lab ColorADD System

25

Transportation Map in Portugal

slide-26
SLIDE 26
  • Introduce unwanted visual artefacts
  • Disrupt the purpose & aesthetic of images
  • Learning curve for colourblind people

Augmenting Visual Information

National University of Singapore NUS-HCI Lab

Downsides

slide-27
SLIDE 27
  • Introduce unwanted visual artefacts
  • Disrupt the purpose & aesthetic of images
  • Learning curve for colourblind people

Augmenting Visual Information

National University of Singapore NUS-HCI Lab

Downsides

slide-28
SLIDE 28

Goals of a Good Colorblind Aid

  • Comfortable to use
  • The “effect” is noticeable
  • Confusing colors are easily distinguishable
  • Does not affect the hues of other non-confusing

colors

  • Does not affect the overall hues for people with

normal color vision

National University of Singapore NUS-HCI Lab

28

slide-29
SLIDE 29

Scenario

  • show image meeting with colorblind and normal

people

  • illustrate the downsides of the 3 types of colorblind

aid

slide-30
SLIDE 30

Desirable Properties of a Colorblind Aid

  • Confusing colors are easily distinguishable
  • Does not affect the hues of other non-confusing

colours

  • Does not affect the overall hues for people with

normal color vision

  • Comfortable to use

National University of Singapore NUS-HCI Lab

30

slide-31
SLIDE 31

ColorBless & PatternBless

slide-32
SLIDE 32

ColorBless and PatternBless

National University of Singapore NUS-HCI Lab

32

Original Image

slide-33
SLIDE 33

ColorBless and PatternBless

National University of Singapore NUS-HCI Lab

33

Meyer & Greenberg ColorBlind Simulation Algorithm (1998)

Simulated Colorblind Image

slide-34
SLIDE 34

ColorBless and PatternBless

National University of Singapore NUS-HCI Lab

34

Find similar colors in colorblind view

slide-35
SLIDE 35

ColorBless and PatternBless

National University of Singapore NUS-HCI Lab

35

Find colors similar in colorblind view, but different in original image

slide-36
SLIDE 36

ColorBless and PatternBless

National University of Singapore NUS-HCI Lab

36

Apply luster effect to one of the color areas

Area to be “blessed” (Applied luster effect) by increasing its brightness (+dY) in one eye and decrease (-dY) in another

slide-37
SLIDE 37

ColorBless and PatternBless

National University of Singapore NUS-HCI Lab

37

Bless the whole area Bless with patterns

slide-38
SLIDE 38

ColorBless and PatternBless

National University of Singapore NUS-HCI Lab

38

Bless the whole area Bless with patterns

+dY +dY

  • dY
  • dY
slide-39
SLIDE 39

User Study

Determine color distinguishability and subjective evaluation

  • f ColorBless and PatternBless against existing techniques

National University of Singapore NUS-HCI Lab

39

slide-40
SLIDE 40

Techniques Compared

  • Daltonize recoloring technique
  • Sajadi’s pattern augmentation technique
  • ColorBless
  • PatternBless

National University of Singapore NUS-HCI Lab

40

slide-41
SLIDE 41

Experimental Design

  • Ten colorblind participants (mild to medium

deutans)

  • Simple color distinguishing task (shown below)
  • Indicate square with a different color
  • Reaction time (s) and error rate were measured

National University of Singapore NUS-HCI Lab

41

slide-42
SLIDE 42

Experimental Design

  • Post-study questionnaires was given to evaluate all four

colorblind techniques

  • Perceived distinguishing speed, cognitive effort

required to distinguish colors, comfort level, and

  • bviousness of the effects was evaluated via 5-pt Likert
  • Indicate their preference in four common use-cases
  • Brief interviews to understand their needs and issues in

decoding color from digital content

National University of Singapore NUS-HCI Lab

42

slide-43
SLIDE 43

Results

  • Overall significant

difference in mean ERs (p < .05)

  • ColorBless is

significantly faster than all other techniques (all p < .01)

  • Recoloring is also faster

than pattern technique (p < .05)

*Error bars denote standard deviation National University of Singapore NUS-HCI Lab

Color Distinguishing Task

43

slide-44
SLIDE 44

Results

Subjective Evaluation

*Average mean values are plotted, error bars denote standard deviation National University of Singapore NUS-HCI Lab

44

slide-45
SLIDE 45

Results

Preferences in Common Use-cases

Users preference in different use-cases National University of Singapore NUS-HCI Lab

45

slide-46
SLIDE 46

Results

  • Our participants place huge emphasis on reliability

and speed when interpreting color-coded information

  • Participants commented that while recoloring

technique could resolve confusion, they avoid relying on color hues exclusively in everyday task

  • Prefer augmentation over substitution-based aids in

general

Post Interviews

National University of Singapore NUS-HCI Lab

46

slide-47
SLIDE 47

Desirable Properties of a Colorblind Aid

  • Confusing colors are easily distinguishable
  • Does not affect the hues of other non-confusing

colours

  • Does not affect the overall hues for people with

normal color vision

  • Comfortable to use

National University of Singapore NUS-HCI Lab

47

slide-48
SLIDE 48

conclusion slides

  • <<show the scenario again and show the picture

…>>

slide-49
SLIDE 49

Interesting Characteristics of Binocular Luster Effect in HCI

  • Brightness differences (dY, in the YUV color space)

between left and right images can be artificially controlled by software

  • The higher the dY, the stronger the luster effect
  • Possibilities of creating different levels of luster to

represent ordinal data

National University of Singapore NUS-HCI Lab

49

slide-50
SLIDE 50

Potential Issues of Luster

  • One of the most salient and rapidly noticeable

visual effect in everyday phenomena

  • Could be uncomfortable to look at
  • Manifested as visual artefacts to be removed in 3D

movies

National University of Singapore NUS-HCI Lab

50

slide-51
SLIDE 51

National University of Singapore NUS-HCI Lab

Luster Effect

51

slide-52
SLIDE 52
  • Comfortable to use
  • The “effect” is noticeable
  • Confusing colors are easily distinguishable
  • Does not affect the hues of other non-confusing

colors

  • Does not affect the overall hues for people with

normal color vision

National University of Singapore NUS-HCI Lab

?

52

Goals of a Good Colorblind Aid

?

slide-53
SLIDE 53

Guidelines for Using Luster Effect in Stereo 3D

National University of Singapore NUS-HCI Lab

53

Images with same contrast polarity Images with

  • pposite

contrast polarity (i) Minimum dY to start perceiving the binocular luster effect 24 13 (ii) Highest dY within the comfortable level 61 44 (iii) Number of distinguishable luster levels between (i) and (ii) 3 3

slide-54
SLIDE 54
  • Comfortable to use
  • The “effect” is noticeable
  • Confusing colors are easily distinguishable
  • Does not affect the hues of other non-confusing

colors

  • Does not affect the overall hues for people with

normal color vision

National University of Singapore NUS-HCI Lab

54

Goals of a Good Colorblind Aid

? ? ?

slide-55
SLIDE 55
  • Comfortable to use
  • The “effect” is noticeable
  • Confusing colors are easily distinguishable
  • Does not affect the hues of other non-confusing

colors

  • Does not affect the overall hues for people with

normal color vision

National University of Singapore NUS-HCI Lab

55

Goals of a Good Colorblind Aid

? ? ?

slide-56
SLIDE 56

National University of Singapore NUS-HCI Lab

56

User with 3D glasses He can see the luster effect User without 3D glasses Can he see the image without significant changes to the hue?

slide-57
SLIDE 57

User Study #1

Evaluate the color differences of various colorblind techniques seen by normal color vision people

National University of Singapore NUS-HCI Lab

57

slide-58
SLIDE 58

Colorblind Techniques Compared

Daltonize Recoloring Technique Sajadi’s Pattern Technique

National University of Singapore NUS-HCI Lab

58

slide-59
SLIDE 59

Experimental Design

  • Ten normal color vision participants
  • Daltonize, Pattern, Luster technique
  • Evaluate the original color, Daltonize color, Pattern

color, and Luster Color

National University of Singapore NUS-HCI Lab

59

A B C D E

slide-60
SLIDE 60

National University of Singapore NUS-HCI Lab

60

Indicate the perceived color of each technique by using Color Dictionary by Heer and Stone (2012)

A

slide-61
SLIDE 61

National University of Singapore NUS-HCI Lab

61

Color name distance was determined with Color Palette Analyser

slide-62
SLIDE 62

Results

  • Main effects

statistically significant (p < .01) for color A,D, and E

  • Luster effect

significantly better than recoloring for A and D

  • Luster retains the color

better than Daltonize for 3 out of 5 colors

National University of Singapore NUS-HCI Lab

Retaining Colors for Normal Color Vision People

62

slide-63
SLIDE 63
  • Comfortable to use
  • The “effect” is noticeable
  • Confusing colors are easily distinguishable
  • Does not affect the hues of other non-confusing

colors

  • Does not affect the overall hues for people with

normal color vision

National University of Singapore NUS-HCI Lab

63

Goals of a Good Colorblind Aid

? ?

slide-64
SLIDE 64
  • Comfortable to use
  • The “effect” is noticeable
  • Confusing colors are easily distinguishable
  • Does not affect the hues of other non-confusing

colors

  • Does not affect the overall hues for people with

normal color vision

National University of Singapore NUS-HCI Lab

64

Goals of a Good Colorblind Aid

? ?

slide-65
SLIDE 65
  • Comfortable to use
  • The “effect” is noticeable
  • Confusing colors are easily distinguishable
  • Does not affect the hues of other non-confusing

colors

  • Does not affect the overall hues for people with

normal color vision

National University of Singapore NUS-HCI Lab

65

Goals of a Good Colorblind Aid

slide-66
SLIDE 66

Potential Applications of Binocular Luster Beyond Colorblind Aids

National University of Singapore NUS-HCI Lab Augmented visualisations in scenarios where one party requires an additional layer of information

66

slide-67
SLIDE 67

Potential Applications of Binocular Luster Beyond Colorblind Aids

Visualisation of additional spatial scalar function without obscuring or changing the color or texture of the image National University of Singapore NUS-HCI Lab

67

slide-68
SLIDE 68

Questions?

National University of Singapore NUS-HCI Lab

chuasoonhau@nus.edu.sg

68