OBSERVING INTERACTION human-computer interaction CSE 440 WINTER - - PowerPoint PPT Presentation

▶
observing
SMART_READER_LITE
LIVE PREVIEW

OBSERVING INTERACTION human-computer interaction CSE 440 WINTER - - PowerPoint PPT Presentation

Jan 27, 2024 266 likes •826 views

Maya Cakmak, Matt Kay, Brad Jacobson, King Xia EVALUATION: OBSERVING INTERACTION human-computer interaction CSE 440 WINTER 2015 University of FEB 12 - WEEK 6 - THURSDAY Washington Today Evaluation Heuristic evaluation recap and

slide-1
SLIDE 1

University of Washington

human-computer interaction

CSE 440 WINTER 2015

FEB 12 - WEEK 6 - THURSDAY

EVALUATION:

OBSERVING

INTERACTION

Maya Cakmak, Matt Kay, Brad Jacobson, King Xia

slide-2
SLIDE 2

University of Washington

Today

  • Evaluation

–Heuristic evaluation recap and reflection –Observing interaction

  • Presentation feedback

Tomorrow (section):

  • Usability testing checkin: Bring your paper

prototypes!

2

slide-3
SLIDE 3

University of Washington

Heuristic evaluation

3

get information from the user expert

slide-4
SLIDE 4

University of Washington

Heuristic evaluation

  • Have evaluators go through the UI twice
  • Ask them to see if it complies with heuristics

–note where it doesn’t & say why

  • Combine the findings from 3 to 5 evaluators
  • Have evaluators independently rate severity

4

slide-5
SLIDE 5

University of Washington

Nielsen’s heuristics

  • Visibility of system status
  • Match between system and the real world
  • User control and freedom
  • Consistency and standards
  • Error prevention
  • Recognition rather than recall
  • Flexibility and efficiency of use
  • Aesthetic and minimalist design
  • Help recognize, diagnose, and recover from errors
  • Help and documentation

5

slide-6
SLIDE 6

University of Washington

6

slide-7
SLIDE 7

University of Washington

7

slide-8
SLIDE 8

University of Washington

8

slide-9
SLIDE 9

University of Washington

9

slide-10
SLIDE 10

University of Washington

10

slide-11
SLIDE 11

University of Washington

11

slide-12
SLIDE 12

University of Washington

12

slide-13
SLIDE 13

University of Washington

13

Bill Moggridge

slide-14
SLIDE 14

University of Washington

14

Heuristic evaluation User testing

  • vs-
slide-15
SLIDE 15

University of Washington

14

Much faster Doesn’t require

Heuristic evaluation User testing

  • vs-
slide-16
SLIDE 16

University of Washington

14

Much faster Doesn’t require interpreting user actions Far more accurate

Heuristic evaluation User testing

  • vs-
slide-17
SLIDE 17

University of Washington

14

Much faster Doesn’t require interpreting user actions Far more accurate

Heuristic evaluation User testing

  • vs-
slide-18
SLIDE 18

University of Washington

14

Much faster Doesn’t require interpreting user actions Far more accurate

Heuristic evaluation User testing

  • vs-

Combine two methods!

slide-19
SLIDE 19

University of Washington

OBSERVING

INTERACTION

slide-20
SLIDE 20

University of Washington

Evaluation Techniques (re-cap)

16

  • Asking users

–Questionnaires, interviews, focus groups

  • Observing users

–Passive observation, think-aloud protocol, ethnography, empirical user studies

  • Make users observe themselves

–Diaries, experience sampling

  • Ask experts

–Heuristic evaluation, cognitive walkthrough

slide-21
SLIDE 21

University of Washington

Evaluation Techniques (re-cap)

16

  • Asking users

–Questionnaires, interviews, focus groups

  • Observing users

–Passive observation, think-aloud protocol, ethnography, empirical user studies

  • Make users observe themselves

–Diaries, experience sampling

  • Ask experts

–Heuristic evaluation, cognitive walkthrough

slide-22
SLIDE 22

University of Washington

Evaluation Techniques (re-cap)

16

  • Asking users

–Questionnaires, interviews, focus groups

  • Observing users

–Passive observation, think-aloud protocol, ethnography, empirical user studies

  • Make users observe themselves

–Diaries, experience sampling

  • Ask experts

–Heuristic evaluation, cognitive walkthrough

slide-23
SLIDE 23

University of Washington

What to measure or observe?

17

...at what granularity?

Attitudinal (subjective)

Behavioral (objective)

Data Source Data type

Qualitative (direct) Quantitative (indirect)

slide-24
SLIDE 24

University of Washington

What to measure or observe?

17

...at what granularity?

Depends on your goal!

Attitudinal (subjective)

Behavioral (objective)

Data Source Data type

Qualitative (direct) Quantitative (indirect)

slide-25
SLIDE 25

University of Washington

User Satisfaction vs. Performance Metrics

18

slide-26
SLIDE 26

University of Washington

Methods for observing interaction

19

Passive observation Comparative study Think-aloud protocol

hmmmm blah blah blah bla
slide-27
SLIDE 27

University of Washington

Methods for observing interaction

20

Passive observation Think-aloud protocol Comparative study

hmmmm blah blah blah bla
slide-28
SLIDE 28

University of Washington

Use case: “If this then that”

21

slide-29
SLIDE 29

Attitudinal (subjective)

Behavioral (objective)

Data Source Data type

Qualitative (direct) Quantitative (indirect)

University of Washington

Passive observation

22

2 4 3 1

slide-30
SLIDE 30

University of Washington

Exercise

  • Get in pairs

–one person is the observer –the other is the participant

  • Participant task: Write a recipe that sends you an

email everyday at 9pm to tell you tomorrow’s weather

  • Observer: Observe, take notes, and ask questions

at the end

23

slide-31
SLIDE 31

University of Washington

Methods for observing interaction

24

Passive observation Think-aloud protocol

hmmmm blah blah blah bla

Comparative study

slide-32
SLIDE 32

University of Washington

Think-aloud

25

"Thinking aloud may be the single most valuable usability engineering method."

  • J. Nielsen
slide-33
SLIDE 33

University of Washington

Explaining the think-aloud

26

slide-34
SLIDE 34

University of Washington

Explaining the think-aloud

  • We have found that we get a great deal of information from

these informal tests if we ask people to think aloud as they work through the exercises.

26

slide-35
SLIDE 35

University of Washington

Explaining the think-aloud

  • We have found that we get a great deal of information from

these informal tests if we ask people to think aloud as they work through the exercises.

  • It may be a bit awkward at first, but it's really very easy
  • nce you get used to it.

26

slide-36
SLIDE 36

University of Washington

Explaining the think-aloud

  • We have found that we get a great deal of information from

these informal tests if we ask people to think aloud as they work through the exercises.

  • It may be a bit awkward at first, but it's really very easy
  • nce you get used to it.
  • All you do is speak your thoughts as you work.

26

slide-37
SLIDE 37

University of Washington

Explaining the think-aloud

  • We have found that we get a great deal of information from

these informal tests if we ask people to think aloud as they work through the exercises.

  • It may be a bit awkward at first, but it's really very easy
  • nce you get used to it.
  • All you do is speak your thoughts as you work.
  • If you forget to think aloud, I'll remind you to keep talking.

26

slide-38
SLIDE 38

University of Washington

Explaining the think-aloud

  • We have found that we get a great deal of information from

these informal tests if we ask people to think aloud as they work through the exercises.

  • It may be a bit awkward at first, but it's really very easy
  • nce you get used to it.
  • All you do is speak your thoughts as you work.
  • If you forget to think aloud, I'll remind you to keep talking.
  • Would you like me to demonstrate?

26

slide-39
SLIDE 39

University of Washington

Think-aloud observation

27

Attitudinal (subjective)

Behavioral (objective)

Data Source Data type

Qualitative (direct) Quantitative (indirect)

2 4 3 1

slide-40
SLIDE 40

University of Washington

Exercise

28

  • Get in pairs

–one person is the observer –the other is the participant

  • Participant task: Write a recipe that sends you an

email when a new listing for “mountain bike, seattle” is posted on Craigslist. Think aloud!

  • Observer: Observe, take notes, and ask questions

at the end

slide-41
SLIDE 41

University of Washington

Methods for observing interaction

29

Passive observation Think-aloud protocol

hmmmm blah blah blah bla

Comparative study

slide-42
SLIDE 42

University of Washington

A/B testing

30

Key performance indicators?

slide-43
SLIDE 43

University of Washington

A/B testing

31

Recommendations based on cart content?

Pro: cross-sell more items Con: distract people at check out

slide-44
SLIDE 44

University of Washington

A/B testing

31

Recommendations based on cart content?

Pro: cross-sell more items Con: distract people at check out

Highest Paid Person’s Opinion “Stop the project!”

slide-45
SLIDE 45

University of Washington

A/B testing

31

Recommendations based on cart content?

Pro: cross-sell more items Con: distract people at check out

Highest Paid Person’s Opinion “Stop the project!” Simple experiment was run, wildly successful

slide-46
SLIDE 46

University of Washington

A/B testing

32

slide-47
SLIDE 47

University of Washington

What is being compared?

33

“conditions”

slide-48
SLIDE 48

University of Washington

What is being compared?

33

Independent variable

interval

  • rdinal

categorical

Continuous values Ordered discrete values Unordered discrete values

“conditions”

slide-49
SLIDE 49

University of Washington

Comparative observation

34

Attitudinal (subjective)

Behavioral (objective)

Data Source Data type

Qualitative (direct) Quantitative (indirect)

2 4 3 1

slide-50
SLIDE 50

University of Washington

PRESENTATION FEEDBACK

slide-51
SLIDE 51

University of Washington

Timing

  • 8 minutes is very short
  • Plan the timing ahead
  • Practice, practice, practice

36

slide-52
SLIDE 52

University of Washington

Nerves

  • Even Obama gets nervous before a speech
  • More severe at the beginning

–Think about exactly what to say on the first few slides

  • Practice, practice, practice

37

slide-53
SLIDE 53

University of Washington

Respecting other presenters

  • When someone is presenting, put your phones

and laptops away

  • Presented in class, did not come to section :-(

38

slide-54
SLIDE 54

University of Washington

Content

  • Contextual inquiry

–Participants, process –Findings, themes –Implications for tasks –Implications for design

  • Tasks as verbs from user perspective
  • Storyboards: walk-through

39

slide-55
SLIDE 55

University of Washington

Ooops, out of time!

40

  • Will come back to it next week...