Maya Cakmak, Matt Kay, Brad Jacobson, King Xia EVALUATION: OBSERVING INTERACTION human-computer interaction CSE 440 WINTER 2015 University of FEB 12 - WEEK 6 - THURSDAY Washington
Today •Evaluation –Heuristic evaluation recap and reflection –Observing interaction •Presentation feedback Tomorrow (section): •Usability testing checkin: Bring your paper prototypes! University of 2 Washington
Heuristic evaluation expert get information from the user University of 3 Washington
Heuristic evaluation •Have evaluators go through the UI twice •Ask them to see if it complies with heuristics –note where it doesn’t & say why •Combine the findings from 3 to 5 evaluators •Have evaluators independently rate severity University of 4 Washington
Nielsen’s heuristics • Visibility of system status • Match between system and the real world • User control and freedom • Consistency and standards • Error prevention • Recognition rather than recall • Flexibility and efficiency of use • Aesthetic and minimalist design • Help recognize, diagnose, and recover from errors • Help and documentation University of 5 Washington
University of 6 Washington
University of 7 Washington
University of 8 Washington
University of 9 Washington
University of 10 Washington
University of 11 Washington
University of 12 Washington
Bill Moggridge University of 13 Washington
Heuristic User -vs- evaluation testing University of 14 Washington
Heuristic User -vs- evaluation testing Much faster Doesn’t require University of 14 Washington
Heuristic User -vs- evaluation testing Much faster Doesn’t require interpreting user actions Far more accurate University of 14 Washington
Heuristic User -vs- evaluation testing Much faster Doesn’t require interpreting user actions Far more accurate University of 14 Washington
Heuristic User -vs- evaluation testing Much faster Doesn’t require interpreting user actions Far more accurate Combine two methods! University of 14 Washington
OBSERVING INTERACTION University of Washington
Evaluation Techniques (re-cap) •Asking users –Questionnaires, interviews, focus groups •Observing users –Passive observation, think-aloud protocol, ethnography, empirical user studies •Make users observe themselves –Diaries, experience sampling •Ask experts –Heuristic evaluation, cognitive walkthrough University of 16 Washington
Evaluation Techniques (re-cap) •Asking users –Questionnaires, interviews, focus groups •Observing users –Passive observation, think-aloud protocol, ethnography, empirical user studies •Make users observe themselves –Diaries, experience sampling •Ask experts –Heuristic evaluation, cognitive walkthrough University of 16 Washington
Evaluation Techniques (re-cap) •Asking users –Questionnaires, interviews, focus groups •Observing users –Passive observation, think-aloud protocol, ethnography, empirical user studies •Make users observe themselves –Diaries, experience sampling •Ask experts –Heuristic evaluation, cognitive walkthrough University of 16 Washington
What to measure or observe? ...at what granularity? Behavioral (objective) Data Source Attitudinal (subjective) Data type Qualitative Quantitative (direct) (indirect) University of 17 Washington
What to measure or observe? ...at what granularity? Behavioral (objective) Data Source Attitudinal (subjective) Data type Qualitative Quantitative (direct) (indirect) Depends on your goal! University of 17 Washington
User Satisfaction vs. Performance Metrics University of 18 Washington
Methods for observing interaction hmmmm blah blah blah bla Passive observation Think-aloud protocol Comparative study University of 19 Washington
Methods for observing interaction hmmmm blah blah blah bla Passive observation Think-aloud protocol Comparative study University of 20 Washington
Use case: “If this then that” University of 21 Washington
Passive observation 1 2 Behavioral (objective) Data Source 3 4 Attitudinal (subjective) Data type Qualitative Quantitative (direct) (indirect) University of 22 Washington
Exercise •Get in pairs –one person is the observer –the other is the participant •Participant task: Write a recipe that sends you an email everyday at 9pm to tell you tomorrow’s weather •Observer: Observe, take notes, and ask questions at the end University of 23 Washington
Methods for observing interaction hmmmm blah blah blah bla Passive observation Think-aloud protocol Comparative study University of 24 Washington
Think-aloud "Thinking aloud may be the single most valuable usability engineering method." J. Nielsen University of 25 Washington
Explaining the think-aloud University of 26 Washington
Explaining the think-aloud • We have found that we get a great deal of information from these informal tests if we ask people to think aloud as they work through the exercises. University of 26 Washington
Explaining the think-aloud • We have found that we get a great deal of information from these informal tests if we ask people to think aloud as they work through the exercises. • It may be a bit awkward at first, but it's really very easy once you get used to it. University of 26 Washington
Explaining the think-aloud • We have found that we get a great deal of information from these informal tests if we ask people to think aloud as they work through the exercises. • It may be a bit awkward at first, but it's really very easy once you get used to it. • All you do is speak your thoughts as you work. University of 26 Washington
Explaining the think-aloud • We have found that we get a great deal of information from these informal tests if we ask people to think aloud as they work through the exercises. • It may be a bit awkward at first, but it's really very easy once you get used to it. • All you do is speak your thoughts as you work. • If you forget to think aloud, I'll remind you to keep talking. University of 26 Washington
Explaining the think-aloud • We have found that we get a great deal of information from these informal tests if we ask people to think aloud as they work through the exercises. • It may be a bit awkward at first, but it's really very easy once you get used to it. • All you do is speak your thoughts as you work. • If you forget to think aloud, I'll remind you to keep talking. • Would you like me to demonstrate? University of 26 Washington
Think-aloud observation 1 2 Behavioral (objective) Data Source 3 4 Attitudinal (subjective) Data type Qualitative Quantitative (direct) (indirect) University of 27 Washington
Exercise •Get in pairs –one person is the observer –the other is the participant •Participant task: Write a recipe that sends you an email when a new listing for “mountain bike, seattle” is posted on Craigslist. Think aloud! •Observer: Observe, take notes, and ask questions at the end University of 28 Washington
Methods for observing interaction hmmmm blah blah blah bla Passive observation Think-aloud protocol Comparative study University of 29 Washington
A/B testing Key performance indicators? University of 30 Washington
A/B testing Recommendations based on cart content? Pro: cross-sell more items Con: distract people at check out University of 31 Washington
A/B testing Recommendations based on cart content? Pro: cross-sell more items Con: distract people at check out Highest Paid Person’s Opinion “Stop the project!” University of 31 Washington
A/B testing Recommendations based on cart content? Pro: cross-sell more items Con: distract people at check out Highest Paid Person’s Opinion “Stop the project!” Simple experiment was run, wildly successful University of 31 Washington
A/B testing University of 32 Washington
What is being compared? “conditions” University of 33 Washington
What is being compared? interval Continuous values Independent variable ordinal Ordered discrete values categorical Unordered discrete values “conditions” University of 33 Washington
Comparative observation 1 2 Behavioral (objective) Data Source 3 4 Attitudinal (subjective) Data type Qualitative Quantitative (direct) (indirect) University of 34 Washington
PRESENTATION FEEDBACK University of Washington
Timing •8 minutes is very short •Plan the timing ahead •Practice, practice, practice University of 36 Washington
Nerves •Even Obama gets nervous before a speech •More severe at the beginning –Think about exactly what to say on the first few slides •Practice, practice, practice University of 37 Washington
Respecting other presenters •When someone is presenting, put your phones and laptops away •Presented in class, did not come to section :-( University of 38 Washington
Recommend
More recommend
