Usability and Eye Tracking Marco Pretorius Usability Manager & - - PowerPoint PPT Presentation

Usability and Eye Tracking

Marco Pretorius

Usability Manager & Researcher UNISA: School of Computing

Agenda

• Introduction

– What is usability? – User-centered design – Eye tracking

• Benefits and ROI
• UNISA usability laboratory
• Studies and results
• Other Interesting figures and examples
• Questions

Introduction

What is usability?

• Lack of usability:

– Play video

What is usability?

• Usability means making products and

systems easier to use, and matching them more closely to user needs and requirements

• ISO9241: Usability is the

– effectiveness – efficiency and – satisfaction – with which specified users – achieve specified goals – in particular environments

What is usability?

• Effectiveness

– Can users achieve what they need by using the product?

• Ease of learning

– How fast can a user who has never seen the interface learn to use it?

• Efficiency of use

– How fast can users complete the task?

• Memorability

– Can users remember enough to reuse the interface effectively?

• Error prevention

– Can users complete tasks without making errors?

• Satisfaction

– How much does the user like using the system?

User-centered design

• ISO 13407: User-centered design is

characterised by: – the active involvement of users and a clear understanding of user and task requirements – an appropriate allocation of function between users and technology – the iteration of design solutions

User-centered design

• Goal

– To develop easy-to-use products – that lead to increased user satisfaction – meet your organizational or business objectives

• Difference between other design

philosophies

– user-centered design tries to optimize the user interface around how people can, want, or need to work – rather than forcing users to change how they work to accommodate the system or function

User-centered design

• ISO13407
• 4 essential activities in UCD

http://www.upassoc.org/usability_resources/about_usability/ what_is_ucd.html

User-centered design

• 1. Specify context of use

– Identify the people who will use the product, what they will use it for, and under what conditions they will use it

User-centered design

• 2. Specify requirements

– Identify any business requirements or user goals that must be met for the product to be successful

User-centered design

• 3. Design

– Producing designs and prototypes – This part of the process may be done in stages, building from a rough concept to a complete design.

User-centered design

• 4. Evaluation

– The most important part of this process is that evaluation - usability testing with actual users - is as integral as quality testing is to good software development

User-centered design

• The stages are carried out in an iterative

fashion – Cycle repeated until the project's usability objectives have been attained – Critical - participants accurately reflect the profile of your actual users – Talk directly to the user at key points in the project to make sure the site will deliver upon their requirements

User-centered design

• Methods used:

– Requirements and data gathering

• Interview
• User observation
• Contextual inquiry
• User group meeting
• Focus groups
• JAD sessions
• Surveys
• Bulletin boards/discussion groups
• Web logs

User-centered design

• Methods used:

– Usability testing

• Task walkthroughs
• Performance-based tests
• Heuristic evaluations
• Preference tests
• A-B testing

What is usability?

• Usability testing involves

– measuring the performance of users on tasks with regard to

• the ease of use,
• the task time, and
• the user‟s perception of the experience
• of the product, software application, website
• r systems

– Based on Performance – Purpose: Feedback

What is usability?

• What Usability testing is not

– Not market research – gathering opinions – not just a milestone to be checked off on the project schedule – not finished when the last participant leaves

• Involves systematic observation under controlled

conditions to determine how well people can use the product

– watching people trying to use something for its intended purpose

• Consider the findings, set priorities, and CHANGE

the prototype or site based on what happened in the usability test

• A technique to determine eye movement

and eye fixation patterns of a person

• The human eye moves by alternating

between

• Saccades
• Quick movement of the eye
• Move focus from one area to the next
• Fixation
• Time spent looking at the newly found area

Eye tracking

Benefits and ROI

Benefits and ROI

• 63% of software projects exceeded their estimates

because (Lederer and Prassad ‟92)

• Frequent request for changes by users
• Overlooked tasks
• Users‟ lack of understanding of their own tasks
• Insufficient user <> analyst communication
• 80% of software life cycle costs occur during

maintenance phase (Pressman „92)

• 80% of maintenance comes from unforseen/unmet user

requirements (Martin „83)

• 60% of maintenance phase is due to re-work because

user requirements were not clear at start (Standish „83)

• Benefits for an organisation:
• Save on development and maintenance by:
• Saving on development costs
• Decreasing the development time
• Reducing maintenance costs
• User-centred design
• Usability testing very early and throughout

development

• With a rigorous user-centered methodology, you

can ensure successful, on-time delivery – and avoid the rework that's necessary later if you don't talk to users in the first place

Benefits: usability

• Benefits for an organisation:
• Increase Total Revenue by increasing the

number of:

• Transactions
• Conversion and hence turnover
• Returning customers
• New customers through word of mouth

advertising

Benefits: usability

• Benefits for an organisation:
• Stimulate the use of the website by:
• Increasing the success ratio for a visitor
• Decreasing the number of errors and show

stoppers

• Increasing efficiency for users (less time to

complete a task)

• Increasing users satisfaction

Benefits: usability

• Benefits for a user:
• Understand a site faster and achieve their

goals faster

• Have a positive experience using the

website

• Develop trust in the company
• Perform tasks without help from telephone
• r e-mail
• http://www.2c.nl/en/what_is_usability/advantages_usability_test.php

Benefits: usability

• Usability and eye tracking:
• All the benefits of traditional usability results
• Eye tracking can add new and interesting insights
• Provide insights that are not available from

traditional usability testing methods

• Scan paths; Pattern of fixations
• Time spent looking at various display elements
• Insight into deployment of visual attention
• Participant strategies

Benefits: combination

Benefits - Examples:

• Human Factors International
• Staples.com
• 67% more repeat customers
• 31-45% reduced drop-off rates
• 10% better shopping experience
• 80% increased traffic
• Increased revenue – sales up by 491%
• Utility company in Canada – Intranet
• Training hours from 8 hours to 15 minutes
• Energy company in US – Intranet
• 300 support calls per day to ZERO

UNISA usability lab

UNISA usability lab

• A usability laboratory is a state-of-the-art

facility designed to support the observation

• f HCI
• Users are brought into a controlled

environment, in which they are asked to do specific tasks within specific timeframes

• Evaluators

– observe the problem(s) the participant might have – videotape the participant – analyse the data

UNISA usability lab

• Observer room
• Participant room
• Separated by one-way mirror
• Tobii 1750 eye tracker
• Audio and Video Recording equipment
• Event logging, eye tracking software

Typical data collected

• Live video recordings

– Screen – Mouse and keyboard movements – Facial expression

• Audio
• Eye tracking video with cursor
• Eye tracking data files
• Post-test questionnaire
• Monitoring of tasks

Usability Measures

• Effectiveness
• Task completion rate
• Number and percentage of tasks completed with

and without assistance

• Error rate recovery
• Efficiency
• Task completion time
• Real-time events
• Satisfaction
• Post-test questionnaire

• Number of fixations
• Fixation duration
• Number of fixations on each Area of Interest
• Number of gazes on each Area of Interest
• Time to 1st fixation on target Area of Interest
• Scan path

Eye tracking measures

Studies and results

MyUnisa – Assignment Submission Tool

• Participant Profile

– Full-time UNISA students who have to submit assignments online – 10 participants – 5 male, 5 female – 7 different languages – 5 expert, 5 non-expert Web users

MyUnisa – Assignment Submission Tool

Task example: Submit a PDF file

• Task completion: 70%
• Assistance needed: 40%
• Errors made: 100%
• Median task completion time: 115.60 seconds
• Play video: (PDF)

MyUnisa – Assignment Submission Tool

• All participants made errors

– Did not select appropriate file format – Error message:

• “ERROR: The type of file does NOT match the selected file type.

(PDF!=DOC)”

• All non-expert received error at least twice
• 3 experts received error once

– Error message not comprehended

• Difference between PDF and Word not

understood

MyUnisa – Assignment Submission Tool

• Eye Tracking

– Step 1 page: File selection till “continue” button click for Step 2

• Main goal: did participant read “File Format”

– Only 3 participants with several fixations on “File Format” label and drop- down box

• Still made the error

Eye tracking results: Task 1

MyUnisa – Assignment Submission Tool

• Eye Tracking:

– Step 1: Error message

• Main goal: did participant read the error message

– All participants read error message

• Only 3 participants got it right after one error

Eye tracking results: Task 1

MyUnisa – Assignment Submission Tool

• PDF task – Recommendations:

– File Type

• Should be picked up automatically

– Error message

• Simple language to be used
• PDF!=DOC is a computer term

– FAQ and instructions should contain more information about file format and PDF – Help Video File to teach new users

MyUnisa – Assignment Submission Tool

• Post-test questionnaire: Participant comments:

– “MyUnisa is a very good tool in my life, saves time driving to UNISA or Post Office to submit assignments.” – “MyUnisa is at the cutting edge in educational pile drive.” – “Think it would be difficult for a person who rarely uses computers. They may have a problem understanding things like the file type.” – “It will be very useful if users could be shown how to use this

• website. I had a friend who had to show me how to work it.”

• Prototype tool
• Allows network managers at Nelson Mandela Metropolitan

University to analyse and explore application performance on network

• Novel visualisation techniques
• Visualise application delay performance
• Integrated Tertiary Software application implemented at University
• Participants: Sound knowledge in domain of network

management

• Pilot study: 6 participants
• Case study: 9 participants

Network Management tool

Results

• Network Overview Task
• Determine which Virtual Local Area Network has the highest

maximum total delay

• Answer obtained from graph or text
• Usability results
• 100% task completion rate
• 100% correct answers
• Eye tracking results
• Graphical AOI
• Most number of fixations
• Greatest fixation percentage
• Very few fixations on textual AOI

Results

• Subnet View Task
• Determine the highest server delay
• Usability results
• 100% task completion rate
• Graph was filtered quick
• 0% correct answers
• Long time to find answers
• Eye tracking results
• Mean of 34.33 fixations on graph to find answer
• Difficulty of extracting information from the graph
• Participants fixated on wrong areas of Y-axis

Other interesting figures & examples

Other interesting figures & examples

• Telkom corporate website – eye tracking video
• Karnaugh maths application – usability video

Incorporate usability in your workspace!!

Conclusions

• Learn about usability

– Websites

• Usability Professionals Association
• Human Factors International

– Books

• Prioritizing Web Usability – Nielsen, Loranger
• Institutionalization of Usability – Schaffer

– Join a Usability forum

• CUA, HFI forums
• SA UX Forum
• LinkedIn

– Take a University course

Conclusions

• Encourage usability in your project lifecycle

– User-centered design

• Third-party usability

– Write usability guarantees into vendor contracts – Work with vendors to ensure usability (from the start!!!)

• Its about the user!

Conclusions

• Usability can help you!

– Websites – Intranets – Systems – Products

• UNISA – usability facility available
• To discuss usability testing/research potential, contact:

– Marco Pretorius – pretomc@unisa.ac.za / marco.pretorius@gmail.com

• Play video