������������������������� If you can’t read this, go to http://uid.csail.mit.edu/6.831/L16.gif ������������������������ Source: Interface Hall of Shame Fall 2005 6.831 UI Design and Implementation 1 Fall 2005 6.831 UI Design and Implementation 2 �������������� ������������������ � � User testing Formative evaluation � Find problems for next iteration of design � Evaluates prototype or implementation, in lab, on chosen tasks � Ethics � Qualitative observations (usability problems) � Field study � Formative evaluation � Find problems in context � Evaluates working implementation, in real context, on real tasks � Mostly qualitative observations � Controlled experiment � Tests a hypothesis (e.g., interface X is faster than interface Y) � Evaluates working implementation, in controlled lab environment, on chosen tasks � Mostly quantitative observations (time, error rate, satisfaction) Fall 2005 6.831 UI Design and Implementation 3 Fall 2005 6.831 UI Design and Implementation 4 1
!��������������������� "������������������ � Users are human beings � Performance anxiety � Human subjects have been seriously abused in � Feels like an intelligence test the past � Nazi concentration camps � Comparing self with other subjects � Tuskegee syphilis study � Feeling stupid in front of observers � MIT Fernald School study: feeding radioactive isotopes to mentally retarded children � Competing with other subjects � Yale electric shock study � Research involving user testing is now subject to close scrutiny � MIT Committee on Use of Humans as Experimental Subjects (COUHES) must approve research-related user studies Fall 2005 6.831 UI Design and Implementation 5 Fall 2005 6.831 UI Design and Implementation 6 ���������������#����$������ %������������ � Time � Time � Pilot-test all materials and tasks � Don � t waste it � Comfort � Comfort � � We � re testing the system; we � re not testing you. � � Make the user comfortable � � Any difficulties you encounter are the system � s fault. We need your help to find these problems. � � Informed consent � Privacy � Inform the user as fully as possible � � Your test results will be completely confidential. � � Information � Privacy � Brief about purpose of study � Preserve the user � s privacy � Inform about audiotaping, videotaping, other observers � Control � Answer any questions beforehand (unless biasing) � Control � The user can stop at any time � � You can stop at any time. � Fall 2005 6.831 UI Design and Implementation 7 Fall 2005 6.831 UI Design and Implementation 8 2
&�������������� '������������� � Time � Comfort � Eliminate unnecessary tasks � Comfort � Say what they � ve helped you do � Calm, relaxed atmosphere � Information � Take breaks in long session � Never act disappointed � Give tasks one at a time � Answer questions that you had to defer to � First task should be easy, for an early success experience avoid biasing the experiment � Privacy � User � s boss shouldn � t be watching � Privacy � Information � Answer questions (again, where they won � t bias) � Don � t publish user-identifying information � Control � User can give up a task and go on to the next � Don � t show video or audio without user � s � User can quit entirely permission Fall 2005 6.831 UI Design and Implementation 9 Fall 2005 6.831 UI Design and Implementation 10 �������(��!(�������� $���������������(��!(�������� � Find some users � User � Should be representative of the target user � Facilitator class(es), based on user analysis � Observers � Give each user some tasks � Should be representative of important tasks, based on task analysis � Watch user do the tasks Fall 2005 6.831 UI Design and Implementation 11 Fall 2005 6.831 UI Design and Implementation 12 3
�������$��� ��������������$��� � User should think aloud � Does the briefing � What they think is happening � Provides the tasks � What they � re trying to do � Why they took an action � Coaches the user to think aloud by � Problems asking questions � Feels weird � � What are you thinking? � � Thinking aloud may alter behavior � Disrupts concentration � � Why did you try that? � � Another approach: pairs of users � Controls the session and prevents � Two users working together are more likely to converse naturally interruptions by observers � Also called co-discovery, constructive interaction Fall 2005 6.831 UI Design and Implementation 13 Fall 2005 6.831 UI Design and Implementation 14 )*���(�����$��� $���������)*���(������ � Be quiet! � Pen & paper notes � Prepared forms can help � Don � t help, don � t explain, don � t point out mistakes � Audio recording � Sit on your hands if it helps � For think-aloud � Take notes � Video recording � Watch for critical incidents: events that strongly � Usability labs often set up with two cameras, one for user � s affect task performance or satisfaction face, one for screen � Usually negative � User may be self-conscious � Good for closed-circuit view by observers in another room � Errors � Generates too much data � Repeated attempts � Retrospective testing: go back through the video with the � Curses user, discussing critical incidents � May be positive � Screen capture & event logging � � Cool! � � Cheap and unobtrusive � � Oh, now I see. � � Camtasia, CamStudio Fall 2005 6.831 UI Design and Implementation 15 Fall 2005 6.831 UI Design and Implementation 16 4
Recommend
More recommend
