Experimental Design & Evaluation 4. Contextual Inquiry - PowerPoint PPT Presentation

Experimental Design & Evaluation 4. Contextual Inquiry Sunyoung�Kim,�PhD�

Contextual Inquiry

Contextual Inquiry Go where the customer works, immerse yourself in the context, observe the customer (user) as he or she works, and talk to the customer about the work. Think yourself as an apprentice and the user as a master. You want to learn about the users’ work from them • A master teaches by doing the work and talking about it while working •

Conducting Contextual Inquiry Use Work-based interview when: Product or process already exists • User is able to complete a task while you observe • Work can be interrupted •

Contextual Inquiry An approach to answering the question, “What should we build to help people do their work better? Key elements Contextual Inquiry – Gather data from users while they do their • work to help decide what system should do. Work modeling – Use data to build models of work that are • explicit and sharable. Work redesign – Use data and models to design work model for • how users will work in the future. The core design problem is work design, rather than technology design. Use work model and iterative prototyping to drive detailed user • interface design.

Why Contextual Inquiry? Need to understand what will help people do their work better while fitting into their lives and matching their culture.

Summary vs. Ongoing Experience Retrospective accounts are often summaries “I got to work, checked my email and had a cup of coffee” By being present in the time and place of activity, we can access much richer data from ongoing activities “I got to work, looked over my email, answered messages from my boss, decided to have some coffee, walked to the coffee machine, found there was no coffee, so I made coffee…” à Abstract vs. Concrete data

Purposes of Contextual Inquiry To obtain data data from users in their context insights about the users’ environment • insights about their many tasks tasks • insights about the people people they work with • insights about cultural influences cultural influences on work (expectations, desires, • policies, values, etc.) understanding of br breakdowns eakdowns in current processes • To help define requirements, plans and designs and to prioritize

Attitudes during Contextual Inquiry Master/Apprentice model You are the Apprentice; the user is the Master - keeps investigator • humble, and customer in charge Investigator is not afraid to ask questions • Teaching while doing means user doesn’t have to think in advance • what to convey Allows discovery of subtle details, which may be overlooked in formal • discussion Reduces need for a formal set of interview questions •

Attitudes during Contextual Inquiry Be open to possibilities • Be committed to expand and ground the focus (more about focus • soon) Learn rather than teach • Be willing to modify your thinking • Pay attention to the participant’s needs • Pay attention to your needs •

Principles of Contextual Inquiry 1. Context: Understand users' needs in their work environment 2. Partnership: Work with users as co-investigators 3. Interpretation: Assign meaning to the observations 4. Focus: Listen & probe from a clearly defined set of concerns

1. Context The interrelated conditions within which something occurs or exists Understand work in its natural environment Go to the stakeholders (users, agents, etc.) • Observe real work • Interview while she/he is working • Interview Interview, Survey , Survey Contextual Inquiry Contextual Inquiry • Remembered experience (or • Ongoing experience & concrete summary data & abstractions data where work is happening • Subjective • Objective • Limited by reliability of human • Limited by ability to observe memory directly • What customers think & say • What customers do they do

Context: Do’s & Dont’s Go to the workplace & see the work as it unfolds • Seek concrete data • o Direct conversation to ongoing work User: "I usually record appointments in my calendar” Interviewer: "Could you go ahead & walk me through the process” o If direct observation is not possible in area of focus, elicit retrospective accounts” Think about the last time you used System-X. Take me through the steps you went through with the system to get the job done.”

Context: Do’s & Dont’s Vet your design ideas • o CI is also a fine time to get initial feedback on design ideas “If you had a technology that did X, would that solve this problem?” o Users will quickly understand the intent of your suggestion, and will be able to provide direct feedback o This will also demonstrate your understanding of the problem, providing opportunity for brainstorming and/or clarification

Context: Do’s & Dont’s Avoid summary information • Avoid abstract data • o Watch out for words like "usually", "generally" etc.: Abstractions are open to interpretation and can compromise the concrete data needed to guide design o Instead, do direct conversation towards ongoing work or recent past events You want responses that include words like: “Let me show you…” - “Here’s how I do it ...” - “Last time, I …” -

Pay attention to User's work space • User's work • User's work intentions • User's words • Tools used • How people work together • Business goals • Organizational and cultural structure • Be guided by focus (and open to changing focus if the changed focus • falls within another area of concern)

2. Partnership A relationship characterized by close cooperation. Build an equitable relationship with the user Not traditional interviewer-interviewee • Not master-apprentice model • An approximation for partnership, but has limitations • Not expert-novice (unless they are experts) • Not host-guest (be nosy) • AIM FOR EQUALITY CI Interviewer must built a partnership •

Establishing Partnership 1. Share control 2. Use open-ended questions that invite users to talk: "What are you doing?” • "Is that what you expect?” • "Why are you doing X ...?” • 3. Let the user lead the conversation (provided on focus) 4. Listen and listen! 5. Pay attention to communication that is non-verbal

Partnership: Do’s & Dont’s • Engage user as a partner in design o Be nosy -- clarify, probe, and watch o Test design ideas in work context in which they might be relevant-- “would it be useful if …” o Use caution on “would it be” questions • Do not assume role of interviewer o Questions should be in context of ongoing work

Partnership: Do’s & Dont’s • Do not assume role of expert o Answering user questions about system alters their behavior o Instead: "Why don’t you go ahead & do what you'd do if I weren't here, I'll answer any questions you may have at the end.” • Do not assume role of guest o Being overly polite may inhibit the kind of interaction that may lead to a better understanding of work. Instead, be nosy!

3. Interpretation Check your interpretation to make sure they are accurate • “I saw you just do X. Is that because of Y?” “I believe X. Is that correct?” As long as you check your interpretations in-context, participants will • respond accurately Outside of context, they may be more inclined to agree or answer in • generalities rather than specifics

4. Focus Focus is a perspective • Better to make it explicit • Characteristics of focus: • Helps keep conversation on useful topics - Set of pre-conceived assumptions & beliefs - Reveals & conceals - Role of focus Directs the selection of participants • Directs questioning • Creates understanding • Provides rich data •

Focus: Do’s & Dont’s Directs the selection of participants • Directs questioning • Creates understanding • Provides rich data •

Focus: Do’s & Dont’s DO allow focus to steer conversation • DO allow focus to expand • o Probe surprises & contradictions o Probe what you don’t know or understand o Be wary of nodding in agreement (Do you REALLY understand?) DO challenge focus assumptions • o DON’T try to validate them DO avoid “expert blind spot” •

Focus differs in different stages Br Broad focus oad focus Artifact Usefulness Usefulness Narr Narrow focus ow focus Usability Usability Time in system development Early Late

Information you want to know Understanding people’s underlying goals goals • What are people trying to accomplish? - Understanding their current behavior • How do they spend their time time? - What are their priorities priorities? - What problems do they currently have? What inef inefficiencies ficiencies or - costs are they putting up with? How do they feel about different experiences experiences they have? -

Pay attention to: Flow of work: Direction of communication and coordination • Sequence of work: Detailed sequence of work steps • Culture of work: External influences • Artifacts used in the work: Physical objects that support the work • Physical environment of the work: Layout of the work environment •