CMSC 20370/30370 Winter 2020 MidTerm Review Jan 31, 2020 - - PowerPoint PPT Presentation

CMSC 20370/30370 Winter 2020 MidTerm Review

Jan 31, 2020

Administrivia

• GP1 due on Monday

Today’s Agenda

• Survey Design
• Midterm review

USER NEEDS DESIGN/PROTOTYPE IMPLEMENT EVALUATE USER-CENTERED DESIGN

Surveys

• What is it?

– Tool to find out who your users are and what opinions they hold – Allows large group of people to describe themselves, their interests, and their preferences in structure way

• When to conduct a survey?

– Could be when you have existing product to understand current user base i.e. profile survey – Could be to check before redesign i.e. satisfaction survey – Could be to see what people find important i.e. value survey – Often better after some initial qualitative research (observations, field studies, focus groups, interviews) – Can also do surveys over time, pre/post, tracking surveys – Can follow up with qualitative research

How to set up a survey

• Let’s say you want to assess how staff at other

homeless shelters deal with S&P issues for their residents?

• Setting the schedule

– Includes time to write and test questions, recruiting, and fielding it

• Writing the survey

– Brainstorm questions – Survey goals

• Descriptive – profile the audience, summarize audience

composition

• Explanatory – explain peoples beliefs and behaviors

Write the questions

• Unlike interviews most closed ended
• Multiple choice question
• Checklist
• Specific, exhaustive, mutually exclusive

Types of questions

• Characteristics– describe who someone is
• Behavior – describe how someone

behaves

• Attitudinal – describes what people want

and believe

Characteristic categories

– Demographic – who are the respondents (age, gender, income etc.) – Technological – ask about digital setup and experience

Question Instructions Answers Reasons How many years have you worked at your shelter? Please select one

• f the options

below Radio buttons for no of years Find out how long they have been employed at the shelter What is your job description? Please select one

• f the options

below Radio buttons Staff Counselor Other Find out what their role is in the shelter

Behavior categories

• Technology use – how they (and

residents) use the technologies you care about

Question Instructions Answers Reasons What devices do your residents typically have? Please check all that apply Checkboxes Mobile, tablet, laptop, smartwatch, etc Find out what technologies their residents use

Attitudinal categories

• Satisfaction – do they like their current

technology setup?

• Preference – what do they find most

compelling about the current setup?

• Desire – what would they like to be

improved?

Question Instructions Answers Reasons How satisfied are you with your laptop for helping shelter residents? Please rate your satisfaction according to the following scale Likert scale 1 – totally unsatisfied 5 – totall satisfied Find out how happy they are with current system for specific features

Edit and order the questions

• Have an introduction, beginning, middle, and

end

• Instructions

– Emphasize that the survey is important – What it is for – Why peoples answers are safe – What the reward is – Who is responsible for the survey – How long the survey is running – Who to contact with questions

• Question instructions

Lay out the report

• Goals, methods, how you’re running things
• Especially in research, doesn’t hurt to start

early

• Recommended for your group project

reports too

Testing an online survey

• Error checking
• Functionality
• Usability
• Timing
• Response rate
• Test the survey
• The incentive

Fielding the survey

• Sample – people who fill out the survey
• Sampling frame

– Subset of the group of people your sampling method can put you in contact with – Ideally matches the whole of your target population e.g. people who work at homeless shelters – If not, can be inaccurate or misleading

Sample and Sampling Frame

Target Audience Sampling Frame Sample

Slide from Ken Fleischmann, UT-Austin

Population and Sample Size

1000 150

Slide from Ken Fleischmann, UT-Austin

Population and Sample Size

10,000 300

Slide from Ken Fleischmann, UT-Austin

Population and Sample Size

100,000 800

Slide from Ken Fleischmann, UT-Austin

Sampling Bias

Target Audience Sampling Frame Sample

Slide from Ken Fleischmann, UT-Austin

Bias

• When people you thought would respond are not

members of target population

– i.e. people not working at a homeless shelter, residents at a homeless shelter

• Non responder bias
• Timing bias
• Duration bias
• Invitation bias
• Self selection
• Presentation bias
• Expectation bias

Field the survey options

• Invitation
• Invitation link
• Email
• Random selection
• Telephone in person and standard mail

surveys

Survey Pitfalls

• People want everything
• People exaggerate
• People will choose answer even if they

don’t feel strongly about it

• People try to outguess the survey
• People lie
• Can use attention check questions to

mitigate

Midterm review: Topics 1

• User-centered design process

– Why do we need each stage – What happens at each stage – What techniques do we use and why

• Includes:

– Gathering User Needs

• surveys/interviews/observation/field work, types of data analysis, qualitative data coding,

being respectful and accommodating users, project ethics and need for IRB

– Design

• Making choices, getting right design, brainstorming, sketching

– Prototyping/Implementation

• Lo-fi and high-fi prototypes, Wizard of Oz, video prototyping, evolutionary vs throwaway

– Evaluation (includes survey design)

• Usability testing, inspection methods, qualitative techniques, formative vs summative,

evaluation planning, metrics for evaluation, quantitative techniques, experimental design

• Mapping from findings to design, working with community partners, setting

expectations

• Finding User Needs

– Determine target audience, recruiting, screeners, scheduling, interviewing basics, survey basics

Midterm review: Topics 2

• All required readings up to next Wed

– Users and Abuse: Intimate Partner Violence – Older Adults: Memory Music Box – Low income: DreamGigs – Accessibility: Sound Awareness – Low income: Tough Times – Accessibility: SeeingVR

• All assignment materials including Norman’s 7

principles of design

• Midterm will have a few short answer questions but
• thers will require you to use what you learned in class

Summary

• Need-finding for inclusive tech often depends on

interviews and surveys

• Surveys need to be carefully designed and piloted
• Class has covered basic research methods for

user-centered design

• In the next half of the class, we are going to

diver deeper into research on various underserved/marginalized communities

– We will ask why there is a need for research – We will give an overview of the types of research that are possible and considerations for each target user group

Coming up next class

• Final set of project proposal presentations
• Feel free to offer other teams offline

feedback

Get in touch:

Office hours: Fridays 2-4pm (Sign up in advance) or by appointment JCL 355 Email: marshini@uchicago.edu