Human factors, usability studies and validation testing. Where do - PowerPoint PPT Presentation

Human factors, usability studies and validation testing. Where do you start? How do you approach it? Ann Blandford Professor of Human – Computer Interaction

New technologies and changing expectations create new opportunities • New technologies create new possibilities. – New technologies for telehealth and new models of shared care – Networks of devices support data integration – Quantities of health and social data offer potential for new understanding • Miniaturisation makes more products feasible and affordable – E.g. portable breathalyser kits and vital signs monitoring equipment • Changing culture: “No decision about me without me” 2

It all looks so easy if you developed it… Video from UHN Toronto 3

Interactive healthcare technologies are not all fit for purpose • They need to work reliably – Hardware and software • “If the user can‟t use it, it doesn‟t work” – Basic usability • If the user doesn‟t use it, it doesn‟t work – E.g. medication reminder apps • If the user uses it “off label”, this introduces vulnerabilities into the system 4

Example: out and about with people taking oral contraceptives Work with Kathy Stawarz • What are the extent and causes of unintentional medication non-adherence? – Focus initially on oral contraceptives, and forgetting • Can technology help? • Methods: – Review of existing medication reminder apps – Online survey 5 http://www.flickr.com/photos/marklincoln/5255361533/

Study I: Smartphone apps review Classified and analysed the functionality of 229 smartphone medication reminder apps • 123 iPhone apps • 106 Android apps 25 dedicated contraception reminder apps • Finding: focus is on just-in-time reminders

Study II: Online survey Women taking oral contraception • N=1,083 • 75% age 18-25 years old http://www.flickr.com/photos/ambersphoto/2067610356/

Study II: Online survey What do you use to help you remember to take your pills on time? N=1,083

Study II: Online survey 90 80 70 60 50 Use (N=260) 40 Don't use (N=823) 30 20 80 70 10 60 0 50 forgot took late Forgot Took late 40 Last week Last month 30 20 Last month 90 10 0 80 Last week 70 Completely forgot at least once Took late at least 60 once 50 Routine (N=644) 40 No routine (N=439) 30 20 10 0 N=1,083 forgot took late Forgot Took late Last week Last month

Discussion • Current technology focuses on just-in-time reminders • Technology doesn‟t support routines • People rely on routines • People do not use the apps • Technology not fit for purpose?

What are people‟s current practices and future needs/options? A classic What design solutions address software those needs? development lifecycle How have needs and practices changed post-implementation? Does the proposed design solution still address the needs? Source: http://en.wikibooks.org/wik i/Introduction_to_Software _Engineering/Process/Life _Cycle Does the implemented system work as envisioned? 11

Understanding current practices and needs • To design the future, you need to understand the present • Methods for understanding the present include: – Reference to existing resources (systems, manuals, protocols, descriptions of how people live their lives) – Interviews / focus groups / surveys – Observation of work / life – Contextual Inquiry 12

Contextual Inquiry in a nutshell • Context: Takes place in the context of use • Ongoing Experience: Involves observation and questioning • Concrete data: Focus on real artefacts and events • Partnership: Data gatherer works with user on understanding their work / life – Recognises that they are experts in this • See Beyer & Holtzblatt (1998) “Contextual Design”, or other texts by same authors 13

Identifying possible design solutions • Inherently creative • Pure intuition can work – You‟re designing for yourself – How typical of the intended user population are you? • Strategies include: – Use of personas and scenarios – Sketching and prototyping – Use of storyboards – Drawing on design patterns / analogous solutions 14

Personas • A persona is a rich (evidence-based) description of a typical user of the intended system • The two temptations for developers envisioning users are: – The users are just like me. They know as much as I do about my design, so it will be obvious to them how it works. – The users are exploring the system to see what is possible with it, and need little knowledge to do that. • Personas provide a „tool for thinking ‟ that encapsulate features of Real Users. 15

Scenarios • Scenarios are “informal narrative descriptions” of human activities and tasks that illustrate how systems support work. • Focus on user activity and motivations, rather than system functions – Similar to but not the same as use cases. 16

Sketches and prototypes • A prototype is a representation of some aspects of design that helps with idea-generation, communication and evaluation. E.g. – Cardboard physical mock-up – Paper sketches of screens – Complex demonstrator software 15:27 17

Checking proposed solutions against needs • Formative testing of prototypes. Approaches include: – Expert review – Laboratory / simlab testing with users – Think-aloud testing with users • Involves iteration • For medical devices seeking regulatory approval, may also involve clinical trial or other pre-market approval processes 18

Expert review • Expert review of an early prototype or implemented system involves… – Defining some user tasks (typical and edge-case) – Walking through the system interaction systematically looking for usability issues – E.g. Heuristic Evaluation • Jakob Nielsen: www.nngroup.com • Expert review typically assesses usability, not utility 19

User testing of implemented system • Approaches include: – Formal user testing – Post-market surveillance – User feedback – Observations / interviews of practice 20

Testing with users • User testing involves working with representative users of a system to identify areas of difficulty with a system – Usually relatively late in development process – Tasks may be defined by analyst or user – Setting may be more or less realistic – Measures may be quantitative • E.g. time to complete task; number of errors – …or qualitative • E.g. user difficulties or perceptions – May involve user diaries for „home‟ systems 21

New designs create new opportunities for use • There is a co-evolution between design and use – Some new uses may be positively regarded – Others (workarounds, violations) expose design limitations • Methods for understanding evolved practices include: – Interviews – Observations – Contextual Inquiry 22

So you know what the problems are… • Prioritise them: – How easy to fix? – How critical / beneficial? – In the current development cycle or the next one? • But also make sure you really understand user requirements, values and practices 23

Example: weight management through iCrave Work with Anne Hsu • Started with an interview study to find out people‟s challenges and motivations for weight management • Focused in on one challenge that people reported: overcoming cravings – Exploited and tested theory about mental imagery 24

iCrave: the prototype � 25

iCrave: the outcome of user testing � • Found to be easy to use • In a one-week user test, it was found to reduce unhealthy snacking (according to self-report) • Appears to have helped foster longer-term change in attitudes to unhealthy snacking • Need longer- term testing… and don‟t assume that “one size fits all” 26

To design the future, you need to understand the present • Revolutions in use are rare, and are generally just fast evolutions! • Innovation is essential, but needs to be shaped by an understanding of people‟s needs and practices • There are many resources to support user-centred design 27

More on situated studies in healthcare • Volume 1 now available: Fieldwork for Healthcare: Case Studies Investigating Human Factors in Computing Systems • Volume 2 (on strategies for conducting field studies in healthcare) coming soon… • Encyclopaedia article on doing qualitative studies: http://www.interaction- design.org/encyclopedia/semi- structured_qualitative_studies.html 28

References • Hsu, A. & Blandford, A. (2014) Designing for Psychological Change: Individuals‟ Reward and Cost Valuations in Weight Management. Journal of Medical Internet Research. 16.6:e138. http://www.jmir.org/2014/6/e138/ • Hsu, A., Yang, J., Yilmaz, Y., Haque, M. S., Can, C. & Blandford, A. (2014) Applying Decision Theory To Persuasive Design: Psychological Rewards And Costs In Weight Management in Proc. CHI'14. 3403-3412. • Stawarz, K., Cox, A. L., & Blandford, A. (2014). Don't forget your pill!: designing effective medication reminder apps that support users' daily routines. In Proc. CHI’14. 2269-2278. 29

Thank you! Questions? A.Blandford@ucl.ac.uk http://www.chi-med.ac.uk 30