Designing People+AI Systems Human-AI Interaction Luigi De Russis - PowerPoint PPT Presentation

Designing People+AI Systems Human-AI Interaction Luigi De Russis Academic Year 2019/2020

AI: Risks, Benefits, and User Tolerance 2 Human-AI Interaction

Human-AI Interaction Fall 19 . Uncertainty & Unpredictability for Users - Relinquishing control to an AI/ML agent can be helpful, but can be much harder to correct or understand if things go wrong - “Unpredictability” can joyful in one kind of experience, and a terrible idea in another 3

Human-AI Interaction Fall 19 . Risk: Severe Failure 4

Human-AI Interaction Fall 19 . What was the error?: Severe Failure ● Tay’s earlier version XiaoIce ran on China’s most widespread instant messaging app Wechat … without any major ethical incidents ● What makes Twitter a different environment? ● Tay had no moral agency . To her, words like Hitler or Holocaust are not different from words like chair or Oklahoma 5

Human-AI Interaction Fall 19 . Mitigating: Severe Failure 2017 Tay used some black-listing of 2018 Zoe uses both black-listing of ‘bad ‘bad words’ but could make no moral words’ and makes moral judgements. judgements. 6

Human-AI Interaction Fall 19 . Mitigating: Severe Failure “ It’s easier to program trigger-blindness than teach a bot how to recognize nuance. But the line between casual use (“We’re all Jews here”) and anti-Semitism (“They’re all Jews here”) can be difficult even for humans to parse. ” … “ Zo’s uncompromising approach to a whole cast of topics represents a troubling trend in AI: censorship without context” - Chloe Rose Stuart-Ulin, Quartz 7

Human-AI Interaction Fall 19 . Uncertainty & Unpredictability for companies & designers - It can be very hard (sometimes today impossible) to predict all the kinds of scenarios your system could wind up in - An unknown-unknown is a situation out there in the world that your system won’t handle correctly but won’t know it’s wrong - Model performance will likely change/degrade as time passes Because users change how they interact with the system - Because it sees new data that looks less like it’s training data - 8

Human-AI Interaction Fall 19 . What are some everyday errors we can expect? 9

Human-AI Interaction Fall 19 . ML/AI error: Poor model performance - Usually solvable by acquiring more training data for the situations the model is weakest at - Data is expensive to collect, and your company or organization has limited resources. Prioritizing what specific data to collect is essential - Designers can use rule or non-ML based fallbacks to still deliver the user some value when model performance isn’t good enough for some cases 10

Human-AI Interaction Fall 19 . ML/AI error: Low confidence or false High confidence in a prediction - Low confidence predictions can mean that the model has lower performance, or the phenomena itself is just… less predictable - Communicating with the user or providing good non-AI/ML fallbacks is key - High confidence ( when the model is really wrong ) is worse Unkown unkown errors - - Need to give the user some error correction or feedback method to deal when this happens 11

Human-AI Interaction Fall 19 . ML/AI error: Relevance errors - Airbnb suggesting ‘fun local activities’ when you’re traveling for a funeral - Exercise app suggesting ‘time to get up and walk!’ when you’re seated on a long flight - Amazon suggesting products that you are allergic to or can’t eat 12

Human-AI Interaction Fall 19 . ML/AI error: Multiple users and kinds of input that look the same to the system 1. Use Spotify to play 1970s pop jams at your Mum’s party 2. Use Spotify to play your favorite study jams 3. Use Spotify to hate-listen to Ariana Grande (sorry) with your roommate 4. Your roommate also controls the same Spotify account to play their favorite study jams What music should Spotify recommend this account play? 13

To recap: what are the stakes for AI failure? User: high stakes o AI causes active harm (e.g. recidivism prediction or hiring prediction) o AI reveals information someone wanted kept private o AI shows offensive content User: low stakes o AI/ML feature is annoying or interrupting o AI/ML feature is often wrong o AI/ML feature is useless Product/Service organization o Users stop using your app/service because of poor AI/ML performance o Bad press or legal troubles o Bad reviews discouraging others from using the app/service 14 Human-AI Interaction

To recap: what AI limitations may look like During Development Deployed in the wild o Insufficient data o ML features fails silently o Poor model accuracy o ML feature reduces or does not add engagement o ML task is more expensive than it’s worth What to do o User scenarios not sufficiently mapped o Design performance metrics for deployment around engagement, use, What to do accuracy § Start with scenarios, involving different o Collect (private and appropriate) user data stakeholders (no tech push) o If possible, do a field study with a special § Start with de-risking group of beta users § Purposefully design (quantitative) metrics to match scenarios and user studies 15 Human-AI Interaction

User Tolerance To AI Failures § What is the role of an AI feature? § Should it be: o Critical or Complimentary • if a system can still work without the feature that AI enables, AI is complementary o Proactive or Reactive • Proactive feature: it provides results without people requesting it to do so • Reactive feature: it provides results when people ask for them or when they take certain actions o Visible or Invisible o Dynamic or Static • how features evolve over time (source: https://developer.apple.com/design/human-interface-guidelines/machine-learning/overview/roles/) 16 Human-AI Interaction

User Tolerance: Critical or Complimentary § In general, the more critical an app feature is, the more people need accurate and reliable results § On the other hand, if a complementary feature delivers results that are not always of the highest quality, people may be more forgiving 17 Human-AI Interaction

User Tolerance: Proactive or Reactive § Proactive features can prompt new tasks and interactions by providing unexpected, sometimes serendipitous results § Reactive features typically help people as they perform their current task § Because people do not ask for the results that a proactive feature provides, they may have less tolerance for low-quality information o such features have more potential to be annoying 18 Human-AI Interaction

User Tolerance: Proactive or Reactive § Proactive features can be helpful o in small amounts o at the "right" moment o if they are easy to dismiss 19 Human-AI Interaction

User Tolerance: Visible or Invisible § People's impression of the reliability of results can differ depending on whether a feature is visible or invisible § With a visible feature, people form an opinion about the feature's reliability as they choose from among its results § It is harder for an invisible feature to communicate its reliability - and potentially receive feedback - because people may not be aware of the feature at all 20 Human-AI Interaction

User Tolerance: Dynamic or Static § In addition to the frequency of a system updates, static or dynamic improvements affect other parts of the user experience § For example, dynamic features often incorporate forms of calibration and feedback (either implicit or explicit), whereas static features may not 21 Human-AI Interaction

User Tolerance To Give Feedback § Do not overuse feedback requests or users will get annoyed o People won't like to feel like the AI is so stupid that it needs their help § Save for high stakes failure, is possible 22 Human-AI Interaction

User Tolerance To AI Failures § How should an AI system best react to failure not to lose the user's trust? o or not to be turned off/abandoned § Which roles AI features should have? And when? § Examples: o YouTube Recommendation are Visible + Complementary + Proactive o Smart home features could be Invisible + Critical • some of them could be reactive, other proactive 23 Human-AI Interaction

"To AI or not to AI?" source: https://pair.withgoogle.com/worksheet/user-needs.pdf 24 Human-AI Interaction

Choosing the People+AI Path Mitigating Risks, Increasing Tolerance, Highlighting Benefits 25 Human-AI Interaction

Guidelines § By Microsoft Research o https://www.microsoft.c om/en- us/research/project/guid elines-for-human-ai- interaction/ § Saleema Amershi et al. Guidelines for Human-AI Interaction. ACM CHI 2019 o https://doi.org/10.1145/32 90605.3300233 26 Human-AI Interaction

Sources § Slides with the "Human-AI Interaction Fall 19" banner are taken from the Human-AI Interaction class at Carnegie Mellon University o http://www.humanaiclass.org § All the other sources are reported when they first occurred 27 Human-AI Interaction